A Computational Introduction to Number Theory and Algebra (Version 2.2)
Victor Shoup
Version History 2.2: Mar. 6, 2008. More typos fixed, some improvements to exposition. 2.1: Jan. 21, 2008. Fixed some typos, changed to Times Roman font. 2.0: Dec. 18, 2007. Extensive organizational and notational changes, as well as corrections, some additional material, and many additional exercises. 1.0: Jan. 15, 2005. Corresponds to the first print edition, published by Cambridge University Press (2005).
Navigation. This PDF document contains hyperlinks, and one may navigate through it by clicking on theorem, definition, lemma, equation, and page numbers, as well as URLs, and chapter and section titles in the table of contents; most PDF viewers should also display a list of “bookmarks” that allow direct access to chapters and sections.
Copyright © 2007 by Victor Shoup
The electronic version of this work is distributed under the terms and conditions of a Creative Commons license (Attribution-NonCommercial-NoDerivs 2.0): You are free to copy, distribute, and display the electronic version of this work under the following conditions: Attribution. You must give the original author credit. Noncommercial. You may not use the electronic version of this work for commercial purposes. No Derivative Works. You may not alter, transform, or build upon the electronic version of this work. For any reuse or distribution, you must make clear to others the license terms of this work. Any of these conditions can be waived if you get permission from the author. For more information about the license, visit creativecommons.org/licenses/by-nd-nc/2.0. All other rights, including the right to distribute of this work in print (or any other) form, are reserved.
Contents
page x xiv
Preface Preliminaries 1
Basic properties of the integers 1.1 Divisibility and primality 1.2 Ideals and greatest common divisors 1.3 Some consequences of unique factorization
1 1 5 10
2
Congruences 2.1 Equivalence relations 2.2 Definitions and basic properties of congruences 2.3 Solving linear congruences 2.4 The Chinese remainder theorem 2.5 Residue classes 2.6 Euler’s phi function 2.7 Euler’s theorem and Fermat’s little theorem 2.8 Quadratic residues 2.9 Summations over divisors
14 14 15 18 21 23 30 32 35 45
3
Computing with large integers 3.1 Asymptotic notation 3.2 Machine models and complexity theory 3.3 Basic integer arithmetic 3.4 Computing in Zn 3.5 Faster integer arithmetic ./ 3.6 Notes
49 49 52 54 64 68 70
4
Euclid’s algorithm 4.1 The basic Euclidean algorithm 4.2 The extended Euclidean algorithm 4.3 Computing modular inverses and Chinese remaindering
73 73 76 80
v
vi
Contents
4.4 4.5 4.6 4.7 4.8
Speeding up algorithms via modular computation An effective version of Fermat’s two squares theorem Rational reconstruction and applications The RSA cryptosystem Notes
82 85 88 98 101
5
The distribution of primes 5.1 Chebyshev’s theorem on the density of primes 5.2 Bertrand’s postulate 5.3 Mertens’ theorem 5.4 The sieve of Eratosthenes 5.5 The prime number theorem . . . and beyond 5.6 Notes
103 103 107 109 114 115 123
6
Abelian groups 6.1 Definitions, basic properties, and examples 6.2 Subgroups 6.3 Cosets and quotient groups 6.4 Group homomorphisms and isomorphisms 6.5 Cyclic groups 6.6 The structure of finite abelian groups ./
125 125 131 136 141 152 162
7
Rings 7.1 Definitions, basic properties, and examples 7.2 Polynomial rings 7.3 Ideals and quotient rings 7.4 Ring homomorphisms and isomorphisms 7.5 The structure of Zn
165 165 175 184 190 202
8
Finite and discrete probability distributions 8.1 Basic definitions 8.2 Conditional probability and independence 8.3 Random variables 8.4 Expectation and variance 8.5 Some useful bounds 8.6 Balls and bins 8.7 Hash functions 8.8 Statistical distance 8.9 Measures of randomness and the leftover hash lemma ./ 8.10 Discrete probability distributions 8.11 Notes
206 206 212 220 233 240 244 251 259 264 269 274
Contents
vii
9
Probabilistic algorithms 9.1 Basic definitions 9.2 Generating a random number from a given interval 9.3 The generate and test paradigm 9.4 Generating a random prime 9.5 Generating a random non-increasing sequence 9.6 Generating a random factored number 9.7 Some complexity theory 9.8 Notes
275 276 283 285 290 293 296 300 302
10
Probabilistic primality testing 10.1 Trial division 10.2 The Miller–Rabin test 10.3 Generating random primes using the Miller–Rabin test 10.4 Factoring and computing Euler’s phi function 10.5 Notes
304 304 305 309 318 322
11
Finding generators and discrete logarithms in Zp 11.1 Finding a generator for Zp 11.2 Computing discrete logarithms in Zp 11.3 The Diffie–Hellman key establishment protocol 11.4 Notes
325 325 327 333 338
12
Quadratic reciprocity and computing modular square roots 12.1 The Legendre symbol 12.2 The Jacobi symbol 12.3 Computing the Jacobi symbol 12.4 Testing quadratic residuosity 12.5 Computing modular square roots 12.6 The quadratic residuosity assumption 12.7 Notes
340 340 344 346 347 348 354 355
13
Modules and vector spaces 13.1 Definitions, basic properties, and examples 13.2 Submodules and quotient modules 13.3 Module homomorphisms and isomorphisms 13.4 Linear independence and bases 13.5 Vector spaces and dimension
357 357 359 361 366 369
14
Matrices 14.1 Basic definitions and properties 14.2 Matrices and linear maps 14.3 The inverse of a matrix
376 376 380 385
viii
Contents
14.4 Gaussian elimination 14.5 Applications of Gaussian elimination 14.6 Notes
387 391 397
15
Subexponential-time discrete logarithms and factoring 15.1 Smooth numbers 15.2 An algorithm for discrete logarithms 15.3 An algorithm for factoring integers 15.4 Practical improvements 15.5 Notes
398 398 399 406 413 418
16
More rings 16.1 Algebras 16.2 The field of fractions of an integral domain 16.3 Unique factorization of polynomials 16.4 Polynomial congruences 16.5 Minimal polynomials 16.6 General properties of extension fields 16.7 Formal derivatives 16.8 Formal power series and Laurent series 16.9 Unique factorization domains ./ 16.10 Notes
420 420 425 428 433 436 438 442 444 448 462
17
Polynomial arithmetic and applications 17.1 Basic arithmetic 17.2 Computing minimal polynomials in F ŒX=.f / (I) 17.3 Euclid’s algorithm 17.4 Computing modular inverses and Chinese remaindering 17.5 Rational function reconstruction and applications 17.6 Faster polynomial arithmetic ./ 17.7 Notes
463 463 466 467 470 472 476 482
18
Linearly generated sequences and applications 18.1 Basic definitions and properties 18.2 Computing minimal polynomials: a special case 18.3 Computing minimal polynomials: a more general case 18.4 Solving sparse linear systems 18.5 Computing minimal polynomials in F ŒX=.f / (II) 18.6 The algebra of linear transformations ./ 18.7 Notes
484 484 488 490 495 498 499 506
19
Finite fields 19.1 Preliminaries
507 507
Contents
20
21
ix
19.2 The existence of finite fields 19.3 The subfield structure and uniqueness of finite fields 19.4 Conjugates, norms and traces
509 513 515
Algorithms for finite fields 20.1 Generating and constructing irreducible polynomials 20.2 Computing minimal polynomials in F ŒX=.f / (III) 20.3 Factoring polynomials: square-free decomposition 20.4 Factoring polynomials: the Cantor–Zassenhaus algorithm 20.5 Factoring polynomials: Berlekamp’s algorithm 20.6 Deterministic factorization algorithms ./ 20.7 Notes
521 521 524 525 529 537 543 545
Deterministic primality testing 21.1 The basic idea 21.2 The algorithm and its analysis 21.3 Notes Appendix: Some useful facts Bibliography Index of notation Index
547 547 548 558 559 564 570 572
Preface
Number theory and algebra play an increasingly significant role in computing and communications, as evidenced by the striking applications of these subjects to such fields as cryptography and coding theory. My goal in writing this book was to provide an introduction to number theory and algebra, with an emphasis on algorithms and applications, that would be accessible to a broad audience. In particular, I wanted to write a book that would be accessible to typical students in computer science or mathematics who have a some amount of general mathematical experience, but without presuming too much specific mathematical knowledge. Prerequisites. The mathematical prerequisites are minimal: no particular mathematical concepts beyond what is taught in a typical undergraduate calculus sequence are assumed. The computer science prerequisites are also quite minimal: it is assumed that the reader is proficient in programming, and has had some exposure to the analysis of algorithms, essentially at the level of an undergraduate course on algorithms and data structures. Even though it is mathematically quite self contained, the text does presuppose that the reader is comfortable with mathematical formalism and has some experience in reading and writing mathematical proofs. Readers may have gained such experience in computer science courses such as algorithms, automata or complexity theory, or some type of “discrete mathematics for computer science students” course. They also may have gained such experience in undergraduate mathematics courses, such as abstract or linear algebra — these courses overlap with some of the material presented here, but even if the reader already has had some exposure to this material, it nevertheless may be convenient to have all of the relevant material easily accessible in one place, and moreover, the emphasis and perspective here will no doubt be different than in a typical mathematics course on these subjects. Structure of the text. All of the mathematics required beyond basic calculus is dex
Preface
xi
veloped “from scratch.” Moreover, the book generally alternates between “theory” and “applications”: one or two chapters on a particular set of purely mathematical concepts are followed by one or two chapters on algorithms and applications — the mathematics provides the theoretical underpinnings for the applications, while the applications both motivate and illustrate the mathematics. Of course, this dichotomy between theory and applications is not perfectly maintained: the chapters that focus mainly on applications include the development of some of the mathematics that is specific to a particular application, and very occasionally, some of the chapters that focus mainly on mathematics include a discussion of related algorithmic ideas as well. In developing the mathematics needed to discuss certain applications, I tried to strike a reasonable balance between, on the one hand, presenting the absolute minimum required to understand and rigorously analyze the applications, and on the other hand, presenting a full-blown development of the relevant mathematics. In striking this balance, I wanted to be fairly economical and concise, while at the same time, I wanted to develop enough of the theory so as to present a fairly wellrounded account, giving the reader more of a feeling for the mathematical “big picture.” The mathematical material covered includes the basics of number theory (including unique factorization, congruences, the distribution of primes, and quadratic reciprocity) and abstract algebra (including groups, rings, fields, and vector spaces). It also includes an introduction to discrete probability theory — this material is needed to properly treat the topics of probabilistic algorithms and cryptographic applications. The treatment of all these topics is more or less standard, except that the text only deals with commutative structures (i.e., abelian groups and commutative rings with unity) — this is all that is really needed for the purposes of this text, and the theory of these structures is much simpler and more transparent than that of more general, non-commutative structures. The choice of topics covered in this book was motivated primarily by their applicability to computing and communications, especially to the specific areas of cryptography and coding theory. For example, the book may be useful for reference or self-study by readers who want to learn about cryptography. The book could also be used as a textbook in a graduate or upper-division undergraduate course on (computational) number theory and algebra, perhaps geared towards computer science students. Since this is an introductory textbook, and not an encyclopedic reference for specialists, some topics simply could not be covered. One such topic whose exclusion will undoubtedly be lamented by some is the theory of lattices, along with algorithms for and applications of lattice basis reduction. Another such topic is that of fast algorithms for integer and polynomial arithmetic — although some of
xii
Preface
the basic ideas of this topic are developed in the exercises, the main body of the text deals only with classical, quadratic-time algorithms for integer and polynomial arithmetic. As an introductory text, some topics just had to go; moreover, there are more advanced texts that cover these topics perfectly well, and these texts should be readily accessible to students who have mastered the material in this book. Note that while continued fractions are not discussed, the closely related problem of “rational reconstruction” is covered, along with a number of interesting applications (which could also be solved using continued fractions). Using the text. Here are a few guidelines on using the text. There are a few sections that are marked with a “./,” indicating that the material covered in that section is a bit technical, and is not needed elsewhere. There are many examples in the text. These form an integral part of the text, and should not be skipped. There are a number of exercises in the text that serve to reinforce, as well as to develop important applications and generalizations of, the material presented in the text. Some exercises are underlined. These specially marked exercises develop important (but usually simple) facts, and should be viewed as an integral part of the text. It is highly recommended that the reader work these exercises, or at the very least, read and understand their statements. In solving exercises, the reader is free to use any previously stated results in the text, including those in previous exercises. However, except where otherwise noted, any result in a section marked with a “./,” or in §5.5, need not and should not be used outside the section in which it appears. There is a very brief “Preliminaries” chapter, which fixes a bit of notation and recalls a few standard facts. This should be skimmed over by the reader. There is an appendix that contains a few useful facts; where such a fact is used in the text, there is a reference such as “see §An,” which refers to the item labeled “An” in the appendix. Feedback. I welcome comments on the book (suggestions for improvement, error reports, etc.) from readers. Please send your comments to [email protected]. There is also a web site where further material and information relating to the book (including a list of errata and the latest electronic version of the book) may be found: www.shoup.net/ntb.
Preface
xiii
Acknowledgments. I would like to thank a number of people who volunteered their time and energy in reviewing parts of the book at different stages: Siddhartha Annapureddy, John Black, Carl Bosley, Joshua Brody, Jan Camenisch, Ronald Cramer, Alex Dent, Nelly Fazio, Mark Giesbrecht, Stuart Haber, Gene Itkis, Alfred Menezes, Antonio Nicolosi, Roberto Oliveira, Louis Salvail, and George Stephanides. I am also grateful to the National Science Foundation for their support provided under grant CCR-0310297. Thanks to David Tranah and his colleagues at Cambridge University Press for their progressive attitudes regarding intellectual property and open access. Victor Shoup
Preliminaries
We establish here some terminology, notation, and simple facts that will be used throughout the text. Logarithms and exponentials We write log x for the natural logarithm of x, and logb x for the logarithm of x to the base b. We write e x for the usual exponential function, where e 2:71828 is the base of the natural logarithm. We may also write expŒx instead of e x . Numbers We use standard notation for various sets of numbers: Z WD the set of integers D f: : : ; 2; 1; 0; 1; 2; : : :g; Q WD the set of rational numbers D fa=b W a; b 2 Z; b ¤ 0g; R WD the set of real numbers; C WD the set of complex numbers: We sometimes use the symbols 1 and 1 in simple arithmetic expressions involving real numbers. The interpretation given to such expressions should be obvious: for example, for every x 2 R, we have 1 < x < 1, x C 1 D 1, x 1 D 1, 1 C 1 D 1, and . 1/ C . 1/ D 1. Expressions such as x .˙1/ also make sense, provided x ¤ 0. However, the expressions 1 1 and 0 1 have no sensible interpretation. We use standard notation for specifying intervals of real numbers: for a; b 2 R
xiv
xv
Preliminaries
with a b, Œa; b WD fx 2 R W a x bg;
.a; b/ WD fa 2 R W a < x < bg;
Œa; b/ WD fx 2 R W a x < bg;
.a; b WD fa 2 R W a < x bg:
As usual, this notation is extended to allow a D and b D 1 for intervals Œa; b/ and .a; b/.
1 for intervals .a; b and .a; b/,
Sets and families We use standard set-theoretic notation: ; denotes the empty set; x 2 A means that x is an element, or member, of the set A; for two sets A; B, A B means that A is a subset of B (with A possibly equal to B), and A ¨ B means that A is a proper subset of B (i.e., A B but A ¤ B). Further, A [ B denotes the union of A and B, A \ B the intersection of A and B, and A n B the set of all elements of A that are not in B. If A is a set with a finite number of elements, then we write jAj for its size, or cardinality. We use standard notation for specifying the elements of a set. For example, the set of all even integers could be specified as fz 2 Z W z=2 2 Zg or as f2z W z 2 Zg. We write S1 Sn for the Cartesian product of sets S1 ; : : : ; Sn , that is, the set of all n-tuples .a1 ; : : : ; an /, where ai 2 Si for i D 1; : : : ; n. We write S n for the Cartesian product of n copies of a set S, and for x 2 S , we write x n for the element of S n consisting of n copies of x. (This notation is a bit non-standard, but we reserve the more standard notation S n for other purposes, so as to avoid ambiguity.) A family is a collection of objects, indexed by some set I , called an index set. If for each i 2 I we have an associated object xi , the family of all such objects is denoted by fxi gi 2I . Unlike a set, a family may contain duplicates; that is, we may have xi D xj for some pair of indices i; j with i ¤ j . If the index set I has some natural order, then we may view the family as being ordered in the same way. As a special case, a family indexed by a subset of Z of the form fm; : : : ; ng or fm; m C 1; : : :g is a sequence, which we may write as fxi gniDm or fxi g1 i Dm . Note that while fxi gi 2I denotes a family, fxi W i 2 I g denotes the set whose members are the (distinct) xi ’s. On occasion, if the choice of index set is not important, we may simply define a family by listing or describing its members, without explicitly describing an index set; for example, “the family of objects a; b; c,” means the family fxi g3iD1 , where x1 WD a, x2 WD b, and x3 WD c. Unions and intersections may be generalized to arbitrary families of sets. For a family fSi gi 2I of sets, the union is [ Si WD fx W x 2 Si for some i 2 I g; i 2I
xvi
Preliminaries
and for I ¤ ;, the intersection is \ Si WD fx W x 2 Si for all i 2 I g: i 2I
Note that if I D ;, the union is by definition ;, but the intersection is, in general, not well defined; however, in certain application, one might define it by a special convention; for example, if all sets under consideration are subsets of some “ambient space,” ˝, then the empty intersection is usually taken to be ˝. Two sets A and B are called disjoint if A \ B D ;. A family fSi gi 2I of sets is called pairwise disjoint if Si \ Sj D ; for all i; j 2 I with i ¤ j . A pairwise disjoint family of non-empty sets whose union is S is called a partition of S ; equivalently, fSi gi 2I is a partition of a set S if each Si is a non-empty subset of S , and each element of S belongs to exactly one Si . Functions We write f W A ! B to indicate that f is a function (also called a map) from a set A to a set B. If A0 A, then f .A0 / WD ff .a/ W a 2 A0 g is the image of A0 under f , and f .A/ is simply referred to as the image of f ; if B 0 B, then f 1 .B 0 / WD fa 2 A W f .a/ 2 B 0 g is the pre-image of B 0 under f . A function f W A ! B is called one-to-one or injective if f .a/ D f .b/ implies a D b. The function f is called onto or surjective if f .A/ D B. The function f is called bijective if it is both injective and surjective; in this case, f is called a bijection, or a one-to-one correspondence. If f is bijective, then we may define the inverse function f 1 W B ! A, where for b 2 B, f 1 .b/ is defined to be the unique a 2 A such that f .a/ D b; in this case, f 1 is also a bijection, and .f 1 / 1 D f . If A0 A, then the inclusion map from A0 to A is the function i W A0 ! A given by i.a/ WD a for a 2 A0 ; when A0 D A, this is called the identity map on A. If A0 A, f 0 W A0 ! B, f W A ! B, and f 0 .a/ D f .a/ for all a 2 A0 , then we say that f 0 is the restriction of f to A0 , and that f is an extension of f 0 to A. If f W A ! B and g W B ! C are functions, their composition is the function g B f W A ! C given by .g B f /.a/ WD g.f .a// for a 2 A. If f W A ! B is a bijection, then f 1 B f is the identity map on A, and f B f 1 is the identity map on B. Conversely, if f W A ! B and g W B ! A are functions such that g B f is the identity map on A and f B g is the identity map on B, then f and g are bijections, each being the inverse of the other. If f W A ! B and g W B ! C are bijections, then so is g B f , and .g B f / 1 D f 1 B g 1 . Function composition is associative; that is, for all functions f W A ! B, g W B ! C , and h W C ! D, we have .h B g/ B f D h B .g B f /. Thus, we
Preliminaries
xvii
can simply write h B g B f without any ambiguity. More generally, if we have functions fi W Ai ! AiC1 for i D 1; : : : ; n, where n 2, then we may write their composition as fn B B f1 without any ambiguity. If each fi is a bijection, then so is fn B B f1 , its inverse being f1 1 B B fn 1 . As a special case of this, if Ai D A and fi D f for i D 1; : : : ; n, then we may write fn B B f1 as f n . It is understood that f 1 D f , and that f 0 is the identity map on A. If f is a bijection, then so is f n for every non-negative integer n, the inverse function of f n being .f 1 /n , which one may simply write as f n . If f W I ! S is a function, then we may view f as the family fxi gi 2I , where xi WD f .i/. Conversely, a family fxi gi 2I , where all of the xi ’s belong to some set S, may be viewed as the function f W I ! S given by f .i / WD xi for i 2 I . Really, functions and families are the same thing, the difference being just one of notation and emphasis. Binary operations A binary operation ? on a set S is a function from S S to S, where the value of the function at .a; b/ 2 S S is denoted a ? b. A binary operation ? on S is called associative if for all a; b; c 2 S , we have .a ? b/ ? c D a ? .b ? c/. In this case, we can simply write a ? b ? c without any ambiguity. More generally, for a1 ; : : : ; an 2 S, where n 2, we can write a1 ? ? an without any ambiguity. A binary operation ? on S is called commutative if for all a; b 2 S , we have a ? b D b ? a. If the binary operation ? is both associative and commutative, then not only is the expression a1 ? ? an unambiguous, but its value remains unchanged even if we re-order the ai ’s. If ? is a binary operation on S , and S 0 S , then S 0 is called closed under ? if a ? b 2 S 0 for all a; b 2 S 0 .
1 Basic properties of the integers
This chapter discusses some of the basic properties of the integers, including the notions of divisibility and primality, unique factorization into primes, greatest common divisors, and least common multiples. 1.1 Divisibility and primality A central concept in number theory is divisibility. Consider the integers Z D f: : : ; 2; 1; 0; 1; 2; : : :g. For a; b 2 Z, we say that a divides b if az D b for some z 2 Z. If a divides b, we write a j b, and we may say that a is a divisor of b, or that b is a multiple of a, or that b is divisible by a. If a does not divide b, then we write a − b. We first state some simple facts about divisibility: Theorem 1.1. For all a; b; c 2 Z, we have (i) a j a, 1 j b, and a j 0; (ii) 0 j b if and only if b D 0; (iii) a j b if and only if a j b if and only if a j (iv) a j b and a j c implies a j .b C c/; (v) a j b and b j c implies a j c.
b;
Proof. These properties can be easily derived from the definition of divisibility, using elementary algebraic properties of the integers. For example, a j a because we can write a 1 D a; 1 j b because we can write 1 b D b; a j 0 because we can write a 0 D 0. We leave it as an easy exercise for the reader to verify the remaining properties. We make a simple observation: if a j b and b ¤ 0, then 1 jaj jbj. Indeed, if az D b ¤ 0 for some integer z, then a ¤ 0 and z ¤ 0; it follows that jaj 1, jzj 1, and so jaj jajjzj D jbj. 1
2
Basic properties of the integers
Theorem 1.2. For all a; b 2 Z, we have a j b and b j a if and only if a D ˙b. In particular, for every a 2 Z, we have a j 1 if and only if a D ˙1. Proof. Clearly, if a D ˙b, then a j b and b j a. So let us assume that a j b and b j a, and prove that a D ˙b. If either of a or b are zero, then the other must be zero as well. So assume that neither is zero. By the above observation, a j b implies jaj jbj, and b j a implies jbj jaj; thus, jaj D jbj, and so a D ˙b. That proves the first statement. The second statement follows from the first by setting b WD 1, and noting that 1 j a. The product of any two non-zero integers is again non-zero. This implies the usual cancellation law: if a, b, and c are integers such that a ¤ 0 and ab D ac, then we must have b D c; indeed, ab D ac implies a.b c/ D 0, and so a ¤ 0 implies b c D 0, and hence b D c. Primes and composites. Let n be a positive integer. Trivially, 1 and n divide n. If n > 1 and no other positive integers besides 1 and n divide n, then we say n is prime. If n > 1 but n is not prime, then we say that n is composite. The number 1 is not considered to be either prime or composite. Evidently, n is composite if and only if n D ab for some integers a; b with 1 < a < n and 1 < b < n. The first few primes are 2; 3; 5; 7; 11; 13; 17; : : : : While it is possible to extend the definition of prime and composite to negative integers, we shall not do so in this text: whenever we speak of a prime or composite number, we mean a positive integer. A basic fact is that every non-zero integer can be expressed as a signed product of primes in an essentially unique way. More precisely: Theorem 1.3 (Fundamental theorem of arithmetic). Every non-zero integer n can be expressed as n D ˙p1e1 prer ; where p1 ; : : : ; pr are distinct primes and e1 ; : : : ; er are positive integers. Moreover, this expression is unique, up to a reordering of the primes. Note that if n D ˙1 in the above theorem, then r D 0, and the product of zero terms is interpreted (as usual) as 1. The theorem intuitively says that the primes act as the “building blocks” out of which all non-zero integers can be formed by multiplication (and negation). The reader may be so familiar with this fact that he may feel it is somehow “self evident,” requiring no proof; however, this feeling is simply a delusion, and most
3
1.1 Divisibility and primality
of the rest of this section and the next are devoted to developing a proof of this theorem. We shall give a quite leisurely proof, introducing a number of other very important tools and concepts along the way that will be useful later. To prove Theorem 1.3, we may clearly assume that n is positive, since otherwise, we may multiply n by 1 and reduce to the case where n is positive. The proof of the existence part of Theorem 1.3 is easy. This amounts to showing that every positive integer n can be expressed as a product (possibly empty) of primes. We may prove this by induction on n. If n D 1, the statement is true, as n is the product of zero primes. Now let n > 1, and assume that every positive integer smaller than n can be expressed as a product of primes. If n is a prime, then the statement is true, as n is the product of one prime; otherwise, n is composite, and so there exist a; b 2 Z with 1 < a < n, 1 < b < n, and n D ab; by the induction hypothesis, both a and b can be expressed as a product of primes, and so the same holds for n. The uniqueness part of Theorem 1.3 is the hard part. An essential ingredient in this proof is the following: Theorem 1.4 (Division with remainder property). Let a; b 2 Z with b > 0. Then there exist unique q; r 2 Z such that a D bq C r and 0 r < b. Proof. Consider the set S of non-negative integers of the form a bt with t 2 Z. This set is clearly non-empty; indeed, if a 0, set t WD 0, and if a < 0, set t WD a. Since every non-empty set of non-negative integers contains a minimum, we define r to be the smallest element of S. By definition, r is of the form r D a bq for some q 2 Z, and r 0. Also, we must have r < b, since otherwise, r b would be an element of S smaller than r, contradicting the minimality of r; indeed, if r b, then we would have 0 r b D a b.q C 1/. That proves the existence of r and q. For uniqueness, suppose that a D bq C r and a D bq 0 C r 0 , where 0 r < b and 0 r 0 < b. Then subtracting these two equations and rearranging terms, we obtain r0
r D b.q
q 0 /:
Thus, r 0 r is a multiple of b; however, 0 r < b and 0 r 0 < b implies jr 0 rj < b; therefore, the only possibility is r 0 r D 0. Moreover, 0 D b.q q 0 / and b ¤ 0 implies q q 0 D 0. Theorem 1.4 can be visualized as follows:
0
r
b
2b
3b
a
4b
4
Basic properties of the integers
Starting with a, we subtract (or add, if a is negative) the value b until we end up with a number in the interval Œ0; b/. Floors and ceilings. Let us briefly recall the usual floor and ceiling functions, denoted bc and de, respectively. These are functions from R (the real numbers) to Z. For x 2 R, bxc is the greatest integer m x; equivalently, bxc is the unique integer m such that m x < m C 1, or put another way, such that x D m C for some 2 Œ0; 1/. Also, dxe is the smallest integer m x; equivalently, dxe is the unique integer m such that m 1 < x m, or put another way, such that x D m for some 2 Œ0; 1/. The mod operator. Now let a; b 2 Z with b > 0. If q and r are the unique integers from Theorem 1.4 that satisfy a D bq C r and 0 r < b, we define a mod b WD rI that is, a mod b denotes the remainder in dividing a by b. It is clear that b j a if and only if a mod b D 0. Dividing both sides of the equation a D bq C r by b, we obtain a=b D q C r=b. Since q 2 Z and r=b 2 Œ0; 1/, we see that q D ba=bc. Thus, .a mod b/ D a
bba=bc:
One can use this equation to extend the definition of a mod b to all integers a and b, with b ¤ 0; that is, for b < 0, we simply define a mod b to be a bba=bc. Theorem 1.4 may be generalized so that when dividing an integer a by a positive integer b, the remainder is placed in an interval other than Œ0; b/. Let x be any real number, and consider the interval Œx; x C b/. As the reader may easily verify, this interval contains precisely b integers, namely, dxe; : : : ; dxe C b 1. Applying Theorem 1.4 with a dxe in place of a, we obtain: Theorem 1.5. Let a; b 2 Z with b > 0, and let x 2 R. Then there exist unique q; r 2 Z such that a D bq C r and r 2 Œx; x C b/. E XERCISE 1.1. Let a; b; d 2 Z with d ¤ 0. Show that a j b if and only if da j db. E XERCISE 1.2. Let n be a composite integer. Show that there exists a prime p dividing n, with p n1=2 : E XERCISE 1.3. Let m be a positive integer. Show that for every real number x 1, the number of multiples of m in the interval Œ1; x is bx=mc; in particular, for every integer n 1, the number of multiples of m among 1; : : : ; n is bn=mc.
1.2 Ideals and greatest common divisors
5
E XERCISE 1.4. Let x 2 R. Show that 2bxc b2xc 2bxc C 1. E XERCISE 1.5. Let x 2 R and n 2 Z with n > 0. Show that bbxc=nc D bx=nc; in particular, bba=bc=cc D ba=bcc for all positive integers a; b; c. E XERCISE 1.6. Let a; b 2 Z with b < 0. Show that .a mod b/ 2 .b; 0. E XERCISE 1.7. Show that Theorem 1.5 also holds for intervals of the form .x; x C b. Does it hold in general for the intervals Œx; x C b or .x; x C b/? 1.2 Ideals and greatest common divisors To carry on with the proof of Theorem 1.3, we introduce the notion of an ideal of Z, which is a non-empty set of integers that is closed under addition, and under multiplication by an arbitrary integer. That is, a non-empty set I Z is an ideal if and only if for all a; b 2 I and all z 2 Z, we have a C b 2 I and az 2 I: It is easy to see that every ideal I contains 0: since a 2 I for some integer a, we have 0 D a 0 2 I . Also, note that if an ideal I contains an integer a, it also contains a, since a D a . 1/ 2 I . Thus, if an ideal contains a and b, it also contains a b. It is clear that f0g and Z are ideals. Moreover, an ideal I is equal to Z if and only if 1 2 I ; to see this, note that 1 2 I implies that for every z 2 Z, we have z D 1 z 2 I , and hence I D Z; conversely, if I D Z, then in particular, 1 2 I. For a 2 Z, define aZ WD faz W z 2 Zg; that is, aZ is the set of all multiples of a. If a D 0, then clearly aZ D f0g; otherwise, aZ consists of the distinct integers : : : ; 3a; 2a; a; 0; a; 2a; 3a; : : : : It is easy to see that aZ is an ideal: for all az; az 0 2 aZ and z 00 2 Z, we have az C az 0 D a.z C z 0 / 2 aZ and .az/z 00 D a.zz 00 / 2 aZ. The ideal aZ is called the ideal generated by a, and an ideal of the form aZ for some a 2 Z is called a principal ideal. Observe that for all a; b 2 Z, we have b 2 aZ if and only if a j b. Also observe that for every ideal I , we have b 2 I if and only if bZ I . Both of these observations are simple consequences of the definitions, as the reader may verify. Combining these two observations, we see that bZ aZ if and only if a j b. If I1 and I2 are ideals, then it is not hard to see that the set I1 C I2 WD fa1 C a2 W a1 2 I1 ; a2 2 I2 g is also an ideal. Indeed, suppose a1 Ca2 2 I1 CI2 and b1 Cb2 2 I1 C I2 . Then we have .a1 C a2 / C .b1 C b2 / D .a1 C b1 / C .a2 C b2 / 2 I1 C I2 , and for every z 2 Z, we have .a1 C a2 /z D a1 z C a2 z 2 I1 C I2 .
6
Basic properties of the integers
Example 1.1. Consider the principal ideal 3Z. This consists of all multiples of 3; that is, 3Z D f: : : ; 9; 6; 3; 0; 3; 6; 9; : : :g. Example 1.2. Consider the ideal 3Z C 5Z. This ideal contains 3 2 C 5 . 1/ D 1. Since it contains 1, it contains all integers; that is, 3Z C 5Z D Z. Example 1.3. Consider the ideal 4Z C 6Z. This ideal contains 4 . 1/ C 6 1 D 2, and therefore, it contains all even integers. It does not contain any odd integers, since the sum of two even integers is again even. Thus, 4Z C 6Z D 2Z. In the previous two examples, we defined an ideal that turned out upon closer inspection to be a principal ideal. This was no accident: the following theorem says that all ideals of Z are principal. Theorem 1.6. Let I be an ideal of Z. Then there exists a unique non-negative integer d such that I D d Z. Proof. We first prove the existence part of the theorem. If I D f0g, then d D 0 does the job, so let us assume that I ¤ f0g. Since I contains non-zero integers, it must contain positive integers, since if a 2 I then so is a. Let d be the smallest positive integer in I . We want to show that I D d Z. We first show that I d Z. To this end, let a be any element in I . It suffices to show that d j a. Using the division with remainder property, write a D dq C r, where 0 r < d . Then by the closure properties of ideals, one sees that r D a dq is also an element of I , and by the minimality of the choice of d , we must have r D 0. Thus, d j a. We have shown that I d Z. The fact that d Z I follows from the fact that d 2 I . Thus, I D d Z. That proves the existence part of the theorem. As for uniqueness, note that if d Z D eZ for some non-negative integer e, then d j e and e j d , from which it follows by Theorem 1.2 that d D ˙e; since d and e are non-negative, we must have d D e. Greatest common divisors. For a; b 2 Z, we call d 2 Z a common divisor of a and b if d j a and d j b; moreover, we call such a d a greatest common divisor of a and b if d is non-negative and all other common divisors of a and b divide d . Theorem 1.7. For all a; b 2 Z, there exists a unique greatest common divisor d of a and b, and moreover, aZ C bZ D d Z. Proof. We apply the previous theorem to the ideal I WD aZ C bZ. Let d 2 Z with I D d Z, as in that theorem. We wish to show that d is a greatest common divisor of a and b. Note that a; b; d 2 I and d is non-negative.
7
1.2 Ideals and greatest common divisors
Since a 2 I D d Z, we see that d j a; similarly, d j b. So we see that d is a common divisor of a and b. Since d 2 I D aZ C bZ, there exist s; t 2 Z such that as C bt D d . Now suppose a D a0 d 0 and b D b 0 d 0 for some a0 ; b 0 ; d 0 2 Z. Then the equation as C bt D d implies that d 0 .a0 s C b 0 t / D d , which says that d 0 j d . Thus, any common divisor d 0 of a and b divides d . That proves that d is a greatest common divisor of a and b. As for uniqueness, note that if e is a greatest common divisor of a and b, then d j e and e j d , and hence d D ˙e; since both d and e are non-negative by definition, we have d D e. For a; b 2 Z, we write gcd.a; b/ for the greatest common divisor of a and b. We say that a; b 2 Z are relatively prime if gcd.a; b/ D 1, which is the same as saying that the only common divisors of a and b are ˙1. The following is essentially just a restatement of Theorem 1.7, but we state it here for emphasis: Theorem 1.8. Let a; b; r 2 Z and let d WD gcd.a; b/. Then there exist s; t 2 Z such that as C bt D r if and only if d j r. In particular, a and b are relatively prime if and only if there exist integers s and t such that as C bt D 1. Proof. We have as C bt D r for some s; t 2 Z ” r 2 aZ C bZ ” r 2 d Z (by Theorem 1.7) ” d j r: That proves the first statement. The second statement follows from the first, setting r WD 1. Note that as we have defined it, gcd.0; 0/ D 0. Also note that when at least one of a or b are non-zero, gcd.a; b/ may be characterized as the largest positive integer that divides both a and b, and as the smallest positive integer that can be expressed as as C bt for integers s and t. Theorem 1.9. Let a; b; c 2 Z such that c j ab and gcd.a; c/ D 1. Then c j b. Proof. Suppose that c j ab and gcd.a; c/ D 1. Then since gcd.a; c/ D 1, by Theorem 1.8 we have as C ct D 1 for some s; t 2 Z. Multiplying this equation by b, we obtain abs C cbt D b:
(1.1)
8
Basic properties of the integers
Since c divides ab by hypothesis, and since c clearly divides cbt, it follows that c divides the left-hand side of (1.1), and hence that c divides b. Suppose that p is a prime and a is any integer. As the only divisors of p are ˙1 and ˙p, we have p j a H) gcd.a; p/ D p; and p − a H) gcd.a; p/ D 1: Combining this observation with the previous theorem, we have: Theorem 1.10. Let p be prime, and let a; b 2 Z. Then p j ab implies that p j a or p j b. Proof. Assume that p j ab. If p j a, we are done, so assume that p − a. By the above observation, gcd.a; p/ D 1, and so by Theorem 1.9, we have p j b. An obvious corollary to Theorem 1.10 is that if a1 ; : : : ; ak are integers, and if p is a prime that divides the product a1 ak , then p j ai for some i D 1; : : : ; k. This is easily proved by induction on k. For k D 1, the statement is trivially true. Now let k > 1, and assume that statement holds for k 1. Then by Theorem 1.10, either p j a1 or p j a2 ak ; if p j a1 , we are done; otherwise, by induction, p divides one of a2 ; : : : ; ak . Finishing the proof of Theorem 1.3. We are now in a position to prove the uniqueness part of Theorem 1.3, which we can state as follows: if p1 ; : : : ; pr are primes (not necessarily distinct), and q1 ; : : : ; qs are primes (also not necessarily distinct), such that p1 pr D q1 qs ;
(1.2)
then .p1 ; : : : ; pr / is just a reordering of .q1 ; : : : ; qs /. We may prove this by induction on r. If r D 0, we must have s D 0 and we are done. Now suppose r > 0, and that the statement holds for r 1. Since r > 0, we clearly must have s > 0. Also, as p1 obviously divides the left-hand side of (1.2), it must also divide the right-hand side of (1.2); that is, p1 j q1 qs . It follows from (the corollary to) Theorem 1.10 that p1 j qj for some j D 1; : : : ; s, and moreover, since qj is prime, we must have p1 D qj . Thus, we may cancel p1 from the left-hand side of (1.2) and qj from the right-hand side of (1.2), and the statement now follows from the induction hypothesis. That proves the uniqueness part of Theorem 1.3. E XERCISE 1.8. Let I be a non-empty set of integers that is closed under addition, that is, a C b 2 I for all a; b 2 I . Show that I is an ideal if and only if a 2 I for all a 2 I .
1.2 Ideals and greatest common divisors
9
E XERCISE 1.9. Show that for all integers a; b; c, we have (a) gcd.a; b/ D gcd.b; a/, (b) gcd.a; b/ D jaj ” a j b, (c) gcd.a; 0/ D gcd.a; a/ D jaj and gcd.a; 1/ D 1, (d) gcd.ca; cb/ D jcj gcd.a; b/. E XERCISE 1.10. Show that for all integers a; b with d WD gcd.a; b/ ¤ 0, we have gcd.a=d; b=d / D 1. E XERCISE 1.11. Let n be an integer. Show that if a; b are relatively prime integers, each of which divides n, then ab divides n. E XERCISE 1.12. Show that two integers are relatively prime if and only if there is no one prime that divides both of them. E XERCISE 1.13. Let a; b1 ; : : : ; bk be integers. Show that gcd.a; b1 bk / D 1 if and only if gcd.a; bi / D 1 for i D 1; : : : ; k. E XERCISE 1.14. Let p be a prime and k an integer, with 0 < k < p. Show that the binomial coefficient ! p pŠ D ; k kŠ.p k/Š which is an integer (see §A2), is divisible by p. E XERCISE 1.15. An integer a is called square-free if it is not divisible by the square of any integer greater than 1. Show that (a) a is square-free if and only if a D ˙p1 pr , where the pi ’s are distinct primes; (b) every positive integer n can be expressed uniquely as n D ab 2 , where a and b are positive integers, and a is square-free. E XERCISE 1.16. For each positive integer m, let Im denote f0; : : : ; m a; b be positive integers, and consider the map
1g. Let
W Ib Ia ! Iab .s; t / 7! .as C bt / mod ab: Show is a bijection if and only if gcd.a; b/ D 1. E XERCISE 1.17. Let a; b; c be positive integers, with gcd.a; b/ D 1 and c .a 1/.b 1/. Show that there exist non-negative integers s; t such that c D asCbt .
10
Basic properties of the integers
E XERCISE 1.18. For each positive integer n, let Dn denote the set of positive divisors of n. Let n1 ; n2 be relatively prime, positive integers. Show that the sets Dn1 Dn2 and Dn1 n2 are in one-to-one correspondence, via the map that sends .d1 ; d2 / 2 Dn1 Dn2 to d1 d2 . 1.3 Some consequences of unique factorization The following theorem is a consequence of just the existence part of Theorem 1.3: Theorem 1.11. There are infinitely many primes. Proof. By way of contradiction, suppose that there were only finitely many primes; Q call them p1 ; : : : ; pk . Then set M WD kiD1 pi and N WD M C 1. Consider a prime p that divides N . There must be at least one such prime p, since N 2, and every positive integer can be written as a product of primes. Clearly, p cannot equal any of the pi ’s, since if it did, then p would divide M , and hence also divide N M D 1, which is impossible. Therefore, the prime p is not among p1 ; : : : ; pk , which contradicts our assumption that these are the only primes. For each prime p, we may define the function p , mapping non-zero integers to non-negative integers, as follows: for every integer n ¤ 0, if n D p e m, where p − m, then p .n/ WD e. We may then write the factorization of n into primes as Y p p .n/ ; nD˙ p
where the product is over all primes p; although syntactically this is an infinite product, all but finitely many of its terms are equal to 1, and so this expression makes sense. Observe that if a and b are non-zero integers, then p .a b/ D p .a/ C p .b/ for all primes p;
(1.3)
a j b ” p .a/ p .b/ for all primes p:
(1.4)
and
From this, it is clear that gcd.a; b/ D
Y
p min.p .a/;p .b// :
p
Least common multiples. For a; b 2 Z, a common multiple of a and b is an integer m such that a j m and b j m; moreover, such an m is the least common multiple of a and b if m is non-negative and m divides all common multiples of a and b. It is easy to see that the least common multiple exists and is unique,
1.3 Some consequences of unique factorization
11
and we denote the least common multiple of a and b by lcm.a; b/. Indeed, for all a; b 2 Z, if either a or b are zero, the only common multiple of a and b is 0, and so lcm.a; b/ D 0; otherwise, if neither a nor b are zero, we have Y lcm.a; b/ D p max.p .a/;p .b// ; p
or equivalently, lcm.a; b/ may be characterized as the smallest positive integer divisible by both a and b. It is convenient to extend the domain of definition of p to include 0, defining p .0/ WD 1. If we interpret expressions involving “1” appropriately (see Preliminaries), then for arbitrary a; b 2 Z, both (1.3) and (1.4) hold, and in addition, p .gcd.a; b// D min.p .a/; p .b// and p .lcm.a; b// D max.p .a/; p .b// for all primes p. Generalizing gcd’s and lcm’s to many integers. It is easy to generalize the notions of greatest common divisor and least common multiple from two integers to many integers. Let a1 ; : : : ; ak be integers. We call d 2 Z a common divisor of a1 ; : : : ; ak if d j ai for i D 1; : : : ; k; moreover, we call such a d the greatest common divisor of a1 ; : : : ; ak if d is non-negative and all other common divisors of a1 ; : : : ; ak divide d . The greatest common divisor of a1 ; : : : ; ak is denoted gcd.a1 ; : : : ; ak / and is the unique non-negative integer d satisfying p .d / D min.p .a1 /; : : : ; p .ak // for all primes p: Analogously, we call m 2 Z a common multiple of a1 ; : : : ; ak if ai j m for i D 1; : : : ; k; moreover, such an m is called the least common multiple of a1 ; : : : ; ak if m divides all common multiples of a1 ; : : : ; ak . The least common multiple of a1 ; : : : ; ak is denoted lcm.a1 ; : : : ; ak / and is the unique non-negative integer m satisfying p .m/ D max.p .a1 /; : : : ; p .ak // for all primes p: Finally, we say that the family fai gkiD1 is pairwise relatively prime if gcd.ai ; aj / D 1 for all indices i; j with i ¤ j . Certainly, if fai gkiD1 is pairwise relatively prime, and k > 1, then gcd.a1 ; : : : ; ak / D 1; however, gcd.a1 ; : : : ; ak / D 1 does not imply that fai gkiD1 is pairwise relatively prime. Rational numbers. Consider now the rational numbers Q D fa=b W a; b 2 Z; b ¤ 0g. Given any rational number a=b, if we set d WD gcd.a; b/, and define the integers a0 WD a=d and b0 WD b=d , then we have a=b D a0 =b0 and gcd.a0 ; b0 / D 1. Moreover, if a1 =b1 D a0 =b0 , then we have a1 b0 D a0 b1 , and so b0 j a0 b1 , and since gcd.a0 ; b0 / D 1, we see that b0 j b1 ; if b1 D b0 c, it
12
Basic properties of the integers
follows that a1 D a0 c. Thus, we can represent every rational number as a fraction in lowest terms, that is, a fraction of the form a0 =b0 where a0 and b0 are relatively prime; moreover, the values of a0 and b0 are uniquely determined up to sign, and every other fraction that represents the same rational number is of the form a0 c =b0 c, for some non-zero integer c. E XERCISE 1.19. Let n be an integer. Generalizing Exercise 1.11, show that if fai gkiD1 is a pairwise relatively prime family of integers, where each ai divides n, Q then their product kiD1 ai also divides n. E XERCISE 1.20. Show that for all integers a; b; c, we have (a) lcm.a; b/ D lcm.b; a/, (b) lcm.a; b/ D jaj ” b j a, (c) lcm.a; a/ D lcm.a; 1/ D jaj, (d) lcm.ca; cb/ D jcj lcm.a; b/. E XERCISE 1.21. Show that for all integers a; b, we have (a) gcd.a; b/ lcm.a; b/ D jabj, (b) gcd.a; b/ D 1 H) lcm.a; b/ D jabj. E XERCISE 1.22. Let a1 ; : : : ; ak 2 Z with k > 1. Show that gcd.a1 ; : : : ; ak / D gcd.a1 ; gcd.a2 ; : : : ; ak // D gcd.gcd.a1 ; : : : ; ak lcm.a1 ; : : : ; ak / D lcm.a1 ; lcm.a2 ; : : : ; ak // D lcm.lcm.a1 ; : : : ; ak
1 /; ak /; 1 /; ak /:
E XERCISE 1.23. Let a1 ; : : : ; ak 2 Z with d WD gcd.a1 ; : : : ; ak /. Show that d Z D a1 Z C C ak Z; in particular, there exist integers z1 ; : : : ; zk such that d D a1 z1 C C ak zk : E XERCISE 1.24. Show that if fai gkiD1 is a pairwise relatively prime family of integers, then lcm.a1 ; : : : ; ak / D ja1 ak j. E XERCISE 1.25. Show that every non-zero x 2 Q can be expressed as x D ˙p1e1 prer ; where the pi ’s are distinct primes and the ei ’s are non-zero integers, and that this expression in unique up to a reordering of the primes. E XERCISE 1.26. Let n and k be positive integers, and suppose x 2 Q such that p x k D n for some x 2 Q. Show that x 2 Z. In other words, k n is either an integer or is irrational.
13
1.3 Some consequences of unique factorization
E XERCISE 1.27. Show that gcd.a C b; lcm.a; b// D gcd.a; b/ for all a; b 2 Z. E XERCISE 1.28. Show that for every positive integer k, there exist k consecutive composite integers. Thus, there are arbitrarily large gaps between primes. E XERCISE 1.29. Let a; b 2 Z and let p be a prime. Show that p .a C b/ minfp .a/; p .b/g, and that if p .a/ < p .b/, then p .a C b/ D p .a/. E XERCISE 1.30. For a given prime p, we may extend the domain of definition of p from Z to Q: for non-zero integers a; b, let us define p .a=b/ WD p .a/ p .b/. Show that: (a) this definition of p .a=b/ is unambiguous, in the sense that it does not depend on the particular choice of a and b; (b) for all x; y 2 Q, we have p .xy/ D p .x/ C p .y/; (c) for all x; y 2 Q, we have p .x C y/ minfp .x/; p .y/g, and if p .x/ < p .y/, then p .x C y/ D p .x/; Q (d) for all non-zero x 2 Q, we have x D ˙ p p p .x/ ; where the product is over all primes, and all but a finite number of terms in the product are equal to 1; (e) for all x 2 Q, we have x 2 Z if and only if p .x/ 0 for all primes p. E XERCISE 1.31. Let n be a positive integer, and let 2k be the highest power of 2 in the set S WD f1; : : : ; ng. Show that 2k does not divide any other element in S. P E XERCISE 1.32. Let n 2 Z with n > 1. Show that niD1 1= i is not an integer. E XERCISE 1.33. Let n be a positive integer, and let Cn denote the number of pairs of integers .a; b/ with a; b 2 f1; : : : ; ng and gcd.a; b/ D 1, and let Fn be the number of distinct rational numbers a=b, where 0 a < b n. (a) Show that Fn D .Cn C 1/=2. (b) Show that Cn n2 =4. Hint: first show that Cn n2 .1 P and then show that d 2 1=d 2 3=4.
P
d 2 1=d
2 /,
E XERCISE 1.34. This exercise develops a characterization of least common multiples in terms of ideals. (a) Arguing directly from the definition of an ideal, show that if I and J are ideals of Z, then so is I \ J . (b) Let a; b 2 Z, and consider the ideals I WD aZ and J WD bZ. By part (a), we know that I \J is an ideal. By Theorem 1.6, we know that I \J D mZ for some uniquely determined non-negative integer m. Show that m D lcm.a; b/.
2 Congruences
This chapter introduces the basic properties of congruences modulo n, along with the related notion of residue classes modulo n. Other items discussed include the Chinese remainder theorem, Euler’s phi function, Euler’s theorem, Fermat’s little theorem, quadratic residues, and finally, summations over divisors. 2.1 Equivalence relations Before discussing congruences, we review the definition and basic properties of equivalence relations. Let S be a set. A binary relation on S is called an equivalence relation if it is reflexive: a a for all a 2 S, symmetric: a b implies b a for all a; b 2 S , and transitive: a b and b c implies a c for all a; b; c 2 S. If is an equivalence relation on S , then for a 2 S one defines its equivalence class as the set fx 2 S W x ag. Theorem 2.1. Let be an equivalence relation on a set S , and for a 2 S , let Œa denote its equivalence class. Then for all a; b 2 S , we have (i) a 2 Œa; (ii) a 2 Œb implies Œa D Œb. Proof. (i) follows immediately from reflexivity. For (ii), suppose a 2 Œb, so that a b by definition. We want to show that Œa D Œb. To this end, consider any
14
2.2 Definitions and basic properties of congruences
15
x 2 S . We have x 2 Œa H) x a (by definition) H) x b (by transitivity, and since x a and a b) H) x 2 Œb: Thus, Œa Œb. By symmetry, we also have b a, and reversing the roles of a and b in the above argument, we see that Œb Œa. This theorem implies that each equivalence class is non-empty, and that each element of S belongs to a unique equivalence class; in other words, the distinct equivalence classes form a partition of S (see Preliminaries). A member of an equivalence class is called a representative of the class. E XERCISE 2.1. Consider the relations D, , and < on the set R. Which of these are equivalence relations? Explain your answers. E XERCISE 2.2. Let S WD R R n f.0; 0/g. For .x; y/; .x 0 ; y 0 / 2 S , let us say .x; y/ .x 0 ; y 0 / if there exists a real number > 0 such that .x; y/ D .x 0 ; y 0 /. Show that is an equivalence relation; moreover, show that each equivalence class contains a unique representative that lies on the unit circle (i.e., the set of points .x; y/ such that x 2 C y 2 D 1). 2.2 Definitions and basic properties of congruences Let n be a positive integer. For integers a and b, we say that a is congruent to b modulo n if n j .a b/, and we write a b .mod n/. If n − .a b/, then we write a 6 b .mod n/. Equivalently, a b .mod n/ if and only if a D b C ny for some y 2 Z. The relation a b .mod n/ is called a congruence relation, or simply, a congruence. The number n appearing in such congruences is called the modulus of the congruence. This usage of the “mod” notation as part of a congruence is not to be confused with the “mod” operation introduced in §1.1. If we view the modulus n as fixed, then the following theorem says that the binary relation “ .mod n/” is an equivalence relation on the set Z. Theorem 2.2. Let n be a positive integer. For all a; b; c 2 Z, we have: (i) a a .mod n/; (ii) a b .mod n/ implies b a .mod n/; (iii) a b .mod n/ and b c .mod n/ implies a c .mod n/. Proof. For (i), observe that n divides 0 D a
a. For (ii), observe that if n divides
16
a a
Congruences
b, then it also divides .a b/ D b a. For (iii), observe that if n divides b and b c, then it also divides .a b/ C .b c/ D a c.
Another key property of congruences is that they are “compatible” with integer addition and multiplication, in the following sense: Theorem 2.3. Let a; a0 ; b; b 0 ; n 2 Z with n > 0. If a a0 .mod n/ and b b 0 .mod n/; then a C b a0 C b 0 .mod n/ and a b a0 b 0 .mod n/: Proof. Suppose that a a0 .mod n/ and b b 0 .mod n/. This means that there exist integers x and y such that a D a0 C nx and b D b 0 C ny. Therefore, a C b D a0 C b 0 C n.x C y/; which proves the first congruence of the theorem, and ab D .a0 C nx/.b 0 C ny/ D a0 b 0 C n.a0 y C b 0 x C nxy/; which proves the second congruence. Theorems 2.2 and 2.3 allow one to work with congruence relations modulo n much as one would with ordinary equalities: one can add to, subtract from, or multiply both sides of a congruence modulo n by the same integer; also, if b is congruent to a modulo n, one may substitute b for a in any simple arithmetic expression (involving addition, subtraction, and multiplication) appearing in a congruence modulo n. Now suppose a is an arbitrary, fixed integer, and consider the set of integers z that satisfy the congruence z a .mod n/. Since z satisfies this congruence if and only if z D a C ny for some y 2 Z, we may apply Theorems 1.4 and 1.5 (with a as given, and b WD n) to deduce that every interval of n consecutive integers contains exactly one such z. This simple fact is of such fundamental importance that it deserves to be stated as a theorem: Theorem 2.4. Let a; n 2 Z with n > 0. Then there exists a unique integer z such that z a .mod n/ and 0 z < n, namely, z WD a mod n. More generally, for every x 2 R, there exists a unique integer z 2 Œx; x C n/ such that z a .mod n/. Example 2.1. Let us find the set of solutions z to the congruence 3z C 4 6 .mod 7/:
(2.1)
2.2 Definitions and basic properties of congruences
17
Suppose that z is a solution to (2.1). Subtracting 4 from both sides of (2.1), we obtain 3z 2 .mod 7/:
(2.2)
Next, we would like to divide both sides of this congruence of 3, to get z by itself on the left-hand side. We cannot do this directly, but since 5 3 1 .mod 7/, we can achieve the same effect by multiplying both sides of (2.2) by 5. If we do this, and then replace 5 3 by 1, and 5 2 by 3, we obtain z 3 .mod 7/: Thus, if z is a solution to (2.1), we must have z 3 .mod 7/; conversely, one can verify that if z 3 .mod 7/, then (2.1) holds. We conclude that the integers z that are solutions to (2.1) are precisely those integers that are congruent to 3 modulo 7, which we can list as follows: : : : ; 18; 11; 4; 3; 10; 17; 24; : : : In the next section, we shall give a systematic treatment of the problem of solving linear congruences, such as the one appearing in the previous example. E XERCISE 2.3. Let a; b; n 2 Z with n > 0. Show that a b .mod n/ if and only if .a mod n/ D .b mod n/. E XERCISE 2.4. Let a; b; n 2 Z with n > 0 and a b .mod n/. Also, let c0 ; c1 ; : : : ; ck 2 Z. Show that c0 C c1 a C C ck ak c0 C c1 b C C ck b k .mod n/: E XERCISE 2.5. Let a; b; n; n0 2 Z with n > 0, n0 > 0, and n0 j n. Show that if a b .mod n/, then a b .mod n0 /. E XERCISE 2.6. Let a; b; n; n0 2 Z with n > 0, n0 > 0, and gcd.n; n0 / D 1. Show that if a b .mod n/ and a b .mod n0 /, then a b .mod nn0 /. E XERCISE 2.7. Let a; b; n 2 Z with n > 0 and a b .mod n/. Show that gcd.a; n/ D gcd.b; n/. E XERCISE 2.8. Let a be a positive integer whose base-10 representation is a D .ak 1 a1 a0 /10 . Let b be the sum of the decimal digits of a; that is, let b WD a0 C a1 C C ak 1 . Show that a b .mod 9/. From this, justify the usual “rules of thumb” for determining divisibility by 9 and 3: a is divisible by 9 (respectively, 3) if and only if the sum of the decimal digits of a is divisible by 9 (respectively, 3).
18
Congruences
E XERCISE 2.9. Let e be a positive integer. For a 2 f0; : : : ; 2e 1g, let aQ denote the integer obtained by inverting the bits in the e-bit, binary representation of a (note that aQ 2 f0; : : : ; 2e 1g). Show that aQ C 1 a .mod 2e /. This justifies the usual rule for computing negatives in 2’s complement arithmetic (which is really just arithmetic modulo 2e ). E XERCISE 2.10. Show that the equation 7y 3 C 2 D z 3 has no solutions y; z 2 Z. E XERCISE 2.11. Show that there are 14 distinct, possible, yearly (Gregorian) calendars, and show that all 14 calendars actually occur. 2.3 Solving linear congruences In this section, we consider the general problem of solving linear congruences. More precisely, for a given positive integer n, and arbitrary integers a and b, we wish to determine the set of integers z that satisfy the congruence az b .mod n/:
(2.3)
Observe that if (2.3) has a solution z, and if z z 0 .mod n/, then z 0 is also a solution to (2.3). However, (2.3) may or may not have a solution, and if it does, such solutions may or may not be uniquely determined modulo n. The following theorem precisely characterizes the set of solutions of (2.3); basically, it says that (2.3) has a solution if and only if d WD gcd.a; n/ divides b, in which case the solution is uniquely determined modulo n=d . Theorem 2.5. Let a; n 2 Z with n > 0, and let d WD gcd.a; n/. (i) For every b 2 Z, the congruence az b .mod n/ has a solution z 2 Z if and only if d j b. (ii) For every z 2 Z, we have az 0 .mod n/ if and only if z 0 .mod n=d /. (iii) For all z; z 0 2 Z, we have az az 0 .mod n/ if and only if z z 0 .mod n=d /. Proof. For (i), let b 2 Z be given. Then we have az b .mod n/ for some z 2 Z ” az D b C ny for some z; y 2 Z (by definition of congruence) ” az
ny D b for some z; y 2 Z
” d j b (by Theorem 1.8): For (ii), we have n j az ” n=d j .a=d /z ” n=d j z:
19
2.3 Solving linear congruences
All of these implications follow rather trivially from the definition of divisibility, except that for the implication n=d j .a=d /z H) n=d j z, we use Theorem 1.9 and the fact that gcd.a=d; n=d / D 1. For (iii), we have az az 0 .mod n/ ” a.z ” z
z 0 / 0 .mod n/ z 0 0 .mod n=d / (by part (ii))
” z z 0 .mod n=d /: We can restate Theorem 2.5 in more concrete terms as follows. Let a; n 2 Z with n > 0, and let d WD gcd.a; n/. Let In WD f0; : : : ; n 1g and consider the “multiplication by a” map a W I n ! I n z 7! az mod n: The image of a consists of the n=d integers i d .i D 0; : : : ; n=d
1/:
Moreover, every element b in the image of a has precisely d pre-images z0 C j .n=d / .j D 0; : : : ; d where z0 2 f0; : : : ; n=d are relatively prime.
1/;
1g. In particular, a is a bijection if and only if a and n
Example 2.2. The following table illustrates what Theorem 2.5 says for n D 15 and a D 1; 2; 3; 4; 5; 6.
2z 3z 4z 5z 6z
z mod 15 mod 15 mod 15 mod 15 mod 15
0 0 0 0 0 0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 2 4 6 8 10 12 14 1 3 5 7 9 11 13 3 6 9 12 0 3 6 9 12 0 3 6 9 12 4 8 12 1 5 9 13 2 6 10 14 3 7 11 5 10 0 5 10 0 5 10 0 5 10 0 5 10 6 12 3 9 0 6 12 3 9 0 6 12 3 9
In the second row, we are looking at the values 2z mod 15, and we see that this row is just a permutation of the first row. So for every b, there exists a unique z such that 2z b .mod 15/. This is implied by the fact that gcd.2; 15/ D 1. In the third row, the only numbers hit are the multiples of 3, which follows from the fact that gcd.3; 15/ D 3. Also note that the pattern in this row repeats every five columns; that is, 3z 3z 0 .mod 15/ if and only if z z 0 .mod 5/.
20
Congruences
In the fourth row, we again see a permutation of the first row, which follows from the fact that gcd.4; 15/ D 1. In the fifth row, the only numbers hit are the multiples of 5, which follows from the fact that gcd.5; 15/ D 5. Also note that the pattern in this row repeats every three columns; that is, 5z 5z 0 .mod 15/ if and only if z z 0 .mod 3/. In the sixth row, since gcd.6; 15/ D 3, we see a permutation of the third row. The pattern repeats after five columns, although the pattern is a permutation of the pattern in the third row. We develop some further consequences of Theorem 2.5. A cancellation law. Let a; n 2 Z with n > 0. Part (iii) of Theorem 2.5 gives us a cancellation law for congruences: if gcd.a; n/ D 1 and az az 0 .mod n/, then z z 0 .mod n/. Example 2.3. Observe that 5 2 5 . 4/ .mod 6/:
(2.4)
Theorem 2.5 tells us that since gcd.5; 6/ D 1, we may cancel the common factor of 5 from both sides of (2.4), obtaining 2 4 .mod 6/, which one can also verify directly. Next observe that 3 5 3 3 .mod 6/:
(2.5)
We cannot simply cancel the common factor of 3 from both sides of (2.5); indeed, 5 6 3 .mod 6/. However, gcd.3; 6/ D 3, and as Theorem 2.5 guarantees, we do indeed have 5 3 .mod 2/. Modular inverses. Again, let a; n 2 Z with n > 0. We say that z 2 Z is a multiplicative inverse of a modulo n if az 1 .mod n/. Part (i) of Theorem 2.5 says that a has a multiplicative inverse modulo n if and only if gcd.a; n/ D 1. Moreover, part (iii) of Theorem 2.5 says that the multiplicative inverse of a, if it exists, is uniquely determined modulo n; that is, if z and z 0 are multiplicative inverses of a modulo n, then z z 0 .mod n/. Note that if z is a multiplicative inverse of a modulo n, then a is a multiplicative inverse of z modulo n. Also note that if a a0 .mod n/, then z is a multiplicative inverse of a modulo n if and only if z is a multiplicative inverse of a0 modulo n. Now suppose that a; b; n 2 Z with n > 0, a ¤ 0, and gcd.a; n/ D 1. Theorem 2.5 says that there exists a unique integer z satisfying az b .mod n/ and 0 z < n: Setting s WD b=a 2 Q, we may generalize the “mod” operation, defining s mod n
2.4 The Chinese remainder theorem
21
to be this value z. As the reader may easily verify, this definition of s mod n does not depend on the particular choice of fraction used to represent the rational number s. With this notation, we can simply write a 1 mod n to denote the unique multiplicative inverse of a modulo n that lies in the interval 0; : : : ; n 1. Example 2.4. Looking back at the table in Example 2.2, we see that 2
1
mod 15 D 8 and 4
1
mod 15 D 4;
and that neither 3, 5, nor 6 have modular inverses modulo 15. E XERCISE 2.12. Let a1 ; : : : ; ak ; b; n be integers with n > 0, and let d WD gcd.a1 ; : : : ; ak ; n/. Show that the congruence a1 z1 C C ak zk b .mod n/ has a solution z1 ; : : : ; zk 2 Z if and only if d j b. E XERCISE 2.13. Let p be a prime, and let a; b; c; e be integers, such that e > 0, a 6 0 .mod p e /, and 0 c < p e . Let N be the number of integers z 2 f0; : : : ; p 2e 1g such that j ı k .az C b/ mod p 2e p e D c: Show that N D p e . 2.4 The Chinese remainder theorem Next, we consider systems of linear congruences with respect to moduli that are relatively prime in pairs. The result we state here is known as the Chinese remainder theorem, and is extremely useful in a number of contexts. Theorem 2.6 (Chinese remainder theorem). Let fni gkiD1 be a pairwise relatively prime family of positive integers, and let a1 ; : : : ; ak be arbitrary integers. Then there exists a solution a 2 Z to the system of congruences a ai .mod ni / .i D 1; : : : ; k/: Moreover, any a0 2 Z is a solution to this system of congruences if and only if Q a a0 .mod n/, where n WD kiD1 ni . Proof. To prove the existence of a solution a to the system of congruences, we first show how to construct integers e1 ; : : : ; ek such that for i; j D 1; : : : ; k, we have 1 .mod ni / if j D i , ej (2.6) 0 .mod ni / if j ¤ i .
22
Congruences
If we do this, then setting a WD
k X
ai ei ;
i D1
one sees that for j D 1; : : : ; k, we have a
k X
ai ei aj .mod nj /;
i D1
since all the terms in this sum are zero modulo nj , except for the term i D j , which is congruent to aj modulo nj . Q To construct e1 ; : : : ; ek satisfying (2.6), let n WD kiD1 ni as in the statement of the theorem, and for i D 1; : : : ; k, let ni WD n=ni ; that is, ni is the product of all the moduli nj with j ¤ i . From the fact that fni gkiD1 is pairwise relatively prime, it follows that for i D 1; : : : ; k, we have gcd.ni ; ni / D 1, and so we may define ti WD .ni / 1 mod ni and ei WD ni ti . One sees that ei 1 .mod ni /, while for j ¤ i , we have ni j nj , and so ej 0 .mod ni /. Thus, (2.6) is satisfied. That proves the existence of a solution a to the given system of congruences. If a a0 .mod n/, then since ni j n for i D 1; : : : ; k, we see that a0 a ai .mod ni / for i D 1; : : : ; k, and so a0 also solves the system of congruences. Finally, if a0 is a solution to the given system of congruences, then a ai 0 a .mod ni / for i D 1; : : : ; k. Thus, ni j .a a0 / for i D 1; : : : ; k. Since fni gkiD1 is pairwise relatively prime, this implies that n j .a a0 /, or equivalently, a a0 .mod n/. We can restate Theorem 2.6 in more concrete terms, as follows. For each positive integer m, let Im denote f0; : : : ; m 1g. Suppose fni gkiD1 is a pairwise relatively prime family of positive integers, and set n WD n1 nk . Then the map W In ! In1 Ink a 7! .a mod n1 ; : : : ; a mod nk / is a bijection. Example 2.5. The following table illustrates what Theorem 2.6 says for n1 D 3 and n2 D 5. a 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 a mod 3 0 1 2 0 1 2 0 1 2 0 1 2 0 1 2 a mod 5 0 1 2 3 4 0 1 2 3 4 0 1 2 3 4 We see that as a ranges from 0 to 14, the pairs .a mod 3; a mod 5/ range over
2.5 Residue classes
23
all pairs .a1 ; a2 / with a1 2 f0; 1; 2g and a2 2 f0; : : : ; 4g, with every pair being hit exactly once. E XERCISE 2.14. Compute the values e1 ; e2 ; e3 in the proof of Theorem 2.6 in the case where k D 3, n1 D 3, n2 D 5, and n3 D 7. Also, find an integer a such that a 1 .mod 3/, a 1 .mod 5/, and a 5 .mod 7/. E XERCISE 2.15. If you want to show that you are a real nerd, here is an ageguessing game you might play at a party. You ask a fellow party-goer to divide his age by each of the numbers 3, 4, and 5, and tell you the remainders. Show how to use this information to determine their age. E XERCISE 2.16. Let fni gkiD1 be a pairwise relatively prime family of positive integers. Let a1 ; : : : ; ak and b1 ; : : : ; bk be integers, and set di WD gcd.ai ; ni / for i D 1; : : : ; k. Show that there exists an integer z such that ai z bi .mod ni / for i D 1; : : : ; k if and only if di j bi for i D 1; : : : ; k. E XERCISE 2.17. For each prime p, let p ./ be defined as in §1.3. Let p1 ; : : : ; pr be distinct primes, a1 ; : : : ; ar be arbitrary integers, and e1 ; : : : ; er be arbitrary nonnegative integers. Show that there exists an integer a such that pi .a ai / D ei for i D 1; : : : ; r. E XERCISE 2.18. Suppose n1 and n2 are positive integers, and let d WD gcd.n1 ; n2 /. Let a1 and a2 be arbitrary integers. Show that there exists an integer a such that a a1 .mod n1 / and a a2 .mod n2 / if and only if a1 a2 .mod d /. 2.5 Residue classes As we already observed in Theorem 2.2, for any fixed positive integer n, the binary relation “ .mod n/” is an equivalence relation on the set Z. As such, this relation partitions the set Z into equivalence classes. We denote the equivalence class containing the integer a by Œan , and when n is clear from context, we simply write Œa. By definition, we have z 2 Œa ” z a .mod n/ ” z D a C ny for some y 2 Z; and hence Œa D a C nZ WD fa C ny W y 2 Zg: Historically, these equivalence classes are called residue classes modulo n, and we shall adopt this terminology here as well. Note that a given residue class modulo n has many different “names”; for example, the residue class Œn 1 is the same as
24
Congruences
the residue class Œ 1. Any member of a residue class is called a representative of that class. We define Zn to be the set of residue classes modulo n. The following is simply a restatement of Theorem 2.4: Theorem 2.7. Let n be a positive integer. Then Zn consists of the n distinct residue classes Œ0; Œ1; : : : ; Œn 1. Moreover, for every x 2 R, each residue class modulo n contains a unique representative in the interval Œx; x C n/. When working with residue classes modulo n, one often has in mind a particular set of representatives. Typically, one works with the set of representatives f0; 1; : : : ; n 1g. However, sometimes it is convenient to work with another set of representatives, such as the representatives in the interval Œ n=2; n=2/. In this case, if n is odd, we can list the elements of Zn as Œ .n
1/=2; : : : ; Œ 1; Œ0; Œ1; : : : ; Œ.n
1/=2;
and when n is even, we can list the elements of Zn as Œ n=2; : : : ; Œ 1; Œ0; Œ1; : : : ; Œn=2
1:
We can “equip” Zn with binary operations defining addition and multiplication in a natural way as follows: for a; b 2 Z, we define Œa C Œb WD Œa C b; Œa Œb WD Œa b: Of course, one has to check this definition is unambiguous, in the sense that the sum or product of two residue classes should not depend on which particular representatives of the classes are chosen in the above definitions. More precisely, one must check that if Œa D Œa0 and Œb D Œb 0 , then Œa C b D Œa0 C b 0 and Œa b D Œa0 b 0 . However, this property follows immediately from Theorem 2.3. Observe that for all a; b; c 2 Z, we have Œa C Œb D Œc ” a C b c .mod n/; and Œa Œb D Œc ” a b c .mod n/;
25
2.5 Residue classes
Example 2.6. Consider the residue classes modulo 6. These are as follows: Œ0 D f: : : ; 12; 6; 0; 6; 12; : : :g Œ1 D f: : : ; 11; 5; 1; 7; 13; : : :g Œ2 D f: : : ; 10; 4; 2; 8; 14; : : :g Œ3 D f: : : ; 9; 3; 3; 9; 15; : : :g Œ4 D f: : : ; 8; 2; 4; 10; 16; : : :g Œ5 D f: : : ; 7; 1; 5; 11; 17; : : :g Let us write down the addition and multiplication tables for Z6 . The addition table looks like this: C Œ0 Œ1 Œ2 Œ3 Œ4 Œ5
Œ0 Œ0 Œ1 Œ2 Œ3 Œ4 Œ5
Œ1 Œ1 Œ2 Œ3 Œ4 Œ5 Œ0
Œ2 Œ2 Œ3 Œ4 Œ5 Œ0 Œ1
Œ3 Œ3 Œ4 Œ5 Œ0 Œ1 Œ2
Œ4 Œ4 Œ5 Œ0 Œ1 Œ2 Œ3
Œ5 Œ5 Œ0 Œ1 Œ2 Œ3 Œ4
Œ2 Œ0 Œ2 Œ4 Œ0 Œ2 Œ4
Œ3 Œ0 Œ3 Œ0 Œ3 Œ0 Œ3
Œ4 Œ0 Œ4 Œ2 Œ0 Œ4 Œ2
Œ5 Œ0 Œ5 Œ4 Œ3 Œ2 Œ1
The multiplication table looks like this: Œ0 Œ1 Œ2 Œ3 Œ4 Œ5
Œ0 Œ0 Œ0 Œ0 Œ0 Œ0 Œ0
Œ1 Œ0 Œ1 Œ2 Œ3 Œ4 Œ5
Instead of using representatives in the interval Œ0; 6/, we could just as well use representatives from another interval, such as Œ 3; 3/. Then, instead of naming the residue classes Œ0; Œ1; Œ2; Œ3; Œ4; Œ5, we would name them Œ 3; Œ 2; Œ 1; Œ0; Œ1; Œ2. Observe that Œ 3 D Œ3, Œ 2 D Œ4, and Œ 1 D Œ5.
Algebraic properties These operations on Zn yield a very natural algebraic structure. For example, addition and multiplication are commutative and associative; that is, for all
26
Congruences
˛; ˇ; 2 Zn , we have ˛ C ˇ D ˇ C ˛; .˛ C ˇ/ C D ˛ C .ˇ C /; ˛ˇ D ˇ˛; .˛ˇ/ D ˛.ˇ /: Note that we have adopted here the usual convention of writing ˛ˇ in place of ˛ ˇ. Furthermore, multiplication distributes over addition; that is, for all ˛; ˇ; 2 Zn , we have ˛.ˇ C / D ˛ˇ C ˛ : All of these properties follow from the definitions, and the corresponding properties for Z; for example, the fact that addition in Zn is commutative may be seen as follows: if ˛ D Œa and ˇ D Œb, then ˛ C ˇ D Œa C Œb D Œa C b D Œb C a D Œb C Œa D ˇ C ˛: Because addition and multiplication in Zn are associative, for ˛1 ; : : : ; ˛k 2 Zn , we may write the sum ˛1 C C ˛k and the product ˛1 ˛k without any parenthesis, and there is no ambiguity; moreover, since both addition and multiplication are commutative, we may rearrange the terms in such sums and products without changing their values. Further, from the distributive law, for all ˇ 2 Zn , we have ˇ.˛1 C C ˛k / D ˇ˛1 C C ˇ˛k : The residue class Œ0 acts as an additive identity; that is, for all ˛ 2 Zn , we have ˛ C Œ0 D ˛; indeed, if ˛ D Œa, then a C 0 a .mod n/. Moreover, Œ0 is the only element of Zn that acts as an additive identity; indeed, if a C z a .mod n/ holds for all integers a, then it holds in particular for a D 0, which implies z 0 .mod n/. The residue class Œ0 also has the property that ˛ Œ0 D Œ0 for all ˛ 2 Zn . Every ˛ 2 Zn has an additive inverse, that is, an element ˇ 2 Zn such that ˛ C ˇ D Œ0; indeed, if ˛ D Œa, then clearly ˇ WD Œ a does the job, since a C . a/ 0 .mod n/. Moreover, ˛ has at most one additive inverse; indeed, if a C z 0 .mod n/, then subtracting a from both sides of this congruence yields z a .mod n/. We naturally denote the additive inverse of ˛ by ˛. Observe that the additive inverse of ˛ is ˛; that is . ˛/ D ˛. Also, we have the identities .˛ C ˇ/ D . ˛/ C . ˇ/; . ˛/ˇ D
.˛ˇ/ D ˛. ˇ/; . ˛/. ˇ/ D ˛ˇ:
For ˛; ˇ 2 Zn , we naturally write ˛ ˇ for ˛ C . ˇ/. The residue class Œ1 acts as a multiplicative identity; that is, for all ˛ 2 Zn , we have ˛ Œ1 D ˛; indeed, if ˛ D Œa, then a 1 a .mod n/. Moreover, Œ1 is the only element of Zn that acts as a multiplicative identity; indeed, if a z
27
2.5 Residue classes
a .mod n/ holds for all integers a, then in particular, it holds for a D 1, which implies z 1 .mod n/. For ˛ 2 Zn , we call ˇ 2 Zn a multiplicative inverse of ˛ if ˛ˇ D Œ1. Not all ˛ 2 Zn have multiplicative inverses. If ˛ D Œa and ˇ D Œb, then ˇ is a multiplicative inverse of ˛ if and only if ab 1 .mod n/. Theorem 2.5 implies that ˛ has a multiplicative inverse if and only if gcd.a; n/ D 1, and that if it exists, it is unique. When it exists, we denote the multiplicative inverse of ˛ by ˛ 1 . We define Zn to be the set of elements of Zn that have a multiplicative inverse. By the above discussion, we have Zn D fŒa W a D 0; : : : ; n
1; gcd.a; n/ D 1g:
If n is prime, then gcd.a; n/ D 1 for a D 1; : : : ; n 1, and we see that Zn D Zn n fŒ0g. If n is composite, then Zn ¨ Zn n fŒ0g; for example, if d j n with 1 < d < n, we see that Œd is not zero, nor does it belong to Zn . Observe that if ˛; ˇ 2 Zn , then so are ˛ 1 and ˛ˇ; indeed, .˛
1
/
1
D ˛ and .˛ˇ/
1
D˛
1
ˇ
1
:
For ˛ 2 Zn and ˇ 2 Zn , we naturally write ˛=ˇ for ˛ˇ 1 . Suppose ˛; ˇ; are elements of Zn that satisfy the equation ˛ˇ D ˛ : If ˛ 2 Zn , we may multiply both sides of this equation by ˛
1
to infer that
ˇ D : This is the cancellation law for Zn . We stress the requirement that ˛ 2 Zn , and not just ˛ ¤ Œ0. Indeed, consider any ˛ 2 Zn n Zn . Then we have ˛ D Œa with d WD gcd.a; n/ > 1. Setting ˇ WD Œn=d and WD Œ0, we see that ˛ˇ D ˛ and ˇ ¤ : Example 2.7. We list the elements of Z15 , and for each ˛ 2 Z15 , we also give ˛ 1. ˛ ˛
1
[1] [1]
[2] [8]
[4] [4]
[7] [13]
[8] [2]
[11] [11]
[13] [7]
[14] [14]
Notational conventions P For ˛1 ; : : : ; ˛k 2 Zn , we may naturally write their sum as kiD1 ˛i . By convenPk Pk tion, this sum is Œ0 when k D 0. It is easy to see that i D1 ˛i D i D1 . ˛i /;
28
Congruences
that is, the additive inverse of the sum is the sum of the additive inverses. In the P special case where all the ˛i ’s have the same value ˛, we define k˛ WD kiD1 ˛; thus, 0˛ D Œ0, 1˛ D ˛, 2˛ D ˛ C ˛, 3˛ D ˛ C ˛ C ˛, and so on. The additive inverse of k˛ is k. ˛/, which we may also write as . k/˛; thus, . 1/˛ D ˛, . 2/˛ D . ˛/ C . ˛/ D .˛ C ˛/, and so on. Therefore, the notation k˛ is defined for all integers k. Note that for all integers k and a, we have kŒa D Œka D ŒkŒa. For all ˛; ˇ 2 Zn and k; ` 2 Z, we have the identities: k.`˛/ D .k`/˛ D `.k˛/; .k C `/˛ D k˛ C `˛; k.˛ C ˇ/ D k˛ C kˇ; .k˛/ˇ D k.˛ˇ/ D ˛.kˇ/: Q Analogously, for ˛1 ; : : : ; ˛k 2 Zn , we may write their product as kiD1 ˛i . By convention, this product is Œ1 when k D 0. It is easy to see that if all of the Q ˛i ’s belong to Zn , then so does their product, and in particular, . kiD1 ˛i / 1 D Qk 1 i D1 ˛i ; that is, the multiplicative inverse of the product is the product of the multiplicative inverses. In the special case where all the ˛i ’s have the same value Q ˛, we define ˛ k WD kiD1 ˛; thus, ˛ 0 D Œ1, ˛ 1 D ˛, ˛ 2 D ˛˛, ˛ 3 D ˛˛˛, and so on. If ˛ 2 Zn , then the multiplicative inverse of ˛ k is .˛ 1 /k , which we may also write as ˛ k ; for example, ˛ 2 D ˛ 1 ˛ 1 D .˛˛/ 1 . Therefore, when ˛ 2 Zn , the notation ˛ k is defined for all integers k. For all ˛; ˇ 2 Zn and all non-negative integers k and `, we have the identities: .˛ ` /k D ˛ k` D .˛ k /` ; ˛ kC` D ˛ k ˛ ` ; .˛ˇ/k D ˛ k ˇ k :
(2.7)
If ˛; ˇ 2 Zn , the identities in (2.7) hold for all k; ` 2 Z. One last notational convention. As already mentioned, when the modulus n is clear from context, we usually write Œa instead of Œan . Although we want to maintain a clear distinction between integers and their residue classes, occasionally even the notation Œa is not only redundant, but distracting; in such situations, we may simply write a instead of Œa. For example, for every ˛ 2 Zn , we have the identity .˛ C Œ1n /.˛ Œ1n / D ˛ 2 Œ1n ; which we may write more simply as .˛ C Œ1/.˛ Œ1/ D ˛ 2 Œ1; or even more simply, and hopefully more clearly, as .˛ C 1/.˛ 1/ D ˛ 2 1: Here, the only reasonable interpretation of the symbol “1” is Œ1, and so there can be no confusion. Summary In summary, algebraic expressions involving residue classes may be manipulated in much the same way as expressions involving ordinary numbers. Extra complications arise only because when n is composite, some non-zero elements of Zn do
2.5 Residue classes
29
not have multiplicative inverses, and the usual cancellation law does not apply for such elements. In general, one has a choice between working with congruences modulo n, or with the algebraic structure Zn ; ultimately, the choice is one of taste and convenience, and it depends on what one prefers to treat as “first class objects”: integers and congruence relations, or elements of Zn . An alternative, and somewhat more concrete, approach to defining Zn is to directly define it to consist of the n “symbols” Œ0; Œ1; : : : ; Œn 1, with addition and multiplication defined as Œa C Œb WD Œ.a C b/ mod n; Œa Œb WD Œ.a b/ mod n; for a; b 2 f0; : : : ; n 1g. Such a definition is equivalent to the one we have given here. One should keep this alternative characterization of Zn in mind; however, we prefer the characterization in terms of residue classes, as it is mathematically more elegant, and is usually more convenient to work with. The Chinese remainder map We close this section with a reinterpretation of the Chinese remainder theorem (Theorem 2.6) in terms of residue classes. Theorem 2.8 (Chinese remainder map). Let fni gkiD1 be a pairwise relatively Q prime family of positive integers, and let n WD kiD1 ni . Define the map W
Zn ! Zn1 Znk Œan 7! .Œan1 ; : : : ; Œank /:
(i) The definition of is unambiguous. (ii) is bijective. (iii) For all ˛; ˇ 2 Zn , if .˛/ D .˛1 ; : : : ; ˛k / and .ˇ/ D .ˇ1 ; : : : ; ˇk /, then (a) .˛ C ˇ/ D .˛1 C ˇ1 ; : : : ; ˛k C ˇk /, (b) . ˛/ D . ˛1 ; : : : ; ˛k /, (c) .˛ˇ/ D .˛1 ˇ1 ; : : : ; ˛k ˇk /, (d) ˛ 2 Zn if and only if ˛i 2 Zni for i D 1; : : : ; k, in which case .˛ 1 / D .˛1 1 ; : : : ; ˛k 1 /: Proof. For (i), note that a a0 .mod n/ implies a a0 .mod ni / for i D 1; : : : ; k, and so the definition of is unambiguous (it does not depend on the choice of a). (ii) follows directly from the statement of the Chinese remainder theorem.
30
Congruences
For (iii), let ˛ D Œan and ˇ D Œbn , so that for i D 1; : : : ; k, we have ˛i D Œani and ˇi D Œbni . Then we have .˛Cˇ/ D .ŒaCbn / D .ŒaCbn1 ; : : : ; ŒaCbnk / D .˛1 Cˇ1 ; : : : ; ˛k Cˇk /; . ˛/ D .Œ an / D .Œ an1 ; : : : ; Œ ank / D . ˛1 ; : : : ; ˛k /; and .˛ˇ/ D .Œabn / D .Œabn1 ; : : : ; Œabnk / D .˛1 ˇ1 ; : : : ; ˛k ˇk /: That proves parts (a), (b), and (c). For part (d), we have ˛ 2 Zn ” gcd.a; n/ D 1 ” gcd.a; ni / D 1 for i D 1; : : : ; k ” ˛i 2 Zni for i D 1; : : : ; k: Moreover, if ˛ 2 Zn and ˇ D ˛
1,
then
.˛1 ˇ1 ; : : : ; ˛k ˇk / D .˛ˇ/ D .Œ1n / D .Œ1n1 ; : : : ; Œ1nk /; and so for i D 1; : : : ; k, we have ˛i ˇi D Œ1ni , which is to say ˇi D ˛i 1 . Theorem 2.8 is very powerful conceptually, and is an indispensable tool in many situations. It says that if we want to understand what happens when we add or multiply ˛; ˇ 2 Zn , it suffices to understand what happens when we add or multiply their “components” ˛i ; ˇi 2 Zni . Typically, we choose n1 ; : : : ; nk to be primes or prime powers, which usually simplifies the analysis. We shall see many applications of this idea throughout the text. E XERCISE 2.19. Let W Zn ! Zn1 Znk be as in Theorem 2.8, and suppose that .˛/ D .˛1 ; : : : ; ˛k /. Show that for every non-negative integer m, we have .˛ m / D .˛1m ; : : : ; ˛km /. Moreover, if ˛ 2 Zn , show that this identity holds for all integers m. P P E XERCISE 2.20. Let p be an odd prime. Show that ˇ 2Zp ˇ 1 D ˇ 2Zp ˇ D 0. Pp 1 E XERCISE 2.21. Let p be an odd prime. Show that the numerator of i D1 1= i is divisible by p. E XERCISE 2.22. Suppose n is square-free (see Exercise 1.15), and let ˛; ˇ; 2 Zn . Show that ˛ 2 ˇ D ˛ 2 implies ˛ˇ D ˛ . 2.6 Euler’s phi function For each positive integer n, we define .n/ WD jZn j. Equivalently, .n/ is equal to the number of integers between 0 and n 1 that are relatively prime to n. For
31
2.6 Euler’s phi function
example, .1/ D 1, .2/ D 1, .3/ D 2, and .4/ D 2. The function is called Euler’s phi function (or Euler’s totient function). Using the Chinese remainder theorem, more specifically Theorem 2.8, it is easy to get a nice formula for .n/ in terms for the prime factorization of n, as we establish in the following sequence of theorems. Theorem 2.9. Let fni gkiD1 be a pairwise relatively prime family of positive inteQ gers, and let n WD kiD1 ni . Then .n/ D
k Y
.ni /:
i D1
Proof. Consider the map W Zn ! Zn1 Znk in Theorem 2.8. By parts (ii) and (iii.d) of that theorem, restricting to Zn yields a one-to-one correspondence between Zn and Zn1 Znk . The theorem now follows immediately. We already know that .p/ D p 1 for every prime p, since the integers 1; : : : ; p 1 are not divisible by p, and hence are relatively prime to p. The next theorem generalizes this, giving us a formula for Euler’s phi function at prime powers. Theorem 2.10. Let p be a prime and e be a positive integer. Then .p e / D p e
1
.p
Proof. The multiples of p among 0; 1; : : : ; p e 0 p; 1 p; : : : ; .p e of which there are precisely p e
1.
1/: 1 are
1
1/ p;
Thus, .p e / D p e
pe
1
D pe
1 .p
1/:
If n D p1e1 prer is the factorization of n into primes, then the family of prime powers fpiei griD1 is pairwise relatively prime, and so Theorem 2.9 implies .n/ D .p1e1 / .prer /. Combining this with Theorem 2.10, we have: Theorem 2.11. If n D p1e1 prer is the factorization of n into primes, then .n/ D
r Y i D1
piei
1
.pi
1/ D n
r Y
.1
1=pi /:
i D1
E XERCISE 2.23. Show that .nm/ D gcd.n; m/ .lcm.n; m//. E XERCISE 2.24. Show that if n is divisible by r distinct odd primes, then 2r j .n/.
32
Congruences
E XERCISE 2.25. For every positive integer n, define 2 .n/ to be the number of integers a 2 f0; : : : ; n 1g such that gcd.a; n/ D gcd.a C 1; n/ D 1. Suppose n D Q p1e1 prer is the factorization of n into primes. Show that 2 .n/ D n riD1 .1 2=pi /: 2.7 Euler’s theorem and Fermat’s little theorem Let n be a positive integer, and let ˛ 2 Zn . Consider the sequence of powers of ˛: 1 D ˛0; ˛1; ˛2; : : : : Since each such power is an element of Zn , and since Zn is a finite set, this sequence of powers must start to repeat at some point; that is, there must be a positive integer k such that ˛ k D ˛ i for some i D 0; : : : ; k 1. Let us assume that k is chosen to be the smallest such positive integer. We claim that i D 0, or equivalently, ˛ k D 1. To see this, suppose by way of contradiction that ˛ k D ˛ i , for some i D 1; : : : ; k 1. Then we can cancel ˛ from both sides of the equation ˛ k D ˛ i , obtaining ˛ k 1 D ˛ i 1 , and this contradicts the minimality of k. This value k is called the multiplicative order of ˛, and can be characterized as the smallest positive integer k such that ˛ k D 1: If ˛ D Œa with a 2 Z (and gcd.a; n/ D 1, since ˛ 2 Zn ), then k is also called the multiplicative order of a modulo n, and can be characterized as the smallest positive integer k such that ak 1 .mod n/: From the above discussion, we see that the first k powers of ˛, that is, are distinct. Moreover, other powers of ˛ simply repeat this pattern. The following is an immediate consequence of this observation. ˛0; ˛1; : : : ; ˛k 1,
Theorem 2.12. Let n be a positive integer, and let ˛ be an element of Zn of multiplicative order k. Then for every i 2 Z, we have ˛ i D 1 if and only if k divides i . More generally, for all i; j 2 Z, we have ˛ i D ˛ j if and only if i j .mod k/. Example 2.8. Let n D 7. For each value a D 1; : : : ; 6, we can compute successive powers of a modulo n to find its multiplicative order modulo n.
2.7 Euler’s theorem and Fermat’s little theorem
1i 2i 3i 4i 5i 6i
i mod 7 mod 7 mod 7 mod 7 mod 7 mod 7
1 1 2 3 4 5 6
2 1 4 2 2 4 1
3 1 1 6 1 6 6
4 1 2 4 4 2 1
5 1 4 5 2 3 6
33
6 1 1 1 1 1 1
So we conclude that modulo 7: 1 has order 1; 6 has order 2; 2 and 4 have order 3; and 3 and 5 have order 6. Theorem 2.13 (Euler’s theorem). Let n be a positive integer and ˛ 2 Zn . Then ˛ .n/ D 1. In particular, the multiplicative order of ˛ divides .n/. Proof. Since ˛ 2 Zn , for every ˇ 2 Zn we have ˛ˇ 2 Zn , and so we may define the “multiplication by ˛” map ˛ W Zn ! Zn ˇ 7! ˛ˇ: It is easy to see that ˛ is a bijection: Injectivity: If ˛ˇ D ˛ˇ 0 , then cancel ˛ to obtain ˇ D ˇ 0 . 1
Surjectivity: For every 2 Zn , ˛
is a pre-image of under ˛ .
Zn ,
Thus, as ˇ ranges over the set so does ˛ˇ, and we have Y Y Y .n/ ˇD .˛ˇ/ D ˛ ˇ : ˇ 2Z n
Canceling the common factor (2.8), we obtain
ˇ 2Z n
Q
ˇ 2Z n
(2.8)
ˇ 2Z n
ˇ 2 Zn from the left- and right-hand side of
1 D ˛ .n/ : That proves the first statement of the theorem. The second follows immediately from Theorem 2.12. As a consequence of this, we obtain: Theorem 2.14 (Fermat’s little theorem). For every prime p, and every ˛ 2 Zp , we have ˛ p D ˛. Proof. If ˛ D 0, the statement is obviously true. Otherwise, ˛ 2 Zp , and by Theorem 2.13 we have ˛ p 1 D 1. Multiplying this equation by ˛ yields ˛ p D ˛.
34
Congruences
In the language of congruences, Fermat’s little theorem says that for every prime p and every integer a, we have ap a .mod p/: For a given positive integer n, we say that a 2 Z with gcd.a; n/ D 1 is a primitive root modulo n if the multiplicative order of a modulo n is equal to .n/. If this is the case, then for ˛ WD Œa 2 Zn , the powers ˛ i range over all elements of Zn as i ranges over the interval 0; : : : ; .n/ 1. Not all positive integers have primitive roots — we will see in §7.5 that the only positive integers n for which there exists a primitive root modulo n are n D 1; 2; 4; p e ; 2p e ; where p is an odd prime and e is a positive integer. The following theorem is sometimes useful in determining the multiplicative order of an element in Zn . Theorem 2.15. Suppose ˛ 2 Zn has multiplicative order k. Then for every m 2 Z, the multiplicative order of ˛ m is k= gcd.m; k/. Proof. Applying Theorem 2.12 to ˛ m , we see that the multiplicative order of ˛ m is the smallest positive integer ` such that ˛ m` D 1. But we have ˛ m` D 1 ” m` 0 .mod k/ (applying Theorem 2.12 to ˛) ” ` 0 .mod k= gcd.m; k// (by Theorem 2.5): E XERCISE 2.26. Find all elements of Z19 of multiplicative order 18. E XERCISE 2.27. Let n 2 Z with n > 1. Show that n is prime if and only if ˛ n 1 D 1 for every non-zero ˛ 2 Zn . E XERCISE 2.28. Let n D pq where p and q are distinct primes, and let m WD lcm.p 1; q 1/. Show that ˛ m D 1 for all ˛ 2 Zn . E XERCISE 2.29. Let p be any prime other than 2 or 5. Show that p divides infinitely many of the numbers 9, 99, 999, etc. E XERCISE 2.30. Let n be an integer greater than 1. Show that n does not divide 2n 1. E XERCISE 2.31. Prove the following generalization of Fermat’s little theorem: for every positive integer n, and every ˛ 2 Zn , we have ˛ n D ˛ n .n/ .
2.8 Quadratic residues
35
E XERCISE 2.32. This exercise develops an alternative proof of Fermat’s little theorem. (a) Using Exercise 1.14, show that for all primes p and integers a, we have .a C 1/p ap C 1 .mod p/. (b) Now derive Fermat’s little theorem from part (a). 2.8 Quadratic residues In §2.3, we studied linear congruences. It is natural to study congruences of higher degree as well. In this section, we study a special case of this more general problem, namely, congruences of the form z 2 a .mod n/. The theory we develop here nicely illustrates many of the ideas we have discussed earlier, and has a number of interesting applications as well. We begin with some general, preliminary definitions and general observations about powers in Zn . For each integer m, we define .Zn /m WD fˇ m W ˇ 2 Zn g; the set of mth powers in Zn . The set .Zn /m is non-empty, as it obviously contains Œ1. Theorem 2.16. Let n be a positive integer, let ˛; ˇ 2 Zn , and let m be any integer. (i) If ˛ 2 .Zn /m , then ˛
1
2 .Zn /m .
(ii) If ˛ 2 .Zn /m and ˇ 2 .Zn /m , then ˛ˇ 2 .Zn /m . (iii) If ˛ 2 .Zn /m and ˇ … .Zn /m , then ˛ˇ … .Zn /m . Proof. For (i), if ˛ D m , then ˛ 1 D . 1 /m . For (ii), if ˛ D m and ˇ D ı m , then ˛ˇ D . ı/m . (iii) follows from (i) and (ii). Suppose that ˛ 2 .Zn /m , ˇ … .Zn /m , and ˛ˇ 2 .Zn /m . Then by (i), ˛ 1 2 .Zn /m , and by (ii), ˇ D ˛ 1 .˛ˇ/ 2 .Zn /m , a contradiction. Theorem 2.17. Let n be a positive integer. For each ˛ 2 Zn , and all `; m 2 Z with gcd.`; m/ D 1, if ˛ ` 2 .Zn /m , then ˛ 2 .Zn /m . Proof. Suppose ˛ ` D ˇ m 2 .Zn /m . Since gcd.`; m/ D 1, there exist integers s and t such that `s C mt D 1. We then have ˛ D ˛ `sCmt D ˛ `s ˛ mt D ˇ ms ˛ mt D .ˇ s ˛ t /m 2 .Zn /m : We now focus on the on squares in Zn , rather than general powers. An integer a is called a quadratic residue modulo n if gcd.a; n/ D 1 and a b 2 .mod n/ for
36
Congruences
some integer b; in this case, we say that b is a square root of a modulo n. In terms of residue classes, a is a quadratic residue modulo n if and only if Œa 2 .Zn /2 . To avoid some annoying technicalities, from now on, we shall consider only the case where n is odd. 2.8.1 Quadratic residues modulo p We first study quadratic residues modulo an odd prime p, and we begin by determining the square roots of 1 modulo p. Theorem 2.18. Let p be an odd prime and ˇ 2 Zp . Then ˇ 2 D 1 if and only if ˇ D ˙1. Proof. Clearly, if ˇ D ˙1, then ˇ 2 D 1. Conversely, suppose that ˇ 2 D 1. Write ˇ D Œb, where b 2 Z. Then we have b 2 1 .mod p/, which means that p j .b 2
1/ D .b
1/.b C 1/;
and since p is prime, we must have p j .b 1/ or p j .b C 1/. This implies b ˙1 .mod p/, or equivalently, ˇ D ˙1. This theorem says that modulo p, the only square roots of 1 are 1 and 1, which obviously belong to distinct residue classes (since p > 2). From this seemingly trivial fact, a number of quite interesting and useful results may be derived. Theorem 2.19. Let p be an odd prime and ; ˇ 2 Zp . Then 2 D ˇ 2 if and only if D ˙ˇ. Proof. This follows from the previous theorem:
2 D ˇ 2 ” . =ˇ/2 D 1 ” =ˇ D ˙1 ” D ˙ˇ: This theorem says that if ˛ D ˇ 2 for some ˇ 2 Zp , then ˛ has precisely two square roots: ˇ and ˇ. Theorem 2.20. Let p be an odd prime. Then j.Zp /2 j D .p
1/=2.
Proof. By the previous theorem, the “squaring map” W Zp ! Zp that sends ˇ to ˇ 2 is a two-to-one map: every element in the image of has precisely two pre-images. As a general principle, if we have a function f W A ! B, where A is a finite set and every element in f .A/ has exactly d pre-images, then jf .A/j D jAj=d . Applying this general principle to our setting, we see that the image of is half the size of Zp . Thus, for every odd prime p, exactly half the elements of Zp are squares, and
37
2.8 Quadratic residues
half are non-squares. If we choose our representatives for the residue classes modulo p from the interval Πp=2; p=2/, we may list the elements of Zp as Π.p
1/=2; : : : ; Œ 1; Œ0; Œ1; : : : ; Œ.p
1/=2:
We then see that Zp consists of the residue classes Œ˙1; : : : ; Œ˙.p
1/=2;
and so .Zp /2 consists of the residue classes Œ12 ; : : : ; Œ.p
1/=22 ;
which must be distinct, since we know that j.Zp /2 j D .p
1/=2.
Example 2.9. Let p D 7. We can list the elements of Zp as Œ˙1; Œ˙2; Œ˙3: Squaring these, we see that .Zp /2 D fŒ12 ; Œ22 ; Œ32 g D fŒ1; Œ4; Œ2g: We next derive an extremely important characterization of quadratic residues. Theorem 2.21 (Euler’s criterion). Let p be an odd prime and ˛ 2 Zp . (i) ˛ .p
1/=2
D ˙1.
(ii) If ˛ 2 .Zp /2 then ˛ .p (iii) If ˛ …
.Zp /2
then
1/=2
˛ .p 1/=2
Proof. For (i), let D ˛ .p
1/=2 .
D 1. D
1.
By Euler’s theorem (Theorem 2.13), we have
2 D ˛p
1
D 1;
and hence by Theorem 2.18, we have D ˙1. For (ii), suppose that ˛ D ˇ 2 . Then again by Euler’s theorem, we have ˛ .p
1/=2
D .ˇ 2 /.p
1/=2
D ˇp
1
D 1:
For (iii), let ˛ 2 Zp n .Zp /2 . We study the product Y WD ˇ: ˇ 2Zp
We shall show that, on the one hand, D ˛ .p 1/=2 , while on the other hand, D 1. To show that D ˛ .p 1/=2 , we group elements of Zp into pairs of distinct elements whose product is ˛. More precisely, let P WD fS Zp W jS j D 2g, and
38
Congruences
define C WD f f; g 2 P W D ˛g. Note that for every 2 Zp , there is a unique 2 Zp such that D ˛, namely, WD ˛=; moreover, ¤ , since otherwise, we would have 2 D ˛, contradicting the assumption that ˛ … .Zp /2 . Thus, every element of Zp belongs to exactly one pair in C; in other words, the elements of C form a partition of Zp . It follows that Y Y . / D ˛ D ˛ .p 1/=2 : D f;g2C
f;g2C
To show that D 1, we group elements of Zp into pairs of distinct elements whose product is Œ1. Define D WD f f; g 2 P W D 1g. For every 2 Zp , there exists a unique 2 Zp such that D 1, namely, WD 1 ; moreover, D if and only if 2 D 1, and by Theorem 2.18, this happens if and only if D ˙1. Thus, every element of Zp except for Œ˙1 belongs to exactly one pair in D; in other words, the elements of D form a partition of Zp n fŒ˙1g. It follows that Y Y D Œ1 Œ 1 . / D Œ 1 Œ1 D 1: f;g2D
f;g2D
Thus, Euler’s criterion says that for every ˛ 2 Zp , we have ˛ .p ˛ 2 .Zp /2 ” ˛ .p
1/=2
1/=2
D ˙1 and
D 1:
In the course of proving Euler’s criterion, we proved the following result, which we state here for completeness: Q Theorem 2.22 (Wilson’s theorem). Let p be an odd prime. Then ˇ 2Zp ˇ D 1. In the language of congruences, Wilson’s theorem may be stated as follows: 1/Š
.p
1 .mod p/:
We also derive the following simple consequence of Theorem 2.21: Theorem 2.23. Let p be an odd prime and ˛; ˇ 2 Zp . If ˛ … .Zp /2 and ˇ … .Zp /2 , then ˛ˇ 2 .Zp /2 . Proof. Suppose ˛ … .Zp /2 and ˇ … .Zp /2 . Then by Euler’s criterion, we have ˛ .p
1/=2
D
1 and ˇ .p
1/=2
D
1:
Therefore, .˛ˇ/.p
1/=2
D ˛ .p
1/=2
ˇ .p
1/=2
D Π1 Π1 D 1;
which again by Euler’s criterion implies that ˛ˇ 2 .Zp /2 .
2.8 Quadratic residues
39
This theorem, together with parts (ii) and (iii) of Theorem 2.16, gives us the following simple rules regarding squares in Zp : square square D square; square non-square D non-square; non-square non-square D square: 2.8.2 Quadratic residues modulo p e We next study quadratic residues modulo p e , where p is an odd prime. The key is to establish the analog of Theorem 2.18: Theorem 2.24. Let p be an odd prime, e be a positive integer, and ˇ 2 Zpe . Then ˇ 2 D 1 if and only if ˇ D ˙1. Proof. Clearly, if ˇ D ˙1, then ˇ 2 D 1. Conversely, suppose that ˇ 2 D 1. Write ˇ D Œb, where b 2 Z. Then we have b 2 1 .mod p e /, which means that p e j .b 2
1/ D .b
1/.b C 1/:
In particular, p j .b 1/.b C 1/, and so p j .b 1/ or p j .b C 1/. Moreover, p cannot divide both b 1 and b C 1, as otherwise, it would divide their difference .b C 1/ .b 1/ D 2, which is impossible (because p is odd). It follows that p e j .b 1/ or p e j .b C 1/, which means ˇ D ˙1. Theorems 2.19–2.23 generalize immediately from Zp to Zpe : we really used nothing in the proofs of these theorems other than the fact that ˙1 are the only square roots of 1 modulo p. As such, we state the analogs of these theorems for Zpe without proof. Theorem 2.25. Let p be an odd prime, e be a positive integer, and ; ˇ 2 Zpe . Then 2 D ˇ 2 if and only if D ˙ˇ. Theorem 2.26. Let p be an odd prime and e be a positive integer. Then j.Zpe /2 j D .p e /=2. Theorem 2.27. Let p be an odd prime, e be a positive integer, and ˛ 2 Zpe . (i) ˛ .p
e /=2
D ˙1.
(ii) If ˛ 2 .Zpe /2 then ˛ .p (iii) If ˛ …
.Zpe /2
then
e /=2
e ˛ .p /=2
D 1. D
1.
Theorem 2.28. Let p be an odd prime and e be a positive integer. Q ˇ 2Ze ˇ D 1. p
Then
40
Congruences
Theorem 2.29. Let p be an odd prime, e be a positive integer, and ˛; ˇ 2 Zpe . If ˛ … .Zpe /2 and ˇ … .Zpe /2 , then ˛ˇ 2 .Zpe /2 . It turns out that an integer is a quadratic residue modulo p e if and only if it is a quadratic residue modulo p. Theorem 2.30. Let p be an odd prime, e be a positive integer, and a be any integer. Then a is a quadratic residue modulo p e if and only if a is a quadratic residue modulo p. Proof. Suppose that a is a quadratic residue modulo p e . Then a is not divisible by p and a b 2 .mod p e / for some integer b. It follows that a b 2 .mod p/, and so a is a quadratic residue modulo p. Suppose that a is not a quadratic residue modulo p e . If a is divisible by p, then by definition a is not a quadratic residue modulo p. So suppose a is not divisible by p. By Theorem 2.27, we have ap
e 1 .p
1/=2
1 .mod p e /:
This congruence holds modulo p as well, and by Fermat’s little theorem (applied e 1 times), 2
a ap ap ap
e 1
.mod p/;
and so 1 ap
e 1 .p
1/=2
a.p
1/=2
.mod p/:
Theorem 2.21 therefore implies that a is not a quadratic residue modulo p. 2.8.3 Quadratic residues modulo n We now study quadratic residues modulo n, where n is an arbitrary, odd integer, with n > 1. Let n D p1e1 prer be the prime factorization of n. Our main tools here are the Chinese remainder map W Zn ! Zpe1 Zprer ; 1
introduced in Theorem 2.8, together with the results developed so far for quadratic residues modulo odd prime powers. Let ˛ 2 Zn with .˛/ D .˛1 ; : : : ; ˛r /.
41
2.8 Quadratic residues
On the one hand, suppose that ˛ D ˇ 2 for some ˇ 2 Zn . If .ˇ/ D .ˇ1 ; : : : ; ˇr /, we have .˛1 ; : : : ; ˛r / D .˛/ D .ˇ 2 / D .ˇ12 ; : : : ; ˇr2 /; where we have used part (iii.c) of Theorem 2.8. It follows that ˛i D ˇi2 for each i . On the other hand, suppose that for each i , ˛i D ˇi2 for some ˇi 2 Zp ei . i Then setting ˇ WD 1 .ˇ1 ; : : : ; ˇr /, we have .ˇ 2 / D .ˇ12 ; : : : ; ˇr2 / D .˛1 ; : : : ; ˛r / D .˛/; where we have again used part (iii.c) of Theorem 2.8, along with the fact that is bijective (to define ˇ). Thus, .˛/ D .ˇ 2 /, and again since is bijective, it follows that ˛ D ˇ 2 . We have shown that 2 ˛ 2 .Zn /2 ” ˛i 2 Zei for i D 1; : : : ; r: pi
In particular, restricting to .Zn /2 yields a one-to-one correspondence between .Zn /2 and 2 2 Ze1 Zpr er ; p1
and therefore, by Theorem 2.26 (and Theorem 2.9), we have j.Zn /2 j
D
r Y
. .piei /=2/ D .n/=2r :
i D1
Now suppose that ˛ D ˇ 2 , with ˇ 2 Zn and .ˇ/ D .ˇ1 ; : : : ; ˇr /. Consider an arbitrary element 2 Zn , with . / D . 1 ; : : : ; r /. Then we have
2 D ˇ 2 ” . 2 / D .ˇ 2 / ” . 12 ; : : : ; r2 / D .ˇ12 ; : : : ; ˇr2 / ” . 1 ; : : : ; r / D .˙ˇ1 ; : : : ; ˙ˇr / (by Theorem 2.25): Therefore, ˛ has precisely 2r square roots, namely,
1 .˙ˇ ; : : : ; ˙ˇ /: 1 r
2.8.4 Square roots of 1 modulo p Using Euler’s criterion, we can easily characterize those primes modulo which 1 is a quadratic residue. This turns out to have a number of nice applications. Consider an odd prime p. The following theorem says that the question of whether 1 is a quadratic residue modulo p is decided by the residue class of p modulo 4. Since p is odd, either p 1 .mod 4/ or p 3 .mod 4/.
42
Congruences
Theorem 2.31. Let p be an odd prime. Then 1 is a quadratic residue modulo p if and only p 1 .mod 4/. Proof. By Euler’s criterion, 1 is a quadratic residue modulo p if and only if . 1/.p 1/=2 1 .mod p/. If p 1 .mod 4/, then .p 1/=2 is even, and so . 1/.p 1/=2 D 1. If p 3 .mod 4/, then .p 1/=2 is odd, and so . 1/.p 1/=2 D 1. In fact, when p 1 .mod 4/, any non-square in Zp yields a square root of 1 modulo p, as follows: Theorem 2.32. Let p be a prime with p 1 .mod 4/, 2 Zp n .Zp /2 , and ˇ WD .p 1/=4 . Then ˇ 2 D 1. Proof. This is a simple calculation, based on Euler’s criterion: ˇ 2 D .p
1/=2
D
1:
The fact that 1 is a quadratic residue modulo primes p 1 .mod 4/ can be used to prove Fermat’s theorem that such primes may be written as the sum of two squares. To do this, we first need the following technical lemma: Theorem 2.33 (Thue’s lemma). Let n; b; r ; t 2 Z, with 0 < r n < r t . Then there exist r; t 2 Z with r bt .mod n/; jrj < r ; and 0 < jtj < t : Proof. For i D 0; : : : ; r 1 and j D 0; : : : ; t 1, we define the number vij WD i bj . Since we have defined r t numbers, and r t > n, two of these numbers must lie in the same residue class modulo n; that is, for some .i1 ; j1 / ¤ .i2 ; j2 /, we have vi1 j1 vi2 j2 .mod n/. Setting r WD i1 i2 and t WD j1 j2 , this implies r bt .mod n/, jrj < r , jtj < t , and that either r ¤ 0 or t ¤ 0. It only remains to show that t ¤ 0. Suppose to the contrary that t D 0. This would imply that r 0 .mod n/ and r ¤ 0, which is to say that r is a non-zero multiple of n; however, this is impossible, since jrj < r n. Theorem 2.34 (Fermat’s two squares theorem). Let p be an odd prime. Then p D r 2 C t 2 for some r; t 2 Z if and only if p 1 .mod 4/. Proof. One direction is easy. Suppose p 3 .mod 4/. It is easy to see that the square of every integer is congruent to either 0 or 1 modulo 4; therefore, the sum of two squares is congruent to either 0, 1, or 2 modulo 4, and so can not be congruent to p modulo 4 (let alone equal to p). For the other direction, suppose p 1 .mod 4/. We know that 1 is a quadratic residue modulo p, so let b be an integer such that b 2 1 .mod p/. Now
43
2.8 Quadratic residues
p
apply Theorem 2.33 with n WD p, b as just defined, and r WD t WD b pc C 1. p p p Evidently, b pc C 1 > p, and hence r t > p. Also, since p is prime, p is p p p not an integer, and so b pc < p < p; in particular, r D b pc C 1 p. Thus, the hypotheses of that theorem are satisfied, and therefore, there exist integers r and t such that p p p p r bt .mod p/; jrj b pc < p; and 0 < jtj b pc < p: It follows that r 2 b2t 2
t 2 .mod p/:
Thus, r 2 C t 2 is a multiple of p and 0 < r 2 C t 2 < 2p. The only possibility is that r 2 C t 2 D p. The fact that 1 is a quadratic residue modulo an odd prime p only if p 1 .mod 4/ can be used so show there are infinitely many such primes. Theorem 2.35. There are infinitely many primes p 1 .mod 4/. Proof. Suppose there were only finitely many such primes, p1 ; : : : ; pk . Set M WD Qk 2 i D1 pi and N WD 4M C 1. Let p be any prime dividing N . Evidently, p is not among the pi ’s, since if it were, it would divide both N and 4M 2 , and so also N 4M 2 D 1. Also, p is clearly odd, since N is odd. Moreover, .2M /2 1 .mod p/; therefore, 1 is a quadratic residue modulo p, and so p 1 .mod 4/, contradicting the assumption that p1 ; : : : ; pk are the only such primes. For completeness, we also state the following fact: Theorem 2.36. There are infinitely many primes p 3 .mod 4/. Proof. Suppose there were only finitely many such primes, p1 ; : : : ; pk . Set M WD Qk 1. Since N 3 .mod 4/, there must be some prime i D1 pi and N WD 4M p 3 .mod 4/ dividing N (if all primes dividing N were congruent to 1 modulo 4, then so too would be their product N ). Evidently, p is not among the pi ’s, since if it were, it would divide both N and 4M , and so also 4M N D 1. This contradicts the assumption that p1 ; : : : ; pk are the only primes congruent to 3 modulo 4. E XERCISE 2.33. Let n be a positive integer, let m be an integer, and let d WD gcd.m; .n//. Show that: (a) if d D 1, then .Zn /m D .Zn /; (b) if ˛ 2 .Zn /m , then ˛ .n/=d D 1. E XERCISE 2.34. Calculate the sets C and D in the proof of Theorem 2.21 in the case p D 11 and ˛ D 1.
44
Congruences
E XERCISE 2.35. Calculate the square roots of 1 modulo 4, 8, and 16. E XERCISE 2.36. Let n 2 Z with n > 1. Show that n is prime if and only if .n 1/Š 1 .mod n/. E XERCISE 2.37. Let p be a prime with p 1 .mod 4/, and b WD ..p Show that b 2 1 .mod p/.
1/=2/Š.
E XERCISE 2.38. Let n WD pq, where p and q are distinct, odd primes. Show that there exist ˛; ˇ 2 Zn such that ˛ … .Zn /2 , ˇ … .Zn /2 , and ˛ˇ … .Zn /2 . E XERCISE 2.39. Let n be an odd positive integer, and let a be any integer. Show that a is a quadratic residue modulo n if and only if a is a quadratic residue modulo p for each prime p j n. E XERCISE 2.40. Show that if p is an odd prime, with p 3 .mod 4/, then .Zp /4 D .Zp /2 . More generally, show that if n is an odd positive integer, where p 3 .mod 4/ for each prime p j n, then .Zn /4 D .Zn /2 . E XERCISE 2.41. Let p be an odd prime, and let e 2 Z with e > 1. Let a be an integer of the form a D p f b, where 0 f < e and p − b. Consider the integer solutions z to the congruence z 2 a .mod p e /. Show that a solution exists if and only if f is even and b is a quadratic residue modulo p, in which case there are exactly 2p f distinct solutions modulo p e . E XERCISE 2.42. Suppose p is an odd prime, and that r 2 C t 2 D p for some integers r; t. Show that if x; y are integers such that x 2 C y 2 D p, then .x; y/ 2 f.˙r; ˙t/; .˙t; ˙r/g. E XERCISE 2.43. Show that if both u and v are the sum of two squares of integers, then so is their product uv. E XERCISE 2.44. Suppose r 2 C t 2 0 .mod n/, where n is a positive integer, and suppose p is an odd prime dividing n. Show that: (a) if p divides neither r nor t , then p 1 .mod 4/; (b) if p divides one of r or t , then it divides the other, and moreover, p 2 divides n, and .r=p/2 C .t =p/2 0 .mod n=p 2 /. E XERCISE 2.45. Let n be a positive integer, and write n D ab 2 where a and b are positive integers, and a is square-free (see Exercise 1.15). Show that n is the sum of two squares of integers if and only if no prime p 3 .mod 4/ divides a. Hint: use the previous two exercises.
45
2.9 Summations over divisors
2.9 Summations over divisors We close this chapter with a brief treatment of summations over divisors. To this end, we introduce some terminology and notation. By an arithmetic function, we simply mean a function from the positive integers into the reals (actually, one usually considers complex-valued functions as well, but we shall not do so here). Let f and g be arithmetic functions. The Dirichlet product of f and g, denoted f ? g, is the arithmetic function whose value at n is defined by the formula X .f ? g/.n/ WD f .d /g.n=d /; d jn
the sum being over all positive divisors d of n. Another, more symmetric, way to write this is X .f ? g/.n/ D f .d1 /g.d2 /; nDd1 d2
the sum being over all pairs .d1 ; d2 / of positive integers with d1 d2 D n. The Dirichlet product is clearly commutative (i.e., f ? g D g ? f ), and is associative as well, which one can see by checking that X .f ? .g ? h//.n/ D f .d1 /g.d2 /h.d3 / D ..f ? g/ ? h/.n/; nDd1 d2 d3
the sum being over all triples .d1 ; d2 ; d3 / of positive integers with d1 d2 d3 D n. We now introduce three special arithmetic functions: ı, 1, and . The functions ı and 1 are defined as follows: 1 if n D 1; ı.n/ WD 1.n/ WD 1: 0 if n > 1; The Möbius function is defined as follows: if n D p1e1 prer is the prime factorization of n, then 0 if ei > 1 for some i D 1; : : : ; rI .n/ WD r . 1/ otherwise: In other words, .n/ D 0 if n is not square-free (see Exercise 1.15); otherwise, .n/ is . 1/r where r is the number of distinct primes dividing n. Here are some examples: .1/ D 1; .2/ D
1; .3/ D
1; .4/ D 0; .5/ D
1; .6/ D 1:
It is easy to see from the definitions that for every arithmetic function f , we have X ı ? f D f and .1 ? f /.n/ D f .d /: d jn
46
Congruences
Thus, ı acts as a multiplicative identity with respect to the Dirichlet product, while “1 ? ” acts as a “summation over divisors” operator. An arithmetic function f is called multiplicative if f .1/ D 1 and for all positive integers n; m with gcd.n; m/ D 1, we have f .nm/ D f .n/f .m/. The reader may easily verify that ı, 1, and are multiplicative functions. Theorem 2.9 says that Euler’s function is multiplicative. The reader may also verify the following: Theorem 2.37. If f is a multiplicative arithmetic function, and if n D p1e1 prer is the prime factorization of n, then f .n/ D f .p1e1 / f .prer /: Proof. Exercise. A key property of the Möbius function is the following: Theorem 2.38. Let f be a multiplicative arithmetic function. If n D p1e1 prer is the prime factorization of n, then X .d /f .d / D .1 f .p1 // .1 f .pr //: (2.9) d jn
Proof. The only non-zero terms appearing in the sum on the left-hand side of (2.9) are those corresponding to divisors d of the form pi1 pi` , where pi1 ; : : : ; pi` are distinct; the value contributed to the sum by such a term is . 1/` f .pi1 pi` / D . 1/` f .pi1 / f .pi` /. These are the same as the terms in the expansion of the product on the right-hand side of (2.9). If we set f WD 1 in the previous theorem, then we see that X 1 if n D 1; .d / D 0 if n > 1. d jn
Translating this into the language of Dirichlet products, we have 1 ? D ı: Thus, with respect to the Dirichlet product, the functions 1 and are multiplicative inverses of one another. Based on this, we may easily derive the following: Theorem 2.39 (Möbius inversion formula). Let f and F be arithmetic functions. Then F D 1 ? f if and only if f D ? F . Proof. If F D 1 ? f , then ? F D ? .1 ? f / D . ? 1/ ? f D ı ? f D f;
2.9 Summations over divisors
47
and conversely, if f D ? F , then 1 ? f D 1 ? . ? F / D .1 ? / ? F D ı ? F D F: The Möbius inversion formula says this: X F .n/ D f .d / for all positive integers n d jn
” f .n/ D
X
.d /F .n=d / for all positive integers n.
d jn
The Möbius inversion formula is a useful tool. As an application, we use it to obtain a simple proof of the following fact: P Theorem 2.40. For every positive integer n, we have d jn .d / D n: Proof. Let us define the arithmetic functions N.n/ WD n and M.n/ WD 1=n. Our goal is to show that N D 1 ? , and by Möbius inversion, it suffices to show that ? N D . If n D p1e1 prer is the prime factorization of n, we have X X . ? N /.n/ D .d /.n=d / D n .d /=d d jn r Y
Dn
d jn
.1
1=pi / (applying Theorem 2.38 with f WD M )
i D1
D .n/ (by Theorem 2.11): E XERCISE 2.46. In our definition of a multiplicative function f , we made the requirement that f .1/ D 1. Show that if we dropped this requirement, the only other function that would satisfy the definition would be the zero function (i.e., the function that is everywhere zero). E XERCISE 2.47. Let f be a polynomial with integer coefficients, and for every positive integer n, define !f .n/ to be the number of integers x 2 f0; : : : ; n 1g such that f .x/ 0 .mod n/. Show that !f is multiplicative. E XERCISE 2.48. Show that if f and g are multiplicative, then so is f ? g. Hint: use Exercise 1.18. E XERCISE 2.49. Let .n/ be the number of positive divisors of n. Show that: (a) is a multiplicative function; Q (b) .n/ D riD1 .ei C 1/; where n D p1e1 prer is the prime factorization of n;
48
Congruences
(c)
P
(d)
P
d jn .d / .n=d / d jn .d / .d /
D 1;
D . 1/r ; where n D p1e1 prer is the prime factorization
of n. E XERCISE 2.50. Define .n/ WD
P
d jn d .
Show that:
(a) is a multiplicative function; Q (b) .n/ D riD1 .piei C1 1/=.pi 1/; where n D p1e1 prer is the prime factorization of n; P (c) d jn .d /.n=d / D n; P e1 er r (d) d jn .d /.d / D . 1/ p1 pr ; where n D p1 pr is the prime factorization of n. E XERCISE 2.51. The Mangoldt function ƒ.n/ is defined for all positive integers n as follows: ƒ.n/ WD log p, if n D p k for some prime p and positive integer k, P and ƒ.n/ WD 0, otherwise. Show that d jn ƒ.d / D log n, and from this, deduce P that ƒ.n/ D d jn .d / log d . E XERCISE 2.52. Show that if f is multiplicative, and if n D p1e1 prer is the P prime factorization of n, then d jn .d /2 f .d / D .1 C f .p1 // .1 C f .pr //: P E XERCISE 2.53. Show that n is square-free if and only if d jn .d /2 .d / D n. E XERCISE 2.54. Show that for every arithmetic function f with f .1/ ¤ 0, there is a unique arithmetic function g, called the Dirichlet inverse of f , such that f ? g D ı. Also, show that if f .1/ D 0, then f has no Dirichlet inverse. E XERCISE 2.55. Show that if f is a multiplicative function, then so is its Dirichlet inverse (as defined in the previous exercise). E XERCISE 2.56. This exercise develops an alternative proof of Theorem 2.40 that does not depend on Theorem 2.11. Let n be a positive integer. Define Fn WD fi=n 2 Q W i D 0; : : : ; n 1g. Also, for each positive integer d , define Gd WD fa=d 2 Q W a 2 Z; gcd.a; d / D 1g. (a) Show that for each x 2 Fn , there exists a unique positive divisor d of n such that x 2 Gd . (b) Show that for each positive divisor d of n, we have Fn \ Gd D fa=d W a 2 f0; : : : ; d 1g; gcd.a; d / D 1g. P (c) Using (a) and (b), show that d jn .d / D n. E XERCISE 2.57. Using Möbius inversion, directly derive Theorem 2.11 from Theorem 2.40.
3 Computing with large integers
In this chapter, we review standard asymptotic notation, introduce the formal computational model we shall use throughout the rest of the text, and discuss basic algorithms for computing with large integers. 3.1 Asymptotic notation We review some standard notation for relating the rate of growth of functions. This notation will be useful in discussing the running times of algorithms, and in a number of other contexts as well. Let f and g be real-valued functions, both defined either on the set of nonnegative integers, or on the set of non-negative reals. Actually, as we are only concerned about the behavior of f .x/ and g.x/ as x ! 1, we only require that f .x/ and g.x/ are defined for all sufficiently large x. We further assume that g is eventually positive, meaning that g.x/ > 0 for all sufficiently large x. Then f D O.g/ means that jf .x/j cg.x/ for some positive constant c and all sufficiently large x (read, “f is big-O of g”), f D .g/ means that f .x/ cg.x/ for some positive constant c and all sufficiently large x (read, “f is big-Omega of g”), f D ‚.g/ means that cg.x/ f .x/ dg.x/ for some positive constants c and d and all sufficiently large x (read, “f is big-Theta of g”), f D o.g/ means that f .x/=g.x/ ! 0 as x ! 1 (read, “f is little-o of g”), and f g means that f .x/=g.x/ ! 1 as x ! 1 (read, “f is asymptotically equal to g”). Example 3.1. Let f .x/ WD x 2 and g.x/ WD 2x 2 f D .g/. Indeed, f D ‚.g/. 49
10x C 1. Then f D O.g/ and
50
Computing with large integers
Example 3.2. Let f .x/ WD x 2 and g.x/ WD x 2
10x C 1. Then f g.
Example 3.3. Let f .x/ WD 100x 2 and g.x/ WD x 3 . Then f D o.g/. Note that by definition, if we write f D .g/, f D ‚.g/, or f g, it must be the case that f (in addition to g) is eventually positive; however, if we write f D O.g/ or f D o.g/, then f need not be eventually positive. When one writes “f D O.g/,” one should interpret “ D O./” as a binary relation between f with g. Analogously for “f D .g/,” “f D ‚.g/,” and “f D o.g/.” One may also write “O.g/” in an expression to denote an anonymous function f such that f D O.g/. Analogously, .g/, ‚.g/, and o.g/ may denote anonymous functions. The expression O.1/ denotes a function bounded in absolute value by a constant, while the expression o.1/ denotes a function that tends to zero in the limit. Example 3.4. Let f .x/ WD x 3 2x 2 Cx 3. One could write f .x/ D x 3 CO.x 2 /: Here, the anonymous function is g.x/ WD 2x 2 Cx 3, and clearly g.x/ D O.x 2 /. One could also write f .x/ D x 3 .2 C o.1//x 2 : Here, the anonymous function is g.x/ WD 1=x C 3=x 2 . While g D o.1/, it is only defined for x > 0. This is acceptable, as we will only regard statements such as this asymptotically, as x ! 1. As an even further use (abuse?) of the notation, one may use the big-O, -Omega, and -Theta notation for functions on an arbitrary domain, in which case the relevant inequalities should hold throughout the entire domain. This usage includes functions of several independent variables, as well as functions defined on sets with no natural ordering. E XERCISE 3.1. Show that: (a) f D o.g/ implies f D O.g/ and g ¤ O.f /; (b) f D O.g/ and g D O.h/ implies f D O.h/; (c) f D O.g/ and g D o.h/ implies f D o.h/; (d) f D o.g/ and g D O.h/ implies f D o.h/. E XERCISE 3.2. Let f and g be eventually positive functions. Show that: (a) f g if and only if f D .1 C o.1//g; (b) f g implies f D ‚.g/; (c) f D ‚.g/ if and only if f D O.g/ and f D .g/; (d) f D .g/ if and only if g D O.f /.
51
3.1 Asymptotic notation
E XERCISE 3.3. Suppose f1 D O.g1 / and f2 D O.g2 /. Show that f1 C f2 D O.max.g1 ; g2 //, f1 f2 D O.g1 g2 /, and that for every positive constant c, cf1 D O.g1 /. E XERCISE 3.4. Suppose that f .x/ c C dg.x/ for some positive constants c and d , and for all sufficiently large x. Show that if g D .1/, then f D O.g/. E XERCISE 3.5. Suppose f and g are defined on the integers i k, and that g.i / > 0 for all i k. Show that if f D O.g/, then there exists a positive constant c such that jf .i /j cg.i / for all i k. E XERCISE 3.6. Let f and g be eventually positive functions, and suppose f .x/=g.x/ tends to a limit L (possibly L D 1) as x ! 1. Show that: (a) if L D 0, then f D o.g/; (b) if 0 < L < 1, then f D ‚.g/; (c) if L D 1, then g D o.f /. E XERCISE 3.7. Let f .x/ WD x ˛ .log x/ˇ and g.x/ WD x .log x/ı , where ˛; ˇ; ; ı are non-negative constants. Show that if ˛ < , or if ˛ D and ˇ < ı, then f D o.g/. E XERCISE 3.8. Order the following functions in x so that for each adjacent pair f; g in the ordering, we have f D O.g/, and indicate if f D o.g/, f g, or g D O.f /: p x 3 ; e x x 2 ; 1=x; x 2 .x C 100/ C 1=x; x C x; log2 x; log3 x; 2x 2 ; x; e
x
; 2x 2
10x C 4; e xC
p
x
; 2x ; 3x ; x
2
; x 2 .log x/1000 :
E XERCISE 3.9. Show that: (a) the relation “” is an equivalence relation on the set of eventually positive functions; (b) for all eventually positive functions f1 ; f2 ; g1 ; g2 , if f1 g1 and f2 g2 , then f1 ? f2 g1 ? g2 , where “?” denotes addition, multiplication, or division; (c) for all eventually positive functions f; g, and every ˛ > 0, if f g, then f ˛ g˛ ; (d) for all eventually positive functions f; g, and every function h such that h.x/ ! 1 as x ! 1, if f g, then f B h g B h, where “B” denotes function composition. E XERCISE 3.10. Show that all of the claims in the previous exercise also hold when the relation “” is replaced with the relation “ D ‚./.”
52
Computing with large integers
E XERCISE 3.11. Let f; g be eventually positive functions. Show that: (a) f D ‚.g/ if and only if log f D log g C O.1/; (b) f g if and only if log f D log g C o.1/. E XERCISE 3.12. Suppose that f and g are functions defined on the integers k; kC P 1; : : : ; and that g is eventually positive. For n k, define F .n/ WD niDk f .i / P and G.n/ WD niDk g.i /. Show that if f D O.g/ and G is eventually positive, then F D O.G/. E XERCISE 3.13. Suppose that f and g are piece-wise continuous Ron Œa; 1/ (see x §A4), andRthat g is eventually positive. For x a, define F .x/ WD a f .t /dt and x G.x/ WD a g.t/dt . Show that if f D O.g/ and G is eventually positive, then F D O.G/. E XERCISE 3.14. Suppose that f and g are functions defined on the integers k; k C 1; : : : ; both of which are eventually positive. For n k, define F .n/ WD Pn Pn i Dk f .i/ and G.n/ WD i Dk g.i /. Show that if f g and G.n/ ! 1 as n ! 1, then F G. E XERCISE 3.15. Suppose that f and g are piece-wise continuous on Œa; R x1/ (see WD §A4), both of which are eventually positive. For x a, define F .x/ a f .t /dt Rx and G.x/ WD a g.t/dt . Show that if f g and G.x/ ! 1 as x ! 1, then F G. E XERCISE 3.16. Give an example of two non-decreasing functions f and g, both mapping positive integers to positive integers, such that f ¤ O.g/ and g ¤ O.f /. 3.2 Machine models and complexity theory When presenting an algorithm, we shall always use a high-level, and somewhat informal, notation. However, all of our high-level descriptions can be routinely translated into the machine-language of an actual computer. So that our theorems on the running times of algorithms have a precise mathematical meaning, we formally define an “idealized” computer: the random access machine or RAM. A RAM consists of an unbounded sequence of memory cells mŒ0; mŒ1; mŒ2; : : : each of which can store an arbitrary integer, together with a program. A program consists of a finite sequence of instructions I0 ; I1 ; : : :, where each instruction is of one of the following types: arithmetic This type of instruction is of the form
˛ ? ˇ, where ? represents
3.2 Machine models and complexity theory
53
one of the operations addition, subtraction, multiplication, or integer division (i.e., b=c). The values ˛ and ˇ are of the form c, mŒa, or mŒmŒa, and is of the form mŒa or mŒmŒa, where c is an integer constant and a is a non-negative integer constant. Execution of this type of instruction causes the value ˛ ? ˇ to be evaluated and then stored in . branching This type of instruction is of the form IF ˛ Þ ˇ GOTO i , where i is the index of an instruction, and where Þ is one of the comparison operations D; ¤; <; >; ; , and ˛ and ˇ are as above. Execution of this type of instruction causes the “flow of control” to pass conditionally to instruction Ii . halt The HALT instruction halts the execution of the program. A RAM executes by executing instruction I0 , and continues to execute instructions, following branching instructions as appropriate, until a HALT instruction is executed. We do not specify input or output instructions, and instead assume that the input and output are to be found in memory cells at some prescribed locations, in some standardized format. To determine the running time of a program on a given input, we charge 1 unit of time to each instruction executed. This model of computation closely resembles a typical modern-day computer, except that we have abstracted away many annoying details. However, there are two details of real machines that cannot be ignored; namely, any real machine has a finite number of memory cells, and each cell can store numbers only in some fixed range. The first limitation must be dealt with by either purchasing sufficient memory or designing more space-efficient algorithms. The second limitation is especially annoying, as we will want to perform computations with quite large integers — much larger than will fit into any single memory cell of an actual machine. To deal with this limitation, we shall represent such large integers as vectors of digits to some fixed base, so that each digit is bounded so as to fit into a memory cell. This is discussed in more detail in the next section. The only other numbers we actually need to store in memory cells are “small” numbers representing array indices, counters, and the like, which hopefully will fit into the memory cells of actual machines. Below, we shall make a more precise, formal restriction on the magnitude of numbers that may be stored in memory cells. Even with these caveats and restrictions, the running time as we have defined it for a RAM is still only a rough predictor of performance on an actual machine. On a real machine, different instructions may take significantly different amounts
54
Computing with large integers
of time to execute; for example, a division instruction may take much longer than an addition instruction. Also, on a real machine, the behavior of the cache may significantly affect the time it takes to load or store the operands of an instruction. Finally, the precise running time of an algorithm given by a high-level description will depend on the quality of the translation of this algorithm into “machine code.” However, despite all of these problems, it still turns out that measuring the running time on a RAM as we propose here is a good “first order” predictor of performance on real machines in many cases. Also, we shall only state the running time of an algorithm using a big-O estimate, so that implementation-specific constant factors are anyway “swept under the rug.” If we have an algorithm for solving a certain problem, we expect that “larger” instances of the problem will require more time to solve than “smaller” instances, and a general goal in the analysis of any algorithm is to estimate the rate of growth of the running time of the algorithm as a function of the size of its input. For this purpose, we shall simply measure the size of an input as the number of memory cells used to represent it. Theoretical computer scientists sometimes equate the notion of “efficient” with “polynomial time” (although not everyone takes theoretical computer scientists very seriously, especially on this point): a polynomial-time algorithm is one whose running time on inputs of size n is at most anb C c, for some constants a, b, and c (a “real” theoretical computer scientist will write this as nO.1/ ). Furthermore, we also require that for a polynomial-time algorithm, all 0 numbers stored in memory are at most a0 nb C c 0 in absolute value, for some constants a0 , b 0 , and c 0 . Even for algorithms that are not polynomial time, we shall insist that after executing t instructions, all numbers stored in memory are at most 0 a0 .n C t/b C c 0 in absolute value, for some constants a0 , b 0 , and c 0 . Note that in defining the notion of polynomial time on a RAM, it is essential that we restrict the magnitude of numbers that may be stored in the machine’s memory cells, as we have done above. Without this restriction, a program could perform arithmetic on huge numbers, being charged just one unit of time for each arithmetic operation— not only is this intuitively “wrong,” it is possible to come up with programs that solve some problems using a polynomial number of arithmetic operations on huge numbers, and these problems cannot otherwise be solved in polynomial time (see §3.6). 3.3 Basic integer arithmetic We will need algorithms to manipulate very large integers. Since such integers will exceed the word-size of actual machines, and to satisfy the formal requirements of our random access model of computation, we shall represent large integers as
3.3 Basic integer arithmetic
55
vectors of digits to some base B, along with a bit indicating the sign. That is, for a 2 Z, if we write aD˙
k X1
ai B i D ˙.ak
1 a1 a0 /B ;
i D0
where 0 ai < B for i D 0; : : : ; k 1, then a will be represented in memory as a data structure consisting of the vector of base-B digits a0 ; : : : ; ak 1 , along with a “sign bit” to indicate the sign of a. To ensure a unique representation, if a is non-zero, then the high-order digit ak 1 in this representation should be non-zero. For our purposes, we shall consider B to be a constant, and moreover, a power of 2. The choice of B as a power of 2 is convenient for a number of technical reasons. A note to the reader: If you are not interested in the low-level details of algorithms for integer arithmetic, or are willing to take them on faith, you may safely skip ahead to §3.3.5, where the results of this section are summarized. We now discuss in detail basic arithmetic algorithms for unsigned (i.e., nonnegative) integers — these algorithms work with vectors of base-B digits, and except where explicitly noted, we do not assume the high-order digits of the input vectors are non-zero, nor do these algorithms ensure that the high-order digit of the output vector is non-zero. These algorithms can be very easily adapted to deal with arbitrary signed integers, and to take proper care that the high-order digit of the vector representing a non-zero number is non-zero (the reader is asked to fill in these details in some of the exercises below). All of these algorithms can be implemented directly in a programming language that provides a “built-in” signed integer type that can represent all integers of absolute value less than B 2 , and that provides the basic arithmetic operations (addition, subtraction, multiplication, integer division). So, for example, using the C or Java programming language’s int type on a typical 32-bit computer, we could take B D 215 . The resulting software would be reasonably efficient, but certainly not the best possible. Suppose we have the base-B representations of two unsigned integers a and b. We present algorithms to compute the base-B representation of a C b, a b, a b, ba=bc, and a mod b. To simplify the presentation, for integers x; y with y ¤ 0, we write QuoRem.x; y/ to denote the quotient/remainder pair .bx=yc; x mod y/. 3.3.1 Addition Let a D .ak 1 a0 /B and b D .b` 1 b0 /B be unsigned integers. Assume that k ` 1 (if k < `, then we can just swap a and b). The sum c WD a C b is of the form c D .ck ck 1 c0 /B . Using the standard “paper-and-pencil”
56
Computing with large integers
method (adapted from base-10 to base-B, of course), we can compute the base-B representation of a C b in time O.k/, as follows: 0 0 to ` tmp ai for i ` to k tmp ai ck carry carry for i
1 do C bi C carry, .carry; ci / QuoRem.tmp; B/ 1 do C carry, .carry; ci / QuoRem.tmp; B/
Note that in every loop iteration, the value of carry is 0 or 1, and the value tmp lies between 0 and 2B 1. 3.3.2 Subtraction Let a D .ak 1 a0 /B and b D .b` 1 b0 /B be unsigned integers. Assume that k ` 1. To compute the difference c WD a b, we may use the same algorithm as above, but with the expression “ai C bi ” replaced by “ai bi .” In every loop iteration, the value of carry is 0 or 1, and the value of tmp lies between B and B 1. If a b, then ck D 0 (i.e., there is no carry out of the last loop iteration); otherwise, ck D 1 (and b a D B k .ck 1 c0 /B , which can be computed with another execution of the subtraction routine). 3.3.3 Multiplication Let a D .ak 1 a0 /B and b D .b` 1 b0 /B be unsigned integers, with k 1 and ` 1. The product c WD a b is of the form .ckC` 1 c0 /B , and may be computed in time O.k`/ as follows: for i for i
0 to k C ` 1 do ci 0 0 to k 1 do carry 0 for j 0 to ` 1 do tmp ai bj C ci Cj C carry .carry; ci Cj / QuoRem.tmp; B/ ci C` carry
Note that at every step in the above algorithm, the value of carry lies between 0 and B 1, and the value of tmp lies between 0 and B 2 1.
3.3 Basic integer arithmetic
57
3.3.4 Division with remainder Let a D .ak 1 a0 /B and b D .b` 1 b0 /B be unsigned integers, with k 1, ` 1, and b` 1 ¤ 0. We want to compute q and r such that a D bq C r and 0 r < b. Assume that k `; otherwise, a < b, and we can just set q 0 and r a. The quotient q will have at most m WD k ` C 1 base-B digits. Write q D .qm 1 q0 /B . At a high level, the strategy we shall use to compute q and r is the following: r a for i m 1 down to 0 do qi br =B i bc r r B i qi b One easily verifies by induction that at the beginning of each loop iteration, we have 0 r < B i C1 b, and hence each qi will be between 0 and B 1, as required. Turning the above strategy into a detailed algorithm takes a bit of work. In particular, we want an easy way to compute br =B i bc. Now, we could in theory just try all possible choices for qi — this would take time O.B`/, and viewing B as a constant, this is O.`/. However, this is not really very desirable from either a practical or theoretical point of view, and we can do much better with just a little effort. We shall first consider a special case; namely, the case where ` D 1. In this case, the computation of the quotient br =B i bc is facilitated by the following, which essentially tells us that this quotient is determined by the two high-order digits of r: Theorem 3.1. Let x and y be integers such that 0 x D x 0 2n C s and 0 < y D y 0 2n for some integers n; s; x 0 ; y 0 , with n 0 and 0 s < 2n . Then bx=yc D bx 0 =y 0 c. Proof. We have x x0 s x0 D 0 C 0 n 0: y y y2 y It follows immediately that bx=yc bx 0 =y 0 c. We also have 0 x x0 s x0 1 x y0 1 1 C C 0: D 0C 0 n < 0C 0 0 0 y y y2 y y y y y Thus, we have x=y < bx 0 =y 0 c C 1; and hence, bx=yc bx 0 =y 0 c.
58
Computing with large integers
From this theorem, one sees that the following algorithm correctly computes the quotient and remainder in time O.k/ (in the case ` D 1): r 0 for i k 1 down to 0 do tmp r B C ai .qi ; r/ QuoRem.tmp; b0 / output the quotient q D .qk 1 q0 /B and the remainder r Note that in every loop iteration, the value of r lies between 0 and b0 B and the value of tmp lies between 0 and B b0 C .B 1/ B 2 1.
1,
That takes care of the special case where ` D 1. Now we turn to the general case ` 1. In this case, we cannot so easily get the digits qi of the quotient, but we can still fairly easily estimate these digits, using the following: Theorem 3.2. Let x and y be integers such that 0 x D x 0 2n C s and 0 < y D y 0 2n C t for some integers n; s; t; x 0 ; y 0 with n 0, 0 s < 2n , and 0 t < 2n . Further, suppose that 2y 0 x=y. Then bx=yc bx 0 =y 0 c bx=yc C 2: Proof. We have x=y x =y 0 2n , and so bx=yc bx =y 0 2n c, and by the previous theorem, bx =y 0 2n c D bx 0 =y 0 c. That proves the first inequality. For the second inequality, first note that from the definitions, we have x=y 0 x =.y 0 C 1/, which implies x 0 y xy 0 x 0. Further, 2y 0 x=y implies 2yy 0 x 0. So we have 2yy 0 x 0 x 0 y xy 0 x, which implies x=y x 0 =y 0 2, and hence bx=yc bx 0 =y 0 c 2. Based on this theorem, we first present an algorithm for division with remainder that works assuming that b is appropriately “normalized,” meaning that b` 1 2w 1 , where B D 2w . This algorithm is shown in Fig. 3.1. Some remarks are in order: 1. In line 4, we compute qi , which by Theorem 3.2 is greater than or equal to the true quotient digit, but exceeds this value by at most 2. 2. In line 5, we reduce qi if it is obviously too big. 3. In lines 6–10, we compute .ri C` ri /B
.ri C` ri /B
qi b:
In each loop iteration, the value of tmp lies between .B 2 and the value carry lies between .B 1/ and 0.
B/ and B
1,
59
3.3 Basic integer arithmetic
1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18.
for i 0 to k 1 do ri ai rk 0 for i k ` down to 0 do qi b.ri C` B C ri C` 1 /=b` 1 c if qi B then qi B 1 carry 0 for j 0 to ` 1 do tmp ri Cj qi bj C carry .carry; ri Cj / QuoRem.tmp; B/ ri C` ri C` C carry while ri C` < 0 do carry 0 for j 0 to ` 1 do tmp ri Cj C bi C carry .carry; ri Cj / QuoRem.tmp; B/ ri C` ri C` C carry qi qi 1 output the quotient q D .qk ` q0 /B and the remainder r D .r` 1 r0 /B
Fig. 3.1. Division with Remainder Algorithm 4. If the estimate qi is too large, this is manifested by a negative value of ri C` at line 10. Lines 11–17 detect and correct this condition: the loop body here executes at most twice; in lines 12–16, we compute .ri C` ri /B
.ri C` ri /B C .b`
1 b0 /B :
Just as in the algorithm in §3.3.1, in every iteration of the loop in lines 13–15, the value of carry is 0 or 1, and the value tmp lies between 0 and 2B 1. It is quite easy to see that the running time of the above algorithm is O.` .k ` C 1//. Finally, consider the general case, where b may not be normalized. We multiply 0 both a and b by an appropriate value 2w , with 0 w 0 < w, obtaining a0 WD 0 0 a2w and b 0 WD 2w , where b 0 is normalized; alternatively, we can use a more efficient, special-purpose “left shift” algorithm to achieve the same effect. We then compute q and r 0 such that a0 D b 0 q C r 0 , using the above division algorithm
60
Computing with large integers 0
for the normalized case. Observe that q D ba0 =b 0 c D ba=bc, and r 0 D r2w , 0 where r D a mod b. To recover r, we simply divide r 0 by 2w , which we can do either using the above “single precision” division algorithm, or by using a specialpurpose “right shift” algorithm. All of this normalizing and denormalizing takes time O.k C `/. Thus, the total running time for division with remainder is still O.` .k ` C 1//. E XERCISE 3.17. Work out the details of algorithms for arithmetic on signed integers, using the above algorithms for unsigned integers as subroutines. You should give algorithms for addition, subtraction, multiplication, and division with remainder of arbitrary signed integers (for division with remainder, your algorithm should compute ba=bc and a mod b). Make sure your algorithm correctly computes the sign bit of the result, and also strips leading zero digits from the result. E XERCISE 3.18. Work out the details of an algorithm that compares two signed integers a and b, determining which of a < b, a D b, or a > b holds. E XERCISE 3.19. Suppose that we run the division with remainder algorithm in Fig. 3.1 for ` > 1 without normalizing b, but instead, we compute the value qi in line 4 as follows: qi
b.ri C` B 2 C ri C`
1B
C riC`
2 /=.b` 1 B
C b`
2 /c:
Show that qi is either equal to the correct quotient digit, or the correct quotient digit plus 1. Note that a limitation of this approach is that the numbers involved in the computation are larger than B 2 . E XERCISE 3.20. Work out the details for an algorithm that shifts a given unsigned integer a to the left by a specified number of bits s (i.e., computes b WD a 2s ). The running time of your algorithm should be linear in the number of digits of the output. E XERCISE 3.21. Work out the details for an algorithm that shifts a given unsigned integer a to the right by a specified number of bits s (i.e., computes b WD ba=2s c). The running time of your algorithm should be linear in the number of digits of the output. Now modify your algorithm so that it correctly computes ba=2s c for signed integers a. E XERCISE 3.22. This exercise is for C/Java programmers. Evaluate the C/Java expressions (-17) % 4;
(-17) & 3;
and compare these values with . 17/ mod 4. Also evaluate the C/Java expressions
3.3 Basic integer arithmetic
(-17) / 4;
61
(-17) >> 2;
and compare with b 17=4c. Explain your findings. E XERCISE 3.23. This exercise is also for C/Java programmers. Suppose that values of type int are stored using a 32-bit 2’s complement representation, and that all basic arithmetic operations are computed correctly modulo 232 , even if an “overflow” happens to occur. Also assume that double precision floating point has 53 bits of precision, and that all basic arithmetic operations give a result with a relative error of at most 2 53 . Also assume that conversion from type int to double is exact, and that conversion from double to int truncates the fractional part. Now, suppose we are given int variables a, b, and n, such that 1 < n < 230 , 0 a < n, and 0 b < n. Show that after the following code sequence is executed, the value of r is equal to .a b/ mod n: int q; q = (int) ((((double) a) * ((double) b)) / ((double) n)); r = a*b - q*n; if (r >= n) r = r - n; else if (r < 0) r = r + n;
3.3.5 Summary We now summarize the results of this section. For an integer a, we define its bit length, or simply, its length, which we denote by len.a/, to be the number of bits in the binary representation of jaj; more precisely, blog2 jajc C 1 if a ¤ 0, len.a/ WD 1 if a D 0. If len.a/ D `, we say that a is an `-bit integer. Notice that if a is a positive, `-bit integer, then log2 a < ` log2 a C 1, or equivalently, 2` 1 a < 2` . Assuming that arbitrarily large integers are represented as described at the beginning of this section, with a sign bit and a vector of base-B digits, where B is a constant power of 2, we may state the following theorem. Theorem 3.3. Let a and b be arbitrary integers. (i) We can compute a ˙ b in time O.len.a/ C len.b//. (ii) We can compute a b in time O.len.a/ len.b//. (iii) If b ¤ 0, we can compute the quotient q WD ba=bc and the remainder r WD a mod b in time O.len.b/ len.q//.
62
Computing with large integers
Note the bound O.len.b/ len.q// in part (iii) of this theorem, which may be significantly less than the bound O.len.a/ len.b//. A good way to remember this bound is as follows: the time to compute the quotient and remainder is roughly the same as the time to compute the product bq appearing in the equality a D bq C r. This theorem does not explicitly refer to the base B in the underlying implementation. The choice of B affects the values of the implied big-O constants; while in theory, this is of no significance, it does have a significant impact in practice. From now on, we shall (for the most part) not worry about the implementation details of long-integer arithmetic, and will just refer directly this theorem. However, we will occasionally exploit some trivial aspects of our data structure for representing large integers. For example, it is clear that in constant time, we can determine the sign of a given integer a, the bit length of a, and any particular bit of the binary representation of a; moreover, as discussed in Exercises 3.20 and 3.21, multiplications and divisions by powers of 2 can be computed in linear time via “left shifts” and “right shifts.” It is also clear that we can convert between the base2 representation of a given integer and our implementation’s internal representation in linear time (other conversions may take longer — see Exercise 3.32). We wish to stress the point that efficient algorithms on large integers should run in time bounded by a polynomial in the bit lengths of the inputs, rather than their magnitudes. For example, if the input to an algorithm is an `-bit integer n, and if the algorithm runs in time O.`2 /, it will easily be able to process 1000-bit inputs in a reasonable amount of time (a fraction of a second) on typical, modern computer. However, if the algorithm runs in time, say, O.n1=2 /, this means that on 1000-bit inputs, it will take roughly 2500 computing steps, which even on the fastest computer available today or in the foreseeable future, will still be running long after our solar system no longer exists. A note on notation: “len” and “log.” In expressing the running times of algorithms in terms of an input a, we generally prefer to write len.a/ rather than log a. One reason is esthetic: writing len.a/ stresses the fact that the running time is a function of the bit length of a. Another reason is technical: for big-O estimates involving functions on an arbitrary domain, the appropriate inequalities should hold throughout the domain, and for this reason, it is very inconvenient to use functions, like log, which vanish or are undefined on some inputs.
E XERCISE 3.24. Let a; b 2 Z with a b > 0, and let q WD ba=bc. Show that len.a/ len.b/ 1 len.q/ len.a/ len.b/ C 1.
63
3.3 Basic integer arithmetic
E XERCISE 3.25. Let n1 ; : : : ; nk be positive integers. Show that k X i D1
len.ni /
k len
Y k
ni
i D1
k X
len.ni /:
i D1
E XERCISE 3.26. Show that the product n of integers n1 ; : : : ; nk , with each ni > 1, can be computed in time O.len.n/2 /. E XERCISE 3.27. Show that given integers n1 ; : : : ; nk , with each ni > 1, and an Q integer a, where 0 a < n and n WD i ni , we can compute the k integers a mod ni , for i D 1; : : : ; k, in time O.len.n/2 /. E XERCISE 3.28. Show that given integers n1 ; : : : ; nk , with each ni > 1, we Q can compute the k integers n=ni , for i D 1; : : : ; k, where n WD i ni , in time O.len.n/2 /. p E XERCISE 3.29. This exercise develops an algorithm to compute b nc for a given positive integer n. Consider the following algorithm: k b.len.n/ 1/=2c, m 2k for i k 1 down to 0 do if .m C 2i /2 n then m m C 2i output m p (a) Show that this algorithm correctly computes b nc. (b) In a straightforward implementation of this algorithm, each loop iteration takes time O.len.n/2 /, yielding a total running time of O.len.n/3 /. Give a more careful implementation, so that each loop iteration takes time O.len.n//, yielding a total running time is O.len.n/2 /. E XERCISE 3.30. Modify the algorithm given on the previous exercise so that that given positive integers n and e, with n 2e , it computes bn1=e c in time O.len.n/3 =e/. E XERCISE 3.31. An integer n > 1 is called a perfect power if n D ab for integers a > 1 and b > 1. Using the algorithm from the previous exercise, design an efficient algorithm that determines if a given n is a perfect power, and if so, also computes a and b such that n D ab , where a > 1, b > 1, and a is as small as possible. Your algorithm should run in time O.`3 len.`//, where ` WD len.n/. E XERCISE 3.32. Show how to convert (in both directions) between the base-10 representation and our implementation’s internal representation of an integer n in time O.len.n/2 /.
64
Computing with large integers
3.4 Computing in Zn Let n > 1. For every ˛ 2 Zn , there exists a unique integer a 2 f0; : : : ; n 1g such that ˛ D Œan ; we call this integer a the canonical representative of ˛, and denote it by rep.˛/. For computational purposes, we represent elements of Zn by their canonical representatives. Addition and subtraction in Zn can be performed in time O.len.n//: given ˛; ˇ 2 Zn , to compute rep.˛ C ˇ/, we simply compute the integer sum rep.˛/ C rep.ˇ/, subtracting n if the result is greater than or equal to n; similarly, to compute rep.˛ ˇ/, we compute the integer difference rep.˛/ rep.ˇ/, adding n if the result is negative. Multiplication in Zn can be performed in time O.len.n/2 /: given ˛; ˇ 2 Zn , we compute rep.˛ ˇ/ as rep.˛/ rep.ˇ/ mod n, using one integer multiplication and one division with remainder. A note on notation: “rep,” “mod,” and “Œn .” In describing algorithms, as well as in other contexts, if ˛; ˇ are elements of Zn , we may write, for example, ˛ C ˇ or ˛ˇ, and it is understood that elements of Zn are represented by their canonical representatives as discussed above, and arithmetic on canonical representatives is done modulo n. Thus, we have in mind a “strongly typed” language for our pseudo-code that makes a clear distinction between integers in the set f0; : : : ; n 1g and elements of Zn . If a 2 Z, we can convert a to an object ˛ 2 Zn by writing ˛ Œan , and if a 2 f0; : : : ; n 1g, this type conversion is purely conceptual, involving no actual computation. Conversely, if ˛ 2 Zn , we can convert ˛ to an object a 2 f0; : : : ; n 1g, by writing a rep.˛/; again, this type conversion is purely conceptual, and involves no actual computation. It is perhaps also worthwhile to stress the distinction between a mod n and Œan — the former denotes an element of the set f0; : : : ; n 1g, while the latter denotes an element of Zn .
Another interesting problem is exponentiation in Zn : given ˛ 2 Zn and a nonnegative integer e, compute ˛ e 2 Zn . Perhaps the most obvious way to do this is to iteratively multiply by ˛ a total of e times, requiring time O.e len.n/2 /. For small values of e, this is fine; however, a much faster algorithm, the repeated-squaring algorithm, computes ˛ e using just O.len.e// multiplications in Zn , thus taking time O.len.e/ len.n/2 /. This method is based on the following observation. Let e D .b` 1 b0 /2 be the binary expansion of e (where b0 is the low-order bit). For i D 0; : : : ; `, define ei WD be=2i c; the binary expansion of ei is ei D .b` 1 bi /2 . Also define ˇi WD ˛ ei for i D 0; : : : ; `, so ˇ` D 1 and ˇ0 D ˛ e . Then we have ei D 2ei C1 C bi and ˇi D ˇi2C1 ˛ bi for i D 0; : : : ; `
1:
This observation yields the following algorithm for computing ˛ e : The repeated-squaring algorithm. On input ˛; e, where ˛ 2 Zn and e is a non-
3.4 Computing in Zn
negative integer, do the following, where e D .b` of e:
1 b0 /2
65
is the binary expansion
ˇ Œ1n for i ` 1 down to 0 do ˇ ˇ2 if bi D 1 then ˇ ˇ˛ output ˇ It is clear that when this algorithm terminates, we have ˇ D ˛ e , and that the running-time estimate is as claimed above. Indeed, the algorithm uses ` squarings in Zn , and at most ` additional multiplications in Zn . Example 3.5. Suppose e D 37 D .100101/2 . The above algorithm performs the following operations in this case: ˇ ˇ ˇ ˇ ˇ ˇ ˇ
Œ1 ˇ2, ˇ ˇ2 ˇ2 ˇ2, ˇ ˇ2 ˇ2, ˇ
ˇ˛
ˇ˛ ˇ˛
// computed exponent (in binary) // 0 // 1 // 10 // 100 // 1001 // 10010 // 100101
The repeated-squaring algorithm has numerous applications. We mention a few here, but we will see many more later on. Computing multiplicative inverses in Zp . Suppose we are given a prime p and an element ˛ 2 Zp , and we want to compute ˛ 1 . By Euler’s theorem (Theorem 2.13), we have ˛ p 1 D 1, and multiplying this equation by ˛ 1 , we obtain ˛ p 2 D ˛ 1 . Thus, we can use the repeated-squaring algorithm to compute ˛ 1 by raising ˛ to the power p 2. This algorithm runs in time O.len.p/3 /. While this is a reasonably efficient algorithm, we will develop an even more efficient method in the next chapter, using Euclid’s algorithm (which also works with any modulus, not just a prime modulus). Testing quadratic residuosity. Suppose we are given an odd prime p and an element ˛ 2 Zp , and we want to test whether ˛ 2 .Zp /2 . By Euler’s criterion (Theorem 2.21), we have ˛ 2 .Zp /2 if and only if ˛ .p 1/=2 D 1. Thus, we can use the repeated-squaring algorithm to test if ˛ 2 .Zp /2 by raising ˛ to the power .p 1/=2. This algorithm runs in time O.len.p/3 /. While this is a reasonably
66
Computing with large integers
efficient algorithm, we will develop an even more efficient method later in the text (in Chapter 12). Testing for primality. Suppose we are given an integer n > 1, and we want to determine whether n is prime or composite. For large n, searching for prime factors of n is hopelessly impractical. A better idea is to use Euler’s theorem, combined with the repeated-squaring algorithm: we know that if n is prime, then every non-zero ˛ 2 Zn satisfies ˛ n 1 D 1. Conversely, if n is composite, there exists a non-zero ˛ 2 Zn such that ˛ n 1 ¤ 1 (see Exercise 2.27). This suggests the following “trial and error” strategy for testing if n is prime: repeat k times choose ˛ 2 Zn n fŒ0g compute ˇ ˛n 1 if ˇ ¤ 1 output “composite” and halt output “maybe prime” As stated, this is not a fully specified algorithm: we have to specify the loopiteration parameter k, and more importantly, we have to specify a procedure for choosing ˛ in each loop iteration. One approach might be to just try ˛ D Œ1; Œ2; Œ3; : : : : Another might to be to choose ˛ at random in each loop iteration: this would be an example of a probabilistic algorithm (a notion we shall discuss in detail in Chapter 9). In any case, if the algorithm outputs “composite,” we may conclude that n is composite (even though the algorithm does not find a non-trivial factor of n). However, if the algorithm completes all k loop iterations and outputs “maybe prime,” it is not clear what we should conclude: certainly, we have some reason to suspect that n is prime, but not really a proof; indeed, it may be the case that n is composite, but we were just unlucky in all of our choices for ˛. Thus, while this rough idea does not quite give us an effective primality test, it is not a bad start, and is the basis for several effective primality tests (a couple of which we shall discuss in detail in Chapters 10 and 21). E XERCISE 3.33. The repeated-squaring algorithm we have presented here processes the bits of the exponent from left to right (i.e., from high order to low order). Develop an algorithm for exponentiation in Zn with similar complexity that processes the bits of the exponent from right to left. E XERCISE 3.34. Show that given a prime p, ˛ 2 Zp , and an integer e p, we can compute ˛ e in time O.len.e/ len.p/ C len.p/3 /. The following exercises develop some important efficiency improvements to the basic repeated-squaring algorithm.
3.4 Computing in Zn
67
E XERCISE 3.35. The goal of this exercise is to develop a “2t -ary” variant of the above repeated-squaring algorithm, in which the exponent is effectively treated as a number in base 2t , for some parameter t , rather than in base 2. Let ˛ 2 Zn and let e be a positive integer of length `. Let us write e in base 2t as e D .ek e0 /2t , where ek ¤ 0. Consider the following algorithm: compute a table of values T Œ0 : : : 2t 1, where T Œj WD ˛ j for j D 0; : : : ; 2t ˇ T Œek for i k 1 down to 0 do t ˇ ˇ 2 T Œei
1
(a) Show that this algorithm correctly computes ˛ e , and work out the implementation details, showing that it may be implemented so as to use at most ` squarings and 2t C `=t C O.1/ additional multiplications in Zn . (b) Show that by appropriately choosing the parameter t, we can bound the number of multiplications in Zn (besides the squarings) by O.`= len.`//. Thus, from an asymptotic point of view, the cost of exponentiation is essentially the cost of about ` squarings in Zn . (c) Improve the algorithm so that it only uses at most ` squarings and 2t 1 C `=t C O.1/ additional multiplications in Zn . Hint: build a table that cont tains only the odd powers of ˛ among ˛ 0 ; ˛ 1 ; : : : ; ˛ 2 1 . E XERCISE 3.36. Suppose we are given ˛1 ; : : : ; ˛k 2 Zn , along with non-negative integers e1 ; : : : ; ek , where len.ei / ` for i D 1; : : : ; k. Show how to compute ˇ WD ˛1e1 ˛kek , using at most ` squarings and ` C 2k additional multiplications in Zn . Your algorithm should work in two phases: the first phase uses only the values ˛1 ; : : : ; ˛k , and performs at most 2k multiplications in Zn ; in the second phase, the algorithm computes ˇ, using the exponents e1 ; : : : ; ek , along with the data computed in the first phase, and performs at most ` squarings and ` additional multiplications in Zn . E XERCISE 3.37. Suppose that we are to compute ˛ e , where ˛ 2 Zn , for many exponents e of length at most `, but with ˛ fixed. Show that for every positive integer parameter k, we can make a pre-computation (depending on ˛, `, and k) that uses at most ` squarings and 2k additional multiplications in Zn , so that after the pre-computation, we can compute ˛ e for every exponent e of length at most ` using at most `=k C O.1/ squarings and `=k C O.1/ additional multiplications in Zn . Hint: use the algorithm in the previous exercise. E XERCISE 3.38. Suppose we are given ˛ 2 Zn , along with non-negative integers
68
Computing with large integers
e1 ; : : : ; er , where len.ei / ` for i D 1; : : : ; r, and r D O.len.`//. Using the previous exercise, show how to compute the r values ˛ e1 ; : : : ; ˛ er , using O.`/ multiplications in Zn . E XERCISE 3.39. Suppose we are given ˛ 2 Zn , along with integers m1 ; : : : ; mr , each greater than 1, whose product is m. Also, for i D 1; : : : ; r, define mi WD m=mi . Show how to compute the r values ˛ m1 ; : : : ; ˛ mr , using a total of O.len.r/`/ multiplications in Zn , where ` WD len.m/. Hint: divide and conquer. Note that if r D O.len.`//, then using the previous exercise, we can do this using just O.`/ multiplications. E XERCISE 3.40. Let k be a constant, positive integer. Suppose we are given ˛1 ; : : : ; ˛k 2 Zn , along with non-negative integers e1 ; : : : ; ek , where len.ei / ` for i D 1; : : : ; k. Show how to compute the value ˛1e1 ˛kek , using at most ` squarings and O.`= len.`// additional multiplications in Zn . Hint: develop a 2t ary version of the algorithm in Exercise 3.36. 3.5 Faster integer arithmetic ./ The quadratic-time algorithms presented in §3.3 for integer multiplication and division are by no means the fastest possible. The next exercise develops a faster multiplication algorithm. E XERCISE 3.41. Suppose we have two positive integers a and b, each of length at most `, such that a D a1 2k C a0 and b D b1 2k C b0 , where 0 a0 < 2k and 0 b0 < 2k . Then ab D a1 b1 22k C .a0 b1 C a1 b0 /2k C a0 b0 : Show how to compute the product ab in time O.`/, given the products a0 b0 , a1 b1 , and .a0 a1 /.b0 b1 /. From this, design a recursive algorithm that computes ab in time O.`log2 3 /. (Note that log2 3 1:58:) The algorithm in the previous is also not the best possible. In fact, it is possible to multiply two integers of length at most ` on a RAM in time O.`/, but we do not explore this any further here (see §3.6). The following exercises explore the relationship between integer multiplication and related problems. We assume that we have an algorithm that multiplies two integers of length at most ` in time at most M.`/. It is convenient (and reasonable) to assume that M is a well-behaved complexity function. By this, we mean that M maps positive integers to positive real numbers, such that for some constant
3.5 Faster integer arithmetic ./
69
1, and all positive integers a and b, we have 1
M.a C b/ : M.a/ C M.b/
E XERCISE 3.42. Show that if M is a well-behaved complexity function, then it is strictly increasing. E XERCISE 3.43. Show that if N.`/ WD M.`/=` is a non-decreasing function, and M.2`/=M.`/ D O.1/, then M is a well-behaved complexity function. E XERCISE 3.44. Let ˛ > 0, ˇ 1, 0, ı 0 be real constants. Show that M.`/ WD ˛`ˇ len.`/ len.len.`//ı is a well-behaved complexity function. E XERCISE 3.45. Show that given integers n > 1 and e > 1, we can compute ne in time O.M.len.ne ///. E XERCISE 3.46. Give an algorithm for Exercise 3.26 that runs in time O.M.len.n// len.k//: Hint: divide and conquer. E XERCISE 3.47. In the previous exercise, suppose all the inputs ni have the same length, and that M.`/ D ˛`ˇ , where ˛ and ˇ are constants with ˛ > 0 and ˇ > 1. Show that your algorithm runs in time O.M.len.n///. E XERCISE 3.48. We can represent a “floating point” number zO as a pair .a; e/, where a and e are integers — the value of zO is the rational number a2e , and we call len.a/ the precision of z. O We say that zO is a k-bit approximation of a real number z if zO has precision k and zO D .1 C /z for some jj 2 kC1 . Show that given positive integers b and k, we can compute a k-bit approximation of 1=b in time O.M.k//. Hint: using Newton iteration, show how to go from a t -bit approximation of 1=b to a .2t 2/-bit approximation of 1=b, making use of just the high-order O.t/ bits of b, in time O.M.t //. Newton iteration is a general method of iteratively approximating a root of an equation f .x/ D 0 by starting with an initial approximation x0 , and computing subsequent approximations by the formula xi C1 D xi f .xi /=f 0 .xi /, where f 0 .x/ is the derivative of f .x/. For this exercise, apply Newton iteration to the function f .x/ D x 1 b. E XERCISE 3.49. Using the result of the previous exercise, show that given positive integers a and b of bit length at most `, we can compute ba=bc and a mod b in time O.M.`//. From this, we see that up to a constant factor, division with remainder is no harder than multiplication.
70
Computing with large integers
E XERCISE 3.50. Using the result of the previous exercise, give an algorithm for Exercise 3.27 that runs in time O.M.len.n// len.k//. Hint: divide and conquer. E XERCISE 3.51. Give an algorithm for Exercise 3.29 that runs in time O.M.len.n///. Hint: Newton iteration. E XERCISE 3.52. Suppose we have an algorithm that computes the square of an `-bit integer in time S.`/, where S is a well-behaved complexity function. Show how to use this algorithm to compute the product of two arbitrary integers of length at most ` in time O.S.`//. E XERCISE 3.53. Give algorithms for Exercise 3.32 that run in time O.M.`/ len.`//, where ` WD len.n/. Hint: divide and conquer. 3.6 Notes Shamir [87] shows how to factor an integer in polynomial time on a RAM, but where the numbers stored in the memory cells may have exponentially many bits. As there is no known polynomial-time factoring algorithm on any realistic machine, Shamir’s algorithm demonstrates the importance of restricting the sizes of numbers stored in the memory cells of our RAMs to keep our formal model realistic. The most practical implementations of algorithms for arithmetic on large integers are written in low-level “assembly language,” specific to a particular machine’s architecture (e.g., the GNU Multi-Precision library GMP, available at www.swox.com/gmp). Besides the general fact that such hand-crafted code is more efficient than that produced by a compiler, there is another, more important reason for using such code. A typical 32-bit machine often comes with instructions that allow one to compute the 64-bit product of two 32-bit integers, and similarly, instructions to divide a 64-bit integer by a 32-bit integer (obtaining both the quotient and remainder). However, high-level programming languages do not (as a rule) provide any access to these low-level instructions. Indeed, we suggested in §3.3 using a value for the base B of about half the word-size of the machine, so as to avoid overflow. However, if one codes in assembly language, one can take B to be much closer to, or even equal to, the word-size of the machine. Since our basic algorithms for multiplication and division run in time quadratic in the number of base-B digits, the effect of doubling the bit-length of B is to decrease the running time of these algorithms by a factor of four. This effect, combined with the improvements one might typically expect from using assembly-language code, can easily lead to a five- to ten-fold decrease in the running time, compared to an imple-
3.6 Notes
71
mentation in a high-level language. This is, of course, a significant improvement for those interested in serious “number crunching.” The “classical,” quadratic-time algorithms presented here for integer multiplication and division are by no means the best possible: there are algorithms that are asymptotically faster. We saw this in the algorithm in Exercise 3.41, which was originally invented by Karatsuba [53] (although Karatsuba is one of two authors on this paper, the paper gives exclusive credit for this particular result to Karatsuba). That algorithm allows us to multiply two integers of length at most ` in time O.`log2 3 /. The fastest known algorithm for multiplying such integers on a RAM runs in time O.`/. This algorithm is due to Schönhage, and actually works on a very restricted type of RAM called a “pointer machine” (see Problem 12, Section 4.3.3 of Knuth [55]). See Exercise 17.26 later in this text for a much simpler (but heuristic) O.`/ multiplication algorithm. Another model of computation is that of Boolean circuits. In this model of computation, one considers families of Boolean circuits (with, say, the usual “and,” “or,” and “not” gates) that compute a particular function — for every input length, there is a different circuit in the family that computes the function on inputs that are bit strings of that length. One natural notion of complexity for such circuit families is the size of the circuit (i.e., the number of gates and wires in the circuit), which is measured as a function of the input length. For many years, the smallest known Boolean circuit that multiplies two integers of length at most ` was of size O.` len.`/ len.len.`///. This result was due to Schönhage and Strassen [84]. More recently, Fürer showed how to reduce this to O.` len.`/2O.log `/ / [37]. Here, the value of log n is defined as the minimum number of applications of the function log2 to the number n to obtain a number that less than or equal to 1. The function log is an extremely slow growing function, and is a constant for all practical purposes. It is hard to say which model of computation, the RAM or circuits, is “better.” On the one hand, the RAM very naturally models computers as we know them today: one stores small numbers, like array indices, counters, and pointers, in individual words of the machine, and processing such a number typically takes a single “machine cycle.” On the other hand, the RAM model, as we formally defined it, invites a certain kind of “cheating,” as it allows one to stuff O.len.`//-bit integers into memory cells. For example, even with the simple, quadratic-time algorithms for integer arithmetic discussed in §3.3, we can choose the base B to have len.`/ bits, in which case these algorithms would run in time O..`= len.`//2 /. However, just to keep things simple, we have chosen to view B as a constant (from a formal, asymptotic point of view). In the remainder of this text, unless otherwise specified, we shall always use
72
Computing with large integers
the classical O.`2 / bounds for integer multiplication and division, which have the advantage of being both simple and reasonably reliable predictors of actual performance for small to moderately sized inputs. For relatively large numbers, experience shows that the classical algorithms are definitely not the best — Karatsuba’s multiplication algorithm, and related algorithms for division, start to perform significantly better than the classical algorithms on inputs of a thousand bits or so (the exact crossover depends on myriad implementation details). The even “faster” algorithms discussed above are typically not interesting unless the numbers involved are truly huge, of bit length around 105 –106 . Thus, the reader should bear in mind that for serious computations involving very large numbers, the faster algorithms are very important, even though this text does not discuss them at great length. For a good survey of asymptotically fast algorithms for integer arithmetic, see Chapter 9 of Crandall and Pomerance [30], as well as Chapter 4 of Knuth [55].
4 Euclid’s algorithm
In this chapter, we discuss Euclid’s algorithm for computing greatest common divisors. It turns out that Euclid’s algorithm has a number of very nice properties, and has applications far beyond that of just computing greatest common divisors. 4.1 The basic Euclidean algorithm We consider the following problem: given two non-negative integers a and b, compute their greatest common divisor, gcd.a; b/. We can do this using the well-known Euclidean algorithm, also called Euclid’s algorithm. The basic idea of Euclid’s algorithm is the following. Without loss of generality, we may assume that a b 0. If b D 0, then there is nothing to do, since in this case, gcd.a; 0/ D a. Otherwise, if b > 0, we can compute the integer quotient q WD ba=bc and remainder r WD a mod b, where 0 r < b. From the equation a D bq C r; it is easy to see that if an integer d divides both b and r, then it also divides a; likewise, if an integer d divides a and b, then it also divides r. From this observation, it follows that gcd.a; b/ D gcd.b; r/, and so by performing a division, we reduce the problem of computing gcd.a; b/ to the “smaller” problem of computing gcd.b; r/. The following theorem develops this idea further: Theorem 4.1. Let a; b be integers, with a b 0. Using the division with remainder property, define the integers r0 ; r1 ; : : : ; r`C1 , and q1 ; : : : ; q` , where ` 0, as follows:
73
74
Euclid’s algorithm
a D r0 ; b D r1 ; r0 D r1 q1 C r2 :: :
.0 < r2 < r1 /;
D ri qi C ri C1 :: :
.0 < ri C1 < ri /;
2
D r`
.0 < r` < r`
1
D r` q`
ri
1
r` r`
1 q` 1
C r`
1 /;
.r`C1 D 0/:
Note that by definition, ` D 0 if b D 0, and ` > 0, otherwise. Then we have rp ` D gcd.a; b/. Moreover, if b > 0, then ` log b= log C 1, where WD .1 C 5/=2 1:62. Proof. For the first statement, one sees that for i D 1; : : : ; `, we have ri 1 D ri qi C ri C1 , from which it follows that the common divisors of ri 1 and ri are the same as the common divisors of ri and ri C1 , and hence gcd.ri 1 ; ri / D gcd.ri ; ri C1 /. From this, it follows that gcd.a; b/ D gcd.r0 ; r1 / D gcd.r` ; r`C1 / D gcd.r` ; 0/ D r` : To prove the second statement, assume that b > 0, and hence ` > 0. If ` D 1, the statement is obviously true, so assume ` > 1. We claim that for i D 0; : : : ; ` 1, we have r` i i . The statement will then follow by setting i D ` 1 and taking logarithms. We now prove the above claim. For i D 0 and i D 1, we have r` 1 D 0 and r` For i D 2; : : : ; ` have r`
i
r`
1
r` C 1 2 1 :
1, using induction and applying the fact that 2 D C 1, we .i 1/
C r`
.i 2/
i
1
C i
2
D i
2
.1 C / D i ;
which proves the claim. Example 4.1. Suppose a D 100 and b D 35. Then the numbers appearing in Theorem 4.1 are easily computed as follows: i ri qi
0 100
1 35 2
2 30 1
3 5 6
4 0
4.1 The basic Euclidean algorithm
75
So we have gcd.a; b/ D r3 D 5. We can easily turn the scheme described in Theorem 4.1 into a simple algorithm: Euclid’s algorithm. On input a; b, where a and b are integers such that a b 0, compute d D gcd.a; b/ as follows: r a; r 0 b while r 0 ¤ 0 do r 00 r mod r 0 .r; r 0 / .r 0 ; r 00 / d r output d We now consider the running time of Euclid’s algorithm. Naively, one could estimate this as follows. Suppose a and b are k-bit numbers. The number of divisions performed by the algorithm is the number ` in Theorem 4.1, which is O.k/. Moreover, each division involves numbers of k bits or fewer in length, and so takes time O.k 2 /. This leads to a bound on the running time of O.k 3 /. However, as the following theorem shows, this cubic running time bound is well off the mark. Intuitively, this is because the cost of performing a division depends on the length of the quotient: the larger the quotient, the more expensive the division, but also, the more progress the algorithm makes towards termination. Theorem 4.2. Euclid’s algorithm runs in time O.len.a/ len.b//. Proof. We may assume that b > 0. With notation as in Theorem 4.1, the running time is O.T /, where T D
` X
len.ri / len.qi / len.b/
i D1
` X
len.qi /
i D1
` X len.b/ .len.ri
1/
len.ri / C 1/ (see Exercise 3.24)
i D1
D len.b/.len.r0 /
len.r` / C `/ (telescoping the sum)
len.b/.len.a/ C log b= log C 1/ (by Theorem 4.1) D O.len.a/ len.b//: E XERCISE 4.1. With notation as in Theorem 4.1, show that for each i D 1; : : : ; `, we have ri C1 ri 1 =2. Thus, with every two division steps, the bit length of the remainder drops by at least 1. This leads an alternative way to bound the number of divisions by O.len.b//, but with a larger constant.
76
Euclid’s algorithm
E XERCISE 4.2. Show how to compute lcm.a; b/ in time O.len.a/ len.b//. E XERCISE 4.3. Let a; b 2 Z with a b > 0, and let d WD gcd.a; b/, a0 WD a=d , and b 0 WD b=d . Show that if we run Euclid’s algorithm on input a; b, then (a) it performs at most log b 0 = log C 1 divisions; (b) its running time is O.len.a0 / len.b//. E XERCISE 4.4. Let ` be a positive integer. Show that there exist integers a; b with a b 0 and ` log b= log , such that Euclid’s algorithm on input a; b performs exactly ` divisions. Thus, the bound in Theorem 4.1 on the number of divisions is essentially tight. E XERCISE 4.5. This exercise looks at an alternative algorithm for computing gcd.a; b/, called the binary gcd algorithm. This algorithm avoids complex operations, such as division and multiplication; instead, it relies only on subtraction, and division and multiplication by powers of 2, which assuming a binary representation of integers (as we are) can be very efficiently implemented using “right shift” and “left shift” operations. The algorithm takes positive integers a and b as input, and runs as follows: r a; r 0 b; e 0 while 2 j r and 2 j r 0 do r r=2; r 0 repeat while 2 j r do r r=2 0 0 while 2 j r do r r 0 =2 0 0 if r < r then .r; r / .r 0 ; r/ r0 r0 r 0 until r D 0 d 2e r output d
r 0 =2; e
eC1
Show that this algorithm correctly computes gcd.a; b/, and runs in time O.k 2 /, where k WD max.len.a/; len.b//. 4.2 The extended Euclidean algorithm Let a and b be integers, and let d WD gcd.a; b/. We know by Theorem 1.8 that there exist integers s and t such that as C bt D d . The extended Euclidean algorithm allows us to efficiently compute s and t. The following theorem defines the quantities computed by this algorithm, and states a number of important facts about them—these will play a crucial role, both in the analysis of the running time
77
4.2 The extended Euclidean algorithm
of the algorithm, as well as in applications of the algorithm that we will discuss later. Theorem 4.3. Let a, b, r0 ; : : : ; r`C1 and q1 ; : : : ; q` be as in Theorem 4.1. Define integers s0 ; : : : ; s`C1 and t0 ; : : : ; t`C1 as follows: s0 WD 1;
t0 WD 0;
s1 WD 0;
t1 WD 1;
si C1 WD si
ti C1 WD ti
si qi ;
1
1
.i D 1; : : : ; `/:
ti qi
Then (i) for i D 0; : : : ; ` C 1, we have asi C bti D ri ; in particular, as` C bt` D gcd.a; b/; ti si C1 D . 1/i ;
(ii) for i D 0; : : : ; `, we have si ti C1
(iii) for i D 0; : : : ; ` C 1, we have gcd.si ; ti / D 1; (iv) for i D 0; : : : ; `, we have ti ti C1 0 and jti j jti C1 j; for i D 1; : : : ; `, we have si si C1 0 and jsi j jsi C1 j; (v) for i D 1; : : : ; ` C 1, we have ri
1 jti j
a and ri
1 jsi j
b;
(vi) if a > 0, then for i D 1; : : : ; ` C 1, we have jti j a and jsi j b; if a > 1 and b > 0, then jt` j a=2 and js` j b=2. Proof. (i) is easily proved by induction on i . For i D 0; 1, the statement is clear. For i D 2; : : : ; ` C 1, we have asi C bti D a.si
2
si
D .asi
2
C bti
D ri
ri
2
1 qi 1 / 2/
1 qi 1
C b.ti
.asi
1
2
ti
C bti
1 qi 1 /
1 /qi 1
(by induction)
D ri : (ii) is also easily proved by induction on i . For i D 0, the statement is clear. For i D 1; : : : ; `, we have si ti C1
ti si C1 D si .ti
1
D
.si
D
. 1/i
ti qi /
1 ti
ti 1
ti .si 1 si /
1
si qi /
(after expanding and simplifying)
(by induction)
i
D . 1/ : (iii) follows directly from (ii). For (iv), one can easily prove both statements by induction on i . The statement involving the ti ’s is clearly true for i D 0; for i D 1; : : : ; `, we have ti C1 D
78
Euclid’s algorithm
ti 1 ti qi , and since by the induction hypothesis ti 1 and ti have opposite signs and jti j jti 1 j, it follows that jti C1 j D jti 1 j C jti jqi jti j, and that the sign of ti C1 is the opposite of that of ti . The proof of the statement involving the si ’s is the same, except that we start the induction at i D 1. For (v), one considers the two equations: asi
1
C bti
1
D ri
1;
asi C bti D ri : Subtracting ti 1 times the second equation from ti times the first, and applying (ii), we get ˙a D ti ri 1 ti 1 ri ; consequently, using the fact that ti and ti 1 have opposite sign, we obtain a D jti ri
ti
1
1 ri j
D jti jri
1
C jti
1 jri
jti jri
1:
The inequality involving si follows similarly, subtracting si 1 times the second equation from si times the first. (vi) follows from (v) and the following observations: if a > 0, then ri 1 > 0 for i D 1; : : : ; ` C 1; if a > 1 and b > 0, then ` > 0 and r` 1 2. Example 4.2. We continue with Example 4.1. The si ’s and ti ’s are easily computed from the qi ’s: i ri qi si ti So we have gcd.a; b/ D 5 D
0 100 1 0
1 35 2 0 1
2 30 1 1 -2
3 5 6 -1 3
4 0 7 -20
a C 3b.
We can easily turn the scheme described in Theorem 4.3 into a simple algorithm: The extended Euclidean algorithm. On input a; b, where a and b are integers such that a b 0, compute integers d , s, and t, such that d D gcd.a; b/ and as C bt D d , as follows: r a; r 0 b 0 s 1; s 0 t 0; t 0 1 while r 0 ¤ 0 do q br=r 0 c; r 00 .r; s; t; r 0 ; s 0 ; t 0 / d r output d; s; t
r mod r 0 .r 0 ; s 0 ; t 0 ; r 00 ; s
s 0 q; t
t 0 q/
79
4.2 The extended Euclidean algorithm
Theorem 4.4. The extended Euclidean algorithm runs in time O.len.a/ len.b//: Proof. We may assume that b > 0. It suffices to analyze the cost of computing the sequences fsi g and fti g. Consider first the cost of computing all of the ti ’s, P which is O.T /, where T D `iD1 len.ti / len.qi /: We have t1 D 1 and, by part (vi) of Theorem 4.3, we have jti j a for i D 2; : : : ; `. Arguing as in the proof of Theorem 4.2, we have T len.q1 / C len.a/
` X
len.qi /
i D2
len.a/ C len.a/.len.r1 /
len.r` / C `
1/ D O.len.a/ len.b//:
An analogous argument shows that one can also compute all of the si ’s in time O.len.a/ len.b//, and in fact, in time O.len.b/2 /. For the reader familiar with the basics of the theory of matrices and determinants, it is instructive to view Theorem 4.3 as follows. For i D 1; : : : ; `, we have ri 0 1 ri 1 D : ri C1 1 qi ri Recursively expanding the right-hand side of this equation, we have
À Mi WD
ri ri C1
D
0 1
1 0 qi 1
1 q1
a : b
This defines the 2 2 matrix Mi for i D 1; : : : ; `. If we also define M0 to be the 2 2 identity matrix, then it is easy to see that for i D 0; : : : ; `, we have si ti Mi D : si C1 ti C1 From these observations, part (i) of Theorem 4.3 is immediate, and part (ii) follows from the fact that Mi is the product of i matrices, each of determinant 1, and the determinant of Mi is evidently si tiC1 ti si C1 . E XERCISE 4.6. In our description of the extended Euclidean algorithm, we made the restriction that the inputs a and b satisfy a b 0. Using this restricted algorithm as a subroutine, give an algorithm that works without any restrictions on its input.
80
Euclid’s algorithm
E XERCISE 4.7. Assume notation as in Theorem 4.3. Show that for all i D 2; : : : ; `, we have jti j < jti C1 j and ri 1 jti j < a, and that for all i D 3; : : : ; `, we have jsi j < jsi C1 j and ri 1 jsi j < b. Also show that si ti 0 for i D 0; : : : ; ` C 1. E XERCISE 4.8. Suppose we modify the extended Euclidean algorithm so that it computes balanced remainders; that is, for i D 1; : : : ; `, the values qi and ri C1 are computed so that ri 1 D ri qi C ri C1 and jri j=2 ri C1 < jri j=2. Assume that the si ’s and the ti ’s are computed by the same formula as in Theorem 4.3. Give a detailed analysis of the running time of this algorithm, which should include an analysis of the number of division steps, and the sizes of the si ’s and ti ’s. E XERCISE 4.9. One can extend the binary gcd algorithm discussed in Exercise 4.5 so that in addition to computing d D gcd.a; b/, it also computes s and t such that as Cbt D d . Here is one way to do this (again, we assume that a and b are positive integers): r a; r 0 b; e 0 0 while 2 j r and 2 j r do r aQ r; bQ r 0; s 1; t repeat while 2 j r do r r=2 if 2 j s and 2 j t
r=2; r 0 r 0 =2; e 0; s 0 0; t 0 1
then s else s
eC1
s=2; t t =2 Q .s C b/=2; t
.t
a/=2 Q
r0
while 2 j do r0 r 0 =2 if 2 j s 0 and 2 j t 0 then s 0 s 0 =2; t 0 t 0 =2 0 0 Q else s .s C b/=2; t0 .t 0 if r 0 < r then .r; s; t; r 0 ; s 0 ; t 0 / .r 0 ; s 0 ; t 0 ; r; s; t / 0 0 0 0 0 0 r r r; s s s; t t t 0 until r D 0 d 2e r, output d; s; t
a/=2 Q
Show that this algorithm is correct and runs in time O.k 2 /, where k WD max.len.a/; len.b//. In particular, you should verify that all of the divisions by 2 performed by the algorithm yield integer results. Moreover, show that the outputs s and t are of length O.k/. 4.3 Computing modular inverses and Chinese remaindering One application of the extended Euclidean algorithm is to the problem of computing multiplicative inverses in Zn .
4.3 Computing modular inverses and Chinese remaindering
81
Assume n > 1. Given b 2 f0; : : : ; n 1g, in time O.len.n/2 /, we can determine if b is relatively prime to n, and if so, compute b 1 mod n, as follows. We run the extended Euclidean algorithm on input n; b, obtaining integers d , s, and t, such that d D gcd.n; b/ and ns C bt D d . If d ¤ 1, then b does not have a multiplicative inverse modulo n. Otherwise, if d D 1, then t is a multiplicative inverse of b modulo n; however, it may not lie in the range f0; : : : ; n 1g, as required. By part (vi) of Theorem 4.3, we have jt j n=2 < n; therefore, either t 2 f0; : : : ; n 1g, or t < 0 and t C n 2 f0; : : : ; n 1g. Thus, b 1 mod n is equal to either t or t C n. We also observe that the Chinese remainder theorem (Theorem 2.6) can be made computationally effective: Theorem 4.5 (Effective Chinese remainder theorem). Suppose we are given integers n1 ; : : : ; nk and a1 ; : : : ; ak , where the family fni gkiD1 is pairwise relatively Q prime, and where ni > 1 and 0 ai < ni for i D 1; : : : ; k. Let n WD kiD1 ni . Then in time O.len.n/2 /, we can compute the unique integer a satisfying 0 a < n and a ai .mod ni / for i D 1; : : : ; k. The algorithm is a straightforward implementation of the proof of Theorem 2.6, and runs as follows: Qk n i D1 ni for i 1 to k do ni n=ni , bi ni mod ni , ti bi 1 mod ni , ei ni ti Pk a i D1 ai ei mod n We leave it to the reader to verify the running time bound. E XERCISE 4.10. In this exercise, you are to make the result of Theorem 2.17 effective. Suppose that we are given a positive integer n, two elements ˛; ˇ 2 Zn , and integers ` and m, such that ˛ ` D ˇ m and gcd.`; m/ D 1. Show how to compute
2 Zn such that ˛ D m in time O.len.`/ len.m/ C .len.`/ C len.m// len.n/2 /. E XERCISE 4.11. In this exercise and the next, you are to analyze an “incremental Chinese remaindering algorithm.” Consider the following algorithm, which takes as input integers a1 ; n1 ; a2 ; n2 , such that gcd.n1 ; n2 / D 1; 0 a1 < n1 ; and 0 a2 < n2 : It outputs integers a; n, such that n D n1 n2 ; 0 a < n; a a1 .mod n1 /; and a a2 .mod n2 /; and runs as follows:
82
Euclid’s algorithm
.a2 w n1 1 mod n2 , h a a1 C n1 h, n n1 n2 output a; n
a1 /w mod n2
Show that the algorithm correctly computes a and n as specified, and runs in time O.len.n/ len.n2 //. E XERCISE 4.12. Using the algorithm in the previous exercise as a subroutine, give a simple O.len.n/2 / algorithm that takes as input integers n1 ; : : : ; nk and a1 ; : : : ; ak , where the family fni gkiD1 is pairwise relatively prime, and where ni > 1 and 0 ai < ni for i D 1; : : : ; k, and outputs integers a and n such that Q 0 a < n, n D kiD1 ni , and a ai .mod ni / for i D 1; : : : ; k. The algorithm should be “incremental,” in that it processes the pairs .ni ; ai / one at a time, using time O.len.n/ len.ni // to process each such pair. E XERCISE 4.13. Suppose we are given ˛1 ; : : : ; ˛k 2 Zn . Show how to compute ˛1 1 ; : : : ; ˛k 1 by computing one multiplicative inverse modulo n, and performing fewer than 3k multiplications modulo n. This result is useful, as in practice, if n is several hundred bits long, it may take 10–20 times longer to compute multiplicative inverses modulo n than to multiply modulo n. 4.4 Speeding up algorithms via modular computation An important practical application of the above “computational” version (Theorem 4.5) of the Chinese remainder theorem is a general algorithmic technique that can significantly speed up certain types of computations involving long integers. Instead of trying to describe the technique in some general form, we simply illustrate the technique by means of a specific example: integer matrix multiplication. Suppose we have two m m matrices A and B whose entries are large integers, and we want to compute the product matrix C WD AB. Suppose that for r; s D 1; : : : ; m, the entry of A at row r and column s is ars , and that for s; t D 1; : : : ; m, the entry of B at row s and column t is bst . Then for r; t D 1; : : : ; m, the entry of C at row r and column t is crt , which is given by the usual rule for matrix multiplication: m X crt D ars bst : (4.1) sD1
Suppose further that M is the maximum absolute value of the entries in A and B, so that the entries in C are bounded in absolute value by M 0 WD M 2 m. Let ` WD len.M /. To simplify calculations, let us also assume that m M (this is
4.4 Speeding up algorithms via modular computation
83
reasonable, as we want to consider large values of M , greater than say 2100 , and certainly, we cannot expect to work with 2100 2100 matrices). By just applying the formula (4.1), we can compute the entries of C using m3 multiplications of numbers of length at most `, and m3 additions of numbers of length at most len.M 0 /, where len.M 0 / 2` C len.m/ D O.`/. This yields a running time of O.m3 `2 /:
(4.2)
Using the Chinese remainder theorem, we can actually do much better than this, as follows. For every integer n > 1, and for all r; t D 1; : : : ; m, we have crt
m X
ars bst .mod n/:
(4.3)
sD1 0 such that Moreover, if we compute integers crt 0 crt
m X
ars bst .mod n/
(4.4)
sD1
and if we also have 0 n=2 crt < n=2 and n > 2M 0 ;
(4.5)
then we must have 0 crt D crt :
(4.6)
To see why (4.6) follows from (4.4) and (4.5), observe that (4.3) and (4.4) imply 0 .mod n/, which means that n divides .c 0 /. Then from the that crt crt crt rt bound jcrt j M 0 and from (4.5), we obtain jcrt
0 0 crt j jcrt j C jcrt j M 0 C n=2 < n=2 C n=2 D n:
0 / is a multiple of n, while at the same time So we see that the quantity .crt crt this quantity is strictly less than n in absolute value; hence, this quantity must be zero. That proves (4.6). So from the above discussion, to compute C , it suffices to compute the entries of C modulo n, where we have to make sure that we compute “balanced” remainders in the interval Œ n=2; n=2/, rather than the more usual “least non-negative” remainders. To compute C modulo n, we choose a number of small integers n1 ; : : : ; nk , such Q that the family fni gkiD1 is pairwise relatively prime, and the product n WD kiD1 ni is just a bit larger than 2M 0 . In practice, one would choose the ni ’s to be small primes, and a table of such primes could easily be computed in advance, so that
84
Euclid’s algorithm
all problems up to a given size could be handled. For example, the product of all primes of at most 16 bits is a number that has more than 90; 000 bits. Thus, by simply pre-computing and storing such a table of small primes, we can handle input matrices with quite large entries (up to about 45; 000 bits). Let us assume that we have pre-computed appropriate small primes n1 ; : : : ; nk . Further, we shall assume that addition and multiplication modulo each ni can be done in constant time. This is reasonable from a practical (and theoretical) point of view, since such primes easily “fit” into a machine word, and we can perform modular addition and multiplication using a constant number of built-in machine operations. Finally, we assume that we do not use more ni ’s than are necessary, so that len.n/ D O.`/ and k D O.`/. To compute C , we execute the following steps: 1. For each i D 1; : : : ; k, do the following: .i /
(a) compute aO rs .i / (b) compute bO st
ars mod ni for r; s D 1; : : : ; m, bst mod ni for s; t D 1; : : : ; m,
(c) for r; t D 1; : : : ; m, compute .i / cOrt
m X
.i /
.i / O aO rs bst mod ni :
sD1
2. For each r; t D 1; : : : ; m, apply the Chinese remainder theorem to .1/ .2/ .k/ cOrt ; cOrt ; : : : ; cOrt , obtaining an integer crt , which should be computed as a balanced remainder modulo n, so that n=2 crt < n=2. 3. Output the matrix C , whose entry in row r and column t is crt . Note that in step 2, if our Chinese remainder algorithm happens to be implemented to return an integer a with 0 a < n, we can easily get a balanced remainder by just subtracting n from a if a n=2. The correctness of the above algorithm has already been established. Let us now analyze its running time. The running time of steps 1a and 1b is easily seen to be O.m2 `2 /. Under our assumption about the cost of arithmetic modulo small primes, the cost of step 1c is O.m3 k/, and since k D O.`/, the cost of this step is O.m3 `/. Finally, by Theorem 4.5, the cost of step 2 is O.m2 `2 /. Thus, the total running time of this algorithm is O.m2 `2 C m3 `/: This is a significant improvement over (4.2); for example, if ` m, then the running time of the original algorithm is O.m5 /, while the running time of the modular algorithm is O.m4 /.
4.5 An effective version of Fermat’s two squares theorem
85
E XERCISE 4.14. Apply the ideas above to the problem of computing the product of two polynomials whose coefficients are large integers. First, determine the running time of the “obvious” algorithm for multiplying two such polynomials, then design and analyze a “modular” algorithm. 4.5 An effective version of Fermat’s two squares theorem We proved in Theorem 2.34 (in §2.8.4) that every prime p 1 .mod 4/ can be expressed as a sum of two squares of integers. In this section, we make this theorem computationally effective; that is, we develop an efficient algorithm that takes as input a prime p 1 .mod 4/, and outputs integers r and t such that p D r 2 C t 2 . One essential ingredient in the proof of Theorem 2.34 was Thue’s lemma (Theorem 2.33). Thue’s lemma asserts the existence of certain numbers, and we proved this using the “pigeonhole principle,” which unfortunately does not translate directly into an efficient algorithm to actually find these numbers. However, we can show that these numbers arise as a “natural by-product” of the extended Euclidean algorithm. To make this more precise, let us introduce some notation. For integers a; b, with a b 0, let us define ˚ `C1 EEA.a; b/ WD .ri ; si ; ti / i D0 ; where ri , si , and ti , for i D 0; : : : ; ` C 1, are defined as in Theorem 4.3. Theorem 4.6 (Effective Thue’s lemma). Let n; b; r ; t 2 Z, with 0 b < n and 0 < r n < r t . Further, let EEA.n; b/ D f.ri ; si ; ti /g`C1 i D0 , and let j be the smallest index (among 0; : : : ; ` C 1) such that rj < r . Then setting r WD rj and t WD tj , we have r bt .mod n/; 0 r < r ; and 0 < jtj < t : Proof. Since r0 D n r > 0 D r`C1 , the value of the index j is well defined, and moreover, j 1 and rj 1 r . It follows that jtj j n=rj
1
(by part (v) of Theorem 4.3)
n=r < t (since n < r t ): Since j 1, by part (iv) of Theorem 4.3, we have jtj j jt1 j > 0. Finally, since rj D nsj C btj , we have rj btj .mod n/. What this theorem says is that given n; b; r ; t , to find the desired values r and t , we run the extended Euclidean algorithm on input n; b. This generates a sequence of remainders r0 > r1 > r2 > , where r0 D n and r1 D b. If
86
Euclid’s algorithm
rj is the first remainder in this sequence that falls below r , and if sj and tj are the corresponding numbers computed by the extended Euclidean algorithm, then r WD rj and t WD tj do the job. The other essential ingredient in the proof of Theorem 2.34 was Theorem 2.31, which guarantees the existence of a square root of 1 modulo p when p is a prime congruent to 1 modulo 4. We need an effective version of this result as well. Later, in Chapter 12, we will study the general problem of computing square roots modulo primes. Right now, we develop an algorithm for this special case. Assume we are given a prime p 1 .mod 4/, and we want to compute ˇ 2 Zp such that ˇ 2 D 1. By Theorem 2.32, it suffices to find 2 Zp n.Zp /2 , since then ˇ WD .p 1/=4 (which we can efficiently compute via repeated squaring) satisfies ˇ 2 D 1. While there is no known efficient, deterministic algorithm to find such a , we do know that half the elements of Zp are squares and half are not (see Theorem 2.20), which suggests the following simple “trial and error” strategy to compute ˇ: repeat choose 2 Zp compute ˇ
.p 2 until ˇ D 1 output ˇ
1/=4
As an algorithm, this is not fully specified, as we have to specify a procedure for selecting in each loop iteration. A reasonable approach is to simply choose
at random: this would be an example of a probabilistic algorithm, a notion that we will study in detail in Chapter 9. Let us assume for the moment that this makes sense from a mathematical and algorithmic point of view, so that with each loop iteration, we have a 50% chance of picking a “good” , that is, one in that is not in .Zp /2 . From this, it follows that with high probability, we should find a “good”
in just a few loop iterations (the probability that after k loop iterations we still have not found one is 1=2k ), and that the expected number of loop iterations is just 2. The running time of each loop iteration is dominated by the cost of repeated squaring, which is O.len.p/3 /. It follows that the expected running time of this algorithm (we will make this notion precise in Chapter 9) is O.len.p/3 /. Let us now put all the ingredients together to get an algorithm to find r; t such that p D r 2 C t 2 . 1. Find ˇ 2 Zp such that ˇ 2 D 2. Set b
1, using the above “trial and error” strategy.
rep.ˇ/ (so that ˇ D Œb and b 2 f0; : : : ; p
1g).
3. Run the extended Euclidean algorithm on input p; b to obtain EEA.p; b/,
4.5 An effective version of Fermat’s two squares theorem
87
p and then apply Theorem 4.6 with n WD p, b, and r WD t WD b pc C 1, to obtain the values r and t. 4. Output r; t. When this algorithm terminates, we have r 2 Ct 2 D p, as required: as we argued in the proof of Theorem 2.34, since r bt .mod p/ and b 2 1 .mod p/, it follows that r 2 C t 2 0 .mod p/, and since 0 < r 2 C t 2 < 2p, we must have r 2 C t 2 D p. The (expected) running time of step 1 is O.len.p/3 /. The running p time of step 3 is O.len.p/2 / (note that we can compute b pc in time O.len.p/2 /, using the algorithm in Exercise 3.29). Thus, the total (expected) running time is O.len.p/3 /. Example 4.3. One can check that p WD 1009 is prime and p 1 .mod 4/. Let us express p as a sum of squares using the above algorithm. First, we need to find a square root of 1 modulo p. Let us just try a random number, say 17, and raise this to the power .p 1/=4 D 252. One can calculate that 17252 469 .mod 1009/, and 4692 1 .mod 1009/. So we were lucky with our first try. Now we run the extended Euclidean algorithm on input p D 1009 and b D 469, obtaining the following data: i 0 1 2 3 4 5 6 7 8 9
ri 1009 469 71 43 28 15 13 2 1 0
qi
si 1 0 1 -6 7 -13 20 -33 218 -469
ti 0 2 1 6 -2 1 13 1 -15 1 28 1 -43 6 71 2 -469 1009 p The first rj that falls below the threshold r D b 1009c C 1 D 32 is at j D 4, and so we set r WD 28 and t WD 15. One verifies that r 2 C t 2 D 282 C 152 D 1009 D p. It is natural to ask whether one can solve this problem without resorting to randomization. The answer is “yes” (see §4.8), but the only known deterministic algorithms for this problem are quite impractical (albeit polynomial time). This example illustrates the utility of randomization as an algorithm design technique, which has proved to be invaluable in solving numerous algorithmic problems in number theory; indeed, in §3.4 we already mentioned its use in connection with
88
Euclid’s algorithm
primality testing, and we will explore many other applications as well (after putting the notion of a probabilistic algorithm on firm mathematical ground in Chapter 9). 4.6 Rational reconstruction and applications In the previous section, we saw how to apply the extended Euclidean algorithm to obtain an effective version of Thue’s lemma. Now, Thue’s lemma asserts that for given integers n and b, there exists a pair of integers .r; t / satisfying r bt .mod n/, and contained in a prescribed rectangle, provided the area of the rectangle is large enough, relative to n. In this section, we first prove a corresponding uniqueness theorem, under the assumption that the area of the rectangle is not too large; of course, if r bt .mod n/, then for any non-zero integer q, we also have rq b.tq/, and so we can only hope to guarantee that the ratio r=t is unique. After proving this uniqueness theorem, we show how to make this theorem computationally effective, and then develop several very neat applications. The basic uniqueness statement is as follows: Theorem 4.7. Let n; b; r ; t 2 Z with r 0, t > 0, and n > 2r t . Further, suppose that r; t; r 0 ; t 0 2 Z satisfy r bt .mod n/; jrj r ; 0 < jtj t ;
(4.7)
0
(4.8)
0
0
0
r bt .mod n/; jr j r ; 0 < jt j t : Then r=t D r 0 =t 0 : Proof. Consider the two congruences r bt .mod n/; r 0 bt 0 .mod n/: Subtracting t times the second from t 0 times the first, we obtain rt 0
r 0 t 0 .mod n/:
However, we also have jrt 0
r 0 tj jrjjt 0 j C jr 0 jjt j 2r t < n:
Thus, rt 0 r 0 t is a multiple of n, but less than n in absolute value; the only possibility is that rt 0 r 0 t D 0, which means r=t D r 0 =t 0 . Now suppose that we are given n; b; r ; t 2 Z as in the above theorem; moreover, suppose that there exist r; t 2 Z satisfying (4.7), but that these values are not given to us. Note that under the hypothesis of Theorem 4.7, Thue’s lemma cannot be used to guarantee the existence of such r and t, but in our eventual applications,
89
4.6 Rational reconstruction and applications
we will have other reasons to guarantee this. We would like to find r 0 ; t 0 2 Z satisfying (4.8), and if we do this, then by the theorem, we know that r=t D r 0 =t 0 . We call this the rational reconstruction problem. We can solve this problem efficiently using the extended Euclidean algorithm; indeed, just as in the case of our effective version of Thue’s lemma, the desired values of r 0 and t 0 appear as “natural by-products” of that algorithm. To state the result precisely, let us recall the notation we introduced in the last section: for integers a; b, with a b 0, we defined ˚ `C1 EEA.a; b/ WD .ri ; si ; ti / iD0 ; where ri , si , and ti , for i D 0; : : : :` C 1, are defined as in Theorem 4.3. Theorem 4.8 (Rational reconstruction). Let n; b; r ; t 2 Z with 0 b < n, 0 r < n, and t > 0. Further, suppose that there exist r; s; t 2 Z such that r D ns C bt; jrj r ; and 0 < jt j t : Let EEA.n; b/ D f.ri ; si ; ti /gi`C1 D0 , and let j be the smallest index (among 0; : : : ; ` C 1) such that rj r , and set r 0 WD rj ; s 0 WD sj ; and t 0 WD tj : Then we have: (i) 0 < jt 0 j t ; (ii) if n > 2r t , then for some non-zero integer q, r D r 0 q; s D s 0 q; and t D t 0 q: Proof. Since r0 D n > r 0 D r`C1 , the value of j is well defined, and moreover, j 1, and we have the inequalities 0 rj r < rj
1;
0 < jtj j; jrj r ; and 0 < jtj t ;
rj
1
(4.9)
along with the identities D nsj
1
C btj
1;
(4.10)
rj D nsj C btj ;
(4.11)
r D ns C bt:
(4.12)
We now turn to part (i) of the theorem. Our goal is to prove that jtj j t :
(4.13)
This is the hardest part of the proof. To this end, let WD sj tj
1
sj
1 tj ;
WD .tj
1s
sj
1 t /=;
WD .sj t
tj s/=:
90
Euclid’s algorithm
Since D ˙1, the numbers and are integers; moreover, one may easily verify that they satisfy the equations sj C sj
1
D s;
(4.14)
tj C tj
1
D t:
(4.15)
We now use these identities to prove (4.13). We consider three cases: (i) Suppose D 0. In this case, (4.15) implies tj j t , and since t ¤ 0, this implies jtj j jtj t . (ii) Suppose < 0. In this case, since tj and tj 1 have opposite sign, (4.15) implies jtj D jtj j C jtj 1 j jtj j, and so again, we have jtj j jtj t . (iii) The only remaining possibility is that ¤ 0 and 0. We argue that this is impossible. Adding n times (4.14) to b times (4.15), and using the identities (4.10), (4.11), and (4.12), we obtain rj C rj
1
D r:
If ¤ 0 and and had the same sign, this would imply that jrj D jrj jC jrj 1 j rj 1 , and hence rj 1 jrj r ; however, this contradicts the fact that rj 1 > r . That proves the inequality (4.13). We now turn to the proof of part (ii) of the theorem, which relies critically on this inequality. Assume that n > 2r t :
(4.16)
From (4.11) and (4.12), we have rj btj .mod n/ and r bt .mod n/: Combining this with the inequalities (4.9), (4.13), and (4.16), we see that the hypotheses of Theorem 4.7 are satisfied, and so we may conclude that rtj
rj t D 0:
(4.17)
Subtracting tj times (4.12) from t times (4.11), and using the identity (4.17), we obtain n.stj sj t/ D 0, and hence stj
sj t D 0:
(4.18)
From (4.18), we see that tj j sj t , and since gcd.sj ; tj / D 1, we must have tj j t. So t D tj q for some q, and we must have q ¤ 0 since t ¤ 0. Substituting tj q for t in equations (4.17) and (4.18) yields r D rj q and s D sj q. That proves part (ii) of the theorem. In our applications in this text, we shall only directly use part (ii) of this theorem; however, part (i) has applications as well (see Exercise 4.15).
91
4.6 Rational reconstruction and applications
4.6.1 Application: recovering fractions from their decimal expansions It should be a familiar fact to the reader that every real number has a decimal expansion, and that this decimal expansion is unique, provided one rules out those expansions that end in an infinite run of 9’s (for example, 1=10 D 0:1000 D 0:0999 ). Now suppose that Alice and Bob play a game. Alice thinks of a rational number z WD s=t, where s and t are integers with 0 s < t , and tells Bob some of the high-order digits in the decimal expansion of z. Bob’s goal in the game is to determine z. Can he do this? The answer is “yes,” provided Bob knows an upper bound M on t , and provided Alice gives Bob enough digits. Of course, from grade school, Bob probably remembers that the decimal expansion of z is ultimately periodic, and that given enough digits of z so as to include the periodic part, he can recover z; however, this technique is quite useless in practice, as the length of the period can be huge — ‚.M / in the worst case (see Exercises 4.18–4.20 below). The method we discuss here requires only O.len.M // digits. Suppose Alice gives Bob the high-order k digits of z, for some k 1. That is, if z D 0 : z 1 z2 z3
(4.19)
is the decimal expansion of z, then Alice gives Bob z1 ; : : : ; zk . Now, if 10k is much smaller than M 2 , the number z is not even uniquely determined by these digits, since there are .M 2 / distinct rational numbers of the form s=t, with 0 s < t M (see Exercise 1.33). However, if 10k > 2M 2 , then not only is z uniquely determined by z1 ; : : : ; zk , but using Theorem 4.8, Bob can efficiently compute it. We shall presently describe efficient algorithms for both Alice and Bob, but before doing so, we make a few general observations about the decimal expansion of z. Let e be an arbitrary non-negative integer, and suppose that the decimal expansion of z is as in (4.19). Observe that 10e z D z1 ze : zeC1 zeC2 : It follows that b10e zc D z1 ze : 0 :
(4.20)
Since z D s=t , if we set r WD 10e s mod t , then 10e s D b10e zct C r, and dividing this by t , we have 10e z D b10e zc C r=t , where r=t 2 Œ0; 1/. Therefore, 10e s mod t D 0 : zeC1 zeC2 zeC3 : t
(4.21)
92
Euclid’s algorithm
Next, consider Alice. Based on the above discussion, Alice may use the following simple, iterative algorithm to compute z1 ; : : : ; zk , for arbitrary k 1, after she chooses s and t: x1 s for i 1 to k do yi 10xi zi byi =tc xi C1 yi mod t output z1 ; : : : ; zk Correctness follows from the observation that for each i D 1; 2; : : : ; we have xi D 10i 1 s mod t; therefore, by (4.21) with e D i 1, we have xi =t D 0 : zi zi C1 zi C2 ; and consequently, by (4.20) with e D 1 and xi =t in the role of z, we have b10xi =tc D zi . Since each loop iteration takes time O.len.M //, the total time for Alice’s computation is O.k len.M //. Finally, consider Bob. Given the high-order digits z1 ; : : : ; zk of z D s=t, along with the upper bound M on t, he can compute z as follows: Pk k i. 1. Compute n 10k and b i D1 zi 10 2. Run the extended Euclidean algorithm on input n; b to obtain EEA.n; b/, and then apply Theorem 4.8 with n, b, and r WD t WD M , to obtain the values r 0 ; s 0 ; t 0 . 3. Output the rational number s 0 =t 0 . Let us analyze this algorithm, assuming that 10k > 2M 2 . For correctness, we must show that z D s 0 =t 0 . To prove this, observe that by (4.20) with e D k, we have b D bnzc D bns =tc. Moreover, if we set r WD ns mod t, then we have r D ns
bt; 0 r < t r ; 0 < t t ; and n > 2r t :
It follows that the integers s 0 ; t 0 from Theorem 4.8 satisfy s D s 0 q and t D t 0 q for some non-zero integer q. Thus, s=t D s 0 =t 0 , as required. As a bonus, since the extended Euclidean algorithm guarantees that gcd.s 0 ; t 0 / D 1, not only do we obtain z, but we obtain z expressed as a fraction in lowest terms. We leave it to the reader to verify that Bob’s computation may be performed in time O.k 2 /. We conclude that both Alice and Bob can successfully play this game with k chosen so that k D O.len.M //, in which case, their algorithms run in time O.len.M /2 /. Example 4.4. Alice chooses integers s; t, with 0 s < t 1000, and tells Bob the high-order seven digits in the decimal expansion of z WD s=t , from which
4.6 Rational reconstruction and applications
i 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
ri 10000000 7197183 2802817 1591549 1211268 380281 70425 28156 14113 14043 70 43 27 16 11 5 1 0
qi 1 2 1 1 3 5 2 1 1 200 1 1 1 1 2 5
si 1 0 1 -2 3 -5 18 -95 208 -303 511 -102503 103014 -205517 308531 -514048 1336627 -7197183
93
ti 0 1 -1 3 -4 7 -25 132 -289 421 -710 142421 -143131 285552 -428683 714235 -1857153 10000000
Fig. 4.1. Bob’s data from the extended Euclidean algorithm Bob should be able to compute z. Suppose s D 511 and t D 710. Then s=t D 0:7197183098591549 . Bob receives the digits 7; 1; 9; 7; 1; 8; 3, and computes n D 107 and b D 7197183. Running the extended Euclidean algorithm on input n; b, Bob obtains the data in Fig. 4.1. The first rj that meets the threshold r D 1000 is at j D 10, and Bob reads off s 0 D 511 and t 0 D 710, from which he obtains z D s 0 =t 0 D 511=710. Another interesting phenomenon to observe in Fig. 4.1 is that the fractions si =ti are very good approximations to the fraction b=n D 7197183=10000000; indeed, if we compute the error terms b=n C si =ti for i D 1; : : : ; 5, we get (approximately) 0:72;
0:28; 0:053;
0:03; 0:0054:
Thus, we can approximate the “complicated” fraction 7197183=10000000 by the “very simple” fraction 5=7, introducing an absolute error of less than 0:006. Exercise 4.15 explores this “data compression” capability of Euclid’s algorithm in more generality.
94
Euclid’s algorithm
4.6.2 Application: Chinese remaindering with errors One interpretation of the Chinese remainder theorem is that if we “encode” an integer a, with 0 a < n, as the sequence .a1 ; : : : ; ak /, where ai D a mod ni for i D 1; : : : ; k, then we can efficiently recover a from this encoding. Here, of course, n D n1 nk , and the family fni gkiD1 is pairwise relatively prime. Suppose that Alice encodes a as .a1 ; : : : ; ak /, and sends this encoding to Bob over some communication network; however, because the network is not perfect, during the transmission of the encoding, some (but hopefully not too many) of the values a1 ; : : : ; ak may be corrupted. The question is, can Bob still efficiently recover the original a from its corrupted encoding? To make the problem more precise, suppose that the original, correct encoding of a is .a1 ; : : : ; ak /, and the corrupted encoding is .b1 ; : : : ; bk /. Let us define G f1; : : : ; kg to be the set of “good” positions i with ai D bi , and B f1; : : : ; kg to be the set of “bad” positions i with ai ¤ bi . We shall assume that jBj `, where ` is some specified parameter. Of course, if Bob hopes to recover a, we need to build some redundancy into the system; that is, we must require that 0 a M for some bound M that is somewhat smaller than n. Now, if Bob knew the location of bad positions, and if the product of the ni ’s at the good positions exceeds M , then Bob could simply discard the errors, and reconstruct a by applying the Chinese remainder theorem to the ai ’s and ni ’s at the good positions. However, in general, Bob will not know a priori the locations of the bad positions, and so this approach will not work. Despite these apparent difficulties, Theorem 4.8 may be used to solve the problem quite easily, as follows. Let P be an upper bound on the product of any ` of the integers n1 ; : : : ; nk (e.g., we could take P to be the product of the ` largest numbers among n1 ; : : : ; nk ). Further, let us assume that n > 2MP 2 . Now, suppose Bob obtains the corrupted encoding .b1 ; : : : ; bk /. Here is what Bob does to recover a: 1. Apply the Chinese remainder theorem, obtaining an integer b, with 0 b < n and b bi .mod ni / for i D 1; : : : ; k. 2. Run the extended Euclidean algorithm on input n; b to obtain EEA.n; b/, and then apply Theorem 4.8 with n, b, r WD MP and t WD P , to obtain values r 0 ; s 0 ; t 0 . 3. If t 0 j r 0 , output the integer r 0 =t 0 ; otherwise, output “error.” We claim that the above procedure outputs a, under our assumption that the Q set B of bad positions is of size at most `. To see this, let t WD i 2B ni . By construction, we have 1 t P . Also, let r WD at, and note that 0 r r and
4.6 Rational reconstruction and applications
95
0 < t t . We claim that r bt .mod n/:
(4.22)
To show that (4.22) holds, it suffices to show that at bt .mod ni /
(4.23)
for all i D 1; : : : ; k. To show this, for each index i we consider two cases: Case 1: i 2 G. In this case, we have ai D bi , and therefore, at ai t bi t bt .mod ni /: Case 2: i 2 B. In this case, we have ni j t , and therefore, at 0 bt .mod ni /: Thus, (4.23) holds for all i D 1; : : : ; k, and so it follows that (4.22) holds. Therefore, the values r 0 ; t 0 obtained from Theorem 4.8 satisfy r at r0 D D D a: 0 t t t One easily checks that both the procedures to encode and decode a value a run in time O.len.n/2 /. The above scheme is an example of an error correcting code, and is actually the integer analog of a Reed–Solomon code. Example 4.5. Suppose we want to encode a 1024-bit message as a sequence of 16bit blocks, so that the above scheme can correct up to 3 corrupted blocks. Without any error correction, we would need just 1024=16 D 64 blocks. However, to correct this many errors, we need a few extra blocks; in fact, 7 will do. Of course, a 1024-bit message can naturally be viewed as an integer a in the set f0; : : : ; 21024 1g, and the ith 16-bit block in the encoding can be viewed as an integer ai in the set f0; : : : ; 216 1g. Setting k WD 71, we select k primes, n1 ; : : : ; nk , each 16-bits in length. In fact, let us choose n1 ; : : : ; nk to be the largest k primes under 216 . If we do this, then the smallest prime among the ni ’s turns out to be 64717, which is greater than 215:98 . We may set M WD 21024 , and since we Q want to correct up to 3 errors, we may set P WD 2316 . Then with n WD i ni , we have n > 27115:98 D 21134:58 > 21121 D 21C1024C616 D 2MP 2 : Thus, with these parameter settings, the above scheme will correct up to 3 corrupted blocks. This comes at a cost of increasing the length of the message from 1024 bits to 71 16 D 1136 bits, an increase of about 11%.
96
Euclid’s algorithm
4.6.3 Applications to symbolic algebra Rational reconstruction also has a number of applications in symbolic algebra. We briefly sketch one such application here. Suppose that we want to find the solution v to the equation vA D w; where we are given as input a non-singular square integer matrix A and an integer vector w. The solution vector v will, in general, have rational entries. We stress that we want to compute the exact solution v, and not some floating point approximation to it. Now, we could solve for v directly using Gaussian elimination; however, the intermediate quantities computed by that algorithm would be rational numbers whose numerators and denominators might get quite large, leading to a rather lengthy computation (however, it is possible to show that the overall running time is still polynomial in the input length). Another approach is to compute a solution vector modulo n, where n is a power of a prime that does not divide the determinant of A. Provided n is large enough, one can then recover the solution vector v using rational reconstruction. With this approach, all of the computations can be carried out using arithmetic on integers not too much larger than n, leading to a more efficient algorithm. More of the details of this procedure are developed later, in Exercise 14.18. E XERCISE 4.15. Let n; b 2 Z with 0 b < n, and let EEA.n; b/ D f.ri ; si ; ti /g`C1 i D0 . This exercise develops some key properties of the fractions si =ti as approximations to b=n. For i D 1; : : : ; ` C 1, let i WD b=n C si =ti . (a) Show that i D ri =ti n for i D 1; : : : ; ` C 1. (b) Show that successive i ’s strictly decrease in absolute value, and alternate in sign. (c) Show that ji j < 1=ti2 for i D 1; : : : ; `, and `C1 D 0. (d) Show that for all s; t 2 Z with t ¤ 0, if jb=n s=t j < 1=2t 2 , then s=t D si =ti for some i D 1; : : : ; `C1. Hint: use part (ii) of Theorem 4.8. (e) Consider a fixed index i 2 f2; : : : ; ` C 1g. Show that for all s; t 2 Z, if 0 < jtj jti j and jb=n s=t j ji j, then s=t D si =ti . In this sense, si =ti is the unique, best approximation to b=n among all fractions of denominator at most jti j. Hint: use part (i) of Theorem 4.8. E XERCISE 4.16. Using the decimal approximation 3:141592654, apply Euclid’s algorithm to calculate a rational number of denominator less than 1000 that is within 10 6 of . Illustrate the computation with a table as in Fig. 4.1.
97
4.6 Rational reconstruction and applications
E XERCISE 4.17. Show that given integers s; t; k, with 0 s < t, and k > 0, we can compute the kth digit in the decimal expansion of s=t in time O.len.k/ len.t/2 /. WD fzi g1 For the following exercises, we need a definition: a sequence i D1 of elements drawn from some arbitrary set is called .k; `/-periodic for integers k 0 and ` 1 if zi D zi C` for all i > k. is called ultimately periodic if it is .k; `/-periodic for some .k; `/. E XERCISE 4.18. Show that if a sequence is ultimately periodic, then it is .k ; ` /-periodic for some uniquely determined pair .k ; ` / for which the following holds: for every pair .k; `/ such that is .k; `/-periodic, we have k k and ` j `. The value ` in the above exercise is called the period of , and k is called the pre-period of . If its pre-period is zero, then is called purely periodic. E XERCISE 4.19. Let z be a real number whose decimal expansion is an ultimately periodic sequence. Show that z is rational. E XERCISE 4.20. Let z D s=t 2 Q, where s and t are relatively prime integers with 0 s < t. Show that: 0
(a) there exist integers k; k 0 such that 0 k < k 0 and s10k s10k .mod t /; (b) for all integers k; k 0 with 0 k < k 0 , the decimal expansion of z is .k; k 0 0 k/-periodic if and only if s10k s10k .mod t /; (c) if gcd.10; t/ D 1, then the decimal expansion of z is purely periodic with period equal to the multiplicative order of 10 modulo t; (d) more generally, if k is the smallest non-negative integer such that 10 and t 0 WD t= gcd.10k ; t / are relatively prime, then the decimal expansion of z is ultimately periodic with pre-period k and period equal to the multiplicative order of 10 modulo t 0 . A famous conjecture of Artin postulates that for every integer d , not equal to 1 or to the square of an integer, there are infinitely many primes t such that d has multiplicative order t 1 modulo t. If Artin’s conjecture is true, then by part (c) of the previous exercise, there are infinitely many primes t such that the decimal expansion of s=t, for every s with 0 < s < t, is a purely periodic sequence of period t 1. In light of these observations, the “grade school” method of computing a fraction from its decimal expansion using the period is hopelessly impractical.
98
Euclid’s algorithm
4.7 The RSA cryptosystem One of the more exciting uses of number theory in recent decades is its application to cryptography. In this section, we give a brief overview of the RSA cryptosystem, named after its inventors Rivest, Shamir, and Adleman. At this point in the text, we already have the concepts and tools at our disposal necessary to understand the basic operation of this system, even though a full understanding of the system will require other ideas that will developed later in the text. Suppose that Alice wants to send a secret message to Bob over an insecure network. An adversary may be able to eavesdrop on the network, and so sending the message “in the clear” is not an option. Using older, more traditional cryptographic techniques would require that Alice and Bob share a secret key between them; however, this creates the problem of securely generating such a shared secret. The RSA cryptosystem is an example of a public key cryptosystem. To use the system, Bob simply places a “public key” in the equivalent of an electronic telephone book, while keeping a corresponding “private key” secret. To send a secret message to Bob, Alice obtains Bob’s public key from the telephone book, and uses this to encrypt her message. Upon receipt of the encrypted message, Bob uses his private key to decrypt it, obtaining the original message. Here is how the RSA cryptosystem works. To generate a public key/private key pair, Bob generates two very large, random primes p and q, with p ¤ q. To be secure, p and q should be quite large; in practice, they are chosen to be around 512 bits in length. Efficient algorithms for generating such primes exist, and we shall discuss them in detail later in the text (that there are sufficiently many primes of a given bit length will be discussed in Chapter 5; algorithms for generating them will be discussed at a high level in §9.4, and in greater detail in Chapter 10). Next, Bob computes n WD pq. Bob also selects an integer e > 1 such that gcd.e; .n// D 1. Here, .n/ D .p 1/.q 1/. Finally, Bob computes d WD e 1 mod .n/, using the extended Euclidean algorithm. The public key is the pair .n; e/, and the private key is the pair .n; d /. The integer e is called the “encryption exponent” and d is called the “decryption exponent.” In practice, the integers n and d are about 1024 bits in length, while e is usually significantly shorter. After Bob publishes his public key .n; e/, Alice may send a secret message to Bob as follows. Suppose that a message is encoded in some canonical way as a number between 0 and n 1 — we can always interpret a bit string of length less than len.n/ as such a number. Thus, we may assume that a message is an element ˛ of Zn . To encrypt the message ˛, Alice simply computes ˇ WD ˛ e using repeated squaring. The encrypted message is ˇ. When Bob receives ˇ, he computes WD ˇ d , and interprets as a message. The most basic requirement of any encryption scheme is that decryption should
99
4.7 The RSA cryptosystem
“undo” encryption. In this case, this means that for all ˛ 2 Zn , we should have .˛ e /d D ˛:
(4.24)
If ˛ 2 Zn , then this is clearly the case, since we have ed D 1 C .n/k for some positive integer k, and hence by Euler’s theorem (Theorem 2.13), we have .˛ e /d D ˛ ed D ˛ 1C .n/k D ˛ ˛ .n/k D ˛: To argue that (4.24) holds in general, let ˛ be an arbitrary element of Zn , and suppose ˛ D Œan . If a 0 .mod p/, then trivially aed 0 .mod p/; otherwise, aed a1C .n/k a a .n/k a .mod p/; where the last congruence follows from the fact that .n/k is a multiple of p 1, which is a multiple of the multiplicative order of a modulo p (again by Euler’s theorem). Thus, we have shown that aed a .mod p/. The same argument shows that aed a .mod q/, and these two congruences together imply that aed a .mod n/. Thus, we have shown that equation (4.24) holds for all ˛ 2 Zn . Of course, the interesting question about the RSA cryptosystem is whether or not it really is secure. Now, if an adversary, given only the public key .n; e/, were able to factor n, then he could easily compute the decryption exponent d himself using the same algorithm used by Bob. It is widely believed that factoring n is computationally infeasible, for sufficiently large n, and so this line of attack is ineffective, barring a breakthrough in factorization algorithms. Indeed, while trying to factor n by brute-force search is clearly infeasible, there are much faster algorithms, but even these are not fast enough to pose a serious threat to the security of the RSA cryptosystem. We shall discuss some of these faster algorithms in some detail later in the text (in Chapter 15). Can one break the RSA cryptosystem without factoring n? For example, it is natural to ask whether one can compute the decryption exponent d without having to go to the trouble of factoring n. It turns out that the answer to this question is “no”: if one could compute the decryption exponent d , then ed 1 would be a multiple of .n/, and as we shall see later in §10.4, given any multiple of .n/, we can easily factor n. Thus, computing the decryption exponent is equivalent to factoring n, and so this line of attack is also ineffective. But there still could be other lines of attack. For example, even if we assume that factoring large numbers is infeasible, this is not enough to guarantee that for a given encrypted message ˇ, the adversary is unable to compute ˇ d (although nobody actually knows how to do this without first factoring n). The reader should be warned that the proper notion of security for an encryption scheme is quite subtle, and a detailed discussion of this is well beyond the scope
100
Euclid’s algorithm
of this text. Indeed, the simple version of RSA presented here suffers from a number of security problems (because of this, actual implementations of public-key encryption schemes based on RSA are somewhat more complicated). We mention one such problem here (others are examined in some of the exercises below). Suppose an eavesdropping adversary knows that Alice will send one of a few, known, candidate messages. For example, an adversary may know that Alice’s message is either “let’s meet today” or “let’s meet tomorrow.” In this case, the adversary can encrypt for himself each of the candidate messages, intercept Alice’s actual encrypted message, and then by simply comparing encryptions, the adversary can determine which particular message Alice encrypted. This type of attack works simply because the encryption algorithm is deterministic, and in fact, any deterministic encryption algorithm will be vulnerable to this type of attack. To avoid this type of attack, one must use a probabilistic encryption algorithm. In the case of the RSA cryptosystem, this is often achieved by padding the message with some random bits before encrypting it (but even this must be done carefully). E XERCISE 4.21. This exercise develops an algorithm for speeding up RSA decryption. Suppose that we are given two distinct `-bit primes, p and q, an element ˇ 2 Zn , where n WD pq, and an integer d , where 1 < d < .n/. Using the algorithm from Exercise 3.35, we can compute ˇ d at a cost of essentially 2` squarings in Zn . Show how this can be improved, making use of the factorization of n, so that the total cost is essentially that of ` squarings in Zp and ` squarings in Zq , leading to a roughly four-fold speed-up in the running time. E XERCISE 4.22. Alice submits a bid to an auction, and so that other bidders cannot see her bid, she encrypts it under the public key of the auction service. Suppose that the auction service provides a public key for an RSA encryption scheme, with a modulus n. Assume that bids are encoded simply as integers between 0 and n 1 prior to encryption. Also, assume that Alice submits a bid that is a “round number,” which in this case means that her bid is a number that is divisible by 10. Show how an eavesdropper can submit an encryption of a bid that exceeds Alice’s bid by 10%, without even knowing what Alice’s bid is. In particular, your attack should work even if the space of possible bids is very large. E XERCISE 4.23. To speed up RSA encryption, one may choose a very small encryption exponent. This exercise develops a “small encryption exponent attack” on RSA. Suppose Bob, Bill, and Betty have RSA public keys with moduli n1 , n2 , and n3 , and all three use encryption exponent 3. Assume that fni g3iD1 is pairwise relatively prime. Suppose that Alice sends an encryption of the same message to Bob, Bill, and Betty — that is, Alice encodes her message as an integer
4.8 Notes
101
a, with 0 a < minfn1 ; n2 ; n3 g, and computes the three encrypted messages ˇi WD Œa3 ni , for i D 1; : : : ; 3. Show how to recover Alice’s message from these three encrypted messages. E XERCISE 4.24. To speed up RSA decryption, one might choose a small decryption exponent, and then derive the encryption exponent from this. This exercise develops a “small decryption exponent attack” on RSA. Suppose n D pq, where p and q are distinct primes with len.p/ D len.q/. Let d and e be integers such that 1 < d < .n/, 1 < e < .n/, and de 1 .mod .n//. Further, assume that d < n1=4 =3: Show how to efficiently compute d , given n and e. Hint: since ed 1 .mod .n//, it follows that ed D 1 C .n/k for an integer k with 0 < k < d ; let r WD nk ed , and show that jrj < n3=4 ; next, show how to recover d (along with r and k) using Theorem 4.8. 4.8 Notes The Euclidean algorithm as we have presented it here is not the fastest known algorithm for computing greatest common divisors. The asymptotically fastest known algorithm for computing the greatest common divisor of two numbers of bit length at most ` runs in time O.` len.`// on a RAM. This algorithm is due to Schönhage [83]. The same algorithm leads to Boolean circuits of size O.` len.`/2 len.len.`///, which using Fürer’s result [37], can be reduced to O.` len.`/2 2O.log n/ /. The same complexity results also hold for the extended Euclidean algorithm, as well as for Chinese remaindering, Thue’s lemma, and rational reconstruction. Experience suggests that such fast algorithms for greatest common divisors are not of much practical value, unless the integers involved are very large — at least several tens of thousands of bits in length. The extra “log” factor and the rather large multiplicative constants seem to slow things down too much. The binary gcd algorithm (Exercise 4.5) is due to Stein [98]. The extended binary gcd algorithm (Exercise 4.9) was first described by Knuth [55], who attributes it to M. Penk. Our formulation of both of these algorithms closely follows that of Menezes, van Oorschot, and Vanstone [64]. Experience suggests that the binary gcd algorithm is faster in practice than Euclid’s algorithm. Schoof [85] presents (among other things) a deterministic, polynomial-time algorithm that computes a square root of 1 modulo p for any given prime p 1 .mod 4/. If we use this algorithm in §4.5, we get a deterministic, polynomial-time algorithm to compute integers r and t such that p D r 2 C t 2 . Our Theorem 4.8 is a generalization of one stated in Wang, Guy, and Davenport [101]. One can generalize Theorem 4.8 using the theory of continued fractions. With this, one can generalize Exercise 4.15 to deal with rational approximations to
102
Euclid’s algorithm
irrational numbers. More on this can be found, for example, in the book by Hardy and Wright [45]. The application of Euclid’s algorithm to computing a rational number from the first digits of its decimal expansion was observed by Blum, Blum, and Shub [17], where they considered the possibility of using such sequences of digits as a pseudorandom number generator — the conclusion, of course, is that this is not such a good idea. The RSA cryptosystem was invented by Rivest, Shamir, and Adleman [80]. There is a vast literature on cryptography. One starting point is the book by Menezes, van Oorschot, and Vanstone [64]. The attack in Exercise 4.24 is due to Wiener [108]; this attack was recently strengthened by Boneh and Durfee [19].
5 The distribution of primes
This chapter concerns itself with the question: how many primes are there? In Chapter 1, we proved that there are infinitely many primes; however, we are interested in a more quantitative answer to this question; that is, we want to know how “dense” the prime numbers are. This chapter has a bit more of an “analytical” flavor than other chapters in this text. However, we shall not make use of any mathematics beyond that of elementary calculus. 5.1 Chebyshev’s theorem on the density of primes The natural way of measuring the density of primes is to count the number of primes up to a bound x, where x is a real number. For each real number x 0, we define .x/ to be the number of primes up to (and including) x. For example, .1/ D 0, .2/ D 1, and .7:5/ D 4. The function is an example of a “step function,” that is, a function that changes values only at a discrete set of points. It might seem more natural to define only on the integers, but it is the tradition to define it over the real numbers (and there are some technical benefits in doing so). Let us first take a look at some values of .x/. Table 5.1 shows values of .x/ for x D 103i and i D 1; : : : ; 6. The third column of this table shows the value of x=.x/ (to five decimal places). One can see that the differences between successive rows of this third column are roughly the same — about 6:9 — which suggests that the function x=.x/ grows logarithmically in x. Indeed, as log.103 / 6:9, it would not be unreasonable to guess that x=.x/ log x, or equivalently, .x/ x= log x. (As discussed in the Preliminaries, log x denotes the natural logarithm of x.) The following theorem is a first — and important — step towards making the above guesswork more rigorous:
103
104
The distribution of primes
Table 5.1. Some values of .x/ x .x/ x=.x/ 103 168 5.95238 106 78498 12.73918 109 50847534 19.66664 1012 37607912018 26.59015 1015 29844570422669 33.50693 1018 24739954287740860 40.42045 Theorem 5.1 (Chebyshev’s theorem). We have .x/ D ‚.x= log x/: It is not too difficult to prove this theorem, which we now proceed to do in several steps. We begin with a simple lower bound on binomial coefficients (see §A2): Lemma 5.2. If m is a positive integer, then ! 2m 22m =2m: m Proof. As 2m m is the largest binomial coefficient in the binomial expansion of .1 C 1/2m , we have ! ! ! ! 2m 2m X X1 2m 2m 2m 2m 2m 2 D D 1C C 1 2 C .2m 1/ 2m : i i m m i D0
i D1
Next, recalling that p .n/ denotes the power to which a prime p divides an integer n, we make with the following observation: Lemma 5.3. Let n be a positive integer. For every prime p, we have X p .nŠ/ D bn=p k c: k1
Proof. For positive integers j; k, define dj k WD 1 if p k j j , and dj k WD 0, otherP wise. Observe that p .j / D k1 dj k (this sum is actually finite, since dj k D 0 for all sufficiently large k). So we have p .nŠ/ D
n X j D1
p .j / D
n X X j D1 k1
dj k D
n XX k1 j D1
dj k :
5.1 Chebyshev’s theorem on the density of primes
105
Pn
Finally, note that j D1 dj k is equal to the number of multiples of p k among the integers 1; : : : ; n, which by Exercise 1.3 is equal to bn=p k c. The following theorem gives a lower bound on .x/. Theorem 5.4. .n/ 21 .log 2/n= log n for every integer n 2. Proof. For positive integer m, consider the binomial coefficient ! 2m .2m/Š N WD D : m .mŠ/2 It is clear that N is divisible only by primes p up to 2m. Applying Lemma 5.3 to the identity N D .2m/Š=.mŠ/2 , we have X p .N / D .b2m=p k c 2bm=p k c/: k1
Each term in this sum is either 0 or 1 (see Exercise 1.4), and for k > log.2m/= log p, each term is zero. Thus, p .N / log.2m/= log p. So we have X log.2m/ .2m/ log.2m/ D log p log p p2m X p .N / log p D log N; p2m
where the summations are over the primes p up to 2m. By Lemma 5.2, we have N 22m =2m 2m , and hence .2m/ log.2m/ m log 2 D 12 .log 2/.2m/: That proves the theorem for even n. Now consider odd n 3, so n D 2m 1 for some m 2. It is easily verified that the function x= log x is increasing for x 3; therefore, .2m
1/ D .2m/ 21 .log 2/.2m/= log.2m/ 12 .log 2/.2m
1/= log.2m
1/:
That proves the theorem for odd n. As a consequence of the above theorem, we have .x/ D .x= log x/ for real x ! 1. Indeed, for every real number x 2, setting c WD 21 .log 2/, we have .x/ D .bxc/ cbxc= logbxc c.x
1/= log x D .x= log x/:
106
The distribution of primes
To obtain a corresponding upper bound for .x/, we introduce an auxiliary function, called Chebyshev’s theta function: X #.x/ WD log p; px
where the sum is over all primes p up to x. Chebyshev’s theta function is an example of a summation over primes, and in this chapter, we will be considering a number of functions that are defined in terms of sums or products over primes. To avoid excessive tedium, we adopt the usual convention used by number theorists: if not explicitly stated, summations and products over the variable p are always understood to be over primes. For example, we P may write .x/ D px 1. Theorem 5.5. We have #.x/ D ‚..x/ log x/: Proof. On the one hand, we have X X #.x/ D log p log x 1 D .x/ log x: px
px
On the other hand, we have X X #.x/ D log p px
D
1 2
log p
1 2
log x
x 1=2
log x .x/
.x 1=2 / D
X
1
x 1=2
1
.x 1=2 /=.x/ .x/ log x:
It will therefore suffice to show that .x 1=2 /=.x/ D o.1/. Certainly, .x 1=2 / x 1=2 . Moreover, by the previous theorem, .x/ D .x= log x/. Therefore, .x 1=2 /=.x/ D O.log x=x 1=2 / D o.1/; and the theorem follows. Theorem 5.6. #.x/ < 2.log 2/x for every real number x 1. Proof. It suffices to prove that #.n/ < 2.log 2/n for every positive integer n, since then #.x/ D #.bxc/ < 2.log 2/bxc 2.log 2/x. We prove this by induction on n. For n D 1 and n D 2, this is clear, so assume n > 2. If n is even, then using the induction hypothesis for n 1, we have #.n/ D #.n
1/ < 2.log 2/.n
1/ < 2.log 2/n:
Now consider the case where n is odd. Write n D 2m C 1, where m is a positive
5.2 Bertrand’s postulate
107
integer, and consider the binomial coefficient ! 2m C 1 .2m C 1/Š M WD : D mŠ.m C 1/Š m Observe that M is divisible by all primes p with m C 1 < p 2m C 1. Moreover, as M occurs twice in the binomial expansion of .1 C 1/2mC1 , we have M < 22mC1 =2 D 22m : It follows that X #.2m C 1/ #.m C 1/ D log p log M < 2.log 2/m: mC1
Using this, and the induction hypothesis for m C 1, we obtain #.n/ D #.2m C 1/
#.m C 1/ C #.m C 1/
< 2.log 2/m C 2.log 2/.m C 1/ D 2.log 2/n: Another way of stating the above theorem is: Y p < 4x : px
Theorem 5.1 follows immediately from Theorems 5.4, 5.5 and 5.6. Note that we have also proved: Theorem 5.7. We have #.x/ D ‚.x/: E XERCISE 5.1. For each positive integer n, let pn denote the nth prime. Show that pn D ‚.n log n/. E XERCISE 5.2. For each positive integer n, let !.n/ denote the number of distinct primes dividing n. Show that !.n/ D O.log n= log log n/. P E XERCISE 5.3. Show that px 1= log p D ‚.x=.log x/2 /. 5.2 Bertrand’s postulate Suppose we want to know how many primes there are of a given bit length, or more generally, how many primes there are between m and 2m for a given positive integer m. Neither the statement, nor our proof, of Chebyshev’s theorem imply that there are any primes between m and 2m, let alone a useful density estimate of such primes. Bertrand’s postulate is the assertion that for every positive integer m, there exists a prime between m and 2m. We shall in fact prove a stronger result, namely,
108
The distribution of primes
that not only is there one prime, but the number of primes between m and 2m is .m= log m/. Theorem 5.8 (Bertrand’s postulate). For every positive integer m, we have m .2m/ .m/ > : 3 log.2m/ The proof uses Theorem 5.6, along with a more careful re-working of the proof of Theorem 5.4. The theorem is clearly true for m 2, sowe may assume that m 3. As in the proof of the Theorem 5.4, define N WD 2m m , and recall that N is divisible only by primes less than 2m, and that we have the identity X p .N / D .b2m=p k c 2bm=p k c/; (5.1) k1
where each term in the sum is either 0 or 1. We can characterize the values p .N / a bit more precisely, as follows: Lemma 5.9. Let m 3 and N WD 2m m . For all primes p, we have p p .N / 2m; p if p > 2m, then p .N / 1;
(5.3)
if 2m=3 < p m, then p .N / D 0;
(5.4)
if m < p < 2m, then p .N / D 1.
(5.5)
(5.2)
Proof. For (5.2), all terms with k > log.2m/= log p in (5.1) vanish, and hence p .N / log.2m/= log p, from which it follows that p p .N / 2m. (5.3) follows immediately from (5.2). For (5.4), if 2m=3 < p m, then 2m=p < 3, and we must also have p 3, since p D 2 implies m < 3. We have p 2 > p.2m=3/ D 2m.p=3/ 2m, and hence all terms with k > 1 in (5.1) vanish. The term with k D 1 also vanishes, since 1 m=p < 3=2, from which it follows that 2 2m=p < 3, and hence bm=pc D 1 and b2m=pc D 2. For (5.5), if m < p < 2m, it follows that 1 < 2m=p < 2, so b2m=pc D 1. Also, m=p < 1, so bm=pc D 0. It follows that the term with k D 1 in (5.1) is 1, and it is clear that 2m=p k < 1 for all k > 1, and so all the other terms vanish. We now have the necessary technical ingredients to prove Theorem 5.8. Define Y Pm WD p; m
and define Qm so that N D Qm Pm :
109
5.3 Mertens’ theorem
By (5.4) and (5.5), we see that Y
Qm D
p p .N / :
p2m=3
p 2m, so there are at most Moreover, by (5.3), .N / > 1 for at most those p p p 2m such primes, and by (5.2), the contribution of each such prime to the above product is at most 2m. Combining this with Theorem 5.6, we obtain p 2m
Qm < .2m/
42m=3 :
We now apply Lemma 5.2, obtaining Pm D NQm1 22m .2m/
1
Qm1 > 4m=3 .2m/
p .1C 2m/
:
It follows that .2m/
p m log 4 .1 C 2m/ 3 log.2m/ p m m.log 4 1/ D C .1 C 2m/: 3 log.2m/ 3 log.2m/
.m/ log Pm = log.2m/ >
Clearly, for all sufficiently large m, we have p m.log 4 1/ > 1 C 2m: 3 log.2m/
(5.6)
That proves Theorem 5.8 for all sufficiently large m. Moreover, a simple calculation shows that (5.6) holds for all m 13; 000, and one can verify by brute force (with the aid of a computer) that the theorem holds for m < 13; 000. 5.3 Mertens’ theorem Our next goal is to prove the following theorem, which turns out to have a number of applications. Theorem 5.10. We have X 1 D log log x C O.1/: p px The proof of this theorem, while not difficult, is a bit technical, and we proceed in several steps. Theorem 5.11. We have X log p D log x C O.1/: p px
110
The distribution of primes
Proof. Let n WD bxc. The idea of the proof is to estimate log.nŠ/ in two different ways. By Lemma 5.3, we have XX X XX log.nŠ/ D bn=p k c log p D bn=pc log p C bn=p k c log p: pn k1
pn
k2 pn
We next show that the last sum is O.n/. We have X X X X log p bn=p k c n log p p pn
pn
k2
k
k2
X log p X log p 1 D n Dn p 2 1 1=p p.p 1/ pn pn n
X k2
log k D O.n/: k.k 1/
Thus, we have shown that log.nŠ/ D
X
bn=pc log p C O.n/:
pn
Since bn=pc D n=p C O.1/, applying Theorem 5.6 (and Exercise 3.12), we obtain X X X log p log.nŠ/ D .n=p/ log p C O. log p/ C O.n/ D n C O.n/: (5.7) p pn pn pn We can also estimate log.nŠ/ by estimating a sum by an integral (see §A5): Z n n X log.nŠ/ D log k D log t dt C O.log n/ D n log n n C O.log n/: (5.8) kD1
1
Combining (5.7) and (5.8), and noting that log x log n D o.1/ (see Exercise 3.11), we obtain X log p D log n C O.1/ D log x C O.1/; p px which proves the theorem. We shall also need the following theorem, which is a very useful tool in its own right; it is essentially a discrete variant of “integration by parts.” Theorem 5.12 (Abel’s identity). Let fci g1 be a sequence of real numbers, and iDk for each real number t, define X C.t / WD ci : ki t
111
5.3 Mertens’ theorem
Further, suppose that f .t / is a function with a continuous derivative f 0 .t / on the interval Œk; x, where x is a real number, with x k. Then Z x X ci f .i / D C.x/f .x/ C.t /f 0 .t / dt: k
ki x
Note that since C.t / is a step function, the integrand C.t /f 0 .t / is piece-wise continuous on Œk; x, and hence the integral is well defined (see §A4). Proof. Let n WD bxc. We have n X
ci f .i/ D C.k/f .k/ C ŒC.k C 1/
i Dk
C ŒC.n/
C.n
D C.k/Œf .k/
C.k/f .k C 1/ C
1/f .n/
f .k C 1/ C C C.n
1/Œf .n
1/
f .n/
1/Œf .n
1/
f .n/
C C.n/f .n/ D C.k/Œf .k/
f .k C 1/ C C C.n
C C.n/Œf .n/ Observe that for i D k; : : : ; n so
f .x/ C C.x/f .x/:
1, we have C.t / D C.i / for all t 2 Œi; i C 1/, and i C1
Z C.i/Œf .i /
f .i C 1/ D
C.t /f 0 .t / dtI
i
likewise, Z C.n/Œf .n/
f .x/ D
x
C.t /f 0 .t / dt;
n
from which the theorem directly follows. Proof of Theorem 5.10. For i 2, set .log i /= i ci WD 0
if i is prime, otherwise.
By Theorem 5.11, we have C.t / WD
X 2i t
ci D
X log p pt
p
D log t C R.t /;
where R.t/ D O.1/. Applying Theorem 5.12 with f .t / WD 1= log t (and using
112
The distribution of primes
Exercise 3.13), we obtain Z x X 1 X C.t / C.x/ D C dt ci f .i / D p log x t .log t /2 2 px 2i x Z x Z x R.x/ dt R.t / D1C C C dt 2 log x 2 t log t 2 t .log t / D 1 C O.1= log x/ C .log log x log log 2/ C O.1/ D log log x C O.1/: Using Theorem 5.10, we can easily show the following: Theorem 5.13 (Mertens’ theorem). We have Y .1 1=p/ D ‚.1= log x/: px
Proof. Using parts (i) and (iii) of §A1, for any fixed prime p, we have 1 1 C log.1 p2 p
1=p/ 0:
(5.9)
Moreover, since X 1 X 1 < 1; p2 i2 px i 2
summing the inequality (5.9) over all primes p x yields X 1 C C log g.x/ 0; p px Q where C is a positive constant, and g.x/ WD px .1 1=p/. From this, and from Theorem 5.10, we obtain log g.x/ D log log x C O.1/, which implies that g.x/ D ‚.1= log x/ (see Exercise 3.11) That proves the theorem. E XERCISE 5.4. For each positive integer k, let Pk denote the product of the first k primes. Show that .Pk / D ‚.Pk = log log Pk /. E XERCISE 5.5. The previous exercise showed that .n/ could be as small as (about) n= log log n for infinitely many n. Show that this is the “worst case,” in the sense that .n/ D .n= log log n/ as n ! 1. E XERCISE 5.6. Show that for every positive integer constant k, Z x dt x x D CO : k .log x/k .log x/kC1 2 .log t / This fact may be useful in some of the following exercises.
5.3 Mertens’ theorem
113
E XERCISE 5.7. Use Chebyshev’s theorem and Abel’s identity to prove a stronger version of Theorem 5.5: #.x/ D .x/ log x C O.x= log x/: E XERCISE 5.8. Use Chebyshev’s theorem and Abel’s identity to show that X 1 .x/ D C O.x=.log x/3 /: log p log x px E XERCISE 5.9. Show that Y
.1
2=p/ D ‚.1=.log x/2 /:
2
E XERCISE 5.10. Show that if .x/ cx= log x for some constant c, then we must have c D 1. P E XERCISE 5.11. Strengthen Theorem 5.10: show that px 1=p D log log x C A C o.1/ for some constant A. (Note: A 0:261497212847643.) E XERCISE 5.12. Use the result from the previous exercise to strengthen Mertens’ Q theorem: show that px .1 1=p/ B1 =.log x/ for some constant B1 . (Note: B1 0:561459483566885.) E XERCISE 5.13. Strengthen the result of Exercise 5.9: show that Y .1 2=p/ B2 =.log x/2 2
for some constant B2 . (Note: B2 0:832429065662.) E XERCISE 5.14. Use Abel’s identity to derive Euler’s summation formula: if f .t / has a continuous derivative f 0 .t / on the interval Œa; b, where a and b are integers, then Z b Z b b X f .i/ f .t /dt D f .a/ C .t bt c/f 0 .t /dt: i Da
a
a
E XERCISE 5.15. Use Euler’s summation formula (previous exercise) to show that n C 21 log n C O.1/; p and from this, conclude that nŠ D ‚..n=e/n n/. This ispa weak form of Stirling’s approximation; a sharper form states that nŠ .n=e/n 2 n. log.nŠ/ D n log n
E XERCISE 5.16. Use Stirling’s approximation (previous exercise) to show that ! p 2m D ‚.4m = m/: m
114
The distribution of primes
5.4 The sieve of Eratosthenes As an application of Theorem 5.10, consider the sieve of Eratosthenes. This is an algorithm that generates all the primes up to a given bound n. It uses an array AŒ2 : : : n, and runs as follows. for k for k
2 to n do AŒk 1 p 2 to b nc do if AŒk D 1 then i 2k; while i n do f AŒi
0; i
i Ck g
When the algorithm finishes, we have AŒk D 1 if and only if k is prime, for k D 2; : : : ; n. This can easily be proven using the fact (see Exercise 1.2) that a composite number k between 2 and n must be divisible by a prime that is at most p n, and by proving by induction on k that at the beginning of each iteration of the main loop, AŒi D 0 if and only if i is divisible by a prime less than k, for i D k; : : : ; n. We leave the details of this to the reader. We are more interested in the running time of the algorithm. To analyze the running time, we assume that all arithmetic operations take constant time; this is reasonable, since all the numbers computed are used as array indices and thus should fit in single machine words. Therefore, we can assume that built-in arithmetic instructions are used for operating on such numbers. Every time we execute the inner loop of the algorithm, we perform O.n=k/ steps to clear the entries of A indexed by multiples of k. Pessimistically, then, we could bound the total running time by O.n T .n//, where X T .n/ WD 1=k: p k n
Estimating the sum by an integral (see §A5), we have p b nc
T .n/ D
X kD1
Z 1=k D
1
p b nc
dy 1 C O.1/ log n: y 2
This implies a O.n len.n// bound on the running time of the algorithm. However, this rather crude analysis ignores the fact that the inner loop is executed only for prime values of k; taking this fact into account, we see that the running time is O.n T1 .n//, where X T1 .n/ WD 1=p: p p n
5.5 The prime number theorem . . . and beyond
115
By Theorem 5.10, T1 .n/ D log log n C O.1/, which implies a O.n len.len.n/// bound on the running time of the algorithm. This is a substantial improvement over the above, rather crude analysis. E XERCISE 5.17. Give a detailed proof of the correctness of the above algorithm. E XERCISE 5.18. One drawback of the above algorithm is its use of space: it requires an array of size n. Show how to modify the algorithm, without substantially increasing its running time, so that one can enumerate all the primes up to n, using p an auxiliary array of size just O. n/. E XERCISE 5.19. Design and analyze an algorithm that on input n outputs the table of values .k/ for k D 1; : : : ; n, where .k/ is the number of positive divisors of k. Your algorithm should run in time O.n len.n//. 5.5 The prime number theorem . . . and beyond In this section, we survey a number of theorems and conjectures related to the distribution of primes. This is a vast area of mathematical research, with a number of very deep results. We shall be stating a number of theorems from the literature in this section without proof; while our intent is to keep the text as self contained as possible, and to avoid degenerating into “mathematical tourism,” it nevertheless is a good idea to occasionally have a somewhat broader perspective. In the following chapters, we shall not make any critical use of the theorems in this section. 5.5.1 The prime number theorem The main theorem in the theory of the density of primes is the following. Theorem 5.14 (Prime number theorem). We have .x/ x= log x: Proof. Literature—see §5.6. As we saw in Exercise 5.10, if .x/=.x= log x/ tends to a limit as x ! 1, then the limit must be 1, so in fact the hard part of proving the prime number theorem is to show that .x/=.x= log x/ does indeed tend to some limit. E XERCISE 5.20. Using the prime number theorem, show that #.x/ x. E XERCISE 5.21. Using the prime number theorem, show that pn n log n, where pn denotes the nth prime.
116
The distribution of primes
Table 5.2. Values of .x/, li.x/, and x= log x x 103 106 109 1012 1015 1018
.x/ 168 78498 50847534 37607912018 29844570422669 24739954287740860
li.x/ 176.6 78626.5 50849233.9 37607950279.8 29844571475286.5 24739954309690414.0
x= log x 144.8 72382.4 48254942.4 36191206825.3 28952965460216.8 24127471216847323.8
E XERCISE 5.22. Using the prime number theorem, show that Bertrand’s postulate can be strengthened (asymptotically) as follows: for every > 0, there exist positive constants c and x0 , such that for all x x0 , we have x ..1 C /x/ .x/ c : log x 5.5.2 The error term in the prime number theorem The prime number theorem says that j.x/
x= log xj ı.x/;
where ı.x/ D o.x= log x/. A natural question is: how small is the “error term” ı.x/? It can be shown that .x/ D x= log x C O.x=.log x/2 /:
(5.10)
This bound on the error term is not very impressive. The problem is that x= log x is not really the best “simple” function that approximates .x/. It turns out that a better approximation to .x/ is the logarithmic integral, defined for real numbers x 2 by Z x dt li.x/ WD : 2 log t It is not hard to show (see Exercise 5.6) that li.x/ D x= log x C O.x=.log x/2 /:
(5.11)
Thus, li.x/ x= log x .x/. However, the error term in the approximation of .x/ by li.x/ is much better. This is illustrated numerically in Table 5.2; for example, at x D 1018 , li.x/ approximates .x/ with a relative error just under 10 9 , while x= log x approximates .x/ with a relative error of about 0:025. The sharpest proven result on the error in approximating .x/ by li.x/ is the following:
5.5 The prime number theorem . . . and beyond
Theorem 5.15. Let .x/ WD .log x/3=5 .log log x/ have
1=5 .
c.x/
.x/ D li.x/ C O.xe
117
Then for some c > 0, we
/:
Proof. Literature—see §5.6. Note that the error term xe c.x/ is o.x=.log x/k / for every fixed k 0. Also note that (5.10) follows directly from (5.11) and Theorem 5.15. Although the above estimate on the error term in the approximation of .x/ by li.x/ is pretty good, it is conjectured that the actual error term is much smaller: Conjecture 5.16. For all x 2:01, we have j.x/
li.x/j < x 1=2 log x:
Conjecture 5.16 is equivalent to a famous conjecture called the Riemann hypothesis, which is an assumption about the location of the zeros of a certain function, called Riemann’s zeta function. We give a very brief, high-level account of this conjecture, and its connection to the theory of the distribution of primes. For real numbers s > 1, the zeta function is defined as 1 X 1 .s/ WD : ns
(5.12)
nD1
Note that because s > 1, the infinite series defining .s/ converges. A simple, but important, connection between the zeta function and the theory of prime numbers is the following: Theorem 5.17 (Euler’s identity). For every real number s > 1, we have Y .s/ D .1 p s / 1 ;
(5.13)
p
where the product is over all primes p. Proof. The rigorous interpretation of the infinite product on the right-hand side of (5.13) is as a limit of finite products. Thus, if pi denotes the i th prime, for i D 1; 2; : : : ; then we are really proving that .s/ D lim
r!1
r Y
.1
pi s /
1
i D1
Now, from the identity .1
pi s /
1
D
1 X eD0
pi
es
;
:
118
The distribution of primes
we have r Y
.1
pi s /
1
D 1 C p1 s C p1 2s C 1 C pr s C pr 2s C
i D1
D
1 X hr .n/ ; ns
nD1
where hr .n/ WD
1 if n is divisible only by the primes p1 ; : : : ; pr ; 0 otherwise.
Here, we have made use of the fact (see §A7) that we can multiply term-wise infinite series with non-negative terms. P s < (because the Now, for every > 0, there exists n0 such that 1 nDn0 n series defining .s/ converges). Moreover, there exists an r0 such that hr .n/ D 1 for all n < n0 and r r0 . Therefore, for all r r0 , we have ˇX ˇ 1 X ˇ 1 hr .n/ ˇ ˇ ˇ .s/ n s < : ˇ ˇ ns nDn nD1
0
It follows that 1 X hr .n/ D .s/; lim r!1 ns nD1
which proves the theorem. While Theorem 5.17 is nice, things become much more interesting if one extends the domain of definition of the zeta function to the complex plane. For the reader who is familiar with just a little complex analysis, it is easy to see that the infinite series defining the zeta function in (5.12) converges absolutely for all complex numbers s whose real part is greater than 1, and that (5.13) holds as well for such s. However, it is possible to extend the domain of definition of even further — in fact, one can extend the definition of in a “nice way ” (in the language of complex analysis, analytically continue) to the entire complex plane (except the point s D 1, where there is a simple pole). Exactly how this is done is beyond the scope of this text, but assuming this extended definition of , we can now state the Riemann hypothesis: Conjecture 5.18 (Riemann hypothesis). Let s be a complex number with s D x C yi, x; y 2 R. Then .s/ D 0 and 0 < x < 1=2 implies x D 1=2. A lot is known about the zeros of the zeta function in the “critical strip,” which consists of those points s whose real part is greater than 0 and less than 1: it is
5.5 The prime number theorem . . . and beyond
119
known that there are infinitely many such zeros, and there are even good estimates about their density. It turns out that one can apply standard tools in complex analysis, like contour integration, to the zeta function (and functions derived from it) to answer various questions about the distribution of primes. Indeed, such techniques may be used to prove the prime number theorem. However, if one assumes the Riemann hypothesis, then these techniques yield much sharper results, such as the bound in Conjecture 5.16. E XERCISE 5.23. For any arithmetic function a (mapping positive integers to reals), we can form the Dirichlet series 1 X a.n/ Fa .s/ WD : ns nD1
For simplicity we assume that s takes only real values, even though such series are usually studied for complex values of s. (a) Show that if the Dirichlet series Fa .s/ converges absolutely for some real s, then it converges absolutely for all real s 0 s. (b) From part (a), conclude that for any given arithmetic function a, there is an interval of absolute convergence of the form .s0 ; 1/, where we allow s0 D 1 and s0 D 1, such that Fa .s/ converges absolutely for s > s0 , and does not converge absolutely for s < s0 . (c) Let a and b be arithmetic functions such that Fa .s/ has an interval of absolute convergence .s0 ; 1/ and Fb .s/ has an interval of absolute convergence .s00 ; 1/, and assume that s0 < 1 and s00 < 1. Let c WD a ? b be the Dirichlet product of a and b, as defined in §2.9. Show that for all s 2 .max.s0 ; s00 /; 1/, the series Fc .s/ converges absolutely and, moreover, that Fa .s/Fb .s/ D Fc .s/. 5.5.3 Explicit estimates Sometimes, it is useful to have explicit estimates for .x/, as well as related functions, like #.x/ and the nth prime function pn . The following theorem presents a number of bounds that have been proved without relying on any unproved conjectures. Theorem 5.19. We have: x 1 x 3 (i) 1C < .x/ < 1C ; for x 59; log x 2 log x log x 2 log x (ii) n.log n C log log n 3=2/ < pn < n.log n C log log n 1=2/; for n 20;
120
The distribution of primes
1 1 < #.x/ < x 1 C ; for x 563; 2 log x 2 log x X 1 1 < 1=p < log log x C A C ; (iv) log log x C A 2 2.log x/ 2.log x/2 px
(iii) x 1
for x 286, where A 0:261497212847643; Y 1 B1 1 B1 1 1 (v) 1 < < 1 C ; log x 2.log x/2 p log x 2.log x/2 px for x 285, where B1 0:561459483566885. Proof. Literature—see §5.6. 5.5.4 Primes in arithmetic progressions In Theorems 2.35 and 2.36, we proved that there are infinitely many primes p 1 .mod 4/ and infinitely many primes p 3 .mod 4/. These results are actually special cases of a much more general result. Let d be a positive integer, and let a be any integer. An arithmetic progression with first term a and common difference d consists of all integers of the form a C d m; m D 0; 1; 2; : : : : The question is: under what conditions does such an arithmetic progression contain infinitely many primes? An equivalent formulation is: under what conditions are there infinitely many primes p a .mod d /? If a and d have a common factor c > 1, then every term in the progression is divisible by c, and so there can be at most one prime in the progression. So a necessary condition for the existence of infinitely many primes p a .mod d / is that gcd.a; d / D 1. A famous theorem due to Dirichlet states that this is a sufficient condition as well. Theorem 5.20 (Dirichlet’s theorem). Let a; d 2 Z with d > 0 and gcd.a; d / D 1. Then there are infinitely many primes p a .mod d /. Proof. Literature—see §5.6. We can also ask about the density of primes in arithmetic progressions. One might expect that for a fixed value of d , the primes are distributed in roughly equal measure among the .d / different residue classes Œad with gcd.a; d / D 1. This is in fact the case. To formulate such assertions, we define .xI d; a/ to be the number of primes p up to x with p a .mod d /. Theorem 5.21. Let a; d 2 Z with d > 0 and gcd.a; d / D 1. Then x .xI d; a/ : .d / log x
5.5 The prime number theorem . . . and beyond
121
Proof. Literature—see §5.6. The above theorem is only applicable in the case where d and a are fixed as x ! 1. For example, it says that roughly half the primes up to x are congruent to 1 modulo 4, and roughly half the primes up to x are congruent to 3 modulo 4. However, suppose d ! 1, and we want to estimate, say, the number of primes p 1 .mod d / up to d 3 . Theorem 5.21 does not help us here. The following conjecture does, however: Conjecture 5.22. Let x 2 R, a; d 2 Z with x 2, d 2, and gcd.a; d / D 1. Then ˇ li.x/ ˇˇ ˇ ˇ.xI d; a/ ˇ x 1=2 .log x C 2 log d /: .d / The above conjecture is in fact a consequence of a generalization of the Riemann hypothesis — see §5.6. This conjecture implies that for every constant ˛ < 1=2, if 2 d x ˛ , then .xI d; a/ is closely approximated by li.x/= .d / (see Exercise 5.24). It can also be used to get an upper bound on the least prime p a .mod d / (see Exercise 5.25). The following theorem is the best rigorously proven upper bound on the smallest prime in an arithmetic progression: Theorem 5.23. There exists a constant c such that for all a; d 2 Z with d 2 and gcd.a; d / D 1, the least prime p a .mod d / is at most cd 11=2 . Proof. Literature—see §5.6. E XERCISE 5.24. Assuming Conjecture 5.22, show that for all ˛; , with 0 < ˛ < 1=2 and 0 < < 1, there exists an x0 , such that for all x > x0 , for all d 2 Z with 2 d x ˛ , and for all a 2 Z relatively prime to d , the number of primes p x such that p a .mod d / is at least .1 / li.x/= .d / and at most .1 C / li.x/= .d /. E XERCISE 5.25. Assuming Conjecture 5.22, show that there exists a constant c such that for all a; d 2 Z with d 2 and gcd.a; d / D 1, the least prime p a .mod d / is at most c .d /2 .log d /4 . 5.5.5 Sophie Germain primes A Sophie Germain prime is a prime p such that 2p C1 is also prime. Such primes are actually useful in a number of practical applications, and so we discuss them briefly here. It is an open problem to prove (or disprove) that there are infinitely many Sophie Germain primes. However, numerical evidence, and heuristic arguments, strongly
122
The distribution of primes
suggest not only that there are infinitely many such primes, but also a fairly precise estimate on the density of such primes. Let .x/ denote the number of Sophie Germain primes up to x. Conjecture 5.24. We have .x/ C
x ; .log x/2
where C is the constant C WD 2
Y q.q 2/ 1:32032; .q 1/2
q>2
and the product is over all primes q > 2. The above conjecture is a special case of the following, more general conjecture. Conjecture 5.25 (Dickson’s conjecture). Let .a1 ; b1 /; : : : ; .ak ; bk / be distinct pairs of integers, where each ai is positive. Let P .x/ be the number of positive integers m up to x such that ai m C bi are simultaneously prime for i D 1; : : : ; k. For each prime p, let !.p/ be the number of integers m 2 f0; : : : ; p 1g that satisfy k Y
.ai m C bi / 0 .mod p/:
i D1
If !.p/ < p for each prime p, then P .x/ D
x ; .log x/k
where D WD
Y1 p
!.p/=p ; .1 1=p/k
the product being over all primes p. In Exercise 5.26 below, you are asked to verify that the quantity D appearing in Conjecture 5.25 satisfies 0 < D < 1. Conjecture 5.24 is implied by Conjecture 5.25 with k WD 2, .a1 ; b1 / WD .1; 0/, and .a2 ; b2 / WD .2; 1/; in this case, !.2/ D 1 and !.p/ D 2 for all p > 2. The above conjecture also includes (a strong version of) the famous twin primes conjecture as a special case: the number of primes p up to x such that p C 2 is also prime is C x=.log x/2 , where C is the same constant as in Conjecture 5.24. A heuristic argument in favor of Conjecture 5.25 runs as follows. In some sense, the chance that a large positive integer m is prime is about 1= log m. Since
5.6 Notes
123
log.ai m C bi / log m, the chance that a1 m C b1 ; : : : ; ak m C bk are all prime should be about 1=.log m/k . But this ignores the fact that a1 m C b1 ; : : : ; ak m C bk are not quite random integers. For each prime p, we must apply a “correction factor” rp =sp , where rp is the chance that for random m, none of a1 mCb1 ; : : : ; ak mC bk is divisible by p, and sp is the chance that for k truly random, large integers, none of them is divisible by p. One sees that rp D 1 !.p/=p and sp D .1 1=p/k . This implies (using §A5 and Exercise 5.6) that P .x/ should be about Z x X D 1=.log m/k D dt =.log t/k Dx=.log x/k : mx
2
Although Conjecture 5.25 is well supported by numerical evidence, there seems little hope of it being proved any time soon, even under the Riemann hypothesis or any of its generalizations. E XERCISE 5.26. Show that the quantity D appearing in Conjecture 5.25 satisfies 0 < D < 1. Hint: first show that !.p/ D k for all sufficiently large p. E XERCISE 5.27. Derive Theorem 5.21 from Conjecture 5.25. E XERCISE 5.28. Show that the constant C appearing in Conjecture 5.24 satisfies 2C D B2 =B12 ; where B1 and B2 are the constants from Exercises 5.12 and 5.13. 5.6 Notes The prime number theorem was conjectured by Gauss in 1791. It was proven independently in 1896 by Hadamard and de la Vallée Poussin. A proof of the prime number theorem may be found, for example, in the book by Hardy and Wright [45]. Theorem 5.19, as well as the estimates for the constants A, B1 , and B2 mentioned in that theorem and Exercises 5.11, 5.12, and 5.13, are from Rosser and Schoenfeld [81]. Theorem 5.15 is from Walfisz [100]. Theorem 5.17, which made the first connection between the theory of prime numbers and the zeta function, was discovered in the 18th century by Euler. The Riemann hypothesis was made by Riemann in 1859, and to this day, remains one of the most vexing conjectures in mathematics. Riemann in fact showed that his conjecture about the zeros of the zeta function is equivalent to the conjecture that
124
The distribution of primes
for each fixed > 0, .x/ D li.x/ C O.x 1=2C /. This was strengthened by von Koch in 1901, who showed that the Riemann hypothesis is true if and only if .x/ D li.x/ C O.x 1=2 log x/. See Chapter 1 of the book by Crandall and Pomerance [30] for more on the connection between the Riemann hypothesis and the theory of prime numbers; in particular, see Exercise 1.36 in that book for an outline of a proof that Conjecture 5.16 follows from the Riemann hypothesis. A warning: some authors (and software packages) define the logarithmic integral using the interval of integration .0; x/, rather than .2; x/, which increases its value by a constant c 1:0452. Theorem 5.20 was proved by Dirichlet in 1837, while Theorem 5.21 was proved by de la Vallée Poussin in 1896. A result of Oesterlé [71] implies that Conjecture 5.22 for d 3 is a consequence of an assumption about the location of the zeros of certain generalizations of Riemann’s zeta function; the case d D 2 follows from the bound in Conjecture 5.16 under the ordinary Riemann hypothesis. Theorem 5.23 is from Heath-Brown [46]. The bound in Exercise 5.25 can be improved to c .d /2 .log d /2 (see Theorem 8.5.8 of [11]). Conjecture 5.25 originates from Dickson [33]. In fact, Dickson only conjectured that the quantity P .x/ defined in Conjecture 5.25 tends to infinity. The conjectured formula for the rate of growth of P .x/ is a special case of a more general conjecture stated by Bateman and Horn [12], which generalizes various, more specific conjectures stated by Hardy and Littlewood [44]. For the reader who is interested in learning more on the topics discussed in this chapter, we recommend the books by Apostol [8] and Hardy and Wright [45]; indeed, many of the proofs presented in this chapter are minor variations on proofs from these two books. Our proof of Bertrand’s postulate is based on the presentation in Section 9.2 of Redmond [78]. See also Bach and Shallit [11] (especially Chapter 8), as well as Crandall and Pomerance [30] (especially Chapter 1), for a more detailed overview of these topics. The data in Tables 5.1 and 5.2 was obtained using the computer program Maple.
6 Abelian groups
This chapter introduces the notion of an abelian group. This is an abstraction that models many different algebraic structures, and yet despite the level of generality, a number of very useful results can be easily obtained. 6.1 Definitions, basic properties, and examples Definition 6.1. An abelian group is a set G together with a binary operation ? on G such that (i) for all a; b; c 2 G, a ? .b ? c/ D .a ? b/ ? c (i.e., ? is associative), (ii) there exists e 2 G (called the identity element) such that for all a 2 G, a ? e D a D e ? a, (iii) for all a 2 G there exists a0 2 G (called the inverse of a) such that a ?a0 D e D a0 ? a, (iv) for all a; b 2 G, a ? b D b ? a (i.e., ? is commutative). While there is a more general notion of a group, which may be defined simply by dropping property (iv) in Definition 6.1, we shall not need this notion in this text. The restriction to abelian groups helps to simplify the discussion significantly. Because we will only be dealing with abelian groups, we may occasionally simply say “group” instead of “abelian group.” Before looking at examples, let us state some very basic properties of abelian groups that follow directly from the definition: Theorem 6.2. Let G be an abelian group with binary operation ?. Then we have: (i) G contains only one identity element; (ii) every element of G has only one inverse.
125
126
Abelian groups
Proof. Suppose e; e 0 are both identities. Then we have e D e ? e0 D e0; where we have used part (ii) of Definition 6.1, once with e 0 as the identity, and once with e as the identity. That proves part (i) of the theorem. To prove part (ii) of the theorem, let a 2 G, and suppose that a has two inverses, 0 a and a00 . Then using parts (i)–(iii) of Definition 6.1, we have a0 D a0 ? e (by part (ii)) D a0 ? .a ? a00 / (by part (iii) with inverse a00 of a) D .a0 ? a/ ? a00 (by part (i)) D e ? a00 (by part (iii) with inverse a0 of a) D a00 (by part (ii)). These uniqueness properties justify use of the definite article in Definition 6.1 in conjunction with the terms “identity element” and “inverse.” Note that we never used part (iv) of the definition in the proof of the above theorem. Abelian groups are lurking everywhere, as the following examples illustrate. Example 6.1. The set of integers Z under addition forms an abelian group, with 0 being the identity, and a being the inverse of a 2 Z. Example 6.2. For each integer n, the set nZ D fnz W z 2 Zg under addition forms an abelian group, again, with 0 being the identity, and n. z/ being the inverse of nz. Example 6.3. The set of non-negative integers under addition does not form an abelian group, since additive inverses do not exist for positive integers. Example 6.4. The set of integers under multiplication does not form an abelian group, since inverses do not exist for integers other than ˙1. Example 6.5. The set of integers f˙1g under multiplication forms an abelian group, with 1 being the identity, and 1 its own inverse. Example 6.6. The set of rational numbers Q D fa=b W a; b 2 Z; b ¤ 0g under addition forms an abelian group, with 0 being the identity, and . a/=b being the inverse of a=b. Example 6.7. The set of non-zero rational numbers Q under multiplication forms an abelian group, with 1 being the identity, and b=a being the inverse of a=b. Example 6.8. The set Zn under addition forms an abelian group, where Œ0n is the identity, and where Œ an is the inverse of Œan .
6.1 Definitions, basic properties, and examples
127
Example 6.9. The set Zn of residue classes Œan with gcd.a; n/ D 1 under multiplication forms an abelian group, where Œ1n is the identity, and if b is a multiplicative inverse of a modulo n, then Œbn is the inverse of Œan . Example 6.10. For every positive integer n, the set of n-bit strings under the “exclusive or” operation forms an abelian group, where the “all zero” bit string is the identity, and every bit string is its own inverse. Example 6.11. The set F of all arithmetic functions f , such that f .1/ ¤ 0, and with the Dirichlet product as the binary operation (see §2.9) forms an abelian group. The special function ı is the identity, and inverses are guaranteed by Exercise 2.54. Example 6.12. The set of all finite bit strings under concatenation does not form an abelian group. Although concatenation is associative and the empty string acts as an identity element, inverses do not exist (except for the empty string), nor is concatenation commutative. Example 6.13. The set of 2 2 integer matrices with determinant ˙1, together with the binary operation of matrix multiplication, is an example of a non-abelian group; that is, it satisfies properties (i)–(iii) of Definition 6.1, but not property (iv). Example 6.14. The set of all permutations on a given set of size n 3, together with the binary operation of function composition, is another example of a nonabelian group (for n D 1; 2, it is an abelian group). Consider an abelian group G with binary operation ?. Since the group operation is associative, for all a1 ; : : : ; ak 2 G, we may write a1 ? ? ak without parentheses, and there can be no ambiguity as to the value of such an expression: any explicit parenthesization of this expression yields the same value. Furthermore, since the group operation is commutative, reordering the ai ’s does not change this value. Note that in specifying a group, one must specify both the underlying set G as well as the binary operation; however, in practice, the binary operation is often implicit from context, and by abuse of notation, one often refers to G itself as the group. For example, when talking about the abelian groups Z and Zn , it is understood that the group operation is addition, while when talking about the abelian group Zn , it is understood that the group operation is multiplication. Typically, instead of using a special symbol like “?” for the group operation, one uses the usual addition (“C”) or multiplication (“”) operations. Additive notation. If an abelian group G is written additively, using “C” as the
128
Abelian groups
group operation, then the identity element is denoted by 0G , and the inverse of an element a 2 G is denoted by a. For a; b 2 G, a b denotes a C . b/. Multiplicative notation. If an abelian group G is written multiplicatively, using “” as the group operation, then the identity element is denoted by 1G , and the inverse of an element a 2 G is denoted by a 1 . As usual, one may write ab in place of a b. Also, one may write a=b for ab 1 . For any particular, concrete abelian group, the most natural choice of notation is clear (e.g., addition for Z and Zn , multiplication for Zn ); however, for a “generic” group, the choice is largely a matter of taste. By convention, whenever we consider a “generic” abelian group, we shall use additive notation for the group operation, unless otherwise specified. The next theorem states a few simple but useful properties of abelian groups (stated using our default, additive notation). Theorem 6.3. Let G be an abelian group. Then for all a; b; c 2 G, we have: (i) if a C b D a C c, then b D c; (ii) the equation a C x D b has a unique solution x 2 G; (iii)
.a C b/ D . a/ C . b/;
(iv)
. a/ D a.
Proof. These statements all follow easily from Definition 6.1 and Theorem 6.2. For (i), just add a to both sides of the equation a C b D a C c. For (ii), the solution is x D b a. For (iii), we have .a C b/ C .. a/ C . b// D .a C . a// C .b C . b// D 0G C 0G D 0G ; which shows that . a/ C . b/ is indeed the inverse of a C b. For (iv), we have . a/ C a D 0G , which means that a is the inverse of a. Part (i) of the above theorem is the cancellation law for abelian groups. P If a1 ; : : : ; ak are elements of an abelian group G, we naturally write kiD1 ai for their sum. By convention, the sum is 0G when k D 0. Part (iii) of Theorem 6.3 Pk Pk obviously generalizes, so that i D1 ai D i D1 . ai /. In the special case where P all the ai ’s have the same value a, we define k a WD kiD1 a. In particular, the inverse of k a is k . a/, which we may write as . k/a. Thus, the notation k a is defined for all integers k. Observe that by definition, 1 a D a and . 1/ a D a. Theorem 6.4. Let G be an abelian group. Then for all a; b 2 G and k; ` 2 Z, we have: (i) k.`a/ D .k`/a D `.ka/; (ii) .k C `/a D ka C `a;
6.1 Definitions, basic properties, and examples
129
(iii) k.a C b/ D ka C kb. Proof. The proof of this is easy, but tedious. We leave the details as an exercise to the reader. Multiplicative notation: It is perhaps helpful to translate the above discussion from additive to multiplicative notation. If a group G is written using multiplicative notation, then Theorem 6.3 says that (i) ab D ac implies b D c, (ii) ax D b has a unique solution, (iii) .ab/ 1 D a 1 b 1 , and (iv) .a 1 / 1 D a. Q If a1 ; : : : ; ak 2 G, we write their product as kiD1 ai , which is 1G when k D 0. Q Q Q We have . kiD1 ai / 1 D kiD1 ai 1 . We define ak WD kiD1 a, and we have .ak / 1 D .a 1 /k , which we may write as a k . Theorem 6.4 says that (i) .a` /k D ak` D .ak /` , (ii) akC` D ak a` , and (iii) .ab/k D ak b k . An abelian group G may be infinite or finite. If the group is finite, we define its order to be the number of elements in the underlying set G; otherwise, we say that the group has infinite order. Example 6.15. The order of the additive group Zn is n. Example 6.16. The order of the multiplicative group Zn is .n/, where is Euler’s phi function, defined in §2.6. Example 6.17. The additive group Z has infinite order. We close this section with two simple constructions for combining groups to build new groups. Example 6.18. If G1 ; : : : ; Gk are abelian groups, we can form the direct product H WD G1 Gk , which consists of all k-tuples .a1 ; : : : ; ak / with a1 2 G1 ; : : : ; ak 2 Gk . We can view H in a natural way as an abelian group if we define the group operation component-wise: .a1 ; : : : ; ak / C .b1 ; : : : ; bk / WD .a1 C b1 ; : : : ; ak C bk /: Of course, the groups G1 ; : : : ; Gk may be different, and the group operation applied in the i th component corresponds to the group operation associated with Gi . We leave it to the reader to verify that H is in fact an abelian group, where 0H D .0G1 ; : : : ; 0Gk / and .a1 ; : : : ; ak / D . a1 ; : : : ; ak /. As a special case, if G D G1 D D Gk , then the k-wise direct product of G is denoted G k . Example 6.19. Let G be an abelian group. An element .a1 ; : : : ; ak / of G k may be identified with the function f W f1; : : : ; kg ! G given by f .i / D ai for i D 1; : : : ; k. We can generalize this, replacing f1; : : : ; kg by an arbitrary set I . We define Map.I; G/ to be the set of all functions f W I ! G, which we naturally
130
Abelian groups
view as a group by defining the group operation point-wise: for f; g 2 Map.I; G/, we define .f C g/.i / WD f .i / C g.i / for all i 2 I : Again, we leave it to the reader to verify that Map.I; G/ is an abelian group, where the identity element is the function that maps each i 2 I to 0G , and for f 2 Map.I; G/, we have . f /.i / D .f .i // for all i 2 I . E XERCISE 6.1. For a finite abelian group, one can completely specify the group by writing down the group operation table. For instance, Example 2.6 presented an addition table for Z6 . (a) Write down group operation tables for the following finite abelian groups: Z5 , Z5 , and Z3 Z4 . (b) Show that the group operation table for every finite abelian group is a Latin square; that is, each element of the group appears exactly once in each row and column. (c) Below is an addition table for an abelian group that consists of the elements fa; b; c; d g; however, some entries are missing. Fill in the missing entries. C a b c d
a a b
b
c
d
a a
E XERCISE 6.2. For a; b 2 R, define a ? b WD ab x > 1g. Show that:
a
b C 2. Let G WD fx 2 R W
(a) G is closed under ?; (b) the set G under the operation ? forms an abelian group. E XERCISE 6.3. Let G be an abelian group, and let g be an arbitrary, fixed element of G. Assume that the group operation of G is written additively. We define a new binary operation ˇ on G, as follows: for a; b 2 G, let a ˇ b WD a C b C g. Show that the set G under ˇ forms an abelian group. E XERCISE 6.4. Let G be a finite abelian group of even order. Show that there exists a 2 G with a ¤ 0G and 2a D 0G . E XERCISE 6.5. Let ? be a binary operation on a finite set G. Assume that ? is associative, commutative, and satisfies the cancellation law: a ? b D a ? c implies b D c. Show that G under ? forms an abelian group.
6.2 Subgroups
131
E XERCISE 6.6. Show that the result of the previous exercise need not hold if G is infinite. 6.2 Subgroups We next introduce the notion of a subgroup. Definition 6.5. Let G be an abelian group, and let H be a non-empty subset of G such that (i) a C b 2 H for all a; b 2 H , and (ii) a 2 H for all a 2 H . Then H is called a subgroup of G. In words: H is a subgroup of G if it is closed under the group operation and taking inverses. Multiplicative notation: if the abelian group G in the above definition is written using multiplicative notation, then H is a subgroup if ab 2 H and a 1 2 H for all a; b 2 H . Theorem 6.6. If G is an abelian group, and H is a subgroup of G, then H contains 0G ; moreover, the binary operation of G, when restricted to H , yields a binary operation that makes H into an abelian group whose identity is 0G . Proof. First, to see that 0G 2 H , just pick any a 2 H , and using both properties of the definition of a subgroup, we see that 0G D a C . a/ 2 H . Next, note that by property (i) of Definition 6.5, H is closed under addition, which means that the restriction of the binary operation “C” on G to H induces a well defined binary operation on H . So now it suffices to show that H , together with this operation, satisfies the defining properties of an abelian group. Associativity and commutativity follow directly from the corresponding properties for G. Since 0G acts as the identity on G, it does so on H as well. Finally, property (ii) of Definition 6.5 guarantees that every element a 2 H has an inverse in H , namely, a. Clearly, for an abelian group G, the subsets G and f0G g are subgroups. These are not very interesting subgroups. Other, more interesting subgroups may sometimes be found by using the following two theorems. Theorem 6.7. Let G be an abelian group, and let m be an integer. Then mG WD fma W a 2 Gg is a subgroup of G. Proof. mG is non-empty, since 0G D m0G 2 mG. For ma; mb 2 mG, we have ma C mb D m.a C b/ 2 mG, and .ma/ D m. a/ 2 mG.
132
Abelian groups
Theorem 6.8. Let G be an abelian group, and let m be an integer. Then Gfmg WD fa 2 G W ma D 0G g is a subgroup of G. Proof. Gfmg is non-empty, since m0G D 0G , and so Gfmg contains 0G . If ma D 0G and mb D 0G , then m.a C b/ D ma C mb D 0G C 0G D 0G and m. a/ D .ma/ D 0G D 0G . Multiplicative notation: if the abelian group G in the above two theorems is written using multiplicative notation, then we write the subgroup of the first theorem as G m WD fam W a 2 Gg. The subgroup in the second theorem is denoted in the same way: Gfmg WD fa 2 G W am D 1G g. Example 6.20. We already proved that .Zn /m is a subgroup of Zn in Theorem 2.16. Also, the proof of Theorem 2.17 clearly works for an arbitrary abelian group G: for each a 2 G, and all `; m 2 Z with gcd.`; m/ D 1, if `a 2 mG, then a 2 mG. Example 6.21. Let p be an odd prime. Then as we saw in Theorem 2.20, .Zp /2 is a subgroup of Zp of order .p 1/=2, and as we saw in Theorem 2.18, Zp f2g D fŒ˙1g. Example 6.22. For every integer m, the set mZ is the subgroup of the additive group Z consisting of all multiples of m. This is the same as the ideal of Z generated by m, which we already studied in some detail in §1.2. Two such subgroups mZ and m0 Z are equal if and only if m D ˙m0 . The subgroup Zfmg is equal to Z if m D 0, and is equal to f0g otherwise. Example 6.23. Let n be a positive integer, let m 2 Z, and consider the subgroup mZn of the additive group Zn . Now, for every residue class Œz 2 Zn , we have mŒz D Œmz. Therefore, Œb 2 mZn if and only if there exists z 2 Z such that mz b .mod n/. By Theorem 2.5, such a z exists if and only if d j b, where d WD gcd.m; n/. Thus, mZn consists precisely of the n=d distinct residue classes Œi d .i D 0; : : : ; n=d
1/;
and in particular, mZn D d Zn . Now consider the subgroup Zn fmg of Zn . The residue class Œz is in Zn fmg if and only if mz 0 .mod n/. By Theorem 2.5, this happens if and only if z 0 .mod n=d /, where d WD gcd.m; n/ as above. Thus, Zn fmg consists precisely of the d residue classes Œi n=d .i D 0; : : : ; d and in particular, Zn fmg D Zn fd g D .n=d /Zn .
1/;
6.2 Subgroups
133
Example 6.24. For n D 15, consider again the table in Example 2.2. For m D 1; 2; 3; 4; 5; 6, the elements appearing in the mth row of that table form the subgroup mZn of Zn , and also the subgroup Zn fn=d g, where d WD gcd.m; n/. Because the abelian groups Z and Zn are of such importance, it is a good idea to completely characterize all subgroups of these abelian groups. As the following two theorems show, the subgroups in Examples 6.22 and 6.23 are the only subgroups of these groups. Theorem 6.9. If G is a subgroup of Z, then there exists a unique non-negative integer m such that G D mZ. Moreover, for two non-negative integers m1 and m2 , we have m1 Z m2 Z if and only if m2 j m1 . Proof. Actually, we have already proven this. One only needs to observe that a subset G of Z is a subgroup if and only if it is an ideal of Z, as defined in §1.2 (see Exercise 1.8). The first statement of the theorem then follows from Theorem 1.6. The second statement follows easily from the definitions, as was observed in §1.2. Theorem 6.10. If G is a subgroup of Zn , then there exists a unique positive integer d dividing n such that G D d Zn . Also, for positive divisors d1 ; d2 of n, we have d1 Zn d2 Zn if and only if d2 j d1 . Proof. Note that the second statement implies the uniqueness part of the first statement, so it suffices to prove just the existence part of the first statement and the second statement. Let G be an arbitrary subgroup of Zn , and let H WD fz 2 Z W Œz 2 Gg. We claim that H is a subgroup of Z. To see this, observe that if a; b 2 H , then Œa and Œb belong to G, and hence so do Œa C b D Œa C Œb and Œ a D Œa, and thus a C b and a belong to H . That proves the claim, and Theorem 6.9 implies that H D d Z for some non-negative integer d . It follows that G D fŒy W y 2 H g D fŒdz W z 2 Zg D d Zn : Evidently, n 2 H D d Z, and hence d j n. That proves the existence part of the first statement of the theorem. To prove the second statement of the theorem, observe that if d1 and d2 are arbitrary integers, then d1 Zn d2 Zn ” d2 z d1 .mod n/ for some z 2 Z ” gcd.d2 ; n/ j d1 (by Theorem 2.5): In particular, if d2 j n, then gcd.d2 ; n/ D d2 , which proves the second statement.
134
Abelian groups
Of course, not all abelian groups have such a simple subgroup structure. Example 6.25. Consider the group G D Z2 Z2 . For every non-zero ˛ 2 G, ˛ C ˛ D 0G . From this, it is clear that the set H D f0G ; ˛g is a subgroup of G. However, for every integer m, mG D G if m is odd, and mG D f0G g if m is even. Thus, the subgroup H is not of the form mG for any m. Example 6.26. Consider the group Z15 . We can enumerate its elements as Z15 D fŒ˙1; Œ˙2; Œ˙4; Œ˙7g. Therefore, the elements of .Z15 /2 are Œ12 D Œ1; Œ22 D Œ4; Œ42 D Œ16 D Œ1; Œ72 D Œ49 D Œ4I thus, .Z15 /2 has order 2, consisting as it does of the two distinct elements Œ1 and Œ4. Going further, one sees that .Z15 /4 D fŒ1g. Thus, ˛ 4 D Œ1 for all ˛ 2 Z15 . By direct calculation, one can determine that .Z15 /3 D Z15 ; that is, cubing simply permutes Z15 . For any given integer m, write m D 4q C r, where 0 r < 4. Then for every ˛ 2 Z15 , we have ˛ m D ˛ 4qCr D ˛ 4q ˛ r D ˛ r . Thus, .Z15 /m is either Z15 , .Z15 /2 , or fŒ1g. However, there are certainly other subgroups of Z15 — for example, the subgroup fŒ˙1g. Example 6.27. Consider the group Z5 D fŒ˙1; Œ˙2g. The elements of .Z5 /2 are Œ12 D Œ1; Œ22 D Œ4 D Œ 1I thus, .Z5 /2 D fŒ˙1g and has order 2. There are in fact no other subgroups of Z5 besides Z5 , fŒ˙1g, and fŒ1g. Indeed, if H is a subgroup containing Œ2, then we must have H D Z5 : Œ2 2 H implies Œ22 D Œ4 D Œ 1 2 H , which implies Œ 2 2 H as well. The same holds if H is a subgroup containing Œ 2. Example 6.28. Consider again the abelian group F of arithmetic functions f , such that f .1/ ¤ 0, and with the Dirichlet product as the binary operation, as discussed in Example 6.11. Exercises 2.48 and 2.55 imply that the subset of all multiplicative functions is a subgroup. We close this section with two theorems that provide useful ways to build new subgroups out of old subgroups. Theorem 6.11. If H1 and H2 are subgroups of an abelian group G, then so is H1 C H2 WD fa1 C a2 W a1 2 H1 ; a2 2 H2 g:
135
6.2 Subgroups
Proof. It is evident that H1 C H2 is non-empty, as it contains 0G C 0G D 0G . Consider two elements in H1 C H2 , which we can write as a1 C a2 and b1 C b2 , where a1 ; b1 2 H1 and a2 ; b2 2 H2 . Then by the closure properties of subgroups, a1 C b1 2 H1 and a2 C b2 2 H2 , and hence .a1 C a2 / C .b1 C b2 / D .a1 C b1 / C .a2 C b2 / 2 H1 C H2 . Similarly, .a1 C a2 / D . a1 / C . a2 / 2 H1 C H2 . Multiplicative notation: if the abelian group G in the above theorem is written multiplicatively, then the subgroup defined in the theorem is written H1 H2 WD fa1 a2 W a1 2 H1 ; a2 2 H2 g. Theorem 6.12. If H1 and H2 are subgroups of an abelian group G, then so is H1 \ H2 . Proof. It is evident that H1 \ H2 is non-empty, as both H1 and H2 contain 0G , and hence so does their intersection. If a 2 H1 \ H2 and b 2 H1 \ H2 , then since a; b 2 H1 , we have a C b 2 H1 , and since a; b 2 H2 , we have a C b 2 H2 ; therefore, a C b 2 H1 \ H2 . Similarly, a 2 H1 and a 2 H2 , and therefore, a 2 H1 \ H2 . Let G be an abelian group and H1 ; H2 ; H3 subgroups of G. The reader may verify that H1 C H2 D H2 C H1 and .H1 C H2 / C H3 D H1 C .H2 C H3 /. It follows that if H1 ; : : : ; Hk are subgroups of G, then we can write H1 C C Hk without any parentheses, and there can be no ambiguity; moreover, the order of the Hi ’s does not matter. The same holds with “C” replaced by “\.” A warning: If H is a subgroup of an abelian group G, then in general, we have H C H ¤ 2H . For example, Z C Z D Z, while 2Z ¤ Z. E XERCISE 6.7. Let G be an abelian group. (a) Show that if H is a subgroup of G and a; b 2 H , then a (b) Suppose that H is a non-empty subset of G such that a a; b 2 H . Show that H is a subgroup of G.
b 2 H. b 2 H for all
E XERCISE 6.8. Let G be an abelian group. (a) Show that if H is a subgroup of G, h 2 H , and g 2 G n H , then h C g 2 G n H. (b) Suppose that H is a non-empty subset of G such that for all h; g 2 G: (i) h 2 H implies h 2 H , and (ii) h 2 H and g 2 G n H implies h C g 2 G n H . Show that H is a subgroup of G. E XERCISE 6.9. Show that if H is a subgroup of an abelian group G, then a set K H is a subgroup of G if and only if K is a subgroup of H .
136
Abelian groups
E XERCISE 6.10. Let G be an abelian group with subgroups H1 and H2 . Show that every subgroup H of G that contains H1 [ H2 contains H1 C H2 , and H1 H2 if and only if H1 C H2 D H2 . E XERCISE 6.11. Let H1 be a subgroup of an abelian group G1 and H2 a subgroup of an abelian group G2 . Show that H1 H2 is a subgroup of G1 G2 . E XERCISE 6.12. Show that if G1 and G2 are abelian groups, and m is an integer, then m.G1 G2 / D mG1 mG2 . E XERCISE 6.13. Let G1 and G2 be abelian groups, and let H be a subgroup of G1 G2 . Define H1 WD fa1 2 G1 W .a1 ; a2 / 2 H for some a2 2 G2 g: Show that H1 is a subgroup of G1 . E XERCISE 6.14. Let I be a set and G be an abelian group, and consider the group Map.I; G/ of functions f W I ! G. Let Map# .I; G/ be the set of functions f 2 Map.I; G/ such that f .i / ¤ 0G for at most finitely many i 2 I . Show that Map# .I; G/ is a subgroup of Map.I; G/. 6.3 Cosets and quotient groups We now generalize the notion of a congruence relation. Let G be an abelian group, and let H be a subgroup of G. For a; b 2 G, we write a b .mod H / if a b 2 H . In other words, a b .mod H / if and only if a D b C h for some h 2 H . Analogously to Theorem 2.2, if we view the subgroup H as fixed, then the following theorem says that the binary relation “ .mod H /” is an equivalence relation on the set G: Theorem 6.13. Let G be an abelian group and H a subgroup of G. For all a; b; c 2 G, we have: (i) a a .mod H /; (ii) a b .mod H / implies b a .mod H /; (iii) a b .mod H / and b c .mod H / implies a c .mod H /. Proof. For (i), observe that H contains 0G D a a. For (ii), observe that if H contains a b, then it also contains .a b/ D b a. For (iii), observe that if H contains a b and b c, then it also contains .a b/ C .b c/ D a c. Since the binary relation “ .mod H /” is an equivalence relation, it partitions G into equivalence classes (see Theorem 2.1). For a 2 G, we denote the
6.3 Cosets and quotient groups
137
equivalence class containing a by ŒaH . By definition, we have x 2 ŒaH ” x a .mod H / ” x D a C h for some h 2 H ; and hence ŒaH D a C H WD fa C h W h 2 H g: It is also clear that Œ0G H D H . Historically, these equivalence classes are called cosets of H in G, and we shall adopt this terminology here as well. Any member of a coset is called a representative of the coset. Multiplicative notation: if G is written multiplicatively, then a b .mod H / means ab 1 2 H , and ŒaH D aH WD fah W h 2 H g. Example 6.29. Let G WD Z and H WD nZ for some positive integer n. Then a b .mod H / if and only if a b .mod n/. The coset ŒaH is exactly the same thing as the residue class Œan 2 Zn . Example 6.30. Let G WD Z6 , which consists of the residue classes Œ0; Œ1; Œ2; Œ3; Œ4; Œ5. Let H be the subgroup 3G D fŒ0; Œ3g of G. The coset of H containing the residue class Œ1 is Œ1 C H D fŒ1; Œ4g, and the coset of H containing the residue class Œ2 is Œ2 C H D fŒ2; Œ5g. The cosets fŒ0; Œ3g, fŒ1; Œ4g, and fŒ2; Œ5g are the only cosets of H in G, and they clearly partition the set Z6 . Note that each coset of H in G contains two elements, each of which is itself a coset of 6Z in Z (i.e., a residue classes modulo 6). In the previous example, we saw that each coset contained the same number of elements. As the next theorem shows, this was no accident. Theorem 6.14. Let G be an abelian group, and H a subgroup of G. For all a; b 2 G, the function f W G!G x 7! b
aCx
is a bijection, which when restricted to the coset ŒaH , yields a bijection from ŒaH to the coset ŒbH . In particular, every two cosets of H in G have the same cardinality. Proof. First, we claim that f is a bijection. Indeed, if f .x/ D f .x 0 /, then b a C x D b a C x 0 , and subtracting b and adding a to both sides of this equation yields x D x 0 . That proves that f is injective. To prove that f is surjective, observe that for any given x 0 2 G, we have f .a b C x 0 / D x 0 . Second, we claim that for all x 2 G, we have x 2 ŒaH if and only if f .x/ 2
138
Abelian groups
ŒbH . On the one hand, suppose that x 2 ŒaH , which means that x D a C h for some h 2 H . Subtracting a and adding b to both sides of this equation yields b a C x D b C h, which means f .x/ 2 ŒbH . Conversely, suppose that f .x/ 2 ŒbH , which means that b a C x D b C h for some h 2 H . Subtracting b and adding a to both sides of this equation yields x D a C h, which means that x 2 ŒaH . The theorem is now immediate from these two claims. An incredibly useful consequence of the above theorem is: Theorem 6.15 (Lagrange’s theorem). If G is a finite abelian group, and H is a subgroup of G, then the order of H divides the order of G. Proof. This is an immediate consequence of the previous theorem, and the fact that the cosets of H in G partition G. Analogous to Theorem 2.3, we have: Theorem 6.16. Let G be an abelian group and H a subgroup. For all a; a0 ; b; b 0 2 G, if a a0 .mod H / and b b 0 .mod H /, then a C b a0 C b 0 .mod H /. Proof. Now, a a0 .mod H / and b b 0 .mod H / means that a D a0 C x and b D b 0 C y for some x; y 2 H . Therefore, a C b D .a0 C x/ C .b 0 C y/ D .a0 Cb 0 /C.x Cy/, and since x Cy 2 H , this means that aCb a0 Cb 0 .mod H /. Let G be an abelian group and H a subgroup. Let G=H denote the set of all cosets of H in G. Theorem 6.16 allows us to define a binary operation on G=H in the following natural way: for a; b 2 G, define ŒaH C ŒbH WD Œa C bH : The fact that this definition is unambiguous follows immediately from Theorem 6.16: if ŒaH D Œa0 H and ŒbH D Œb 0 H , then Œa C bH D Œa0 C b 0 H . We can easily verify that this operation makes G=H into an abelian group. We need to check that the four properties of Definition 6.1 are satisfied: (i) Associativity: ŒaH C .ŒbH C ŒcH / D ŒaH C Œb C cH D Œa C .b C c/H D Œ.a C b/ C cH D Œa C bH C ŒcH D .ŒaH C ŒbH / C ŒcH : Here, we have used the definition of addition of cosets, and the corresponding associativity property for G.
6.3 Cosets and quotient groups
139
(ii) Identity element: the coset Œ0G H D H acts as the identity element, since ŒaH C Œ0G H D Œa C 0G H D ŒaH D Œ0G C aH D Œ0G H C ŒaH : (iii) Inverses: the inverse of the coset ŒaH is Œ aH , since ŒaH C Œ aH D Œa C . a/H D Œ0G H D Œ. a/ C aH D Œ aH C ŒaH : (iv) Commutativity: ŒaH C ŒbH D Œa C bH D Œb C aH D ŒbH C ŒaH : The group G=H is called the quotient group of G modulo H . The order of the group G=H is sometimes denoted ŒG W H and is called the index of H in G. Note that if H D G, then the quotient group G=H consists of just a single element, and so ŒG W H D 1. Multiplicative notation: if G is written multiplicatively, then the definition of the group operation of G=H is expressed ŒaH ŒbH WD Œa bH ; the identity element of G=H is Œ1G H D H , and the inverse of ŒaH is Œa 1 H . Theorem 6.17. Let G be a finite abelian group and H a subgroup. Then ŒG W H D jGj=jH j. Moreover, if K is a subgroup of H , then ŒG W K D ŒG W H ŒH W K: Proof. The fact that ŒG W H D jGj=jH j follows directly from Theorem 6.14. The fact that ŒG W K D ŒG W H ŒH W K follows from a simple calculation: ŒG W H D
jGj jGj=jKj ŒG W K D D : jH j jH j=jKj ŒH W K
Example 6.31. For each n 1, the group Zn is precisely the quotient group Z=nZ. Example 6.32. Continuing with Example 6.30, let G WD Z6 and H WD 3G D fŒ0; Œ3g. The quotient group G=H has order 3, and consists of the cosets ˛ WD fŒ0; Œ3g; ˇ WD fŒ1; Œ4g; WD fŒ2; Œ5g: If we write out an addition table for G, grouping together elements in cosets of H in G, then we also get an addition table for the quotient group G=H :
140
Abelian groups
C Œ0 Œ3 Œ1 Œ4 Œ2 Œ5
Œ0 Œ0 Œ3 Œ1 Œ4 Œ2 Œ5
Œ3 Œ3 Œ0 Œ4 Œ1 Œ5 Œ2
Œ1 Œ1 Œ4 Œ2 Œ5 Œ3 Œ0
Œ4 Œ4 Œ1 Œ5 Œ2 Œ0 Œ3
Œ2 Œ2 Œ5 Œ3 Œ0 Œ4 Œ1
Œ5 Œ5 Œ2 Œ0 Œ3 Œ1 Œ4
This table illustrates quite graphically the point of Theorem 6.16: for every two cosets, if we take any element from the first and add it to any element of the second, we always end up in the same coset. We can also write down just the addition table for G=H : C ˛ ˇ
˛ ˛ ˇ
ˇ ˇ
˛
˛ ˇ
Note that by replacing ˛ with Œ03 , ˇ with Œ13 , and with Œ23 , the addition table for G=H becomes the addition table for Z3 . In this sense, we can view G=H as essentially just a “renaming” of Z3 . Example 6.33. Let us return to Example 6.26. The multiplicative group Z15 , as we saw, is of order 8. The subgroup .Z15 /2 of Z15 has order 2. Therefore, the quotient group Z15 =.Z15 /2 has order 4. Indeed, the cosets are ˛00 WD .Z15 /2 D fŒ1; Œ4g;
˛01 WD Œ 1.Z15 /2 D fŒ 1; Œ 4g;
˛10 WD Œ2.Z15 /2 D fŒ2; Œ 7g;
˛11 WD Œ 2.Z15 /2 D fŒ 2; Œ7g:
We can write down the multiplication table for the quotient group: ˛00 ˛01 ˛10 ˛11
˛00 ˛00 ˛01 ˛10 ˛11
˛01 ˛01 ˛00 ˛11 ˛10
˛10 ˛10 ˛11 ˛00 ˛01
˛11 ˛11 ˛10 ˛01 ˛00
Note that this group is essentially just a “renaming” of the additive group Z2 Z2 . Example 6.34. As we saw in Example 6.27, .Z5 /2 D fŒ˙1g. Therefore, the quotient group Z5 =.Z5 /2 has order 2. The cosets of .Z5 /2 in Z5 are ˛0 WD fŒ˙1g and ˛1 WD fŒ˙2g, and the multiplication table looks like this:
6.4 Group homomorphisms and isomorphisms
˛0 ˛1
˛0 ˛0 ˛1
141
˛1 ˛1 ˛0
We see that the quotient group is essentially just a “renaming” of Z2 . E XERCISE 6.15. Write down the cosets of .Z35 /2 in Z35 , along with the multiplication table for the quotient group Z35 =.Z35 /2 . E XERCISE 6.16. Let n be an odd, positive integer whose factorization into primes is n D p1e1 prer . Show that ŒZn W .Zn /2 D 2r . E XERCISE 6.17. Let n be a positive integer, and let m be any integer. Show that ŒZn W mZn D n= gcd.m; n/. E XERCISE 6.18. Let G be an abelian group and H a subgroup with ŒG W H D 2. Show that if a; b 2 G n H , then a C b 2 H . E XERCISE 6.19. Let H be a subgroup of an abelian group G, and let a; b 2 G with a b .mod H /. Show that ka kb .mod H / for all k 2 Z. E XERCISE 6.20. Let G be an abelian group, and let be an equivalence relation on G. Further, suppose that for all a; a0 ; b 2 G, if a a0 , then a C b a0 C b. Let H WD fa 2 G W a 0G g. Show that H is a subgroup of G, and that for all a; b 2 G, we have a b if and only if a b .mod H /. E XERCISE 6.21. Let H be a subgroup of an abelian group G, and let a; b 2 G. Show that Œa C bH D fx C y W x 2 ŒaH ; y 2 ŒbH g. 6.4 Group homomorphisms and isomorphisms In this section, we study maps that relate the structure of one group to another. Such maps are often very useful, as they may allow us to transfer hard-won knowledge about one group to another, perhaps more mysterious, group. Definition 6.18. A group homomorphism is a function from an abelian group G to an abelian group G 0 such that .a C b/ D .a/ C .b/ for all a; b 2 G. Note that in the equality .a C b/ D .a/ C .b/ in the above definition, the addition on the left-hand side is taking place in the group G while the addition on the right-hand side is taking place in the group G 0 . Two sets play a critical role in understanding a group homomorphism W G ! G 0 . The first set is the image of , that is, the set .G/ D f.a/ W a 2 Gg. The second set is the kernel of , defined as the set of all elements of G that are mapped
142
Abelian groups
to 0G 0 by , that is, the set 1 .f0G 0 g/ D fa 2 G W .a/ D 0G 0 g. We introduce the following notation for these sets: Im denotes the image of , and Ker denotes the kernel of . Example 6.35. If H is a subgroup of an abelian group G, then the inclusion map i W H ! G is obviously a group homomorphism. Example 6.36. Suppose H is a subgroup of an abelian group G. We define the map W G ! G=H a 7! ŒaH : It is not hard to see that this is a group homomorphism. Indeed, this follows almost immediately from the way we defined addition in the quotient group G=H : .a C b/ D Œa C bH D ŒaH C ŒbH D .a/ C .b/: It is clear that is surjective. It is also not hard to see that Ker D H ; indeed, H is the identity element in G=H , and ŒaH D H if and only if a 2 H . The map is called the natural map from G to G=H . Example 6.37. For a given positive integer n, the natural map from Z to Zn sends a 2 Z to the residue class Œan . This map is a surjective group homomorphism with kernel nZ. Example 6.38. Suppose G is an abelian group and m is an integer. The map W G!G a 7! ma is a group homomorphism, since .a C b/ D m.a C b/ D ma C mb D .a/ C .b/: The image of this homomorphism is the subgroup mG and the kernel is the subgroup Gfmg. We call this map the m-multiplication map on G. If G is written multiplicatively, then this map, which sends a 2 G to am 2 G, is called the mpower map on G, and its image is G m . Example 6.39. Let p be an odd prime. Consider the 2-power, or squaring, map on Zp . Then as we saw in Example 6.21, the image .Zp /2 of this map is a subgroup of Zp of order .p 1/=2, and its kernel is Zp f2g D fŒ˙1g. Example 6.40. Consider the m-multiplication map on Z. As we saw in Example 6.22, its image mZ is equal to Z if and only if m D ˙1, while its kernel Zfmg is equal to Z if m D 0, and is equal to f0g otherwise.
6.4 Group homomorphisms and isomorphisms
143
Example 6.41. Consider the m-multiplication map on Zn . As we saw in Example 6.23, if d WD gcd.m; n/, the image mZn of this map is a subgroup of Zn of order n=d , while its kernel Zn fmg is a subgroup of order d . Example 6.42. Suppose G is an abelian group and a is an element of G. It is easy to see that the map W Z!G z 7! za is a group homomorphism, since .z C z 0 / D .z C z 0 /a D za C z 0 a D .z/ C .z 0 /: Example 6.43. As a special case of the previous example, let n be a positive integer and let ˛ be an element of Zn . Let W Z ! Zn be the group homomorphism that sends z 2 Z to ˛ z 2 Zn . That is a group homomorphism means 0 0 that ˛ zCz D ˛ z ˛ z for all z; z 0 2 Z (note that the group operation is addition in Z and multiplication in Zn ). If the multiplicative order of ˛ is equal to k, then as discussed in §2.7, the image of consists of the k distinct group elements ˛ 0 ; ˛ 1 ; : : : ; ˛ k 1 . The kernel of consists of those integers z such that ˛ z D 1. Again by the discussion in §2.7, the kernel of is equal to the subgroup kZ. Example 6.44. Generalizing Example 6.42, the reader may verify that if a1 ; : : : ; ak are fixed elements of an abelian group G, then the map Zk ! G
W
.z1 ; : : : ; zk / 7! z1 a1 C C zk ak is a group homomorphism. Example 6.45. Suppose that H1 ; : : : ; Hk are subgroups of an abelian group G. The reader may easily verify that the map W H1 Hk ! G .a1 ; : : : ; ak / 7! a1 C C ak is a group homomorphism whose image is the subgroup H1 C C Hk . The following theorem summarizes some of the most important properties of group homomorphisms. Theorem 6.19. Let be a group homomorphism from G to G 0 . (i) .0G / D 0G 0 . (ii) . a/ D
.a/ for all a 2 G.
144
Abelian groups
(iii) .na/ D n.a/ for all n 2 Z and a 2 G. (iv) If H is a subgroup of G, then .H / is a subgroup of G 0 ; in particular (setting H WD G), Im is a subgroup of G 0 . (v) If H 0 is a subgroup of G 0 , then 1 .H 0 / is a subgroup of G; in particular (setting H 0 WD f0G 0 g), Ker is a subgroup of G. (vi) For all a; b 2 G, .a/ D .b/ if and only if a b .mod Ker /. (vii) is injective if and only if Ker D f0G g. Proof. These are all straightforward calculations. (i) We have 0G 0 C .0G / D .0G / D .0G C 0G / D .0G / C .0G /: Now cancel .0G / from both sides. (ii) We have 0G 0 D .0G / D .a C . a// D .a/ C . a/; and hence . a/ is the inverse of .a/. (iii) For n D 0, this follows from part (i). For n > 0, this follows from the definitions by induction on n. For n < 0, this follows from the positive case and part (ii). (iv) For all a; b 2 H , we have a C b 2 H and a 2 H ; hence, .H / contains .a C b/ D .a/ C .b/ and . a/ D .a/. (v) 1 .H 0 / is non-empty, since .0G / D 00G 2 H 0 . If .a/ 2 H 0 and .b/ 2 H 0 , then .a C b/ D .a/ C .b/ 2 H 0 , and . a/ D .a/ 2 H 0 . (vi) We have .a/ D .b/ ” .a/ ” a
.b/ D 0G 0 ” .a
b/ D 0G 0
b 2 Ker ” a b .mod Ker /:
(vii) If is injective, then in particular, 1 .f0G 0 g/ cannot contain any other element besides 0G . If is not injective, then there exist two distinct elements a; b 2 G with .a/ D .b/, and by part (vi), Ker contains the element a b, which is non-zero. Part (vii) of the above theorem is particularly useful: to check that a group homomorphism is injective, it suffices to determine if Ker D f0G g. Thus, the injectivity and surjectivity of a given group homomorphism W G ! G 0 may be characterized in terms of its kernel and image: is injective if and only if Ker D f0G g;
6.4 Group homomorphisms and isomorphisms
145
is surjective if and only if Im D G 0 . We next present two very simple theorems that allow us to compose group homomorphisms in simple ways. Theorem 6.20. If W G ! G 0 and 0 W G 0 ! G 00 are group homomorphisms, then so is their composition 0 B W G ! G 00 . Proof. For all a; b 2 G, we have 0 ..a C b// D 0 ..a/ C .b// D 0 ..a// C 0 ..b//: Theorem 6.21. Let i W G ! Gi0 , for i D 1; : : : ; k, be group homomorphisms. Then the map W G ! G10 Gk0 a 7! .1 .a/; : : : ; k .a// is a group homomorphism. Proof. For all a; b 2 G, we have .a C b/ D .1 .a C b/; : : : ; k .a C b// D .1 .a/ C 1 .b/; : : : ; k .a/ C k .b// D .a/ C .b/: Consider a group homomorphism W G ! G 0 . If is bijective, then is called a group isomorphism of G with G 0 . If such a group isomorphism exists, we say that G is isomorphic to G 0 , and write G Š G 0 . Moreover, if G D G 0 , then is called a group automorphism on G. Theorem 6.22. If is a group isomorphism of G with G 0 , then the inverse function 1 is a group isomorphism of G 0 with G. Proof. For all a0 ; b 0 2 G 0 , we have . and hence
1
.a0 / C
1 .a0 /
C
1
.b 0 // D .
1 .b 0 /
D
1 .a0
1
.a0 // C .
1
.b 0 // D a0 C b 0 ;
C b 0 /.
Because of this theorem, if G is isomorphic to G 0 , we may simply say that “G and G 0 are isomorphic.” We stress that a group isomorphism of W G ! G 0 is essentially just a “renaming” of the group elements. This can be visualized as follows. Imagine the addition table for G written out with rows and columns labeled by elements of G, with the entry in row a and column b being a C b. Now suppose we use the function to consistently rename all the elements of G appearing in this table: the label on row a is replaced by .a/, the label on column b by .b/, and the entry in row a
146
Abelian groups
and column b by .a C b/. Because is bijective, every element of G 0 appears exactly once as a label on a row and as a label on a column; moreover, because .a C b/ D .a/ C .b/, what we end up with is an addition table for G 0 . It follows that all structural properties of the group are preserved, even though the two groups might look quite different syntactically. Example 6.46. As was shown in Example 6.32, the quotient group G=H discussed in that example is isomorphic to Z3 . As was shown in Example 6.33, the quotient group Z15 =.Z15 /2 is isomorphic to Z2 Z2 . As was shown in Example 6.34, the quotient group Z5 =.Z5 /2 is isomorphic to Z2 . Example 6.47. If gcd.m; n/ D 1, then the m-multiplication map on Zn is a group automorphism. The next theorem tells us that corresponding to any group homomorphism, there is a natural group isomomorphism. As group isomorphisms are much nicer than group homomorphisms, this is often very useful. Theorem 6.23 (First isomorphism theorem). Let W G ! G 0 be a group homomorphism with kernel K and image H 0 . Then we have a group isomorphism G=K Š H 0 : Specifically, the map N W G=K ! G 0 ŒaK 7! .a/ is an injective group homomorphism whose image is H 0 . Proof. Using part (vi) of Theorem 6.19, we see that for all a; b 2 G, we have ŒaK D ŒbK ” a b .mod K/ ” .a/ D .b/: This immediately implies that the definition of N is unambiguous (since ŒaK D ŒbK implies .a/ D .b/), and that N is injective (since .a/ D .b/ implies ŒaK D ŒbK ). It is clear that N maps onto H 0 , since every element of H 0 is of the form .a/ for some a 2 G, and the map N sends ŒaK to .a/. Finally, to see that N is a group homomorphism, note that .Œa N K CŒbK / D .ŒaCb N N K /C .Œb N K /: K / D .aCb/ D .a/C.b/ D .Œa We can generalize the previous theorem, as follows: Theorem 6.24. Let W G ! G 0 be a group homomorphism. Then for every
6.4 Group homomorphisms and isomorphisms
147
subgroup H of G with H Ker , we may define a group homomorphism N W G=H ! G 0 ŒaH 7! .a/: Moreover, Im N D Im , and N is injective if and only if H D Ker . Proof. Using the assumption that H Ker , we see that N is unambiguously defined, since for all a; b 2 G, we have ŒaH D ŒbH H) a b .mod H / H) a b .mod Ker / H) .a/ D .b/: That N is a group homomorphism, with Im N D Im , follows as in the proof of Theorem 6.23. If H D Ker , then by Theorem 6.23, N is injective, and if H ¨ Ker , then N is not injective, since if we choose a 2 Ker n H , we see that .Œa N H / D 0G 0 , and hence Ker N is non-trivial. The next theorem gives us another important construction of a group isomorphism. Theorem 6.25 (Internal direct product). Let G be an abelian group with subgroups H1 ; H2 , where H1 \ H2 D f0G g. Then we have a group isomorphism H1 H2 Š H1 C H2 given by the map W H1 H2 ! H1 C H2 .a1 ; a2 / 7! a1 C a2 : Proof. We already saw that is a surjective group homomorphism in Example 6.45. To see that is injective, it suffices to show that Ker is trivial; that is, it suffices to show that for all a1 2 H1 and a2 2 H2 , if a1 C a2 D 0G , then a1 D a2 D 0G . But a1 C a2 D 0G implies a1 D a2 2 H2 , and hence a1 2 H1 \ H2 D f0G g, and so a1 D 0G . Similarly, one shows that a2 D 0G , and that finishes the proof. If H1 ; H2 are as in the above theorem, then H1 C H2 is sometimes called the internal direct product of H1 and H2 . Example 6.48. We can use the general theory developed so far to get a quickand-dirty proof of the Chinese remainder theorem (Theorem 2.6). Let fni gkiD1 Q be a pairwise relatively prime family of positive integers, and let n WD kiD1 ni . Consider the map W Z ! Zn1 Znk a 7! .Œan1 ; : : : ; Œank /:
148
Abelian groups
It is easy to see that this map is a group homomorphism; indeed, it is the map constructed in Theorem 6.21 applied with the natural maps i W Z ! Zni , for i D 1; : : : ; k. Evidently, a 2 Ker if and only if ni j a for i D 1; : : : ; k, and since fni gkiD1 is pairwise relatively prime, it follows that a 2 Ker if and only if n j a; that is, Ker D nZ. Theorem 6.23 then gives us an injective group homomorphism N W
Zn ! Zn1 Znk Œan 7! .Œan1 ; : : : ; Œank /:
But since the sets Zn and Zn1 Znk have the same size, injectivity implies surjectivity. From this, Theorem 2.6 is immediate. The map N is a group isomorphism Zn Š Zn1 Znk : In fact, the map N is the same as the map in Theorem 2.8, and so we also immediately obtain parts (i), (ii), (iii.a), and (iii.b) of Theorem 2.8. Observe that parts (iii.c) and (iii.d) of Theorem 2.8 imply that restricting the map to Zn yields an isomorphism of the multiplicative groups Zn Š Zn1 Znk : This fact does not follow from the general theory developed so far; however, in the next chapter, we will see how this fact fits into the broader algebraic picture. One advantage of our original proof of Theorem 2.6 is that it gives us an explicit formula for the inverse map 1 , which is useful in computations. Example 6.49. Let n1 ; n2 be positive integers with n1 j n2 . Consider the natural map W Z ! Zn1 . This is a surjective group homomorphism with Ker D n1 Z. Since H WD n2 Z n1 Z, we may apply Theorem 6.24 with the subgroup H , obtaining the surjective group homomorphism N W
Zn2 ! Zn1 Œan2 7! Œan1 :
Example 6.50. Let us revisit Example 6.23. Let n be a positive integer, and let m be any integer. Let 1 W Z ! Zn be the natural map, and let 2 W Zn ! Zn be the m-multiplication map. The composed map WD 2 B 1 from Z to Zn is also a group homomorphism. For each z 2 Z, we have .z/ D mŒzn D Œmzn . The kernel of consists of those integers z such that mz 0 .mod n/, and so Theorem 2.5 implies that Ker D .n=d /Z, where d WD gcd.m; n/. The image of
6.4 Group homomorphisms and isomorphisms
149
is mZn . Theorem 6.23 therefore implies that the map N W
Zn=d ! mZn Œzn=d 7! mŒzn
is a group isomorphism. Example 6.51. Consider the group Zp where p is an odd prime, and let W Zp ! Zp be the squaring map. By definition, Im D .Zp /2 , and we proved in Theorem 2.18 that Ker D fŒ˙1g. Theorem 2.19 says that for all ; ˇ 2 Zp ,
2 D ˇ 2 if and only if D ˙ˇ. This fact can also be seen to be a special case of part (vi) of Theorem 6.19. Theorem 6.23 says that Zp =Ker Š Im , and since jZp =Ker j D jZp j=jKer j D .p 1/=2, we see that Theorem 2.20, which says that j.Zp /2 j D .p 1/=2, follows from this. Let H WD .Zp /2 , and consider the quotient group Zp =H . Because jH j D .p 1/=2, we know that jZp =H j D jZp j=jH j D 2, and hence Zp =H consists of x WD Zp n H . the two cosets H and H x , and consider the map Let ˛ be an arbitrary, fixed element of H W Z ! Zp =H z 7! Œ˛ z H : It is easy to see that is a group homomorphism; indeed, it is the composition of the homomorphism discussed in Example 6.43 and the natural map from Zp to Zp =H . Moreover, it is easy to see (for example, as a special case of Theorem 2.17) that ˛ z 2 H ” z is even: From this, it follows that Ker D 2Z; also, since Zp =H consists of just the two x , it follows that is surjective. Therefore, Theorem 6.23 says that cosets H and H the map N W
Z2 ! Zp =H Œz2 7! Œ˛ z H
is a group isomorphism, under which Œ02 corresponds to H , and Œ12 corresponds x. to H This isomorphism gives another way to derive Theorem 2.23, which says that in Zp , the product of two non-squares is a square; indeed, the statement “non-zero plus non-zero equals zero in Z2 ” translates via the isomorphism N to the statement “non-square times non-square equals square in Zp .” Example 6.52. Let Q be the multiplicative group of non-zero rational numbers.
150
Abelian groups
Let H1 be the subgroup f˙1g, and let H2 be the subgroup of positive rationals. It is easy to see that Q D H1 H2 and that H1 \ H2 D f1g. Thus, Q is the internal direct product of H1 and H2 , and Theorem 6.25 gives us a group isomorphism Q Š H1 H2 . Let G and G 0 be abelian groups. Recall from Example 6.19 that Map.G; G 0 / is the group of all functions W G ! G 0 , where the group operation is defined pointwise using the group operation of G 0 : . C /.a/ D .a/ C .a/ and . /.a/ D .a/ for all ; 2 Map.G; G 0 / and all a 2 G. The following theorem isolates an important subgroup of this group. Theorem 6.26. Let G and G 0 be abelian groups, and consider the group of functions Map.G; G 0 /. Then Hom.G; G 0 / WD f 2 Map.G; G 0 / W is a group homomorphismg is a subgroup of Map.G; G 0 /. Proof. First, observe that Hom.G; G 0 / is non-empty, as it contains the map that sends everything in G to 0G 0 (this is the identity element of Map.G; G 0 /). Next, we have to show that if and are homomorphisms from G to G 0 , then so are C and . But C D 2 B 1 , where 1 W G ! G 0 G 0 is the map constructed in Theorem 6.21, applied with and , and 2 W G 0 G 0 ! G 0 is as in Example 6.45. Also, D 1 B , where 1 is the . 1/-multiplication map. E XERCISE 6.22. Verify that the “is isomorphic to” relation on abelian groups is an equivalence relation; that is, for all abelian groups G1 ; G2 ; G3 , we have: (a) G1 Š G1 ; (b) G1 Š G2 implies G2 Š G1 ; (c) G1 Š G2 and G2 Š G3 implies G1 Š G3 . E XERCISE 6.23. Let i W Gi ! Gi0 , for i D 1; : : : ; k, be group homomorphisms. Show that the map W G1 Gk ! G10 Gk0 .a1 ; : : : ; ak / 7! .1 .a1 /; : : : ; k .ak // is a group homomorphism. Also show that if each i is an isomorphism, then so is . E XERCISE 6.24. Let W G ! G 0 be a group homomorphism. Let H; K be subgroups of G and let m be a positive integer. Show that .H C K/ D .H / C .K/ and .mH / D m.H /.
6.4 Group homomorphisms and isomorphisms
151
E XERCISE 6.25. Let W G ! G 0 be a group homomorphism. Let H be a subgroup of G, and let W H ! G 0 be the restriction of to H . Show that is a group homomorphism and that Ker D Ker \ H . E XERCISE 6.26. Suppose G1 ; : : : ; Gk are abelian groups. Show that for each i D 1; : : : ; k, the projection map i W G1 Gk ! Gi that sends .a1 ; : : : ; ak / to ai is a surjective group homomorphism. E XERCISE 6.27. Show that if G D G1 G2 for abelian groups G1 and G2 , and H1 is a subgroup of G1 and H2 is a subgroup of G2 , then we have a group isomorphism G=.H1 H2 / Š G1 =H1 G2 =H2 . E XERCISE 6.28. Let G be an abelian group with subgroups H and K. (a) Show that we have a group isomorphism .H C K/=K Š H=.H \ K/. (b) Show that if H and K are finite, then jH C Kj D jH jjKj=jH \ Kj. E XERCISE 6.29. Let G be an abelian group with subgroups H , K, and A, where K H . Show that .H \ A/=.K \ A/ is isomorphic to a subgroup of H=K. E XERCISE 6.30. Let W G ! G 0 be a group homomorphism with kernel K. Let H be a subgroup of G. Show that we have a group isomorphism G=.H C K/ Š .G/=.H /. E XERCISE 6.31. Let W G ! G 0 be a surjective group homomorphism. Let S be the set of all subgroups of G that contain Ker , and let S 0 be the set of all subgroups of G 0 . Show that the sets S and S 0 are in one-to-one correspondence, via the map that sends H 2 S to .H / 2 S 0 . Moreover, show that this correspondence preserves inclusions, that is, for all H1 ; H2 2 S, we have H1 H2 ” .H1 / .H2 /. E XERCISE 6.32. Use the previous exercise, together with Theorem 6.9, to get a short proof of Theorem 6.10. E XERCISE 6.33. Show that the homomorphism of Example 6.44 arises by direct application of Example 6.42, combined with Theorems 6.20 and 6.21. E XERCISE 6.34. Suppose that G, G1 , and G2 are abelian groups, and that W G1 G2 ! G is a group isomorphism. Let H1 WD .G1 f0G2 g/ and H2 WD .f0G1 g G2 /. Show that G is the internal direct product of H1 and H2 . E XERCISE 6.35. Let ZC denote the set of positive integers, and let Q be the multiplicative group of non-zero rational numbers. Consider the abelian groups Map# .ZC ; Z/ and Map# .ZC ; Z2 /, as defined in Exercise 6.14. Show that we have group isomorphisms
152
Abelian groups
(a) Q Š Z2 Map# .ZC ; Z/, and (b) Q =.Q /2 Š Map# .ZC ; Z2 /. E XERCISE 6.36. Let n be an odd, positive integer whose factorization into primes is n D p1e1 prer . Show that: (a) we have a group isomorphism Zn =.Zn /2 Š Zr 2 ; (b) if pi 3 .mod 4/ for each i D 1; : : : ; r, then the squaring map on .Zn /2 is a group automorphism. E XERCISE 6.37. Which of the following pairs of groups are isomorphic? Why or why not? (a) Z2 Z2 and Z4 , (b) Z12 and Z8 , (c) Z5 and Z4 , (d) Z2 Z and Z, (e) Q and Z, (f) Z Z and Z. 6.5 Cyclic groups Let G be an abelian group. For a 2 G, define hai WD fza W z 2 Zg. It is easy to see that hai is a subgroup of G; indeed, it is the image of the group homomorphism discussed in Example 6.42. Moreover, hai is the smallest subgroup of G containing a; that is, hai contains a, and every subgroup H of G that contains a must also contain hai. Indeed, if a subgroup contains a, it must contain a C a D 2a, a C a C a D 3a, and so on; it must also contain 0G D 0a, a D . 1/a, . a/ C . a/ D . 2/a, and so on. The subgroup hai is called the subgroup (of G) generated by a. Also, one defines the order of a to be the order of the subgroup hai. More generally, for a1 ; : : : ; ak 2 G, we define ha1 ; : : : ; ak i WD fz1 a1 C C zk ak W z1 ; : : : ; zk 2 Zg: It is easy to see that ha1 ; : : : ; ak i is a subgroup of G; indeed, it is the image of the group homomorphism discussed in Example 6.44. Moreover, this subgroup is the smallest subgroup of G that contains a1 ; : : : ; ak ; that is, ha1 ; : : : ; ak i contains the elements a1 ; : : : ; ak , and any subgroup H that contains these elements must also contain ha1 ; : : : ; ak i. The subgroup ha1 ; : : : ; ak i it is called the subgroup (of G) generated by a1 ; : : : ; ak . An abelian group G is called cyclic if G D hai for some a 2 G, in which case, a is called a generator for G. An abelian group G is called finitely generated if G D ha1 ; : : : ; ak i for some a1 ; : : : ; ak 2 G. Multiplicative notation: if G is written multiplicatively, then hai WD faz W z 2 Zg, and ha1 ; : : : ; ak i WD fa1z1 akzk W z1 ; : : : ; zk 2 Zg; also, for emphasis and clarity, we use the term multiplicative order of a.
153
6.5 Cyclic groups
Example 6.53. Consider the additive group Z. This is a cyclic group, with 1 being a generator: h1i D fz 1 W z 2 Zg D fz W z 2 Zg D Z: For every m 2 Z, we have hmi D fzm W z 2 Zg D fmz W z 2 Zg D mZ: It follows that the only elements of Z that generate Z are 1 and element generates a subgroup that is strictly contained in Z.
1: every other
Example 6.54. For n > 0, consider the additive group Zn . This is a cyclic group, with Œ1 being a generator: hŒ1i D fzŒ1 W z 2 Zg D fŒz W z 2 Zg D Zn : For every m 2 Z, we have hŒmi D fzŒm W z 2 Zg D fŒzm W z 2 Zg D fmŒz W z 2 Zg D mZn : By Example 6.23, the subgroup mZn has order n= gcd.m; n/. Thus, Œm has order n= gcd.m; n/; in particular, Œm generates Zn if and only if m is relatively prime to n, and hence, the number of generators of Zn is .n/. Implicit in Examples 6.53 and 6.54 is the following general fact: Theorem 6.27. Let G be a cyclic group generated by a. Then for every m 2 Z, we have hmai D mG: Proof. We have hmai D fz.ma/ W z 2 Zg D fm.za/ W z 2 Zg D mhai D mG: The following two examples present some groups that are not cyclic. Example 6.55. Consider the additive group G WD Z Z. Set ˛1 WD .1; 0/ 2 G and ˛2 WD .0; 1/ 2 G: It is not hard to see that G D h˛1 ; ˛2 i, since for all z1 ; z2 2 Z, we have z1 ˛1 C z2 ˛2 D .z1 ; 0/ C .0; z2 / D .z1 ; z2 /: However, G is not cyclic. To see this, let ˇ D .b1 ; b2 / be an arbitrary element of G. We claim that one of ˛1 or ˛2 does not belong to hˇi. Suppose to the contrary
154
Abelian groups
that both ˛1 and ˛2 belong to hˇi. This would imply that there exist integers z and z 0 such that zb1 D 1;
zb2 D 0;
0
z 0 b2 D 1:
z b1 D 0;
Multiplying the upper left equality by the lower right, and the upper right by the lower left, we obtain 1 D zz 0 b1 b2 D 0; which is impossible. Example 6.56. Consider the additive group G WD Zn1 Zn2 . Set ˛1 WD .Œ1n1 ; Œ0n2 / 2 G and ˛2 WD .Œ0n1 ; Œ1n2 / 2 G: It is not hard to see that G D h˛1 ; ˛2 i, since for all z1 ; z2 2 Z, we have z1 ˛1 C z2 ˛2 D .Œz1 n1 ; Œ0n2 / C .Œ0n1 ; Œz2 n2 / D .Œz1 n1 ; Œz2 n2 /: However, G may or may not be cyclic: it depends on d WD gcd.n1 ; n2 /. If d D 1, then G is cyclic, with ˛ WD .Œ1n1 ; Œ1n2 / being a generator. One can see this easily using the Chinese remainder theorem: for all z1 ; z2 2 Z, there exists z 2 Z such that z z1 .mod n1 / and z z2 .mod n2 /; which implies z˛ D .Œzn1 ; Œzn2 / D .Œz1 n1 ; Œz2 n2 /: If d > 1, then G is not cyclic. To see this, let ˇ D .Œb1 n1 ; Œb2 n2 / be an arbitrary element of G. We claim that one of ˛1 or ˛2 does not belong to hˇi. Suppose to the contrary that both ˛1 and ˛2 belong to hˇi. This would imply that there exist integers z and z 0 such that zb1 1 .mod n1 /; 0
z b1 0 .mod n1 /;
zb2 0 .mod n2 /; z 0 b2 1 .mod n2 /:
All of these congruences hold modulo d as well, and multiplying the upper left congruence by the lower right, and the upper right by the lower left, we obtain 1 zz 0 b1 b2 0 .mod d /; which is impossible.
6.5 Cyclic groups
155
It should be clear that since a group isomorphism preserves all structural properties of groups, it preserves the property of being cyclic. We state this, along with related facts, as a theorem. Theorem 6.28. Let W G ! G 0 be a group isomorphism. (i) For all a 2 G, we have .hai/ D h.a/i. (ii) For all a 2 G, a and .a/ have the same order. (iii) G is cyclic if and only if G 0 is cyclic. Proof. For all a 2 G, we have .hai/ D f.za/ W z 2 Zg D fz.a/ W z 2 Zg D h.a/i: That proves (i). (ii) follows from (i) and the fact that is injective. (iii) follows from (i), as follows. If G is cyclic, then G D hai, and since is surjective, we have G 0 D .G/ D h.a/i. The converse follows by applying the same argument to the inverse isomorphism 1 W G 0 ! G. Example 6.57. Consider again the additive group G WD Zn1 Zn2 , discussed in Example 6.56. If gcd.n1 ; n2 / D 1, then one can also see that G is cyclic as follows: by the discussion in Example 6.48, we know that G is isomorphic to Zn1 n2 , and since Zn1 n2 is cyclic, so is G. Example 6.58. Consider again the subgroup mZn of Zn , discussed in Example 6.54. One can also see that this is cyclic of order n=d , where d WD gcd.m; n/, as follows: in Example 6.50, we constructed an isomorphism between Zn=d and mZn , and this implies mZn is cyclic of order n=d . Classification of cyclic groups. Examples 6.53 and 6.54 are extremely important examples of cyclic groups. Indeed, as we shall now demonstrate, every cyclic group is isomorphic either to Z or to Zn for some n > 0. Suppose that G is a cyclic group with generator a. Consider the map W Z ! G that sends z 2 Z to za 2 G. As discussed in Example 6.42, this map is a group homomorphism, and since a is a generator for G, it must be surjective. There are two cases to consider. Case 1: Ker D f0g. In this case, is an isomorphism of Z with G. Case 2: Ker ¤ f0g. In this case, since Ker is a subgroup of Z different from f0g, by Theorem 6.9, it must be of the form nZ for some n > 0. Hence, by Theorem 6.23, the map N W Zn ! G that sends Œzn to za is an isomorphism of Zn with G. Based on the this isomorphism, we immediately obtain:
156
Abelian groups
Theorem 6.29. Let G be an abelian group and let a 2 G. If there exists a positive integer m such that ma D 0G , then the least such positive integer n is the order of a; in this case, we have: for every integer z, za D 0G if and only if n divides z, and more generally, for all integers z1 ; z2 , we have z1 a D z2 a if and only if z1 z2 .mod n/; the subgroup hai consists of the n distinct elements 0 a; 1 a; : : : ; .n
1/ a:
Otherwise, a has infinite order, and every element of hai can be expressed as za for some unique integer z. In the case where the group is finite, we can say more: Theorem 6.30. Let G be a finite abelian group and let a 2 G. Then jGja D 0G and the order of a divides jGj. Proof. Since hai is a subgroup of G, by Lagrange’s theorem (Theorem 6.15), the order of a divides jGj. It then follows by Theorem 6.29 that jGja D 0G . Example 6.59. Let a; n 2 Z with n > 0 and gcd.a; n/ D 1, and let ˛ WD Œa 2 Zn . Theorem 6.29 implies that the definition given in this section of the multiplicative order of ˛ is consistent with that given in §2.7. Moreover, Euler’s theorem (Theorem 2.13) can be seen as just a special case of Theorem 6.30. Also, note that ˛ is a generator for Zn if and only if a is a primitive root modulo p. Example 6.60. As we saw in Example 6.26, all elements of Z15 have multiplicative order dividing 4, and since Z15 has order 8, we conclude that Z15 is not cyclic. Example 6.61. The group Z5 is cyclic, with Œ2 being a generator: Œ22 D Œ4 D Œ 1; Œ23 D Œ 2; Œ24 D Œ1: Example 6.62. Based on the calculations in Example 2.8, we may conclude that Z7 is cyclic, with both Œ3 and Œ5 being generators. Example 6.63. Consider again the additive group G WD Zn1 Zn2 , discussed in Example 6.56. If d WD gcd.n1 ; n2 / > 1, then one can also see that G is not cyclic as follows: for every ˇ 2 G, we have .n1 n2 =d /ˇ D 0G , and hence by Theorem 6.29, the order of ˇ divides n1 n2 =d . The following two theorems completely characterize the subgroup structure of cyclic groups. Actually, we have already proven most of the results in these two theorems, but nevertheless, they deserve special emphasis.
6.5 Cyclic groups
157
Theorem 6.31. Let G be a cyclic group of infinite order. (i) G is isomorphic to Z. (ii) The subgroups of G are in one-to-one correspondence with the nonnegative integers, where each such integer m corresponds to the cyclic group mG. (iii) For every two non-negative integers m; m0 , mG m0 G if and only if m0 j m. Proof. That G Š Z was established in our classification of cyclic groups, and so it suffices to prove the other statements of the theorem for G D Z. As we saw in Example 6.53, for every integer m, the subgroup mZ is cyclic, as it is generated by m. This fact, together with Theorem 6.9, establishes all the other statements. Theorem 6.32. Let G be a cyclic group of finite order n. (i) G is isomorphic to Zn . (ii) The subgroups of G are in one-to-one correspondence with the positive divisors of n, where each such divisor d corresponds to the subgroup dG; moreover, dG is a cyclic group of order n=d . (iii) For each positive divisor d of n, we have dG D Gfn=d g; that is, the kernel of the .n=d /-multiplication map is equal to the image of the d multiplication map; in particular, Gfn=d g has order n=d . (iv) For every two positive divisors d; d 0 of n, we have dG d 0 G if and only if d 0 j d . (v) For every positive divisor d of n, the number of elements of order d in G is .d /. (vi) For every integer m, we have mG D dG and Gfmg D Gfd g, where d WD gcd.m; n/. Proof. That G Š Zn was established in our classification of cyclic groups, and so it suffices to prove the other statements of the theorem for G D Zn . The one-to-one correspondence in part (ii) was established in Theorem 6.10. By the discussion in Example 6.54, it is clear that d Zn is generated by Œd and has order n=d . Part (iii) was established in Example 6.23. Part (iv) was established in Theorem 6.10. For part (v), the elements of order d in Zn are all contained in Zn fd g, and so the number of such elements is equal to the number of generators of Zn fd g. The group Zn fd g is cyclic of order d , and so is isomorphic to Zd , and as we saw in Example 6.54, this group has .d / generators.
158
Abelian groups
Part (vi) was established in Example 6.23. Since cyclic groups are in some sense the simplest kind of abelian group, it is nice to establish some sufficient conditions under which a group must be cyclic. The following three theorems provide such conditions. Theorem 6.33. If G is an abelian group of prime order, then G is cyclic. Proof. Let jGj D p. Let a 2 G with a ¤ 0G , and let k be the order of a. As the order of an element divides the order of the group, we have k j p, and so k D 1 or k D p. Since a ¤ 0G , we must have k ¤ 1, and so k D p, which implies that a generates G. Theorem 6.34. If G1 and G2 are finite cyclic groups of relatively prime order, then G1 G2 is also cyclic. In particular, if G1 is generated by a1 and G2 is generated by a2 , then G1 G2 is generated by .a1 ; a2 /. Proof. We give a direct proof, based on Theorem 6.29. Let n1 WD jG1 j and n2 WD jG2 j, where gcd.n1 ; n2 / D 1. Also, let a1 2 G1 have order n1 and a2 2 G2 have order n2 . We want to show that .a1 ; a2 / has order n1 n2 . Applying Theorem 6.29 to .a1 ; a2 /, we see that the order of .a1 ; a2 / is the smallest positive integer k such that k.a1 ; a2 / D .0G1 ; 0G2 /. Now, for every integer k, we have k.a1 ; a2 / D .ka1 ; ka2 /, and .ka1 ; ka2 / D .0G1 ; 0G2 / ” n1 j k and n2 j k (applying Theorem 6.29 to a1 and a2 ) ” n1 n2 j k (since gcd.n1 ; n2 / D 1): Theorem 6.35. Let G be a cyclic group. Then for every subgroup H of G, both H and G=H are cyclic. Proof. The fact that H is cyclic follows from part (ii) of Theorem 6.31 in the case where G is infinite, and part (ii) of Theorem 6.32 in the case where G is finite. If G is generated by a, then it is easy to see that G=H is generated by ŒaH . The next three theorems are often useful in calculating the order of a group element. The first generalizes Theorem 2.15. Theorem 6.36. Let G be an abelian group, let a 2 G be of finite order n, and let m be an arbitrary integer. Then the order of ma is n= gcd.m; n/. Proof. Let H WD hai, and d WD gcd.m; n/. By Theorem 6.27, we have hmai D mH , and by Theorem 6.32, we have mH D dH , which has order n=d . That proves the theorem. Alternatively, we can give a direct proof, based on Theorem 6.29. Applying Theorem 6.29 to ma, we see that order of ma is the
6.5 Cyclic groups
159
smallest positive integer k such that k.ma/ D 0G . Now, for every integer k, we have k.ma/ D .km/a, and .km/a D 0G ” km 0 .mod n/ (applying Theorem 6.29 to a) ” k 0 .mod n= gcd.m; n// (by Theorem 2.5): Theorem 6.37. Suppose that a is an element of an abelian group, and for some prime p and integer e 1, we have p e a D 0G and p e 1 a ¤ 0G . Then a has order p e . Proof. If m is the order of a, then since p e a D 0G , we have m j p e . So m D p f for some f D 0; : : : ; e. If f < e, then p e 1 a D 0G , contradicting the assumption that p e 1 a ¤ 0G . Theorem 6.38. Suppose G is an abelian group with a1 ; a2 2 G such that a1 is of finite order n1 , a2 is of finite order n2 , and gcd.n1 ; n2 / D 1. Then the order of a1 C a2 is n1 n2 . Proof. Let H1 WD ha1 i and H2 WD ha2 i so that jH1 j D n1 and jH2 j D n2 . First, we claim that H1 \ H2 D f0G g. To see this, observe that H1 \ H2 is a subgroup of H1 , and so jH1 \ H2 j divides n1 ; similarly, jH1 \ H2 j divides n2 . Since gcd.n1 ; n2 / D 1, we must have jH1 \ H2 j D 1, and that proves the claim. Using the claim, we can apply Theorem 6.25, obtaining a group isomorphism between H1 C H2 and H1 H2 . Under this isomorphism, the group element a1 C a2 2 H1 C H2 corresponds to .a1 ; a2 / 2 H1 H2 , which by Theorem 6.34 (again using the fact that gcd.n1 ; n2 / D 1) has order n1 n2 . For an abelian group G, we say that an integer k kills G if kG D f0G g. Consider the set KG of integers that kill G. Evidently, KG is a subgroup of Z, and hence of the form mZ for a uniquely determined non-negative integer m. This integer m is called the exponent of G. If m ¤ 0, then we see that m is the least positive integer that kills G. We first state some basic properties. Theorem 6.39. Let G be an abelian group of exponent m. (i) For every integer k such that kG D f0G g, we have m j k. (ii) If G has finite order, then m divides jGj. (iii) If m ¤ 0, then for every a 2 G, the order of a is finite, and the order of a divides m. (iv) If G is cyclic, then the exponent of G is 0 if G is infinite, and is jGj if G is finite.
160
Abelian groups
Proof. Exercise. The next two theorems develop some crucial properties about the structure of finite abelian groups. Theorem 6.40. If an abelian group G has non-zero exponent m, then G contains an element of order m. In particular, a finite abelian group is cyclic if and only if its order equals its exponent. Proof. The second statement follows immediately from the first. For the first stateQ ment, assume that m > 1, and let m D riD1 piei be the prime factorization of m. First, we claim that for each i D 1; : : : ; r, there exists ai 2 G such that .m=pi /ai ¤ 0G . Suppose the claim were false: then for some i, .m=pi /a D 0G for all a 2 G; however, this contradicts the minimality property in the definition of the exponent m. That proves the claim. Let a1 ; : : : ; ar be as in the above claim. Then by Theorem 6.37, .m=piei /ai has order piei for each i D 1; : : : ; r. Finally, by Theorem 6.38, the group element .m=p1e1 /a1 C C .m=prer /ar has order m. Theorem 6.41. Let G be a finite abelian group of order n. If p is a prime dividing n, then G contains an element of order p. Proof. We can prove this by induction on n. If n D 1, then the theorem is vacuously true. Now assume n > 1 and that the theorem holds for all groups of order strictly less than n. Let a be any non-zero element of G, and let m be the order of a. Since a is non-zero, we must have m > 1. If p j m, then .m=p/a is an element of order p, and we are done. So assume that p − m and consider the quotient group G=H , where H is the subgroup of G generated by a. Since H has order m, G=H has order n=m, which is strictly less than n, and since p − m, we must have p j .n=m/. So we can apply the induction hypothesis to the group G=H and the prime p, which says that there is an element b 2 G such that the coset ŒbH 2 G=H has order p. If ` is the order of b, then `b D 0G , and so `b 0G .mod H /, which implies that the order of ŒbH divides `. Thus, p j `, and so .`=p/b is an element of G of order p. As a corollary, we have: Theorem 6.42. Let G be a finite abelian group. Then the primes dividing the exponent of G are the same as the primes dividing its order.
6.5 Cyclic groups
161
Proof. Since the exponent divides the order, every prime dividing the exponent must divide the order. Conversely, if a prime p divides the order, then since there is an element of order p in the group, the exponent must be divisible by p. E XERCISE 6.38. Find ˛1 ; ˛2 2 Z15 such that Z15 D h˛1 ; ˛2 i. E XERCISE 6.39. Show that Q is not finitely generated. E XERCISE 6.40. Let G be an abelian group, a 2 G, and m 2 Z, such that m > 0 and ma D 0G . Let m D p1e1 prer be the prime factorization of m. For i D 1; : : : ; r, let fi be the largest non-negative integer such that fi ei and f e f e f m=pi i a D 0G . Show that the order of a is equal to p11 1 pr r r . E XERCISE 6.41. Let G be an abelian group of order n, and let m be an integer. Show that mG D G if and only if gcd.m; n/ D 1. E XERCISE 6.42. Let H be a subgroup of an abelian group G. Show that: (a) if H and G=H are both finitely generated, then so is G; (b) if G is finite, gcd.jH j; jG=H j/ D 1, and H and G=H are both cyclic, then G is cyclic. E XERCISE 6.43. Show that for abelian groups G1 ; G2 whose exponents are m1 and m2 , the exponent of G1 G2 is lcm.m1 ; m2 /. E XERCISE 6.44. Let G be an abelian group of exponent m1 m2 , where gcd.m1 ; m2 / D 1. Show that G is the internal direct product of m1 G and m2 G. E XERCISE 6.45. Show how Theorem 2.40 easily follows from Theorem 6.32. E XERCISE 6.46. Consider the quotient group G WD Q=Z. Show that: (a) G has exponent 0; (b) all elements of G have finite order; (c) for all positive integers m, we have mG D G and Gfmg Š Zm ; (d) all finite subgroups of G are cyclic. E XERCISE 6.47. Suppose that G is an abelian group that satisfies the following properties: (i) for all m 2 Z, Gfmg is either equal to G or is of finite order; (ii) for some m 2 Z, f0G g ¨ Gfmg ¨ G. Show that Gfmg is finite for all non-zero m 2 Z.
162
Abelian groups
6.6 The structure of finite abelian groups ./ We next state a theorem that classifies all finite abelian groups up to isomorphism. Theorem 6.43 (Fundamental theorem of finite abelian groups). A finite abelian group (with more than one element) is isomorphic to a direct product of cyclic groups Zpe1 Zprer ; 1
where the pi ’s are primes (not necessarily distinct) and the ei ’s are positive integers. This direct product of cyclic groups is unique up to the order of the factors. An alternative statement of this theorem is the following: Theorem 6.44. A finite abelian group (with more than one element) is isomorphic to a direct product of cyclic groups Zm1 Zm t ; where each mi > 1, and where for i D 1; : : : ; t 1, we have mi j mi C1 . Moreover, the integers m1 ; : : : ; m t are uniquely determined, and m t is the exponent of the group. The statements of these theorems are much more important than their proofs. Even if the reader does not study the proofs, he is urged to understand what the theorems actually say. In an exercise below, you are asked to show that these two theorems are equivalent. We now prove Theorem 6.44, which we break into two lemmas, the first of which proves the existence part of the theorem, and the second of which proves the uniqueness part. Lemma 6.45. A finite abelian group (with more than one element) is isomorphic to a direct product of cyclic groups Zm1 Zm t ; where each mi > 1, and where for i D 1; : : : ; t m t is the exponent of the group.
1, we have mi j mi C1 ; moreover,
Proof. Let G be a finite abelian group with more than one element, and let m be the exponent of G. By Theorem 6.40, there exists an element a 2 G of order m. Let A D hai. Then A Š Zm . Now, if A D G, the lemma is proved. So assume that A ¨ G. We will show that there exists a subgroup B of G such that G D A C B and A \ B D f0G g. From this, Theorem 6.25 gives us an isomorphism of G with
6.6 The structure of finite abelian groups ./
163
A B. Moreover, the exponent of B is clearly a divisor of m, and so the lemma will follow by induction (on the order of the group). So it suffices to show the existence of a subgroup B as above. We prove this by contradiction. Suppose that there is no such subgroup, and among all subgroups B such that A \ B D f0G g, assume that B is maximal, meaning that there is no subgroup B 0 of G such that B ¨ B 0 and A \ B 0 D f0G g. By assumption C WD A C B ¨ G. Let d be any element of G that lies outside of C . Consider the quotient group G=C , and let r be the order of Œd C 2 G=C . Note that r > 1 and r j m. We shall define a group element d 0 with slightly nicer properties than d , as follows. Since rd 2 C , we have rd D sa C b for some s 2 Z and b 2 B. We claim that r j s. To see this, note that 0G D md D .m=r/rd D .m=r/sa C .m=r/b, and since A \ B D f0G g, we have .m=r/sa D 0G , which can only happen if r j s. That proves the claim. This allows us to define d 0 WD d .s=r/a. Since d d 0 .mod C /, we see not only that Œd 0 C 2 G=C has order r, but also that rd 0 2 B. We next show that A\.B Chd 0 i/ D f0G g, which will yield the contradiction we seek, and thus prove the lemma. Because A\B D f0G g, it will suffice to show that A \ .B C hd 0 i/ B. Now, suppose we have a group element b 0 C xd 0 2 A, with b 0 2 B and x 2 Z. Then in particular, xd 0 2 C , and so r j x, since Œd 0 C 2 G=C has order r. Further, since rd 0 2 B, we have xd 0 2 B, whence b 0 C xd 0 2 B. Lemma 6.46. Suppose that G WD Zm1 Zm t and H WD Zn1 Zn t are isomorphic, where the mi ’s and ni ’s are positive integers (possibly 1) such that mi j mi C1 and ni j ni C1 for i D 1; : : : ; t 1. Then mi D ni for i D 1; : : : ; t . Q Q Proof. Clearly, i mi D jGj D jH j D i ni . We prove the lemma by induction on the order of the group. If the group order is 1, then clearly all the mi ’s and ni ’s must be 1, and we are done. Otherwise, let p be a prime dividing the group order. Now, suppose that p divides mr ; : : : ; m t but not m1 ; : : : ; mr 1 , and that p divides ns ; : : : ; n t but not n1 ; : : : ; ns 1 , where r t and s t. Evidently, the groups pG and pH are isomorphic. Moreover, pG Š Zm1 Zmr
1
Zmr =p Zm t =p ;
pH Š Zn1 Zns
1
Zns =p Zn t =p :
and Thus, we see that jpGj D jGj=p t rC1 and jpH j D jH j=p t sC1 , from which it follows that r D s, and the lemma then follows by induction.
164
Abelian groups
E XERCISE 6.48. Show that Theorems 6.43 and 6.44 are equivalent; that is, show that each one implies the other. To do this, give a natural one-to-one correspondence between sequences of prime powers (as in Theorem 6.43) and sequences of integers m1 ; : : : ; m t (as in Theorem 6.44). E XERCISE 6.49. Using the fundamental theorem of finite abelian groups (either form), give short and simple proofs of Theorems 6.40 and 6.41. E XERCISE 6.50. In our proof of Euler’s criterion (Theorem 2.21), we really only used the fact that Zp has a unique element of multiplicative order 2. This exercise develops a proof of a generalization of Euler’s criterion, based on the fundamental theorem of finite abelian groups. Suppose G is an abelian group of even order n that contains a unique element of order 2. (a) Show that G Š Z2e Zm1 Zmk , where e > 0 and the mi ’s are odd integers. (b) Using part (a), show that 2G D Gfn=2g. E XERCISE 6.51. Let G be a non-trivial, finite abelian group. Let s be the smallest positive integer with the following property: G D ha1 ; : : : ; as i for some a1 ; : : : ; as 2 G. Show that s is equal to the value of t in Theorem 6.44. In particular, G is cyclic if and only if t D 1. E XERCISE 6.52. Suppose G Š Zm1 Zm t . Let p be a prime, and let s be the number of mi ’s divisible by p. Show that Gfpg Š Zps . E XERCISE 6.53. Suppose G Š Zm1 Zm t with mi j mi C1 for i D 1; : : : ; t 1, and that H is a subgroup of G. Show that H Š Zn1 Zn t , where ni j ni C1 for i D 1; : : : ; t 1 and ni j mi for i D 1; : : : ; t . E XERCISE 6.54. Suppose that G is an abelian group such that for all m > 0, we have mG D G and jGfmgj D m2 (note that G is not finite). Show that Gfmg Š Zm Zm for all m > 0. Hint: use induction on the number of prime factors of m.
7 Rings
This chapter introduces the notion of a ring, more specifically, a commutative ring with unity. The theory of rings provides a useful conceptual framework for reasoning about a wide class of interesting algebraic structures. Intuitively speaking, a ring is an algebraic structure with addition and multiplication operations that behave as one would expect. While there is a lot of terminology associated with rings, the basic ideas are fairly simple. 7.1 Definitions, basic properties, and examples Definition 7.1. A commutative ring with unity is a set R together with addition and multiplication operations on R, such that: (i) the set R under addition forms an abelian group, and we denote the additive identity by 0R ; (ii) multiplication is associative; that is, for all a; b; c 2 R, we have a.bc/ D .ab/c; (iii) multiplication distributes over addition; that is, for all a; b; c 2 R, we have a.b C c/ D ab C ac and .b C c/a D ba C ca; (iv) there exists a multiplicative identity; that is, there exists an element 1R 2 R, such that 1R a D a D a 1R for all a 2 R; (v) multiplication is commutative; that is, for all a; b 2 R, we have ab D ba. There are other, more general (and less convenient) types of rings — one can drop properties (iv) and (v), and still have what is called a ring. We shall not, however, be working with such general rings in this text. Therefore, to simplify terminology, from now on, by a “ring,” we shall always mean a commutative ring with unity. Let R be a ring. Notice that because of the distributive law, for any fixed a 2 R, 165
166
Rings
the map from R to R that sends b 2 R to ab 2 R is a group homomorphism with respect to the underlying additive group of R. We call this the a-multiplication map. We first state some simple facts: Theorem 7.2. Let R be a ring. Then: (i) the multiplicative identity 1R is unique; (ii) 0R a D 0R for all a 2 R; (iii) . a/b D a. b/ D
.ab/ for all a; b 2 R;
(iv) . a/. b/ D ab for all a; b 2 R; (v) .ka/b D a.kb/ D k.ab/ for all k 2 Z and a; b 2 R. Proof. Part (i) may be proved using the same argument as was used to prove part (i) of Theorem 6.2. Parts (ii), (iii), and (v) follow directly from parts (i), (ii), and (iii) of Theorem 6.19, using appropriate multiplication maps, discussed above. Part (iv) follows from part (iii), along with part (iv) of Theorem 6.3: . a/. b/ D .a. b// D . .ab// D ab. Example 7.1. The set Z under the usual rules of multiplication and addition forms a ring. Example 7.2. For n 1, the set Zn under the rules of multiplication and addition defined in §2.5 forms a ring. Example 7.3. The set Q of rational numbers under the usual rules of multiplication and addition forms a ring. Example 7.4. The set R of real numbers under the usual rules of multiplication and addition forms a ring. Example 7.5. The set C of complex numbers under the usual rules of multiplication and addition forms a ring. Every (uniquely) as p ˛ 2 0C can0 be written 0 ˛ D a C bi, where a; b 2 R and i D 1. If ˛ D a C b i is another complex number, with a0 ; b 0 2 R, then ˛ C ˛ 0 D .a C a0 / C .b C b 0 /i and ˛˛ 0 D .aa0
bb 0 / C .ab 0 C a0 b/i:
The fact that C is a ring can be verified by direct calculation; however, we shall see later that this follows easily from more general considerations. Recall the complex conjugation operation, which sends ˛ to ˛N WD a bi. One can verify by direct calculation that complex conjugation is both additive and multiplicative; that is, ˛ C ˛ 0 D ˛N C ˛N 0 and ˛ ˛ 0 D ˛N ˛N 0 . The norm of ˛ is N.˛/ WD ˛ ˛N D a2 Cb 2 . So we see that N.˛/ is a non-negative
7.1 Definitions, basic properties, and examples
167
real number, and is zero if and only if ˛ D 0. Moreover, from the multiplicativity of complex conjugation, it is easy to see that the norm is multiplicative as well: N.˛˛ 0 / D ˛˛ 0 ˛˛ 0 D ˛˛ 0 ˛N ˛N 0 D N.˛/N.˛ 0 /. Example 7.6. Consider the set F of all arithmetic functions, that is, functions mapping positive integers to reals. Let us define addition of arithmetic functions point-wise (i.e., .f C g/.n/ D f .n/ C g.n/ for all positive integers n) and multiplication using the Dirichlet product, introduced in §2.9. The reader should verify that with addition and multiplication so defined, F forms a ring, where the all-zero function is the additive identity, and the special function ı defined in §2.9 is the multiplicative identity. Example 7.7. Generalizing Example 6.18, if R1 ; : : : ; Rk are rings, then we can form the direct product S WD R1 Rk , which consists of all k-tuples .a1 ; : : : ; ak / with a1 2 R1 ; : : : ; ak 2 Rk . We can view S in a natural way as a ring, with addition and multiplication defined component-wise. The additive identity is .0R1 ; : : : ; 0Rk / and the multiplicative identity is .1R1 ; : : : ; 1Rk /. When R D R1 D D Rk , the k-wise direct product of R is denoted Rk . Example 7.8. Generalizing Example 6.19, if I is an arbitrary set and R is a ring, then Map.I; R/, which is the set of all functions f W I ! R, may be naturally viewed as a ring, with addition and multiplication defined point-wise; that is, for f; g 2 Map.I; R/, we define .f C g/.i/ WD f .i / C g.i / and .f g/.i / WD f .i / g.i / for all i 2 I . We leave it to the reader to verify that Map.I; R/ is a ring, where the additive identity is the all-zero function, and the multiplicative identity is the all-one function. Note that in a ring R, if 1R D 0R , then for all a 2 R, we have a D 1R a D 0R a D 0R , and hence the ring R is trivial, in the sense that it consists of the single element 0R , with 0R C 0R D 0R and 0R 0R D 0R . If 1R ¤ 0R , we say that R is non-trivial. We shall rarely be concerned with trivial rings for their own sake; however, they do sometimes arise in certain constructions. For a1 ; : : : ; ak ; b1 ; : : : ; b` 2 R, the distributive law implies X .a1 C C ak /.b1 C C b` / D ai bj : 1i k 1j `
For a1 ; : : : ; ak 2 R, the product a1 ak needs no parentheses, because multiplication is associative; moreover, we can reorder the ai ’s without changing the value of the product, since multiplication is commutative. We can also write this
168
Rings
Qk
product as i D1 ai . By convention, such a product is defined to be 1R when k D 0. When a D a1 D D ak , we can write this product as ak . The reader may verify the usual power laws: for all a; b 2 R, and all non-negative integers k and `, we have .a` /k D ak` D .ak /` ; akC` D ak a` ; .ab/k D ak b k :
(7.1)
A ring R is in particular an abelian group with respect to addition. We shall call a subgroup of the additive group of R an additive subgroup of R. The characteristic of R is defined as the exponent of this group (see §6.5). Note that for all m 2 Z and a 2 R, we have ma D m.1R a/ D .m 1R /a; so that if m 1R D 0R , then ma D 0R for all a 2 R. Thus, if the additive order of 1R is infinite, the characteristic of R is zero, and otherwise, the characteristic of R is equal to the additive order of 1R . Example 7.9. The ring Z has characteristic zero, Zn has characteristic n, and Zn1 Zn2 has characteristic lcm.n1 ; n2 /. When there is no possibility for confusion, one may write “0” instead of “0R ” and “1” instead of “1R .” Also, one may also write, for example, 2R to denote 21R , 3R to denote 3 1R , and so on; moreover, where the context is clear, one may use an implicit “type cast,” so that m 2 Z really means m 1R . E XERCISE 7.1. Show that the familiar binomial theorem (see §A2) holds in an arbitrary ring R; that is, for all a; b 2 R and every positive integer n, we have ! n X n n k k n a b : .a C b/ D k kD0
E XERCISE 7.2. Let R be a ring. For additive subgroups A and B of R, we define their ring-theoretic product AB as the set of all elements of R that can be expressed as a1 b1 C C ak bk for some a1 ; : : : ; ak 2 A and b1 ; : : : ; bk 2 B; by definition, this set includes the “empty sum” 0R . Show that for all additive subgroups A; B; and C of R: (a) AB is also an additive subgroup of R; (b) AB D BA; (c) A.BC / D .AB/C ; (d) A.B C C / D AB C AC .
7.1 Definitions, basic properties, and examples
169
7.1.1 Divisibility, units, and fields For elements a; b in a ring R, we say that a divides b if ar D b for some r 2 R. If a divides b, we write a j b, and we may say that a is a divisor of b, or that b is a multiple of a, or that b is divisible by a. If a does not divide b, then we write a − b. Note that Theorem 1.1 holds for an arbitrary ring. We call a 2 R a unit if a j 1, that is, if ar D 1 for some r 2 R. Using the same argument as was used to prove part (ii) of Theorem 6.2, it is easy to see that r is uniquely determined; it is called the multiplicative inverse of a, and we denote it by a 1 . Also, for b 2 R, we may write b=a to denote ba 1 . Evidently, if a is a unit, then a j b for every b 2 R. We denote the set of units by R . It is not hard to see that the set R is closed under multiplication, from which it follows that R is an abelian group, called the multiplicative group of units of R. If a 2 R , then of course ak 2 R for all non-negative integers k, and the multiplicative inverse of ak is .a 1 /k , which we may also write as a k (which is consistent with our notation for abelian groups). For all a; b 2 R , the identities (7.1) hold for all integers k and `. If R is non-trivial and every non-zero element of R has a multiplicative inverse, then R is called a field. Example 7.10. The only units in the ring Z are ˙1. Hence, Z is not a field. Example 7.11. Let n be a positive integer. The units in Zn are the residue classes Œan with gcd.a; n/ D 1. In particular, if n is prime, all non-zero residue classes are units, and if n is composite, some non-zero residue classes are not units. Hence, Zn is a field if and only if n is prime. Of course, the notation Zn introduced in this section for the group of units of the ring Zn is consistent with the notation introduced in §2.5. Example 7.12. Every non-zero element of Q is a unit. Hence, Q is a field. Example 7.13. Every non-zero element of R is a unit. Hence, R is a field. Example 7.14. For non-zero ˛ D a C bi 2 C, with a; b 2 R, we have c WD N.˛/ D a2 C b 2 > 0. It follows that the complex number ˛c N 1 D .ac 1 / C 1 D 1. . bc 1 /i is the multiplicative inverse of ˛, since ˛ ˛c N 1 D .˛ ˛/c N Hence, every non-zero element of C is a unit, and so C is a field. Example 7.15. For rings R1 ; : : : ; Rk , it is easy to see that the multiplicative group of units of the direct product R1 Rk is equal to R1 Rk . Indeed, by definition, .a1 ; : : : ; ak / has a multiplicative inverse if and only if each individual ai does.
170
Rings
Example 7.16. If I is an index set and R is a ring, then the units in Map.I; R/ are those functions f W I ! R such that f .i / 2 R for all i 2 I . Example 7.17. Consider the ring F of arithmetic functions defined in Example 7.6. By the result of Exercise 2.54, F D ff 2 F W f .1/ ¤ 0g. 7.1.2 Zero divisors and integral domains Let R be a ring. If a and b are non-zero elements of R such that ab D 0, then a and b are both called zero divisors. If R is non-trivial and has no zero divisors, then it is called an integral domain. Note that if a is a unit in R, it cannot be a zero divisor (if ab D 0, then multiplying both sides of this equation by a 1 yields b D 0). In particular, it follows that every field is an integral domain. Example 7.18. Z is an integral domain. Example 7.19. For n > 1, Zn is an integral domain if and only if n is prime. In particular, if n is composite, so n D ab with 1 < a < n and 1 < b < n, then Œan and Œbn are zero divisors: Œan Œbn D Œ0n , but Œan ¤ Œ0n and Œbn ¤ Œ0n . Example 7.20. Q, R, and C are fields, and hence are also integral domains. Example 7.21. For two non-trivial rings R1 ; R2 , an element .a1 ; a2 / 2 R1 R2 is a zero divisor if and only if a1 is a zero divisor, a2 is a zero divisor, or exactly one of a1 or a2 is zero. In particular, R1 R2 is not an integral domain. The next two theorems establish certain results that are analogous to familiar facts about integer divisiblity. These results hold in a general ring, provided one avoids zero divisors. The first is a cancellation law: Theorem 7.3. If R is a ring, and a; b; c 2 R such that a ¤ 0 and a is not a zero divisor, then ab D ac implies b D c. Proof. ab D bc implies a.b c/ D 0. The fact that a ¤ 0 and a is not a zero divisor implies that we must have b c D 0, and so b D c. Theorem 7.4. Let R be a ring. (i) Suppose a; b 2 R, and that either a or b is not a zero divisor. Then a j b and b j a if and only if ar D b for some r 2 R . (ii) Suppose a; b 2 R, a j b, a ¤ 0, and a is not b zero divisor. Then there exists a unique r 2 R such that ar D b, which we denote by b=a. Proof. For the first statement, if ar D b for some r 2 R , then we also have br 1 D a; thus, a j b and b j a. For the converse, suppose that a j b and b j a.
7.1 Definitions, basic properties, and examples
171
We may assume that b is not a zero divisor (otherwise, exchange the roles of a and b). We may also assume that b is non-zero (otherwise, b j a implies a D 0, and so the conclusion holds with any r). Now, a j b implies ar D b for some r 2 R, and b j a implies br 0 D a for some r 0 2 R, and hence b D ar D br 0 r. Canceling b from both sides of the equation b D br 0 r, we obtain 1 D r 0 r, and so r is b unit. For the second statement, a j b means ar D b for some r 2 R. Moreover, this value of r is unique: if ar D b D ar 0 , then we may cancel a, obtaining r D r 0 . Of course, in the previous two theorems, if the ring is an integral domain, then there are no zero divisors, and so the hypotheses may be simplified in this case, dropping the explicit requirement that certain elements are not zero divisors. In particular, if a, b, and c are elements of an integral domain, such that ab D ac and a ¤ 0, then we can cancel a, obtaining b D c. The next two theorems state some facts pertaining specifically to integral domains. Theorem 7.5. The characteristic of an integral domain is either zero or a prime. Proof. By way of contradiction, suppose that D is an integral domain with characteristic m that is neither zero nor prime. Since, by definition, D is not a trivial ring, we cannot have m D 1, and so m must be composite. Say m D st , where 1 < s < m and 1 < t < m. Since m is the additive order of 1D , it follows that .s 1D / ¤ 0D and .t 1D / ¤ 0D ; moreover, since D is an integral domain, it follows that .s 1D /.t 1D / ¤ 0D . So we have 0D D m 1D D .st / 1D D .s 1D /.t 1D / ¤ 0D ; a contradiction. Theorem 7.6. Every finite integral domain is a field. Proof. Let D be a finite integral domain, and let a be any non-zero element of D. Consider the a-multiplication map that sends b 2 D to ab, which is a group homomorphism on the additive group of D. Since a is not a zero-divisor, it follows that the kernel of the a-multiplication map is f0D g, hence the map is injective, and by finiteness, it must be surjective as well. In particular, there must be an element b 2 D such that ab D 1D . Theorem 7.7. Every finite field F must be of cardinality p w , where p is prime, w is a positive integer, and p is the characteristic of F . Proof. By Theorem 7.5, the characteristic of F is either zero or a prime, and since F is finite, it must be prime. Let p denote the characteristic. By definition, p is the exponent of the additive group of F , and by Theorem 6.42, the primes dividing
172
Rings
the exponent are the same as the primes dividing the order, and hence F must have cardinality p w for some positive integer w. Of course, for every prime p, Zp is a finite field of cardinality p. As we shall see later (in Chapter 19), for every prime p and positive integer w, there exists a field of cardinality p w . Later in this chapter, we shall see some specific examples of finite fields of cardinality p 2 (Examples 7.40, 7.59, and 7.60). E XERCISE 7.3. Let R be a ring, and let a; b 2 R such that ab ¤ 0. Show that ab is a zero divisor if and only if a is a zero divisor or b is a zero divisor. E XERCISE 7.4. Suppose that R is a non-trivial ring in which the cancellation law holds in general: for all a; b; c 2 R, if a ¤ 0 and ab D ac, then b D c. Show that R is an integral domain. E XERCISE 7.5. Let R be a ring of characteristic m > 0, and let n be an integer. Show that: (a) if gcd.n; m/ D 1, then n 1R is a unit; (b) if 1 < gcd.n; m/ < m, then n 1R is a zero divisor; (c) otherwise, n 1R D 0. E XERCISE 7.6. Let D be an integral domain, m 2 Z, and a 2 D. Show that ma D 0 if and only if m is a multiple of the characteristic of D or a D 0. E XERCISE 7.7. Show that for all n 1, and for all a; b 2 Zn , if a j b and b j a, then ar D b for some r 2 Zn . Hint: this result does not follow from part (i) of Theorem 7.4, as we allow a and b to be zero divisors here; first consider the case where n is a prime power. E XERCISE 7.8. Show that the ring F of arithmetic functions defined in Example 7.6 is an integral domain. E XERCISE 7.9. This exercise depends on results in §6.6. Using the fundamental theorem of finite abelian groups, show that the additive group of a finite field of characteristic p and cardinality p w is isomorphic to Zpw . 7.1.3 Subrings Definition 7.8. A subset S of a ring R is called a subring if (i) S is an additive subgroup of R, (ii) S is closed under multiplication, and (iii) 1R 2 S .
7.1 Definitions, basic properties, and examples
173
It is clear that the operations of addition and multiplication on a ring R make a subring S of R into a ring, where 0R is the additive identity of S and 1R is the multiplicative identity of S. One may also call R an extension ring of S . Some texts do not require that 1R belongs to a subring S , and instead require only that S contains a multiplicative identity, which may be different than that of R. This is perfectly reasonable, but for simplicity, we restrict ourselves to the case where 1R 2 S. Expanding the above definition, we see that a subset S of R is a subring if and only if 1R 2 S and for all a; b 2 S, we have a C b 2 S;
a 2 S; and ab 2 S:
In fact, to verify that S is a subring, it suffices to show that 1R 2 S and that S is closed under addition and multiplication; indeed, if 1R 2 S and S is closed under multiplication, then S is closed under negation, and further, 1R D . 1R / 2 S . Example 7.22. Z is a subring of Q. Example 7.23. Q is a subring of R. Example 7.24. R is a subring of C. Note that for all ˛ WD a C bi 2 C, with a; b 2 R, we have ˛N D ˛ ” a C bi D a bi ” b D 0. That is, ˛N D ˛ ” ˛ 2 R. Example 7.25. The set ZŒi of complex numbers of the form aCbi , with a; b 2 Z, is a subring of C. It is called the ring of Gaussian integers. Since C is a field, C contains no zero divisors, and hence ZŒi contains no zero divisors either. Hence, ZŒi is an integral domain. Let us determine the units of ZŒi . If ˛ 2 ZŒi is a unit, then there exists 0 ˛ 2 ZŒi such that ˛˛ 0 D 1. Taking norms, we obtain 1 D N.1/ D N.˛˛ 0 / D N.˛/N.˛ 0 /: Clearly, the norm of a Gaussian integer is a non-negative integer, and so N.˛/N.˛ 0 / D 1 implies N.˛/ D 1. Now, if ˛ D a C bi, with a; b 2 Z, then N.˛/ D a2 C b 2 , and so N.˛/ D 1 implies ˛ D ˙1 or ˛ D ˙i . Conversely, it is clear that ˙1 and ˙i are indeed units, and so these are the only units in ZŒi . Example 7.26. Let m be a positive integer, and let Q.m/ be the set of rational numbers which can be written as a=b, where a and b are integers, and b is relatively prime to m. Then Q.m/ is a subring of Q, since for all a; b; c; d 2 Z with gcd.b; m/ D 1 and gcd.d; m/ D 1, we have a c ad C bc a c ac C D and D ; b d bd b d bd
174
Rings
and since gcd.bd; m/ D 1, it follows that the sum and product of any two elements of Q.m/ is again in Q.m/ . Clearly, Q.m/ contains 1, and so it follows that Q.m/ is a subring of Q. The units of Q.m/ are precisely those rational numbers of the form a=b, where gcd.a; m/ D gcd.b; m/ D 1. Example 7.27. Suppose R is a non-trivial ring. Then the set f0R g is not a subring of R: although it satisfies the first two requirements of the definition of a subring, it does not satisfy the third. Generalizing the argument in Example 7.25, it is clear that every subring of an integral domain is itself an integral domain. However, it is not the case that a subring of a field is always a field: the subring Z of Q is a counter-example. If F 0 is a subring of a field F , and F 0 is itself a field, then we say that F 0 is a subfield of F , and that F is an extension field of F 0 . For example, Q is a subfield of R, which in turn is a subfield of C. E XERCISE 7.10. Show that if S and T are subrings of R, then so is S \ T . E XERCISE 7.11. Let S1 be a subring of R1 , and S2 a subring of R2 . Show that S1 S2 is a subring of R1 R2 . E XERCISE 7.12. Suppose that S and T are subrings of a ring R. Show that their ring-theoretic product S T (see Exercise 7.2) is a subring of R that contains S [ T , and is the smallest such subring. E XERCISE 7.13. Show that the set QŒi of complex numbers of the form a C bi, with a; b 2 Q, is a subfield of C. E XERCISE 7.14. Consider the ring Map.R; R/ of functions f W R ! R, with addition and multiplication defined point-wise. (a) Show that Map.R; R/ is not an integral domain, and that Map.R; R/ consists of those functions that never vanish. (b) Let a; b 2 Map.R; R/. Show that if a j b and b j a, then ar D b for some r 2 Map.R; R/ . (c) Let C be the subset of Map.R; R/ of continuous functions. Show that C is a subring of Map.R; R/, and that all functions in C are either everywhere positive or everywhere negative. (d) Find elements a; b 2 C, such that in the ring C, we have a j b and b j a, yet there is no r 2 C such that ar D b.
175
7.2 Polynomial rings
7.2 Polynomial rings If R is a ring, then we can form the ring of polynomials RŒX, consisting of all polynomials g D a0 C a1 X C C ak Xk in the indeterminate, or “formal” variable, X, with coefficients ai in R, and with addition and multiplication defined in the usual way. Example 7.28. Let us define a few polynomials over the ring Z: a WD 3 C X2 ; b WD 1 C 2X
X3 ; c WD 5; d WD 1 C X; e WD X; f WD 4X3 :
We have aCb D 4C2XCX2 X3 ; ab D 3C6XCX2 X3 X5 ; cd Cef D 5C5XC4X4 : As illustrated in the previous example, elements of R are also polynomials. Such polynomials are called constant polynomials; all other polynomials are called non-constant polynomials. The set R of constant polynomials forms a subring of RŒX. In particular, 0R is the additive identity in RŒX and 1R is the multiplicative identity in RŒX. Note that if R is the trivial ring, then so is RŒX; also, if R is a subring of E, then RŒX is a subring of EŒX. So as to keep the distinction between ring elements and indeterminates clear, we shall use the symbol “X” only to denote the latter. Also, for a polynomial g 2 RŒX, we shall in general write this simply as “g,” and not as “g.X/.” Of course, the choice of the symbol “X” is arbitrary; occasionally, we may use another symbol, such as “Y,” as an alternative. 7.2.1 Formalities For completeness, we present a more formal definition of the ring RŒX. The reader should bear in mind that this formalism is rather tedious, and may be more distracting than it is enlightening. Formally, a polynomial g 2 RŒX is an infinite sequence fai g1 i D0 , where each ai 2 R, but only finitely many of the ai ’s are non-zero. Intuitively, ai represents the coefficient of Xi . For non-negative integer j and c 2 R, it will be convenient to define j .c/ to be the sequence fci g1 i D0 2 RŒX, where cj WD c and ci WD 0R for i ¤ j . For 1 g D fai g1 i D0 2 RŒX and h D fbi gi D0 2 RŒX;
we define 1 g C h WD fsi g1 i D0 and gh WD fpi gi D0 ;
where for i D 0; 1; 2; : : : ; si WD ai C bi
(7.2)
176
Rings
and pi WD
X
aj bk ;
(7.3)
i Dj Ck
the sum being over all pairs .j; k/ of non-negative integers such that i D j C k (which is a finite sum). We leave it to the reader to verify that g C h and gh are polynomials (i.e., only finitely many of the si ’s and pi ’s are non-zero). The reader may also verify that all the requirements of Definition 7.1 are satisfied: the additive identity is the all-zero sequence, and the multiplicative identity is 0 .1R /. One can easily verify that for all c; d 2 R, we have 0 .c C d / D 0 .c/ C 0 .d / and 0 .cd / D 0 .c/0 .d /: We shall identify c 2 R with 0 .c/ 2 RŒX, viewing the ring element c as simply “shorthand” for the polynomial 0 .c/ in contexts where a polynomial is expected. Note that while c and 0 .c/ are not the same mathematical object, there will certainly be no possible confusion in treating them as such. Thus, from a narrow, legalistic point of view, R is not a subring of RŒX, but we shall not let such annoying details prevent us from continuing to speak of it as such. Indeed, by appropriately renaming elements, we can make R a subring of RŒX in the literal sense of the term. We also define X WD 1 .1R /. One can easily verify that Xi D i .1R / for all i 0. More generally, for any polynomial g D fai g1 i D0 , if ai D 0R for all i Pk exceeding some value k, then we have g D i D0 0 .ai /Xi . Writing ai in place of P 0 .ai /, we have g D kiD0 ai Xi , and so we can return to the standard practice of writing polynomials as we did in Example 7.28, without any loss of precision. 7.2.2 Basic properties of polynomial rings P Let R be a ring. For non-zero g 2 RŒX, if g D kiD0 ai Xi with ak ¤ 0, then we call k the degree of g, denoted deg.g/, we call ak the leading coefficient of g, denoted lc.g/, and we call a0 the constant term of g. If lc.g/ D 1, then g is called monic. P P Suppose g D kiD0 ai Xi and h D `iD0 bi Xi are polynomials such that ak ¤ 0 and b` ¤ 0, so that deg.g/ D k and lc.g/ D ak , and deg.h/ D ` and lc.h/ D b` . When we multiply these two polynomials, we get gh D a0 b0 C .a0 b1 C a1 b0 /X C C ak b` XkC` : In particular, deg.gh/ deg.g/Cdeg.h/. If either of ak or b` are not zero divisors, then ak b` is not zero, and hence deg.gh/ D deg.g/ C deg.h/. However, if both ak
177
7.2 Polynomial rings
and b` are zero divisors, then we may have ak b` D 0, in which case, the product gh may be zero, or perhaps gh ¤ 0 but deg.gh/ < deg.g/ C deg.h/. For the zero polynomial, we establish the following conventions: its leading coefficient and constant term are defined to be 0R , and its degree is defined to be 1. With these conventions, we may succinctly state that for all g; h 2 RŒX, we have deg.gh/ deg.g/ C deg.h/, with equality guaranteed to hold unless the leading coefficients of both g and h are zero divisors. In particular, if the leading coefficient of a polynomial is not a zero divisor, then the polynomial is not a zero divisor. In the case where the ring of coefficients is an integral domain, we can be more precise: Theorem 7.9. Let D be an integral domain. Then: (i) for all g; h 2 DŒX, we have deg.gh/ D deg.g/ C deg.h/; (ii) DŒX is an integral domain; (iii) .DŒX/ D D . Proof. Exercise. An extremely important property of polynomials is a division with remainder property, analogous to that for the integers: Theorem 7.10 (Division with remainder property). Let R be a ring. For all g; h 2 RŒX with h ¤ 0 and lc.h/ 2 R , there exist unique q; r 2 RŒX such that g D hq C r and deg.r/ < deg.h/. Proof. Consider the set S WD fg ht W t 2 RŒXg. Let r D g hq be an element of S of minimum degree. We must have deg.r/ < deg.h/, since otherwise, we could subtract an appropriate multiple of h from r so as to eliminate the leading coefficient of r, obtaining r 0 WD r
h .lc.r/ lc.h/
1 deg.r/ deg.h/
X
/ 2 S;
where deg.r 0 / < deg.r/, contradicting the minimality of deg.r/. That proves the existence of r and q. For uniqueness, suppose that g D hq C r and g D hq 0 C r 0 , where deg.r/ < deg.h/ and deg.r 0 / < deg.h/. This implies r 0 r D h .q q 0 /: However, if q ¤ q 0 , then deg.h/ > deg.r 0
r/ D deg.h .q
q 0 // D deg.h/ C deg.q
q 0 / deg.h/;
which is impossible. Therefore, we must have q D q 0 , and hence r D r 0 . If g D hq Cr as in the above theorem, we define g mod h WD r. Clearly, h j g if
178
Rings
and only if g mod h D 0. Moreover, note that if deg.g/ < deg.h/, then q D 0 and r D g; otherwise, if deg.g/ deg.h/, then q ¤ 0 and deg.g/ D deg.h/ C deg.q/. 7.2.3 Polynomial evaluation P Of course, a polynomial g D kiD0 ai Xi defines a polynomial function on R that P sends x 2 R to kiD0 ai x i 2 R, and we denote the value of this function as g.x/ (note that “X” denotes an indeterminate, while “x” denotes an element of R). It is important to regard polynomials over R as formal expressions, and not to identify them with their corresponding functions. In particular, two polynomials are equal if and only if their coefficients are equal, while two functions are equal if and only if their values agree at all points in R. This distinction is important, since there are rings R over which two different polynomials define the same function. One can of course define the ring of polynomial functions on R, but in general, that ring has a different structure from the ring of polynomials over R. Example 7.29. In the ring Zp , for prime p, by Fermat’s little theorem (Theorem 2.14), we have x p D x for all x 2 Zp . However, the polynomials Xp and X are not the same polynomials (in particular, the former has degree p, while the latter has degree 1). P More generally, if R is a subring of a ring E, a polynomial g D kiD0 ai Xi 2 P RŒX defines a polynomial function from E to E that sends ˛ 2 E to kiD0 ai ˛ i 2 E, and the value of this function is denoted g.˛/. We say that ˛ is a root of g if g.˛/ D 0. An obvious, yet important, fact is the following: Theorem 7.11. Let R be a subring of a ring E. For all g; h 2 RŒX and ˛ 2 E, if s WD g C h 2 RŒX and p WD gh 2 RŒX, then we have s.˛/ D g.˛/ C h.˛/ and p.˛/ D g.˛/h.˛/: Also, if c 2 R is a constant polynomial, then c.˛/ D c for all ˛ 2 E. Proof. The statement about evaluating a constant polynomial is clear from the definitions. The proof of the statements about evaluating the sum or product of P i polynomials is really just symbol pushing. Indeed, suppose g D i ai X and P P h D i bi Xi . Then s D i .ai C bi /Xi , and so X X X s.˛/ D .ai C bi /˛ i D ai ˛ i C bi ˛ i D g.˛/ C h.˛/: i
i
i
179
7.2 Polynomial rings
Also, we have pD
X
ai X
i
X
i
bj X
j
D
j
X
ai bj Xi Cj ;
i;j
and employing the result for evaluating sums of polynomials, we have X X X i Cj i j D D g.˛/h.˛/: p.˛/ D ai bj ˛ ai ˛ bj ˛ i;j
i
j
Example 7.30. Consider the polynomial g WD 2X3 2X2 C X 1 2 ZŒX. We can write g D .2X2 C 1/.X 1/. For any element ˛ of Z, or an extension ring of Z, we have g.˛/ D .2˛ 2 C 1/.˛ 1/. From this, it is clear that in Z, g has aproot only at 1; moreover, it has no other roots in R, but in C, it also has roots ˙i= 2. Example 7.31. If E D RŒX, then evaluating a polynomial g 2 RŒX at a point ˛ 2 E amounts to polynomial composition. For example, if g WD X2 C X then g X C 1 D .X C 1/2 C .X C 1/ D X2 C 3X C 2: The reader is perhaps familiar with the fact that over the real or the complex numbers, every polynomial of degree k has at most k distinct roots, and the fact that every set of k points can be interpolated by a unique polynomial of degree less than k. As we will now see, these results extend to much more general, though not completely arbitrary, coefficient rings. Theorem 7.12. Let R be a ring, g 2 RŒX, and x 2 R. Then there exists a unique polynomial q 2 RŒX such that g D .X x/q C g.x/. In particular, x is a root of g if and only if .X x/ divides g. Proof. If R is the trivial ring, there is nothing to prove, so assume that R is nontrivial. Using the division with remainder property for polynomials, there exist unique q; r 2 RŒX such that g D .X x/q C r, with q; r 2 RŒX and deg.r/ < 1, which means that r 2 R. Evaluating at x, we see that g.x/ D .x x/q.x/Cr D r. That proves the first statement. The second follows immediately from the first. Note that the above theorem says that X x divides g g.x/, and the polynomial q in the theorem may be expressed as qD
g
g.x/ : X x
Theorem 7.13. Let D be an integral domain, and let x1 ; : : : ; xk be distinct elements of D. Then for every polynomial g 2 DŒX, the elements x1 ; : : : ; xk are Q roots of g if and only if the polynomial kiD1 .X xi / divides g.
180
Rings
Qk
Proof. One direction is trivial: if i D1 .X xi / divides g, then it is clear that each xi is a root of g. We prove the converse by induction on k. The base case k D 1 is just Theorem 7.12. So assume k > 1, and that the statement holds for k 1. Let g 2 DŒX and let x1 ; : : : ; xk be distinct roots of g. Since xk is a root of g, then by Theorem 7.12, there exists q 2 DŒX such that g D .X xk /q. Moreover, for each i D 1; : : : ; k 1, we have 0 D g.xi / D .xi
xk /q.xi /;
and since xi xk ¤ 0 and D is an integral domain, we must have q.xi / D 0. Q Thus, q has roots x1 ; : : : ; xk 1 , and by induction ki D11 .X xi / divides q, from Q which it then follows that kiD1 .X xi / divides g. Note that in this theorem, we can slightly weaken the hypothesis: we do not need to assume that the coefficient ring is an integral domain; rather, all we really need is that for all i ¤ j , the difference xi xj is not a zero divisor. As an immediate consequence of this theorem, we obtain: Theorem 7.14. Let D be an integral domain, and let g 2 DŒX, with deg.g/ D k 0. Then g has at most k distinct roots. Proof. If g had k C 1 distinct roots x1 ; : : : ; xkC1 , then by the previous theorem, Q the polynomial kC1 i D1 .X xi /, which has degree k C 1, would divide g, which has degree k —an impossibility. Theorem 7.15 (Lagrange interpolation). Let F be a field, let x1 ; : : : ; xk be distinct elements of F , and let y1 ; : : : ; yk be arbitrary elements of F . Then there exists a unique polynomial g 2 F ŒX with deg.g/ < k such that g.xi / D yi for i D 1; : : : ; k, namely Q k X xj / j ¤i .X : g WD yi Q xj / j ¤i .xi i D1
Proof. For the existence part of the theorem, one just has to verify that g.xi / D yi for the given g, which clearly has degree less than k. This is easy to see: for i D 1; : : : ; k, evaluating the i th term in the sum defining g at xi yields yi , while evaluating any other term at xi yields 0. The uniqueness part of the theorem follows almost immediately from Theorem 7.14: if g and h are polynomials of degree less then k such that g.xi / D yi D h.xi / for i D 1; : : : ; k, then g h is a polynomial of degree less than k with k distinct roots, which, by the previous theorem, is impossible. Again, we can slightly weaken the hypothesis of this theorem: we do not need
7.2 Polynomial rings
181
to assume that the coefficient ring is a field; rather, all we really need is that for all i ¤ j , the difference xi xj is a unit. E XERCISE 7.15. Let D be an infinite integral domain, and let g; h 2 DŒX. Show that if g.x/ D h.x/ for all x 2 D, then g D h. Thus, for an infinite integral domain D, there is a one-to-one correspondence between polynomials over D and polynomial functions on D. E XERCISE 7.16. Let F be a field. (a) Show that for all b 2 F , we have b 2 D 1 if and only if b D ˙1. (b) Show that for all a; b 2 F , we have a2 D b 2 if and only if a D ˙b. (c) Show that the familiar quadratic formula holds for F , assuming F has characteristic other than 2, so that 2F ¤ 0F . That is, for all a; b; c 2 F with a ¤ 0, the polynomial g WD aX2 C bX C c 2 F ŒX has a root in F if and only if there exists e 2 F such that e 2 D d , where d is the discriminant of g, defined as d WD b 2 4ac, and in this case the roots of g are . b ˙ e/=2a. E XERCISE 7.17. Let R be a ring, let g 2 RŒX, with deg.g/ D k 0, and let x be an element of R. Show that: (a) there exist an integer m, with 0 m k, and a polynomial q 2 RŒX, such that g D .X
x/m q and q.x/ ¤ 0;
and moreover, the values of m and q are uniquely determined; (b) if we evaluate g at X C x, we have k X g XCx D bi Xi ; i D0
where b0 D D bm
1
D 0 and bm D q.x/ ¤ 0.
Let mx .g/ denote the value m in the previous exercise; for completeness, one can define mx .g/ WD 1 if g is the zero polynomial. If mx .g/ > 0, then x is called a root of g of multiplicity mx .g/; if mx .g/ D 1, then x is called a simple root of g, and if mx .g/ > 1, then x is called a multiple root of g. The following exercise refines Theorem 7.14, taking into account multiplicities. E XERCISE 7.18. Let D be an integral domain, and let g 2 DŒX, with deg.g/ D k 0. Show that X mx .g/ k: x2D
182
Rings
E XERCISE 7.19. Let D be an integral domain, let g; h 2 DŒX, and let x 2 D. Show that mx .gh/ D mx .g/ C mx .h/. 7.2.4 Multi-variate polynomials One can naturally generalize the notion of a polynomial in a single variable to that of a polynomial in several variables. Consider the ring RŒX of polynomials over a ring R. If Y is another indeterminate, we can form the ring RŒXŒY of polynomials in Y whose coefficients are themselves polynomials in X over the ring R. One may write RŒX; Y instead of RŒXŒY. An element of RŒX; Y is called a bivariate polynomial. Consider a typical element g 2 RŒX; Y, which may be written ` X k X i gD aij X Yj : (7.4) j D0 i D0
Rearranging terms, this may also be written as X gD aij Xi Yj ;
(7.5)
0i k 0j `
or as gD
k X ` X
aij Y
j
Xj :
(7.6)
i D0 j D0
If g is written as in (7.5), the terms Xi Yj are called monomials. The total degree of such a monomial Xi Yj is defined to be i C j , and if g is non-zero, then the total degree of g, denoted Deg.g/, is defined to be the maximum total degree among all monomials Xi Yj appearing in (7.5) with a non-zero coefficient aij . We define the total degree of the zero polynomial to be 1. When g is written as in (7.6), one sees that we can naturally view g as an element of RŒYŒX, that is, as a polynomial in X whose coefficients are polynomials in Y. From a strict, syntactic point of view, the rings RŒYŒX and RŒXŒY are not the same, but there is no harm done in blurring this distinction when convenient. We denote by degX .g/ the degree of g, viewed as a polynomial in X, and by degY .g/ the degree of g, viewed as a polynomial in Y. Example 7.32. Let us illustrate, with a particular example, the three different forms — as in (7.4), (7.5), and (7.6) — of expressing a bivariate polynomial. In
7.2 Polynomial rings
183
the ring ZŒX; Y we have g D .5X2
3X C 4/Y C .2X2 C 1/
D 5X2 Y C 2X2
3XY C 4Y C 1
D .5Y C 2/X2 C . 3Y/X C .4Y C 1/: We have Deg.g/ D 3, degX .g/ D 2, and degY .g/ D 1. More generally, if X1 ; : : : ; Xn are indeterminates, we can form the ring RŒX1 ; : : : ; Xn of multi-variate polynomials in n variables over R. Formally, we can define this ring recursively as RŒX1 ; : : : ; Xn 1 ŒXn , that is, the ring of polynomials in the variable Xn , with coefficients in RŒX1 ; : : : ; Xn 1 . A monomial is a term of the form Xe11 Xenn , and the total degree of such a monomial is e1 C C en . Every non-zero multi-variate polynomial g can be expressed uniquely (up to a re-ordering of terms) as a1 1 C C ak k , where each ai is a non-zero element of R, and each i is a monomial; we define the total degree of g, denoted Deg.g/, to be the maximum of the total degrees of the i ’s. As usual, the zero polynomial is defined to have total degree 1. Just as for bivariate polynomials, the order of the indeterminates is not important, and for every i D 1; : : : ; n, one can naturally view any g 2 RŒX1 ; : : : ; Xn as a polynomial in Xi over the ring RŒX1 ; : : : ; Xi 1 ; Xi C1 ; : : : ; Xn , and define degXi .g/ to be the degree of g when viewed in this way. Just as polynomials in a single variable define polynomial functions, so do polynomials in several variables. If R is a subring of E, g 2 RŒX1 ; : : : ; Xn , and ˛1 ; : : : ; ˛n 2 E, we define g.˛1 ; : : : ; ˛n / to be the element of E obtained by evaluating the expression obtained by substituting ˛i for Xi in g. Theorem 7.11 carries over directly to the multi-variate case. E XERCISE 7.20. Let R be a ring, and consider the ring of multi-variate polynomials RŒX1 ; : : : ; Xn . For m 0, define Hm to be the subset of polynomials that can be expressed as a1 1 C C ak k , where each ai belongs to R and each i is a monomial of total degree m (by definition, Hm includes the zero polynomial, and H0 D R). Polynomials that belong to Hm for some m are called homogeneous polynomials. Show that: (a) if g; h 2 Hm , then g C h 2 Hm ; (b) if g 2 H` and h 2 Hm , then gh 2 H`Cm ; (c) every non-zero polynomial g can be expressed uniquely as g0 C C gd , where gi 2 Hi for i D 0; : : : ; d , gd ¤ 0, and d D Deg.g/; (d) for all polynomials g; h, we have Deg.gh/ Deg.g/ C Deg.h/, and if R is an integral domain, then Deg.gh/ D Deg.g/ C Deg.h/.
184
Rings
E XERCISE 7.21. Suppose that D is an integral domain, and g; h are non-zero, multi-variate polynomials over D such that gh is homogeneous. Show that g and h are also homogeneous. E XERCISE 7.22. Let R be a ring, and let x1 ; : : : ; xn be elements of R. Show that every polynomial g 2 RŒX1 ; : : : ; Xn can be expressed as g D .X1
x1 /q1 C C .Xn
xn /qn C g.x1 ; : : : ; xn /;
where q1 ; : : : ; qn 2 RŒX1 ; : : : ; Xn . E XERCISE 7.23. This exercise generalizes Theorem 7.14. Let D be an integral domain, and let g 2 DŒX1 ; : : : ; Xn , with Deg.g/ D k 0. Let T be a finite subset of D. Show that the number of elements .x1 ; : : : ; xn / 2 T n such that g.x1 ; : : : ; xn / D 0 is at most kjT jn 1 . 7.3 Ideals and quotient rings Definition 7.16. Let R be a ring. An ideal of R is an additive subgroup I of R that is closed under multiplication by elements of R, that is, for all a 2 I and r 2 R, we have ar 2 I . Expanding the above definition, we see that a non-empty subset I of R is an ideal of R if and only if for all a; b 2 I and r 2 R, we have a C b 2 I;
a 2 I; and ar 2 I:
Since R is commutative, the condition ar 2 I is equivalent to ra 2 I . The condition a 2 I is redundant, as it is implied by the condition ar 2 I with r D 1R . In the case when R is the ring Z, this definition of an ideal is consistent with that given in §1.2. Clearly, f0R g and R are ideals of R. From the fact that an ideal I is closed under multiplication by elements of R, it is easy to see that I D R if and only if 1R 2 I . Example 7.33. For each m 2 Z, the set mZ is not only an additive subgroup of the ring Z, it is also an ideal of this ring. Example 7.34. For each m 2 Z, the set mZn is not only an additive subgroup of the ring Zn , it is also an ideal of this ring. Example 7.35. In the previous two examples, we saw that for some rings, the notion of an additive subgroup coincides with that of an ideal. Of course, that is the exception, not the rule. Consider the ring of polynomials RŒX. Suppose g is a non-zero polynomial in RŒX. The additive subgroup generated by g contains only polynomials whose degrees are at most that of g. However, this subgroup is not an
7.3 Ideals and quotient rings
185
ideal, since every ideal containing g must also contain g Xi for all i 0, and must therefore contain polynomials of arbitrarily high degree. Example 7.36. Let R be a ring and x 2 R. Consider the set I WD fg 2 RŒX W g.x/ D 0g. It is not hard to see that I is an ideal of RŒX. Indeed, for all g; h 2 I and q 2 RŒX, we have .g C h/.x/ D g.x/ C h.x/ D 0 C 0 D 0 and .gq/.x/ D g.x/q.x/ D 0 q.x/ D 0. Moreover, by Theorem 7.12, we have I D f.X x/q W q 2 RŒXg. We next develop some general constructions of ideals. Theorem 7.17. Let R be a ring and let a 2 R. Then aR WD far W r 2 Rg is an ideal of R. Proof. This is an easy calculation. For all ar; ar 0 2 aR and r 00 2 R, we have ar C ar 0 D a.r C r 0 / 2 aR and .ar/r 00 D a.rr 00 / 2 aR. The ideal aR in the previous theorem is called the ideal of R generated by a. Since R is commutative, one could also write this ideal as Ra WD fra W r 2 Rg. It is easy to see that this ideal contains a, and is the smallest ideal of R with this property. An ideal of this form is called a principal ideal. Corresponding to Theorems 6.11 and 6.12, we have: Theorem 7.18. If I1 and I2 are ideals of a ring R, then so are I1 C I2 and I1 \ I2 . Proof. We already know that I1 C I2 and I1 \ I2 are additive subgroups of R, so it suffices to show that they are closed under multiplication by elements of R. The reader may easily verify that this is the case. Let a1 ; : : : ; ak be elements of a ring R. The ideal a1 R C C ak R is called the ideal of R generated by a1 ; : : : ; ak . When the ring R is clear from context, one often writes .a1 ; : : : ; ak / to denote this ideal. It is easy to see that this ideal contains a1 ; : : : ; ak , and is the smallest ideal of R with this property. Example 7.37. Let n be a positive integer, and let x be any integer. Define I WD fg 2 ZŒX W g.x/ 0 .mod n/g. We claim that I is the ideal .X x; n/ of ZŒX. To see this, consider any fixed g 2 ZŒX. Using Theorem 7.12, we have g D .X x/q C g.x/ for some q 2 ZŒX. Using the division with remainder property for integers, we have g.x/ D nq 0 C r for some r 2 f0; : : : ; n 1g and q 0 2 Z. Thus, g.x/ r .mod n/, and if g.x/ 0 .mod n/, then we must have r D 0, and hence g D .X x/q C nq 0 2 .X x; n/. Conversely, if g 2 .X x; n/, we can write g D .X x/q C nq 0 for some q; q 0 2 ZŒX, and from this, it is clear that g.x/ D nq 0 .x/ 0 .mod n/. Let I be an ideal of a ring R. Since I is an additive subgroup of R, we may adopt
186
Rings
the congruence notation in §6.3, writing a b .mod I / to mean a b 2 I , and we can form the additive quotient group R=I of cosets. Recall that for a 2 R, the coset of R containing a is denoted ŒaI , and that ŒaI D a C I D fa C x W x 2 I g. Also recall that addition in R=I was defined in terms of addition of coset representatives; that is, for a; b 2 I , we defined ŒaI C ŒbI WD Œa C bI : Theorem 6.16 ensured that this definition was unambiguous. Our goal now is to make R=I into a ring by similarly defining multiplication in R=I in terms of multiplication of coset representatives. To do this, we need the following multiplicative analog of Theorem 6.16, which exploits in an essential way the fact that an ideal is closed under multiplication by elements of R; in fact, this is one of the main motivations for defining the notion of an ideal as we did. Theorem 7.19. Let I be an ideal of a ring R, and let a; a0 ; b; b 0 2 R. If a a0 .mod I / and b b 0 .mod I /, then ab a0 b 0 .mod I /. Proof. If a D a0 C x for some x 2 I and b D b 0 C y for some y 2 I , then ab D a0 b 0 Ca0 y Cb 0 x Cxy. Since I is closed under multiplication by elements of R, we see that a0 y; b 0 x; xy 2 I , and since I is closed under addition, a0 y Cb 0 x Cxy 2 I . Hence, ab a0 b 0 2 I . Using this theorem we can now unambiguously define multiplication on R=I as follows: for a; b 2 R, ŒaI ŒbI WD ŒabI : Once that is done, it is straightforward to verify that all the properties that make R a ring are inherited by R=I — we leave the details of this to the reader. In particular, the multiplicative identity of R=I is the coset Œ1R I . The ring R=I is called the quotient ring or residue class ring of R modulo I . Elements of R=I may be called residue classes. Note that if I D dR, then a b .mod I / if and only if d j .a b/, and as a matter of notation, one may simply write this congruence as a b .mod d /. We may also write Œad instead of ŒaI . Finally, note that if I D R, then R=I is the trivial ring. Example 7.38. For each n 1, the ring Zn is precisely the quotient ring Z=nZ. Example 7.39. Let f be a polynomial over a ring R with deg.f / D ` 0 and lc.f / 2 R , and consider the quotient ring E WD RŒX=f RŒX. By the division with remainder property for polynomials (Theorem 7.10), for every g 2 RŒX, there
7.3 Ideals and quotient rings
187
exists a unique polynomial h 2 RŒX such that g h .mod f / and deg.h/ < `. From this, it follows that every element of E can be written uniquely as Œhf , where h 2 RŒX is a polynomial of degree less than `. Note that in this situation, we will generally prefer the more compact notation RŒX=.f /, instead of RŒX=f RŒX. Example 7.40. Consider the polynomial f WD X2 CXC1 2 Z2 ŒX and the quotient ring E WD Z2 ŒX=.f /. Let us name the elements of E as follows: 00 WD Œ0f ; 01 WD Œ1f ; 10 WD ŒXf ; 11 WD ŒX C 1f : With this naming convention, addition of two elements in E corresponds to just computing the bit-wise exclusive-or of their names. More precisely, the addition table for E is the following: C 00 01 10 11
00 00 01 10 11
01 01 00 11 10
10 10 11 00 01
11 11 10 01 00
Note that 00 acts as the additive identity for E, and that as an additive group, E is isomorphic to the additive group Z2 Z2 . As for multiplication in E, one has to compute the product of two polynomials, and then reduce modulo f . For example, to compute 10 11, using the identity X2 X C 1 .mod f /, one sees that X .X C 1/ X2 C X .X C 1/ C X 1 .mod f /I thus, 10 11 D 01. The reader may verify the following multiplication table for E: 00 01 10 11
00 00 00 00 00
01 00 01 10 11
10 00 10 11 01
11 00 11 01 10
Observe that 01 acts as the multiplicative identity for E. Notice that every nonzero element of E has a multiplicative inverse, and so E is in fact a field. Observe that E is cyclic: the reader may verify that both 10 and 11 have multiplicative order 3. This is the first example we have seen of a finite field whose cardinality is not prime. E XERCISE 7.24. Show that if F is a field, then the only ideals of F are f0F g and F.
188
Rings
E XERCISE 7.25. Let a; b be elements of a ring R. Show that a j b ” b 2 aR ” bR aR: E XERCISE 7.26. Let R be a ring. Show that if I is a non-empty subset of RŒX that is closed under addition, multiplication by elements of R, and multiplication by X, then I is an ideal of RŒX. E XERCISE 7.27. Let I be an ideal of R, and S a subring of R. Show that I \ S is an ideal of S . E XERCISE 7.28. Let I be an ideal of R, and S a subring of R. Show that I C S is a subring of R, and that I is an ideal of I C S . E XERCISE 7.29. Let I1 be an ideal of R1 , and I2 an ideal of R2 . Show that I1 I2 is an ideal of R1 R2 . E XERCISE 7.30. Write down the multiplication table for Z2 ŒX=.X2 C X/. Is this a field? E XERCISE 7.31. Let I be an ideal of a ring R, and let x and y be elements of R with x y .mod I /. Let g 2 RŒX. Show that g.x/ g.y/ .mod I /. E XERCISE 7.32. Let R be a ring, and fix x1 ; : : : ; xn 2 R. Let I WD fg 2 RŒX1 ; : : : ; Xn W g.x1 ; : : : ; xn / D 0g. Show that I is an ideal of RŒX1 ; : : : ; Xn , and that I D .X1 x1 ; : : : ; Xn xn /. E XERCISE 7.33. Let p be a prime, and consider the ring Q.p/ (see Example 7.26). Show that every non-zero ideal of Q.p/ is of the form .p i /, for some uniquely determined integer i 0. E XERCISE 7.34. Let p be a prime. Show that in the ring ZŒX, the ideal .X; p/ is not a principal ideal. E XERCISE 7.35. Let F be a field. Show that in the ring F ŒX; Y, the ideal .X; Y/ is not a principal ideal. E XERCISE 7.36. Let R be a ring, and let fIi g1 of ideals of R i D1 be a sequence S such that Ii Ii C1 for all i D 1; 2; 3; : : : : Show that the union 1 i D1 Ii is also an ideal of R. An ideal I of a ring R is called prime if I ¨ R and if for all a; b 2 R, ab 2 I implies a 2 I or b 2 I . An ideal I of a ring R is called maximal if I ¨ R and there are no ideals J of R such that I ¨ J ¨ R. E XERCISE 7.37. Let R be a ring. Show that:
7.3 Ideals and quotient rings
189
(a) an ideal I of R is prime if and only if R=I is an integral domain; (b) an ideal I of R is maximal if and only if R=I is a field; (c) all maximal ideals of R are also prime ideals. E XERCISE 7.38. This exercise explores some examples of prime and maximal ideals. Show that: (a) in the ring Z, the ideal f0g is prime but not maximal, and that the maximal ideals are precisely those of the form pZ; where p is prime. (b) in an integral domain D, the ideal f0g is prime, and this ideal is maximal if and only if D is a field; (c) if p is a prime, then in the ring ZŒX, the ideal .X; p/ is maximal, while the ideals .X/ and .p/ are prime, but not maximal; (d) if F is a field, then in the ring F ŒX; Y, the ideal .X; Y/ is maximal, while the ideals .X/ and .Y/ are prime, but not maximal. E XERCISE 7.39. It is a fact that every non-trivial ring R contain at least one maximal ideal. Showing this in general requires some fancy set-theoretic notions. This exercise develops a simple proof in the case where R is countable (see §A3). (a) Show that if R is non-trivial but finite, then it contains a maximal ideal. (b) Assume that R is countably infinite, and let a1 ; a2 ; a3 ; : : : be an enumeration of the elements of R. Define a sequence of ideals I0 ; I1 ; I2 ; : : : ; as follows. Set I0 WD f0R g, and for each i 0, define Ii C ai R if Ii C ai R ¨ R; Ii C1 WD Ii otherwise. S1 Finally, set I WD i D0 Ii . Show that I is a maximal ideal of R. Hint: first show that I is an ideal; then show that I ¨ R by assuming that 1R 2 I and deriving a contradiction; finally, show that I is maximal by assuming that for some i D 1; 2; : : : ; we have I ¨ I C ai R ¨ R, and deriving a contradiction. E XERCISE 7.40. Let R be a ring, and let I and J be ideals of R. With the ringtheoretic product as defined in Exercise 7.2, show that: (a) IJ is an ideal; (b) if I and J are principal ideals, with I D aR and J D bR, then IJ D abR, and so is also a principal ideal; (c) IJ I \ J ; (d) if I C J D R, then IJ D I \ J .
190
Rings
E XERCISE 7.41. Let R be a subring of E, and I an ideal of R. Show that the ring-theoretic product IE is an ideal of E that contains I , and is the smallest such ideal. E XERCISE 7.42. Let M be a maximal ideal of a ring R, and let a; b 2 R. Show that if ab 2 M 2 and b … M , then a 2 M 2 . Here, M 2 WD MM , the ring-theoretic product. E XERCISE 7.43. Let F be a field, and let f 2 F ŒX; Y. Define V .f / WD f.x; y/ 2 F F W f .x; y/ D 0g. Let E WD F ŒX; Y=.f /. (a) Every element ˛ of E naturally defines a function from V .f / to F , as follows: if ˛ D Œgf , with g 2 F ŒX; Y, then for P D .x; y/ 2 V .f /, we define ˛.P / WD g.x; y/. Show that this definition is unambiguous, that is, g h .mod f / implies g.x; y/ D h.x; y/. (b) For P D .x; y/ 2 V .f /, define MP WD f˛ 2 E W ˛.P / D 0g. Show that MP is a maximal ideal of E, and that MP D ECE, where WD ŒX xf and WD ŒY yf . E XERCISE 7.44. Continuing with the previous exercise, now assume that the characteristic of F is not 2, and that f D Y2 , where 2 F ŒX is a non-zero polynomial with no multiple roots in F . (a) Show that if P D .x; y/ 2 V .f /, then so is PN WD .x; y/, and that P D PN ” y D 0 ” .x/ D 0. (b) Let P D .x; y/ 2 V .f / and WD ŒX xf 2 E. Show that E D MP MPN (the ring-theoretic product). Hint: treat the cases P D PN and P ¤ PN separately, and use Exercise 7.42. E XERCISE 7.45. Let R be a ring, and I an ideal of R. Define Rad.I / WD fa 2 R W an 2 I for some positive integer ng. (a) Show that Rad.I / is an ideal. Hint: show that if an 2 I and b m 2 I , then .a C b/nCm 2 I . (b) Show that if R D Z and I D .d /, where d D p1e1 prer is the prime factorization of d , then Rad.I / D .p1 pr /. 7.4 Ring homomorphisms and isomorphisms Definition 7.20. A function from a ring R to a ring R0 is called a ring homomorphism if (i) is a group homomorphism with respect to the underlying additive groups of R and R0 ,
7.4 Ring homomorphisms and isomorphisms
191
(ii) .ab/ D .a/.b/ for all a; b 2 R, and (iii) .1R / D 1R0 . Expanding the definition, we see that the requirements that must satisfy in order to be a ring homomorphism are that for all a; b 2 R, we have .a C b/ D .a/ C .b/ and .ab/ D .a/.b/; and that .1R / D 1R0 . Note that some texts do not require that a ring homomorphism satisfies part (iii) of our definition (which is not redundant — see Examples 7.49 and 7.50 below). Since a ring homomorphism is also an additive group homomorphism, we use the same notation and terminology for image and kernel. Example 7.41. If S is a subring of a ring R, then the inclusion map i W S ! R is obviously a ring homomorphism. Example 7.42. Suppose I is an ideal of a ring R. Analogous to Example 6.36, we may define the natural map from the ring R to the quotient ring R=I as follows: W R ! R=I a 7! ŒaI : Not only is this a surjective homomorphism of additive groups, with kernel I , it is a ring homomorphism. Indeed, we have .ab/ D ŒabI D ŒaI ŒbI D .a/ .b/; and .1R / D Œ1R I , which is the multiplicative identity in R=I . Example 7.43. For a given positive integer n, the natural map from Z to Zn sends a 2 Z to the residue class Œan . This is a surjective ring homomorphism, whose kernel is nZ. Example 7.44. Let R be a subring of a ring E, and fix ˛ 2 E. The polynomial evaluation map W RŒX ! E g 7! g.˛/ is a ring homomorphism (see Theorem 7.11). The image of consists of all polynomial expressions in ˛ with coefficients in R, and is denoted RŒ˛. As the reader may verify, RŒ˛ is a subring of E containing R and ˛, and is the smallest such subring of E. Example 7.45. We can generalize the previous example to multi-variate polyno-
192
Rings
mials. If R is a subring of a ring E and ˛1 ; : : : ; ˛n 2 E, then the map W RŒX1 ; : : : ; Xn ! E g 7! g.˛1 ; : : : ; ˛n / is a ring homomorphism. Its image consists of all polynomial expressions in ˛1 ; : : : ; ˛n with coefficients in R, and is denoted RŒ˛1 ; : : : ; ˛n . Moreover, this image is a subring of E containing R and ˛1 ; : : : ; ˛n , and is the smallest such subring of E. Note that RŒ˛1 ; : : : ; ˛n D RŒ˛1 ; : : : ; ˛n 1 Œ˛n . Example 7.46. Let W R ! R0 be a ring homomorphism. We can extend the P P domain of definition of from R to RŒX by defining . i ai Xi / WD i .ai /Xi . This yields a homomorphism from RŒX into R0 ŒX. To verify this, suppose g D P P i i in RŒX. Let s WD g C h 2 RŒX and i ai X and h D i bi X are polynomials P P i p WD gh 2 RŒX, and write s D i si X and p D i pi Xi , so that X si D ai C bi and pi D aj bk : i Dj Ck
Then we have .si / D .ai C bi / D .ai / C .bj /; which is the coefficient of Xi in .g/ C .h/, and X X X .pi / D aj bk D .aj bk / D .aj /.bk /; i Dj Ck
i Dj Ck
i Dj Ck
which is the coefficient of Xi in .g/.h/. Sometimes a more compact notation is convenient: we may prefer to write aN for P the image of a 2 R under , and if we do this, then for g D i ai Xi 2 RŒX, we P write gN for the image i aN i Xi of g under the extension of to RŒX. Example 7.47. Consider the natural map that sends a 2 Z to aN WD Œan 2 Zn (see Example 7.43). As in the previous example, we may extend this to a ring P homomorphism from ZŒX to Zn ŒX, which maps g D i ai Xi 2 ZŒX to gN D P N i Xi 2 Zn ŒX. This homomorphism is clearly surjective. Let us determine its ia P kernel. Observe that if g D i ai Xi , then gN D 0 if and only if n j ai for each i ; therefore, the kernel is the ideal nZŒX of ZŒX. Example 7.48. Let R be a ring of prime characteristic p. For all a; b 2 R, we have (see Exercise 7.1) ! p X p .a C b/p D ap k b k : k kD0
7.4 Ring homomorphisms and isomorphisms
193
However, by Exercise 1.14, all of the binomial coefficients are multiples of p, except for k D 0 and k D p, and hence in the ring R, all of these terms vanish, leaving us with .a C b/p D ap C b p : This result is often jokingly referred to as the “freshman’s dream,” for somewhat obvious reasons. Of course, as always, we have p
.ab/p D ap b p and 1R D 1R ; and so it follows that the map that sends a 2 R to ap 2 R is a ring homomorphism from R into R. Example 7.49. Suppose R is a non-trivial ring, and let W R ! R map everything in R to 0R . Then satisfies parts (i) and (ii) of Definition 7.20, but not part (iii). Example 7.50. In special situations, part (iii) of Definition 7.20 may be redundant. One such situation arises when W R ! R0 is surjective. In this case, we know that 1R0 D .a/ for some a 2 R, and by part (ii) of the definition, we have .1R / D .1R / 1R0 D .1R /.a/ D .1R a/ D .a/ D 1R0 : For a ring homomorphism W R ! R0 , all of the results of Theorem 6.19 apply. In particular, .0R / D 0R0 , .a/ D .b/ if and only if a b .mod Ker /, and is injective if and only if Ker D f0R g. However, we may strengthen Theorem 6.19 as follows: Theorem 7.21. Let W R ! R0 be a ring homomorphism. (i) If S is a subring of R, then .S / is a subring of R0 ; in particular (setting S WD R), Im is a subring of R0 . (ii) If S 0 is a subring of R0 , then
1 .S 0 /
is a subring of R.
(ii) If I is an ideal of R, then .I / is an ideal of Im . (iv) If I 0 is an ideal of Im , then 1 .I 0 / is an ideal of R; in particular (setting I 0 WD f0R0 g), Ker is an ideal of R. Proof. In each part, we already know that the relevant object is an additive subgroup, and so it suffices to show that the appropriate additional properties are satisfied. (i) For all a; b 2 S , we have ab 2 S , and hence .S / contains .ab/ D .a/.b/. Also, 1R 2 S, and hence .S / contains .1R / D 1R0 . (ii) If .a/ 2 S 0 and .b/ 2 S 0 , then .ab/ D .a/.b/ 2 S 0 . Also, .1R / D 1R 0 2 S 0 .
194
Rings
(iii) For all a 2 I and r 2 R, we have ar 2 I , and hence .I / contains .ar/ D .a/.r/. (iv) For all a 2 1 .I 0 / and r 2 R, we have .ar/ D .a/.r/, and since .a/ belongs to the ideal I 0 , so does .a/.r/, and hence 1 .I 0 / contains ar. Theorems 6.20 and 6.21 have natural ring analogs — one only has to show that the corresponding group homomorphisms satisfy the additional requirements of a ring homomorphism, which we leave to the reader to verify: Theorem 7.22. If W R ! R0 and 0 W R0 ! R00 are ring homomorphisms, then so is their composition 0 B W R ! R00 . Theorem 7.23. Let i W R ! Ri0 , for i D 1; : : : ; k, be ring homomorphisms. Then the map W R ! R10 Rk0 a 7! .1 .a/; : : : ; k .a// is a ring homomorphism. If a ring homomorphism W R ! R0 is a bijection, then it is called a ring isomorphism of R with R0 . If such a ring isomorphism exists, we say that R is isomorphic to R0 , and write R Š R0 . Moreover, if R D R0 , then is called a ring automorphism on R. Analogous to Theorem 6.22, we have: Theorem 7.24. If is a ring isomorphism of R with R0 , then the inverse function 1 is a ring isomorphism of R0 with R. Proof. Exercise. Because of this theorem, if R is isomorphic to R0 , we may simply say that “R and R0 are isomorphic.” We stress that a ring isomorphism is essentially just a “renaming” of elements; in particular, we have: Theorem 7.25. Let W R ! R0 be a ring isomorphism. (i) For all a 2 R, a is a zero divisor if and only if .a/ is a zero divisor. (ii) For all a 2 R, a is a unit if and only if .a/ is a unit. (iii) The restriction of R to R is a group isomorphism of R with .R0 / . Proof. Exercise. An injective ring homomorphism W R ! E is called an embedding of R in E. In this case, Im is a subring of E and R Š Im . If the embedding is a natural one
7.4 Ring homomorphisms and isomorphisms
195
that is clear from context, we may simply identify elements of R with their images in E under the embedding; that is, for a 2 R, we may simply write “a,” and it is understood that this really means “.a/” if the context demands an element of E. As a slight abuse of terminology, we shall say that R is a subring of E. Indeed, by appropriately renaming elements, we can always make R a subring of E in the literal sense of the term. This practice of identifying elements of a ring with their images in another ring under a natural embedding is very common. We have already seen an example of this, namely, when we formally defined the ring of polynomials RŒX over R in §7.2.1, we defined the map 0 W R ! RŒX that sends c 2 R to the polynomial whose constant term is c, and all other coefficients zero. This map 0 is an embedding, and it was via this embedding that we identified elements of R with elements of RŒX, and so viewed R as a subring of RŒX. We shall see more examples of this later (in particular, Example 7.55 below). Theorems 6.23 and 6.24 also have natural ring analogs — again, one only has to show that the corresponding group homomorphisms are also ring homomorphisms: Theorem 7.26 (First isomorphism theorem). Let W R ! R0 be a ring homomorphism with kernel K and image S 0 . Then we have a ring isomorphism R=K Š S 0 : Specifically, the map N W R=K ! R0 ŒaK 7! .a/ is an injective ring homomorphism whose image is S 0 . Theorem 7.27. Let W R ! R0 be a ring homomorphism. Then for every ideal I of R with I Ker , we may define a ring homomorphism N W R=I ! R0 ŒaI 7! .a/: Moreover, Im N D Im , and N is injective if and only if I D Ker . Example 7.51. Returning again to the Chinese remainder theorem and the discussion in Example 6.48, if fni gkiD1 is a pairwise relatively prime family of positive Q integers, and n WD kiD1 ni , then the map W Z ! Zn1 Znk a 7! .Œan1 ; : : : ; Œank /
196
Rings
is not just a surjective group homomorphism with kernel nZ, it is also a ring homomorphism. Applying Theorem 7.26, we get a ring isomorphism N W
Zn ! Zn1 Znk Œan 7! .Œan1 ; : : : ; Œank /;
which is the same function as the function in Theorem 2.8. By part (iii) of Theorem 7.25, the restriction of to Zn is a group isomorphism of Zn with the multiplicative group of units of Zn1 Znk , which (according to Example 7.15) is Zn1 Znk . Thus, part (iii) of Theorem 2.8 is an immediate consequence of the above observations. Example 7.52. Extending Example 6.49, if n1 and n2 are positive integers with n1 j n2 , then the map N W
Zn2 ! Zn1 Œan2 7! Œan1
is a surjective ring homomorphism. Example 7.53. For a ring R, consider the map W Z ! R that sends m 2 Z to m 1R in R. It is easily verified that is a ring homomorphism. If Ker D f0g, then Im Š Z, and so the ring Z is embedded in R, and R has characteristic zero. If Ker D nZ for some n > 0, then Im Š Zn , and so the ring Zn is embedded in R, and R has characteristic n. Note that Im is the smallest subring of R: any subring of R must contain 1R and be closed under addition and subtraction, and so must contain Im . Example 7.54. We can generalize Example 7.44 by evaluating polynomials at several points. This is most fruitful when the underlying coefficient ring is a field, and the evaluation points belong to the same field. So let F be a field, and let x1 ; : : : ; xk be distinct elements of F . Define the map W F ŒX ! F k g 7! .g.x1 /; : : : ; g.xk //: This is a ring homomorphism. By Theorem 7.13, Ker D .f /, where f WD Qk xi /. By Theorem 7.15, is surjective. Therefore, by Theorem 7.26, we i D1 .X get a ring isomorphism N W F ŒX=.f / ! F k Œgf 7! .g.x1 /; : : : ; g.xk //: Example 7.55. As in Example 7.39, let f be a polynomial over a ring R with deg.f / D ` and lc.f / 2 R , but now assume that ` > 0. Consider the natural
7.4 Ring homomorphisms and isomorphisms
197
map from RŒX to the quotient ring E WD RŒX=.f / that sends g 2 RŒX to Œgf . Let be the restriction of to the subring R of RŒX. Evidently, is a ring homomorphism from R into E. Moreover, since distinct polynomials of degree less than ` belong to distinct residue classes modulo f , we see that is injective. Thus, is an embedding of R into E. As is a very natural embedding, we can identify elements of R with their images in E under , and regard R as a subring P of E. Taking this point of view, we see that if g D i ai Xi , then X X X Œgf D Œ ai Xi f D Œai f .ŒXf /i D ai i D g./; i
i
i
where WD ŒXf 2 E. Therefore, the natural map may be viewed as the polynomial evaluation map (see Example 7.44) that sends g 2 RŒX to g./ 2 E. Note that we have E D RŒ; moreover, every element of E can be expressed uniquely as g./ for some g 2 RŒX of degree less than `, and more generally, for arbitrary g; h 2 RŒX, we have g./ D h./ if and only if g h .mod f /. Finally, note that f ./ D Œf f D Œ0f ; that is, is a root of f . Example 7.56. As a special case of Example 7.55, let f WD X2 C 1 2 RŒX, and consider the quotient ring RŒX=.f /. If we set i WD ŒXf 2 RŒX=.f /, then every element of RŒX=.f / can be expressed uniquely as a C bi, where a; b 2 R. Moreover, we have i 2 D 1, and more generally, for all a; b; a0 ; b 0 2 R, we have .a C bi / C .a0 C b 0 i / D .a C a0 / C .b C b 0 /i and .a C bi / .a0 C b 0 i / D .aa0
bb 0 / C .ab 0 C a0 b/i:
Thus, the rules for arithmetic in RŒX=.f / are precisely the familiar rules of complex arithmetic, and so C and RŒX=.f / are essentially the same, as rings. Indeed, the “algebraically correct” way of defining the complex numbers C is simply to define them to be the quotient ring RŒX=.f / in the first place. This will be our point of view from now on. Example 7.57. Consider the polynomial evaluation map W RŒX ! C D RŒX=.X2 C 1/ that sends g 2 RŒX to g. i /. For every g 2 RŒX, we may write g D .X2 C 1/q C a C bX, where q 2 RŒX and a; b 2 R. Since . i /2 C 1 D i 2 C 1 D 0, we have g. i / D .. i /2 C 1/q. i / C a bi D a bi. Clearly, then, is surjective and the kernel of is the ideal of RŒX generated by the polynomial X2 C 1. By Theorem 7.26, we therefore get a ring automorphism N on C that sends a C bi 2 C to a bi. In fact, N is none other than the complex conjugation map. Indeed, this is the “algebraically correct” way of defining complex conjugation in the first place.
198
Rings
Example 7.58. We defined the ring ZŒi of Gaussian integers in Example 7.25 as a subring of C. Let us verify that the notation ZŒi introduced in Example 7.25 is consistent with that introduced in Example 7.44. Consider the polynomial evaluation map W ZŒX ! C that sends g 2 ZŒX to g.i / 2 C. For every g 2 ZŒX, we may write g D .X2 C 1/q C a C bX, where q 2 ZŒX and a; b 2 Z. Since i 2 C 1 D 0, we have g.i / D .i 2 C 1/q.i / C a C bi D a C bi. Clearly, then, the image of is the set fa C bi W a; b 2 Zg, and the kernel of is the ideal of ZŒX generated by the polynomial X2 C 1. This shows that ZŒi in Example 7.25 is the same as ZŒi in Example 7.44, and moreover, Theorem 7.26 implies that ZŒi is isomorphic to ZŒX=.X2 C 1/. Thus, we can directly construct the Gaussian integers as the quotient ring ZŒX=.X2 C 1/. Likewise the field QŒi (see Exercise 7.13) can be constructed directly as QŒX=.X2 C 1/. Example 7.59. Let p be a prime, and consider the quotient ring E WD Zp ŒX=.X2 C 1/. If we set i WD ŒXX2 C1 2 E, then E D Zp Œi D fa C bi W a; b 2 Zp g. In particular, E is a ring of cardinality p 2 . Moreover, we have i 2 D 1, and the rules for addition and multiplication in E look exactly the same as they do in C: for all a; b; a0 ; b 0 2 Zp , we have .a C bi / C .a0 C b 0 i / D .a C a0 / C .b C b 0 /i and .a C bi / .a0 C b 0 i / D .aa0
bb 0 / C .ab 0 C a0 b/i:
The ring E may or may not be a field. We now determine for which primes p we get a field. If p D 2, then 0 D 1 C i 2 D .1 C i /2 (see Example 7.48), and so in this case, 1 C i is a zero divisor and E is not a field. Now suppose p is odd. There are two subcases: p 1 .mod 4/ and p 3 .mod 4/. Suppose p 1 .mod 4/. By Theorem 2.31, there exists c 2 Zp such that 2 c D 1, and therefore X2 C1 D X2 c 2 D .X c/.XCc/, and by Example 7.45, we have ring isomorphism E Š Zp Zp (which maps aCbi 2 E to .aCbc; a bc/ 2 Zp Zp ); in particular, E is not a field. Indeed, c C i is a zero divisor, since .c C i/.c i/ D c 2 i 2 D c 2 C 1 D 0. Suppose p 3 .mod 4/. By Theorem 2.31, there is no c 2 Zp such that c 2 D 1. It follows that for all a; b 2 Zp , not both zero, we must have a2 C b 2 ¤ 0; indeed, suppose that a2 C b 2 D 0, and that, say, b ¤ 0; then we would have .a=b/2 D 1, contradicting the assumption that 1 has no square root in Zp . Therefore, a2 C b 2 has a multiplicative inverse in Zp , from which it follows that
7.4 Ring homomorphisms and isomorphisms
199
the formula for multiplicative inverses in C applies equally well in E; that is, .a C bi /
1
D
a bi : a2 C b 2
Therefore, in this case, E is a field. In Example 7.40, we saw a finite field of cardinality 4. The previous example provides us with an explicit construction of a finite field of cardinality p 2 , for every prime p congruent to 3 modulo 4. As the next example shows, there exist finite fields of cardinality p 2 for all primes p. Example 7.60. Let p an odd prime, and let d 2 Zp . Let f WD X2 d 2 Zp ŒX, and consider the ring E WD Zp ŒX=.f / D Zp Œ, where WD ŒXf 2 E. We have E D fa C b W a; b 2 Zp g and jEj D p 2 . Note that 2 D d , and the general rules for arithmetic in E look like this: for all a; b; a0 ; b 0 2 Zp , we have .a C b/ C .a0 C b 0 / D .a C a0 / C .b C b 0 / and .a C b/ .a0 C b 0 / D .aa0 C bb 0 d / C .ab 0 C a0 b/: Suppose that d 2 .Zp /2 , so that d D c 2 for some c 2 Zp . Then f D .X c/.XCc/, and like in previous example, we have a ring isomorphism E Š Zp Zp (which maps a C b 2 E to .a C bc; a bc/ 2 Zp Zp ); in particular, E is not a field. Suppose that d … .Zp /2 . This implies that for all a; b 2 Zp , not both zero, we have a2 b 2 d ¤ 0. Using this, we get the following formula for multiplicative inverses in E: a b .a C b/ 1 D 2 : a b2d Therefore, E is a field in this case. By Theorem 2.20, we know that j.Zp /2 j D .p 1/=2, so there exists d 2 Zp n .Zp /2 for all odd primes p. Thus, we have a general (though not explicit) construction for finite fields of cardinality p 2 for all odd primes p. E XERCISE 7.46. Show that if W F ! R is a ring homomorphism from a field F into a ring R, then either R is trivial or is injective. Hint: use Exercise 7.24. E XERCISE 7.47. Verify that the “is isomorphic to” relation on rings is an equivalence relation; that is, for all rings R1 ; R2 ; R3 , we have: (a) R1 Š R1 ; (b) R1 Š R2 implies R2 Š R1 ;
200
Rings
(c) R1 Š R2 and R2 Š R3 implies R1 Š R3 . E XERCISE 7.48. Let i W Ri ! Ri0 , for i D 1; : : : ; k, be ring homomorphisms. Show that the map W R1 Rk ! R10 Rk0 .a1 ; : : : ; ak / 7! .1 .a1 /; : : : ; k .ak // is a ring homomorphism. E XERCISE 7.49. Let W R ! R0 be a ring homomorphism, and let a 2 R. Show that .aR/ D .a/.R/. E XERCISE 7.50. Let W R ! R0 be a ring homomorphism. Let S be a subring of R, and let W S ! R0 be the restriction of to S. Show that is a ring homomorphism and that Ker D Ker \ S . E XERCISE 7.51. Suppose R1 ; : : : ; Rk are rings. Show that for each i D 1; : : : ; k, the projection map i W R1 Rk ! Ri that sends .a1 ; : : : ; ak / to ai is a surjective ring homomorphism. E XERCISE 7.52. Show that if R D R1 R2 for rings R1 and R2 , and I1 is an ideal R1 and I2 is an ideal of R2 , then we have a ring isomorphism R=.I1 I2 / Š R1 =I1 R2 =I2 . E XERCISE 7.53. Let I be an ideal of R, and S a subring of R. As we saw in Exercises 7.27, and 7.28, I \ S is an ideal of S , and I is an ideal of the subring I C S . Show that we have a ring isomorphism .I C S /=I Š S=.I \ S /. E XERCISE 7.54. Let W R ! R0 be a ring homomorphism with kernel K. Let I be an ideal of R. Show that we have a ring isomorphism R=.I CK/ Š .R/=.R/. E XERCISE 7.55. Let n be a positive integer, and consider the natural map that sends a 2 Z to aN WD Œan 2 Zn , which we may extend coefficient-wise to a ring homomorphism from ZŒX to Zn ŒX, as in Example 7.47. Show that for every f 2 ZŒX, we have a ring isomorphism ZŒX=.f; n/ Š Zn ŒX=.fN/. E XERCISE 7.56. Let n be a positive integer. Show that we have ring isomorphisms ZŒX=.n/ Š Zn ŒX, ZŒX=.X/ Š Z, and ZŒX=.X; n/ Š Zn . E XERCISE 7.57. Let n D pq, where p and q are distinct primes. Show that we have a ring isomorphism Zn ŒX Š Zp ŒX Zq ŒX. E XERCISE 7.58. Let p be a prime with p 1 .mod 4/. Show that we have a ring isomorphism ZŒX=.X2 C 1; p/ Š Zp Zp .
7.4 Ring homomorphisms and isomorphisms
201
E XERCISE 7.59. Let W R ! R0 be a surjective ring homomorphism. Let S be the set of all ideals of R that contain Ker , and let S 0 be the set of all ideals of R0 . Show that the sets S and S 0 are in one-to-one correspondence, via the map that sends I 2 S to .I / 2 S 0 . Moreover, show that under this correspondence, prime ideals in S correspond to prime ideals in S 0 , and maximal ideals in S correspond to maximal ideals in S 0 . (See definitions above Exercise 7.37.) E XERCISE 7.60. Let n be a positive integer whose factorization into primes is n D p1e1 prer . What are the prime ideals of Zn ? (See definitions above Exercise 7.37.) E XERCISE 7.61. Let W R ! S be a ring homomorphism. Show that .R / S , and that the restriction of to R yields a group homomorphism W R ! S . E XERCISE 7.62. Let R be a ring, and let x1 ; : : : ; xn be elements of R. Show that the rings R and RŒX1 ; : : : ; Xn =.X1 x1 ; : : : ; Xn xn / are isomorphic. E XERCISE 7.63. This exercise and the next generalize the Chinese remainder theorem to arbitrary rings. Suppose I and J are two ideals of a ring R such that I C J D R. Show that the map W R ! R=I R=J that sends a 2 R to .ŒaI ; ŒaJ / is a surjective ring homomorphism with kernel IJ (see Exercise 7.40). Conclude that R=.IJ / is isomorphic to R=I R=J . E XERCISE 7.64. Generalize the previous exercise, showing that R=.I1 Ik / is isomorphic to R=I1 R=Ik , where R is a ring, and I1 ; : : : ; Ik are ideals of R, provided Ii C Ij D R for all i; j such that i ¤ j . E XERCISE 7.65. Let Q.m/ be the subring of Q defined in Example 7.26. Let us define the map W Q.m/ ! Zm as follows. For a=b 2 Q with b relatively prime to m, .a=b/ WD Œam .Œbm / 1 . Show that is unambiguously defined, and is a surjective ring homomorphism. Also, describe the kernel of . E XERCISE 7.66. Let R be a ring, a 2 R , and b 2 R. Define the map W RŒX ! RŒX that sends g 2 RŒX to g aX C b . Show that is a ring automorphism. E XERCISE 7.67. Consider the subring ZŒ1=2 of Q. Show that ZŒ1=2 D fa=2i W a; i 2 Z; i 0g, that .ZŒ1=2/ D f2i W i 2 Zg, and that every non-zero ideal of ZŒ1=2 is of the form .m/, for some uniquely determined, odd integer m.
202
Rings
7.5 The structure of Zn We are now in a position to precisely characterize the structure of the group Zn , for an arbitrary integer n > 1. This characterization will prove to be very useful in a number of applications. Suppose n D p1e1 prer is the factorization of n into primes. By the Chinese remainder theorem (see Theorem 2.8 and Example 7.51), we have the ring isomorphism W
Zn ! Zpe1 Zprer 1
Œan 7! .Œape1 ; : : : ; Œaprer /; 1
and restricting to
Zn
yields a group isomorphism Zn Š Ze1 Zper : p1
r
Thus, to determine the structure of the group Zn for general n, it suffices to determine the structure for n D p e , where p is prime. By Theorem 2.10, we already know the order of the group Zpe , namely, .p e / D p e 1 .p 1/. The main result of this section is the following: Theorem 7.28. If p is an odd prime, then for every positive integer e, the group Zpe is cyclic. The group Z2e is cyclic for e D 1 or 2, but not for e 3. For e 3, Z2e is isomorphic to the additive group Z2 Z2e 2 . In the case where e D 1, this theorem is a special case the following, more general, theorem: Theorem 7.29. Let D be an integral domain and G a subgroup of D of finite order. Then G is cyclic. Proof. Suppose G is not cyclic. Then by Theorem 6.40, we know that the exponent m of G is strictly less than jGj. It follows that am D 1 for all a 2 G. That is, all the elements of G are roots of the polynomial Xm 1 2 DŒX. But by Theorem 7.14, a polynomial of degree m over an integral domain has at most m roots, and this contradicts the fact that m < jGj. This theorem immediately implies that Zp is cyclic for every prime p, since Zp is a field; however, we cannot directly use this theorem to prove that Zpe is cyclic for e > 1 (and p odd), because Zpe is not a field. To deal with the case e > 1, we need a few simple facts. Lemma 7.30. Let p be a prime. For every positive integer e, if a b .mod p e /, then ap b p .mod p eC1 /.
7.5 The structure of Zn
203
Proof. We have a D b Ccp e for some c 2 Z. Thus, ap D b p Cpb p for an integer d . It follows that ap b p .mod p eC1 /.
1 cp e Cdp 2e
Lemma 7.31. Let p be a prime, and let e be a positive integer such that p e > 2. If a 1 C p e .mod p eC1 /, then ap 1 C p eC1 .mod p eC2 /. Proof. By Lemma 7.30, ap .1 C p e /p .mod p eC2 /. Expanding .1 C p e /p , we have ! p X1 p p ek C p ep : .1 C p e /p D 1 C p p e C k kD2
By Exercise 1.14, all of the terms in the sum on k are divisible by p 1C2e , and 1 C 2e e C 2 for all e 1. For the term p ep , the assumption that p e > 2 means that either p 3 or e 2, which implies ep e C 2. Now consider Theorem 7.28 in the case where p is odd. As we already know that Zp is cyclic, assume e > 1. Let x 2 Z be chosen so that Œxp generates Zp . Suppose the multiplicative order of Œxpe 2 Zpe is m. Then as x m 1 .mod p e / implies x m 1 .mod p/, it must be the case that p 1 divides m, and so Œx m=.p 1/ pe has multiplicative order exactly p 1. By Theorem 6.38, if we find an integer y such that Œype has multiplicative order p e 1 , then Œx m=.p 1/ ype has multiplicative order .p 1/p e 1 , and we are done. We claim that y WD 1 C p does the job. Any integer between 0 and p e 1 can be expressed as an e-digit number in base p; for example, y D .0 0 1 1/p . If we compute successive pth powers of y modulo p e , then by Lemma 7.31 we have y mod p e D .0 y p mod p e D . 2 y p mod p e D . :: : e 2
yp e yp
1
0 1 1/p ; 1 0 1/p ; 1 0 0 1/p ;
mod p e D .1 0 mod p e D .0
0 1/p ; 0 1/p :
Here, “” indicates an arbitrary digit. From this table of values, it is clear (see Theorem 6.37) that Œype has multiplicative order p e 1 . That proves Theorem 7.28 for odd p. We now prove Theorem 7.28 in the case p D 2. For e D 1 and e D 2, the theorem is easily verified. Suppose e 3. Consider the subgroup G Z2e generated by Œ52e . Expressing integers between 0 and 2e 1 as e-digit binary
204
Rings
numbers, and applying Lemma 7.31, we have 5 mod 2e D .0 52 mod 2e D . :: : e 3
52 e 52
2
0 1 0 1/2 ; 1 0 0 1/2 ;
mod 2e D .1 0 mod 2e D .0
0 1/2 ; 0 1/2 :
So it is clear (see Theorem 6.37) that Œ52e has multiplicative order 2e 2 . We claim that Œ 12e … G. If it were, then since it has multiplicative order 2, and since every cyclic group of even order has precisely one element of order 2 (see Theorem 6.32), e 3 it must be equal to Œ52 2e ; however, it is clear from the above calculation that e 3 52 6 1 .mod 2e /. Let H Z2e be the subgroup generated by Œ 12e . Then from the above, G \ H D fŒ12e g, and hence by Theorem 6.25, G H is isomorphic to the subgroup G H of Z2e . But since the orders of G H and Z2e are equal, we must have G H D Z2e . That proves the theorem. Example 7.61. Let p be an odd prime, and let d be a positive integer dividing p 1. Since Zp is a cyclic group of order p 1, Theorem 6.32, implies that .Zp /d is the unique subgroup of Zp of order .p 1/=d , and moreover, .Zp /d D Zp f.p 1/=d g; that is, for all ˛ 2 Zp , we have ˛ D ˇ d for some ˇ 2 Zp ” ˛ .p
1/=d
D 1:
Setting d D 2, we arrive again at Euler’s criterion (Theorem 2.21), but by a very different, and perhaps more elegant, route than that taken in our original proof of that theorem. E XERCISE 7.68. Show that if n is a positive integer, the group Zn is cyclic if and only if n D 1; 2; 4; p e ; or 2p e ; where p is an odd prime and e is a positive integer. E XERCISE 7.69. Let n D pq, where p and q are distinct primes such that p D 2p 0 C 1 and q D 2q 0 C 1, where p 0 and q 0 are themselves prime. Show that the subgroup .Zn /2 of squares is a cyclic group of order p 0 q 0 . E XERCISE 7.70. Let n D pq, where p and q are distinct primes such that p − .q 1/ and q − .p 1/. (a) Show that the map that sends Œan 2 Zn to Œan n2 2 .Zn2 /n is a group isomorphism (in particular, you need to show that this map is unambiguously defined).
7.5 The structure of Zn
205
(b) Consider the element ˛ WD Œ1 C nn2 2 Zn2 ; show that for every nonnegative integer k, ˛ k D Œ1 C k nn2 ; deduce that ˛ has multiplicative order n, and also that the identity ˛ k D Œ1 C k nn2 holds for all integers k. (c) Show that the map from Zn Zn to Zn2 that sends .Œkn ; Œan / to Œ.1 C k n/an n2 is a group isomorphism. E XERCISE 7.71. This exercise develops an alternative proof of Theorem 7.29 that relies on less group theory. Let n be the order of the group. Using Theorem 7.14, show that for all d j n, there are at most d elements in the group whose multiplicative order divides d . From this, deduce that for all d j n, the number of elements of multiplicative order d is either 0 or .d /. Now use Theorem 2.40 to deduce that for all d j n (and in particular, for d D n), the number of elements of multiplicative order d is equal to .d /.
8 Finite and discrete probability distributions
To understand the algorithmic aspects of number theory and algebra, and applications such as cryptography, a firm grasp of the basics of probability theory is required. This chapter introduces concepts from probability theory, starting with the basic notions of probability distributions on finite sample spaces, and then continuing with conditional probability and independence, random variables, and expectation. Applications such as “balls and bins,” “hash functions,” and the “leftover hash lemma” are also discussed. The chapter closes by extending the basic theory to probability distributions on countably infinite sample spaces. 8.1 Basic definitions Let ˝ be a finite, non-empty set. A probability distribution on ˝ is a function P W ˝ ! Œ0; 1 that satisfies the following property:
X
P.!/ D 1:
(8.1)
!2˝
The set ˝ is called the sample space of P. Intuitively, the elements of ˝ represent the possible outcomes of a random experiment, where the probability of outcome ! 2 ˝ is P.!/. For now, we shall only consider probability distributions on finite sample spaces. Later in this chapter, in §8.10, we generalize this to allow probability distributions on countably infinite sample spaces. Example 8.1. If we think of rolling a fair die, then setting ˝ WD f1; 2; 3; 4; 5; 6g, and P.!/ WD 1=6 for all ! 2 ˝, gives a probability distribution that naturally describes the possible outcomes of the experiment. Example 8.2. More generally, if ˝ is any non-empty, finite set, and P.!/ WD 1=j˝j for all ! 2 ˝, then P is called the uniform distribution on ˝. 206
8.1 Basic definitions
207
Example 8.3. A coin toss is an example of a Bernoulli trial, which in general is an experiment with only two possible outcomes: success, which occurs with probability p, and failure, which occurs with probability q WD 1 p. Of course, success and failure are arbitrary names, which can be changed as convenient. In the case of a coin, we might associate success with the outcome that the coin comes up heads. For a fair coin, we have p D q D 1=2; for a biased coin, we have p ¤ 1=2. An event is a subset A of ˝, and the probability of A is defined to be X PŒA WD P.!/:
(8.2)
!2A
While an event is simply a subset of the sample space, when discussing the probability of an event (or other properties to be introduced later), the discussion always takes place relative to a particular probability distribution, which may be implicit from context. For events A and B, their union A [ B logically represents the event that either the event A or the event B occurs (or both), while their intersection A \ B logically represents the event that both A and B occur. For an event A, we define its complement Ax WD ˝ n A, which logically represents the event that A does not occur. In working with events, one makes frequent use of the usual rules of Boolean logic. DeMorgan’s law says that for all events A and B, x A [ B D Ax \ Bx and A \ B D Ax [ B: We also have the Boolean distributive law: for all events A; B; and C, A \ .B [ C/ D .A \ B/ [ .A \ C/ and A [ .B \ C/ D .A [ B/ \ .A [ C/: Example 8.4. Continuing with Example 8.1, the event that the die has an odd value is A WD f1; 3; 5g, and we have PŒA D 1=2. The event that the die has a value greater than 2 is B WD f3; 4; 5; 6g, and PŒB D 2=3. The event that the die x D 1=3. The event that the has a value that is at most 2 is Bx D f1; 2g, and PŒB value of the die is odd or exceeds 2 is A [ B D f1; 3; 4; 5; 6g, and PŒA [ B D 5=6. The event that the value of the die is odd and exceeds 2 is A \ B D f3; 5g, and PŒA \ B D 1=3. Example 8.5. If P is the uniform distribution on a set ˝, and A is a subset of ˝, then PŒA D jAj=j˝j. We next derive some elementary facts about probabilities of certain events, and
208
Finite and discrete probability distributions
relations among them. It is clear from the definitions that PŒ; D 0 and PŒ˝ D 1;
and that for every event A, we have x D1 PŒA
PŒA:
Now consider events A and B, and their union A [ B. We have PŒA [ B PŒA C PŒBI
(8.3)
PŒA [ B D PŒA C PŒB if A and B are disjoint;
(8.4)
moreover,
that is, if A \ B D ;. The exact formula for arbitrary events A and B is: PŒA [ B D PŒA C PŒB
PŒA \ B:
(8.5)
(8.3), (8.4), and (8.5) all follow from the observation that in the expression X X PŒA C PŒB D P.!/ C P.!/; !2A
!2B
the value P.!/ is counted once for each ! 2 A [ B, except for those ! 2 A \ B, for which P.!/ is counted twice. Example 8.6. Alice rolls two dice, and asks Bob to guess a value that appears on either of the two dice (without looking). Let us model this situation by considering the uniform distribution on ˝ WD f1; : : : ; 6g f1; : : : ; 6g, where for each pair .s; t / 2 ˝, s represents the value of the first die, and t the value of the second. For k D 1; : : : ; 6, let Ak be the event that the first die is k, and Bk the event that the second die is k. Let Ck D Ak [ Bk be the event that k appears on either of the two dice. No matter what value k Bob chooses, the probability that this choice is correct is PŒCk D PŒAk [ Bk D PŒAk C PŒBk
D 1=6 C 1=6
PŒAk \ Bk
1=36 D 11=36;
which is slightly less than the estimate PŒAk C PŒBk obtained from (8.3). If fAi gi2I is a family of events, indexed by some set I , we can naturally form S T the union i 2I Ai and intersection i 2I Ai . If I D ;, then by definition, the union is ;, and by special convention, the intersection is the entire sample space ˝.
209
8.1 Basic definitions
Logically, the union represents the event that some Ai occurs, and the intersection represents the event that all the Ai ’s occur. DeMorgen’s law generalizes as follows: [ \ \ [ Ai D Ai D Axi and Axi ; i 2I
i 2I
i 2I
i 2I
and if B is an event, then the Boolean distributive law generalizes as follows: \ \ [ [ .B [ Ai /: .B \ Ai / and B [ Ai D B\ Ai D i 2I
i 2I
i 2I
i 2I
We now generalize (8.3), (8.4), and (8.5) from pairs of events to families of events. Let fAi gi 2I be a finite family of events (i.e., the index set I is finite). Using (8.3), it follows by induction on jI j that h[ i X PŒAi ; (8.6) P Ai i 2I
i 2I
which is known as Boole’s inequality (and also sometimes called the union bound). Analogously, using (8.4), it follows by induction on jI j that h[ i X P Ai D PŒAi ; if fAi gi 2I is pairwise disjoint; (8.7) i 2I
i 2I
that is, if Ai \Aj D ; for all i; j 2 I with i ¤ j . We shall refer to (8.7) as Boole’s equality. Both (8.6) and (8.7) are invaluable tools in calculating or estimating the probability of an event A by breaking A up into a family fAi gi 2I of smaller, and hopefully simpler, events, whose union is A. We shall make frequent use of them. The generalization of (8.5) is messier. Consider first the case of three events, A, B, and C. We have PŒA [ B [ C D PŒA C PŒB C PŒC
PŒA \ B
PŒA \ C
PŒB \ C
C PŒA \ B \ C: Thus, starting with the sum of the probabilities of the individual events, we have to subtract a “correction term” that consists of the sum of probabilities of all intersections of pairs of events; however, this is an “over-correction,” and we have to correct the correction by adding back in the probability of the intersection of all three events. The general statement is as follows: Theorem 8.1 (Inclusion/exclusion principle). Let fAi gi 2I be a finite family of events. Then h[ i h\ i X P Ai D . 1/jJ j 1 P Aj ; i 2I
;¨J I
the sum being over all non-empty subsets J of I .
j 2J
210
Finite and discrete probability distributions
Proof. For ! 2 ˝ and B ˝, define ı! ŒB WD 1 if ! 2 B, and ı! ŒB WD 0 if ! … B. As a function of !, ı! ŒB is simply the characteristic function of B. One may x D 1 ı! ŒB easily verify that for all ! 2 ˝, B ˝, and C ˝, we have ı! ŒB and ı! ŒB \ C D ı! ŒBı! ŒC. It is also easily seen that for every B ˝, we have P ŒB D PŒB. !2˝ P.!/ı S! T Let A WD i 2I Ai , and for J I , let AJ WD j 2J Aj . For every ! 2 ˝, h\ i Y Y x D ı! 1 ı! ŒA D ı! ŒA Axi D ı! ŒAxi D .1 ı! ŒAi / i 2I
D
X
. 1/
Y
jJ j
i 2I
ı! ŒAi D
jJ j
. 1/
ı! ŒAJ ;
J I
j 2J
J I
i 2I
X
and so X
ı! ŒA D
. 1/jJ j
1
ı! ŒAJ :
(8.8)
;¨J I
Multiplying (8.8) by P.!/, and summing over all ! 2 ˝, we have X X X . 1/jJ j 1 ı! ŒAJ P.!/ P.!/ı! ŒA D PŒA D !2˝
!2˝
X
D
. 1/jJ j
1
X
;¨J I
!2˝
;¨J I
X
P.!/ı! ŒAJ D
. 1/jJ j
1
PŒAJ :
;¨J I
One can also state the inclusion/exclusion principle in a slightly different way, splitting the sum into terms with jJ j D 1, jJ j D 2, etc., as follows: h[
P
i 2I
i
Ai D
X i 2I
PŒAi C
jI j X
. 1/k
kD2
1
X J I jJ jDk
h\
P
i Aj ;
j 2J
where the last sum in this formula is taken over all subsets J of I of size k. We next consider a useful way to “glue together” probability distributions. Suppose one conducts two physically separate and unrelated random experiments, with each experiment modeled separately as a probability distribution. What we would like is a way to combine these distributions, obtaining a single probability distribution that models the two experiments as one grand experiment. This can be accomplished in general, as follows. Let P1 W ˝1 ! Œ0; 1 and P2 W ˝2 ! Œ0; 1 be probability distributions. Their product distribution P WD P1 P2 is defined as follows: P W ˝1 ˝2 ! Œ0; 1
.!1 ; !2 / 7! P1 .!1 /P2 .!2 /:
211
8.1 Basic definitions
It is easily verified that P is a probability distribution on the sample space ˝1 ˝2 : X X X X P.!1 ; !2 / D P1 .!1 /P2 .!2 / D P1 .!1 / P2 .!2 / D 11 D 1: !1 ;!2
!1 ;!2
!1
!2
More generally if Pi W ˝i ! Œ0; 1, for i D 1; : : : ; n, are probability distributions, then their product distribution is P WD P1 Pn , where P W ˝1 ˝n ! Œ0; 1
.!1 ; : : : ; !n / 7! P1 .!1 / Pn .!n /: If P1 D P2 D D Pn , then we may write P D Pn1 . It is clear from the definitions that if each Pi is the uniform distribution on ˝i , then P is the uniform distribution on ˝1 ˝n . Example 8.7. We can view the probability distribution P in Example 8.6 as P21 , where P1 is the uniform distribution on f1; : : : ; 6g. Example 8.8. Suppose we have a coin that comes up heads with some probability p, and tails with probability q WD 1 p. We toss the coin n times, and record the outcomes. We can model this as the product distribution P D Pn1 , where P1 is the distribution of a Bernoulli trial (see Example 8.3) with success probability p, and where we identify success with heads, and failure with tails. The sample space ˝ of P is the set of all 2n tuples ! D .!1 ; : : : ; !n /, where each !i is either heads or tails. If the tuple ! has k heads and n k tails, then P.!/ D p k q n k , regardless of the positions of the heads and tails in the tuple. For each k D 0; : : : ; n, let Ak be the event that our coin comes up heads exactly k times. As a set, Ak consists of all those tuples in the sample space with exactly k heads, and so ! n jAk j D ; k from which it follows that ! n k n PŒAk D p q k
k
:
If our coin is a fair coin, so that p D q D 1=2, then P is the uniform distribution on ˝, and for each k D 0; : : : ; n, we have ! n PŒAk D 2 n: k Suppose P W ˝ ! Œ0; 1 is a probability distribution. The support of P is defined to be the set f! 2 ˝ W P.!/ ¤ 0g. Now consider another probability
212
Finite and discrete probability distributions
distribution P0 W ˝ 0 ! Œ0; 1. Of course, these two distributions are equal if and only if ˝ D ˝ 0 and P.!/ D P0 .!/ for all ! 2 ˝. However, it is natural and convenient to have a more relaxed notion of equality. We shall say that P and P0 are essentially equal if the restriction of P to its support is equal to the restriction of P0 to its support. For example, if P the probability distribution on f1; 2; 3; 4g that assigns probability 1=3 to 1, 2, and 3, and probability 0 to 4, we may say that P is essentially the uniform distribution on f1; 2; 3g. E XERCISE 8.1. Show that PŒA \ BPŒA [ B PŒAPŒB for all events A; B. x Show that E XERCISE 8.2. Suppose A; B; C are events such that A \ Cx D B \ C. jPŒA PŒBj PŒC. E XERCISE 8.3. Let m be a positive integer, and let ˛.m/ be the probability that a number chosen at random from f1; : : : ; mg is divisible by either 4, 5, or 6. Write down an exact formula for ˛.m/, and also show that ˛.m/ D 14=30 C O.1=m/. E XERCISE 8.4. This exercise asks you to generalize Boole’s inequality (8.6), proving Bonferroni’s inequalities. Let fAi gi2I be a finite family of events, where n WD jI j. For m D 0; : : : ; n, define ˛m WD
m X
. 1/k
kD1
1
X
h\
P
J I jJ jDk
i Aj :
j 2J
Also, define h[ i ˛ WD P Ai : i 2I
Show that ˛ ˛m if m is odd, and ˛ ˛m if m is even. Hint: use induction on n. 8.2 Conditional probability and independence Let P be a probability distribution on a sample space ˝. For a given event B ˝ with PŒB ¤ 0, and for ! 2 ˝, let us define P.!/=PŒB if ! 2 B, P.! j B/ WD 0 otherwise. Viewing B as fixed, the function P. j B/ is a new probability distribution on the sample space ˝, called the conditional distribution (derived from P) given B. Intuitively, P. j B/ has the following interpretation. Suppose a random experiment produces an outcome according to the distribution P. Further, suppose we
8.2 Conditional probability and independence
213
learn that the event B has occurred, but nothing else about the outcome. Then the distribution P. j B/ assigns new probabilities to all possible outcomes, reflecting the partial knowledge that the event B has occurred. For a given event A ˝, its probability with respect to the conditional distribution given B is X PŒA \ B PŒA j B D P.! j B/ D : PŒB !2A
The value PŒA j B is called the conditional probability of A given B. Again, the intuition is that this is the probability that the event A occurs, given the partial knowledge that the event B has occurred. For events A and B, if PŒA\B D PŒA PŒB, then A and B are called independent events. If PŒB ¤ 0, a simple calculation shows that A and B are independent if and only if PŒA j B D PŒA; intuitively, independence means that the partial knowledge that event B has occurred does not affect the likelihood that A occurs. Example 8.9. Suppose P is the uniform distribution on ˝, and that B ˝ with PŒB ¤ 0. Then the conditional distribution given B is essentially the uniform distribution on B. Example 8.10. Consider again Example 8.4, where A is the event that the value on the die is odd, and B is the event that the value of the die exceeds 2. Then as we calculated, PŒA D 1=2, PŒB D 2=3, and PŒA \ B D 1=3; thus, PŒA \ B D PŒAPŒB, and we conclude that A and B are independent. Indeed, PŒA j B D .1=3/=.2=3/ D 1=2 D PŒA; intuitively, given the partial knowledge that the value on the die exceeds 2, we know it is equally likely to be either 3, 4, 5, or 6, and so the conditional probability that it is odd is 1=2. However, consider the event C that the value on the die exceeds 3. We have PŒC D 1=2 and PŒA \ C D 1=6 ¤ 1=4, from which we conclude that A and C are not independent. Indeed, PŒA j C D .1=6/=.1=2/ D 1=3 ¤ PŒA; intuitively, given the partial knowledge that the value on the die exceeds 3, we know it is equally likely to be either 4, 5, or 6, and so the conditional probability that it is odd is just 1=3, and not 1=2. Example 8.11. In Example 8.6, suppose that Alice tells Bob the sum of the two dice before Bob makes his guess. The following table is useful for visualizing the situation:
214
Finite and discrete probability distributions 6 5 4 3 2 1
7 6 5 4 3 2 1
8 7 6 5 4 3 2
9 8 7 6 5 4 3
10 9 8 7 6 5 4
11 10 9 8 7 6 5
12 11 10 9 8 7 6
For example, suppose Alice tells Bob the sum is 4. Then what is Bob’s best strategy in this case? Let D` be the event that the sum is `, for ` D 2; : : : ; 12, and consider the conditional distribution given D4 . This conditional distribution is essentially the uniform distribution on the set f.1; 3/; .2; 2/; .3; 1/g. The numbers 1 and 3 both appear in two pairs, while the number 2 appears in just one pair. Therefore, PŒC1 j D4 D PŒC3 j D4 D 2=3;
while PŒC2 j D4 D 1=3
and PŒC4 j D4 D PŒC5 j D4 D PŒC6 j D4 D 0:
Thus, if the sum is 4, Bob’s best strategy is to guess either 1 or 3, which will be correct with probability 2=3. Similarly, if the sum is 5, then we consider the conditional distribution given D5 , which is essentially the uniform distribution on f.1; 4/; .2; 3/; .3; 2/; .4; 1/g. In this case, Bob should choose one of the numbers k D 1; : : : ; 4, each of which will be correct with probability PŒCk j D5 D 1=2. Suppose fBi gi 2I is a finite, pairwise disjoint family of events, whose union is ˝. Now consider an arbitrary event A. Since fA \ Bi gi 2I is a pairwise disjoint family of events whose union is A, Boole’s equality (8.7) implies X PŒA D PŒA \ Bi : (8.9) i2I
Furthermore, if each Bi occurs with non-zero probability (so that in particular, fBi gi 2I is a partition of ˝), then we have X PŒA D PŒA j Bi PŒBi : (8.10) i 2I
If, in addition, PŒA ¤ 0, then for each j 2 I , we have PŒBj j A D
PŒA \ Bj PŒA
DP
PŒA j Bj PŒBj
i 2I
PŒA j Bi PŒBi
:
(8.11)
Equations (8.9) and (8.10) are sometimes called the law of total probability, while
8.2 Conditional probability and independence
215
equation (8.11) is known as Bayes’ theorem. Equation (8.10) (resp., (8.11)) is useful for computing or estimating PŒA (resp., PŒBj j A) by conditioning on the events Bi . Example 8.12. Let us continue with Example 8.11, and compute Bob’s overall probability of winning, assuming he follows an optimal strategy. If the sum is 2 or 12, clearly there is only one sensible choice for Bob to make, and it will certainly be correct. If the sum is any other number `, and there are N` pairs in the sample space that sum to that number, then there will always be a value that appears in exactly 2 of these N` pairs, and Bob should choose such a value (see the diagram in Example 8.11). Indeed, this is achieved by the simple rule of choosing the value 1 if ` 7, and the value 6 if ` > 7. This is an optimal strategy for Bob, and if C is the event that Bob wins following this strategy, then by total probability (8.10), we have 12 X PŒC D PŒC j D` PŒD` : `D2
Moreover, 1 1 1 1 D ; PŒC j D12 PŒD12 D 1 D ; 36 36 36 36 and for ` D 3; : : : ; 11, we have PŒC j D2 PŒD2 D 1
PŒC j D` PŒD` D
2 N` 1 D : N` 36 18
Therefore, PŒC D
1 1 9 10 C C D : 36 36 18 18
Example 8.13. Suppose that the rate of incidence of disease X in the overall population is 1%. Also suppose that there is a test for disease X ; however, the test is not perfect: it has a 5% false positive rate (i.e., 5% of healthy patients test positive for the disease), and a 2% false negative rate (i.e., 2% of sick patients test negative for the disease). A doctor gives the test to a patient and it comes out positive. How should the doctor advise his patient? In particular, what is the probability that the patient actually has disease X, given a positive test result? Amazingly, many trained doctors will say the probability is 95%, since the test has a false positive rate of 5%. However, this conclusion is completely wrong. Let A be the event that the test is positive and let B be the event that the patient has disease X. The relevant quantity that we need to estimate is PŒB j A; that is, the probability that the patient has disease X , given a positive test result. We use
216
Finite and discrete probability distributions
Bayes’ theorem to do this: PŒB j A D
PŒA j BPŒB
x PŒB x PŒA j BPŒB C PŒA j B
D
0:98 0:01 0:17: 0:98 0:01 C 0:05 0:99
Thus, the chances that the patient has disease X given a positive test result are just 17%. The correct intuition here is that it is much more likely to get a false positive than it is to actually have the disease. Of course, the real world is a bit more complicated than this example suggests: the doctor may be giving the patient the test because other risk factors or symptoms may suggest that the patient is more likely to have the disease than a random member of the population, in which case the above analysis does not apply. Example 8.14. This example is based on the TV game show “Let’s make a deal,” which was popular in the 1970’s. In this game, a contestant chooses one of three doors. Behind two doors is a “zonk,” that is, something amusing but of little or no value, such as a goat, and behind one of the doors is a “grand prize,” such as a car or vacation package. We may assume that the door behind which the grand prize is placed is chosen at random from among the three doors, with equal probability. After the contestant chooses a door, the host of the show, Monty Hall, always reveals a zonk behind one of the two doors not chosen by the contestant. The contestant is then given a choice: either stay with his initial choice of door, or switch to the other unopened door. After the contestant finalizes his decision on which door to choose, that door is opened and he wins whatever is behind it. The question is, which strategy is better for the contestant: to stay or to switch? Let us evaluate the two strategies. If the contestant always stays with his initial selection, then it is clear that his probability of success is exactly 1=3. Now consider the strategy of always switching. Let B be the event that the contestant’s initial choice was correct, and let A be the event that the contestant wins the grand prize. On the one hand, if the contestant’s initial choice was correct, then switching will certainly lead to failure (in this case, Monty has two doors to choose from, but his choice does not affect the outcome). Thus, PŒA j B D 0. On the other hand, suppose that the contestant’s initial choice was incorrect, so that one of the zonks is behind the initially chosen door. Since Monty reveals the x D 1. other zonk, switching will lead with certainty to success. Thus, PŒA j B Furthermore, it is clear that PŒB D 1=3. So using total probability (8.10), we compute x PŒB x D 0 .1=3/ C 1 .2=3/ D 2=3: PŒA D PŒA j BPŒB C PŒA j B Thus, the “stay” strategy has a success probability of 1=3, while the “switch” strategy has a success probability of 2=3. So it is better to switch than to stay.
8.2 Conditional probability and independence
217
Of course, real life is a bit more complicated. Monty did not always reveal a zonk and offer a choice to switch. Indeed, if Monty only revealed a zonk when the contestant had chosen the correct door, then switching would certainly be the wrong strategy. However, if Monty’s choice itself was a random decision made independent of the contestant’s initial choice, then switching is again the preferred strategy. We next generalize the notion of independence from pairs of events to families of events. Let fAi gi 2I be a finite family of events. For a given positive integer k, we say that the family fAi gi 2I is k-wise independent if the following holds: h\ i Y P Aj D PŒAj for all J I with jJ j k: j 2J
j 2J
The family fAi gi 2I is called pairwise independent if it is 2-wise independent. Equivalently, pairwise independence means that for all i; j 2 I with i ¤ j , we have PŒAi \ Aj D PŒAi PŒAj , or put yet another way, that for all i; j 2 I with i ¤ j , the events Ai and Aj are independent. The family fAi gi 2I is called mutually independent if it is k-wise independent for all positive integers k. Equivalently, mutual independence means that h\ i Y PŒAj for all J I : P Aj D j 2J
j 2J
If n WD jI j > 0, mutual independence is equivalent to n-wise independence; moreover, if 0 < k n, then fAi gi 2I is k-wise independent if and only if fAj gj 2J is mutually independent for every J I with jJ j D k. In defining independence, the choice of the index set I plays no real role, and we can rename elements of I as convenient. Example 8.15. Suppose we toss a fair coin three times, which we formally model using the uniform distribution on the set of all 8 possible outcomes of the three coin tosses: (heads, heads, heads), (heads, heads, tails), etc., as in Example 8.8. For i D 1; 2; 3, let Ai be the event that the i th toss comes up heads. Then fAi g3iD1 is a mutually independent family of events, where each individual Ai occurs with probability 1=2. Now let B12 be the event that the first and second tosses agree (i.e., both heads or both tails), let B13 be the event that the first and third tosses agree, and let B23 be the event that the second and third tosses agree. Then the family of events B12 ; B13 ; B23 is pairwise independent, but not mutually independent. Indeed, the probability that any given individual event occurs is 1=2, and the probability that any given pair of events occurs is 1=4; however, the probability that all three events occur is also 1=4, since if any two events occur, then so does the third.
218
Finite and discrete probability distributions
We close this section with some simple facts about independence of events and their complements. x Theorem 8.2. If A and B are independent events, then so are A and B. Proof. We have x (by total probability (8.9)) PŒA D PŒA \ B C PŒA \ B x (since A and B are independent). D PŒAPŒB C PŒA \ B Therefore, x D PŒA PŒA \ B
PŒAPŒB D PŒA.1
x PŒB/ D PŒAPŒB:
This theorem implies that A and B are independent ” A and Bx are independent ” Ax and B " " ” Ax and Bx "
"
:
The following theorem generalizes this result to families of events. It says that if a family of events is k-wise independent, then the family obtained by complementing any number of members of the given family is also k-wise independent. Theorem 8.3. Let fAi gi 2I be a finite, k-wise independent family of events. Let J be a subset of I , and for each i 2 I , define A0i WD Ai if i 2 J , and A0i WD Axi if i … J . Then fA0i gi 2I is also k-wise independent. Proof. It suffices to prove the theorem for the case where J D I n fd g, for an arbitrary d 2 I : this allows us to complement any single member of the family that we wish, without affecting independence; by repeating the procedure, we can complement any number of them. To this end, it will suffice to show the following: if J I , jJ j < k, d 2 I n J , T and AJ WD j 2J Aj , we have Y xd \ AJ D .1 PŒAd / PŒA PŒAj : (8.12) j 2J
Using total probability (8.9), along with the independence hypothesis (twice), we have Y xd \ AJ PŒAj D PŒAJ D PŒAd \ AJ C PŒA j 2J
D PŒAd
Y j 2J
xd \ AJ ; PŒAj C PŒA
8.2 Conditional probability and independence
219
from which (8.12) follows immediately. E XERCISE 8.5. For events A1 ; : : : ; An , define ˛1 WD PŒA1 , and for i D 2; : : : ; n, define ˛i WD PŒAi j A1 \ \ Ai 1 . Show that PŒA1 \ \ An D ˛1 ˛n . E XERCISE 8.6. Let B be an event, and let fBi gi 2I be a finite, pairwise disjoint family of events whose union is B. Generalizing the law of total probability (equations (8.9) and (8.10)), show that every event A, we have PŒA \ B D P i 2I PŒA \ Bi , and if PŒB ¤ 0, then X PŒA j BPŒB D PŒA j Bi PŒBi ; i 2I
where I WD fi 2 I W PŒBi ¤ 0g. Also show that if PŒA j Bi ˛ for each i 2 I , then PŒA j B ˛. E XERCISE 8.7. Three fair coins are tossed. Let A be the event that at least two coins are heads. Let B be the event that the number of heads is odd. Let C be the event that the third coin is heads. Are A and B independent? A and C? B and C? E XERCISE 8.8. Consider again the situation in Example 8.11, but now suppose that Alice only tells Bob the value of the sum of the two dice modulo 6. Describe an optimal strategy for Bob, and calculate his overall probability of winning. E XERCISE 8.9. Consider again the situation in Example 8.13, but now suppose that the patient is visiting the doctor because he has symptom Y . Furthermore, it is known that everyone who has disease X exhibits symptom Y , while 10% of the population overall exhibits symptom Y . Assuming that the accuracy of the test is not affected by the presence of symptom Y , how should the doctor advise his patient should the test come out positive? E XERCISE 8.10. This exercise develops an alternative proof, based on probability theory, of Theorem 2.11. Let n be a positive integer and consider an experiment in which a number a is chosen at random from f0; : : : ; n 1g. If n D p1e1 prer is the prime factorization of n, let Ai be the event that a is divisible by pi , for i D 1; : : : ; r. (a) Show that .n/=n D PŒAx1 \ \ Axr ; where is Euler’s phi function. (b) Show that if J f1; : : : ; rg, then h\ i .Y P Aj D 1 pj : j 2J
fAi griD1
Conclude that for each i D 1; : : : ; r.
j 2J
is mutually independent, and that PŒAi D 1=pi
220
Finite and discrete probability distributions
(c) Using part (b), deduce that x1 \ \ Axr D PŒA
r Y
.1
1=pi /:
i D1
(d) Combine parts (a) and (c) to derive the result of Theorem 2.11 that .n/ D n
r Y
.1
1=pi /:
i D1
8.3 Random variables It is sometimes convenient to associate a real number, or other mathematical object, with each outcome of a random experiment. The notion of a random variable formalizes this idea. Let P be a probability distribution on a sample space ˝. A random variable X is a function X W ˝ ! S , where S is some set, and we say that X takes values in S . We do not require that the values taken by X are real numbers, but if this is the case, we say that X is real valued. For s 2 S , “X D s” denotes the event f! 2 ˝ W X .!/ D sg. It is immediate from this definition that X PŒX D s D P.!/: !2X
1 .fsg/
More generally, for any predicate on S, we may write “ .X /” as shorthand for the event f! 2 ˝ W .X .!//g. When we speak of the image of X , we simply mean its image in the usual function-theoretic sense, that is, the set X .˝/ D fX .!/ W ! 2 ˝g. While a random variable is simply a function on the sample space, any discussion of its properties always takes place relative to a particular probability distribution, which may be implicit from context. One can easily combine random variables to define new random variables. Suppose X1 ; : : : ; Xn are random variables, where Xi W ˝ ! Si for i D 1; : : : ; n. Then .X1 ; : : : ; Xn / denotes the random variable that maps ! 2 ˝ to .X1 .!/; : : : ; Xn .!// 2 S1 Sn . If f W S1 Sn ! T is a function, then f .X1 ; : : : ; Xn / denotes the random variable that maps ! 2 ˝ to f .X1 .!/; : : : ; Xn .!//. If f is applied using a special notation, the same notation may be applied to denote the resulting random variable; for example, if X and Y are random variables taking values in a set S, and ? is a binary operation on S, then X ? Y denotes the random variable that maps ! 2 ˝ to X .!/ ? Y .!/ 2 S. Let X be a random variable whose image is S . The variable X determines a probability distribution PX W S ! Œ0; 1 on the set S, where PX .s/ WD PŒX D s
221
8.3 Random variables
for each s 2 S . We call PX the distribution of X . If PX is the uniform distribution on S, then we say that X is uniformly distributed over S . Example 8.16. Again suppose we roll two dice, and model this experiment as the uniform distribution on ˝ WD f1; : : : ; 6g f1; : : : ; 6g. We can define the random variable X that takes the value of the first die, and the random variable Y that takes the value of the second; formally, X and Y are functions on ˝, where X .s; t / WD s and Y .s; t / WD t for .s; t / 2 ˝: For each value s 2 f1; : : : ; 6g, the event X D s consists of the 6 pairs .s; 1/; : : : ; .s; 6/, and so PŒX D s D 6=36 D 1=6: Thus, X is uniformly distributed over f1; : : : ; 6g. Likewise, Y is uniformly distributed over f1; : : : ; 6g, and the random variable .X ; Y / is uniformly distributed over ˝. We can also define the random variable Z WD X C Y , which formally is the function on the sample space defined by Z .s; t / WD s C t for .s; t/ 2 ˝: The image of Z is f2; : : : ; 12g, and its distribution is given by the following table: u
2
3
4
5
6
7
8
9
10
11
12
PŒZ D u 1/36 2/36 3/36 4/36 5/36 6/36 5/36 4/36 3/36 2/36 1/36
Example 8.17. If A is an event, we may define a random variable X as follows: X WD 1 if the event A occurs, and X WD 0 otherwise. The variable X is called the indicator variable for A. Formally, X is the function that maps ! 2 A to 1, and ! 2 ˝ n A to 0; that is, X is simply the characteristic function of A. The distribution of X is that of a Bernoulli trial: PŒX D 1 D PŒA and PŒX D 0 D 1 PŒA. x Now suppose B is It is not hard to see that 1 X is the indicator variable for A. another event, with indicator variable Y . Then it is also not hard to see that XY is the indicator variable for A \ B, and that X C Y XY is the indicator variable for A [ B; in particular, if A \ B D ;, then X C Y is the indicator variable for A [ B. Example 8.18. Consider again Example 8.8, where we have a coin that comes up heads with probability p, and tails with probability q WD 1 p, and we toss it n times. For each i D 1; : : : ; n, let Ai be the event that the i th toss comes up heads, and let Xi be the corresponding indicator variable. Let us also define X WD X1 C C Xn , which represents the total number of tosses that come up heads. The image of X is f0; : : : ; ng. By the calculations made in Example 8.8, for
222
Finite and discrete probability distributions
each k D 0; : : : ; n, we have ! n k n PŒX D k D p q k
k
:
The distribution of the random variable X is called a binomial distribution. Such a distribution is parameterized by the success probability p of the underlying Bernoulli trial, and by the number of times n the trial is repeated. Uniform distributions are very nice, simple distributions. It is therefore good to have simple criteria that ensure that certain random variables have uniform distributions. The next theorem provides one such criterion. We need a definition: if S and T are finite sets, then we say that a given function f W S ! T is a regular function if every element in the image of f has the same number of pre-images under f . Theorem 8.4. Suppose f W S ! T is a surjective, regular function, and that X is a random variable that is uniformly distributed over S . Then f .X / is uniformly distributed over T . Proof. The assumption that f is surjective and regular implies that for every t 2 T , the set S t WD f 1 .ft g/ has size jS j=jT j. So, for each t 2 T , working directly from the definitions, we have X X X X PŒf .X / D t D P.!/ D P.!/ D PŒX D s !2X
D
X
1 .S
t/
s2S t !2X
1 .fsg/
s2S t
1=jS j D .jS j=jT j/=jS j D 1=jT j:
s2S t
As a corollary, we have: Theorem 8.5. Suppose that W G ! G 0 is a surjective homomorphism of finite abelian groups G and G 0 , and that X is a random variable that is uniformly distributed over G. Then .X / is uniformly distributed over G 0 . Proof. It suffices to show that is regular. Recall that the kernel K of is a subgroup of G, and that for every g 0 2 G 0 , the set 1 .fg 0 g/ is a coset of K (see Theorem 6.19); moreover, every coset of K has the same size (see Theorem 6.14). These facts imply that is regular. Example 8.19. Let us continue with Example 8.16. Recall that for a given integer a, and positive integer n, Œan 2 Zn denotes the residue class of a modulo n. Let us define X 0 WD ŒX 6 and Y 0 WD ŒY 6 . It is not hard to see that both X 0 and Y 0 are uniformly distributed over Z6 , while .X 0 ; Y 0 / is uniformly distributed
8.3 Random variables
223
over Z6 Z6 . Let us define Z 0 WD X 0 C Y 0 (where addition here is in Z6 ). We claim that Z 0 is uniformly distributed over Z6 . This follows immediately from fact that the map that sends .a; b/ 2 Z6 Z6 to a C b 2 Z6 is a surjective group homomorphism (see Example 6.45). Further, we claim that .X 0 ; Z 0 / is uniformly distributed over Z6 Z6 . This follows immediately from fact that the map that sends .a; b/ 2 Z6 Z6 to .a; aCb/ 2 Z6 Z6 is a surjective group homomorphism (indeed, it is a group isomorphism). Let X be a random variable whose image is S . Let B be an event with PŒB ¤ 0. The conditional distribution of X given B is defined to be the distribution of X relative to the conditional distribution P. j B/, that is, the distribution PX ;B W S ! Œ0; 1 defined by PX ;B .s/ WD PŒX D s j B for s 2 S . Suppose X and Y are random variables, with images S and T , respectively. We say X and Y are independent if for all s 2 S and all t 2 T , the events X D s and Y D t are independent, which is to say, PŒ.X D s/ \ .Y D t / D PŒX D sPŒY D t :
Equivalently, X and Y are independent if and only if the distribution of .X ; Y / is essentially equal to the product of the distribution of X and the distribution of Y . As a special case, if X is uniformly distributed over S , and Y is uniformly distributed over T , then X and Y are independent if and only if .X ; Y / is uniformly distributed over S T . Independence can also be characterized in terms of conditional probabilities. From the definitions, it is immediate that X and Y are independent if and only if for all values t taken by Y with non-zero probability, we have PŒX D s j Y D t D PŒX D s
for all s 2 S; that is, the conditional distribution of X given Y D t is the same as the distribution of X . From this point of view, an intuitive interpretation of independence is that information about the value of one random variable does not reveal any information about the value of the other. Example 8.20. Let us continue with Examples 8.16 and 8.19. The random variables X and Y are independent: each is uniformly distributed over f1; : : : ; 6g, and .X ; Y / is uniformly distributed over f1; : : : ; 6g f1; : : : ; 6g. Let us calculate the conditional distribution of X given Z D 4. We have PŒX D s j Z D 4 D 1=3 for s D 1; 2; 3, and PŒX D s j Z D 4 D 0 for s D 4; 5; 6. Thus, the conditional distribution of X given Z D 4 is essentially the uniform distribution on f1; 2; 3g. Let us calculate the conditional distribution of Z given X D 1. We have PŒZ D u j X D 1 D 1=6 for u D 2; : : : ; 7, and PŒZ D u j X D 1 D 0 for u D 8; : : : ; 12. Thus, the conditional distribution of Z given X D 1 is essentially
224
Finite and discrete probability distributions
the uniform distribution on f2; : : : ; 7g. In particular, it is clear that X and Z are not independent. The random variables X 0 and Y 0 are independent, as are X 0 and Z 0 : each of X 0 , Y 0 , and Z 0 is uniformly distributed over Z6 , and each of .X 0 ; Y 0 / and .X 0 ; Z 0 / is uniformly distributed over Z6 Z6 . We now generalize the notion of independence to families of random variables. Let fXi gi 2I be a finite family of random variables. Let us call a corresponding family of values fsi gi 2I an assignment to fXi gi 2I if si is in the image of Xi for each i 2 I . For a given positive integer k, we say that the family fXi gi2I is kwise independent if for every assignment fsi gi 2I to fXi gi 2I , the family of events fXi D si gi 2I is k-wise independent. The notions of pairwise and mutual independence for random variables are defined following the same pattern that was used for events. The family fXi gi 2I is called pairwise independent if it is 2-wise independent, which means that for all i; j 2 I with i ¤ j , the variables Xi and Xj are independent. The family fXi gi 2I is called mutually independent if it is k-wise independent for all positive integers k. Equivalently, and more explicitly, mutual independence means that for every assignment fsi gi 2I to fXi gi 2I , we have h\ i Y P .Xj D sj / D PŒXj D sj for all J I : (8.13) j 2J
j 2J
If n WD jI j > 0, mutual independence is equivalent to n-wise independence; moreover, if 0 < k n, then fXi gi 2I is k-wise independent if and only if fXj gj 2J is mutually independent for every J I with jJ j D k. Example 8.21. Returning again to Examples 8.16, 8.19, and 8.20, we see that the family of random variables X 0 ; Y 0 ; Z 0 is pairwise independent, but not mutually independent; for example, P .X 0 D Œ06 / \ .Y 0 D Œ06 / \ .Z 0 D Œ06 / D 1=62 ; but P X 0 D Œ06 P Y 0 D Œ06 P Z 0 D Œ06 D 1=63 :
Example 8.22. Suppose fAi gi 2I is a finite family of events. Let fXi gi 2I be the corresponding family of indicator variables, so that for each i 2 I , Xi D 1 if Ai occurs, and Xi D 0, otherwise. Theorem 8.3 immediately implies that for every positive integer k, fAi gi 2I is k-wise independent if and only if fXi gi 2I is k-wise independent. Example 8.23. Consider again Example 8.15, where we toss a fair coin 3 times. For i D 1; 2; 3, let Xi be the indicator variable for the event Ai that the i th toss
225
8.3 Random variables
comes up heads. Then fXi g3iD1 is a mutually independent family of random variables. Let Y12 be the indicator variable for the event B12 that tosses 1 and 2 agree; similarly, let Y13 be the indicator variable for the event B13 , and Y23 the indicator variable for B23 . Then the family of random variables Y12 ; Y13 ; Y23 is pairwise independent, but not mutually independent. We next present a number of useful tools for establishing independence. Theorem 8.6. Let X be a random variable with image S, and Y be a random variable with image T . Further, suppose that f W S ! Œ0; 1 and g W T ! Œ0; 1 are functions such that X X f .s/ D g.t / D 1; (8.14) s2S
t 2T
and that for all s 2 S and t 2 T , we have PŒ.X D s/ \ .Y D t / D f .s/g.t /:
(8.15)
Then X and Y are independent, the distribution of X is f , and the distribution of Y is g. Proof. Since fY D tg t 2T is a partition of the sample space, making use of total probability (8.9), along with (8.15) and (8.14), we see that for all s 2 S, we have X X X g.t / D f .s/: f .s/g.t / D f .s/ PŒ.X D s/ \ .Y D t / D PŒX D s D t 2T
t 2T
t 2T
Thus, the distribution of X is indeed f . Exchanging the roles of X and Y in the above argument, we see that the distribution of Y is g. Combining this with (8.15), we see that X and Y are independent. The generalization of Theorem 8.6 to families of random variables is a bit messy, but the basic idea is the same: Theorem 8.7. Let fXi gi 2I be a finite family of random variables, where each Xi has image Si . Also, let ffi gi 2I be a family of functions, where for each i 2 I , P fi W Si ! Œ0; 1 and si 2Si fi .si / D 1. Further, suppose that h\ i Y P .Xi D si / D fi .si / i 2I
i 2I
for each assignment fsi gi 2I to fXi gi 2I . Then the family fXi gi 2I is mutually independent, and for each i 2 I , the distribution of Xi is fi . Proof. To prove the theorem, it suffices to prove the following statement: for every
226
Finite and discrete probability distributions
subset J of I , and every assignment fsj gj 2J to fXj gj 2J , we have i Y h\ .Xj D sj / D fj .sj /: P j 2J
j 2J
Moreover, it suffices to prove this statement for the case where J D I n fd g, for an arbitrary d 2 I : this allows us to eliminate any one variable from the family, without affecting the hypotheses, and by repeating this procedure, we can eliminate any number of variables. Thus, let d 2 I be fixed, let J WD I n fd g, and let fsj gj 2J be a fixed assignment to fXj gj 2J . Then, since fXd D sd gsd 2Sd is a partition of the sample space, we have i h\ i h [ \ i X h\ P .Xi D si / P .Xj D sj / D P .Xi D si / D j 2J
D
sd 2Sd
sd 2Sd i 2I
X Y
fi .si / D
Y
fj .sj /
j 2J
sd 2Sd i 2I
X sd 2Sd
fd .sd / D
Y
i 2I
fj .sj /:
j 2J
This theorem has several immediate consequences. First of all, mutual independence may be more simply characterized: Theorem 8.8. Let fXi gi 2I be a finite family of random variables. Suppose that for every assignment fsi gi 2I to fXi gi 2I , we have h\ i Y P .Xi D si / D PŒXi D si : i 2I
i 2I
Then fXi gi 2I is mutually independent. Theorem 8.8 says that to check for mutual independence, we only have to consider the index set J D I in (8.13). Put another way, it says that a family of random variables fXi gniD1 is mutually independent if and only if the distribution of .X1 ; : : : ; Xn / is essentially equal to the product of the distributions of the individual Xi ’s. Based on the definition of mutual independence, and its characterization in Theorem 8.8, the following is also immediate: Theorem 8.9. Suppose fXi gniD1 is a family of random variables, and that m is an integer with 0 < m < n. Then the following are equivalent: (i) fXi gniD1 is mutually independent; n (ii) fXi gm i D1 is mutually independent, fXi gi DmC1 is mutually independent, and the two variables .X1 ; : : : ; Xm / and .XmC1 ; : : : ; Xn / are independent. The following is also an immediate consequence of Theorem 8.7 (it also follows easily from Theorem 8.4).
227
8.3 Random variables
Theorem 8.10. Suppose that X1 ; : : : ; Xn are random variables, and that S1 ; : : : ; Sn are finite sets. Then the following are equivalent: (i) .X1 ; : : : ; Xn / is uniformly distributed over S1 Sn ; (ii) fXi gniD1 is mutually independent, with each Xi uniformly distributed over Si . Another immediate consequence of Theorem 8.7 is the following: Theorem 8.11. Suppose P is the product distribution P1 Pn , where each Pi is a probability distribution on a sample space ˝i , so that the sample space of P is ˝ D ˝1 ˝n . For each i D 1; : : : ; n, let Xi be the random variable that projects on the i th coordinate, so that Xi .!1 ; : : : ; !n / D !i . Then fXi gniD1 is mutually independent, and for each i D 1; : : : ; n, the distribution of Xi is Pi . Theorem 8.11 is often used to synthesize independent random variables “out of thin air,” by taking the product of appropriate probability distributions. Other arguments may then be used to prove the independence of variables derived from these. Example 8.24. Theorem 8.11 immediately implies that in Example 8.18, the family of indicator variables X1 ; : : : ; Xn is mutually independent, where for each i D 1; : : : ; n, PŒXi D 1 D p. The following theorem gives us yet another way to establish independence. Theorem 8.12. Suppose fXi gniD1 is a mutually independent family of random variables. Further, suppose that for i D 1; : : : ; n, Yi WD gi .Xi / for some function gi . Then fYi gniD1 is mutually independent. Proof. It suffices to prove the theorem for n D 2. The general case follows easily by induction, using Theorem 8.9. For i D 1; 2, let ti be any value in the image of Yi , and let Si0 WD gi 1 .fti g/. We have h [ [ i PŒ.Y1 D t1 / \ .Y2 D t2 / D P .X1 D s1 / \ .X2 D s2 / s1 2S10
h [
DP
s2 2S20
[
i .X1 D s1 / \ .X2 D s2 /
s1 2S10 s2 2S20
D
X X s1 2S10
D
PŒ.X1 D s1 / \ .X2 D s2 /
s2 2S20
X X s1 2S10 s2 2S20
PŒX1 D s1 PŒX2 D s2
228
Finite and discrete probability distributions
D
X
PŒX1 D s1
X
PŒX2 D s2
s1 2S10
s2 2S20
h [
i i h [ .X2 D s2 / .X1 D s1 / P
DP
s1 2S10
s2 2S20
D PŒY1 D t1 PŒY2 D t2 : As a special case of the above theorem, if each gi is the characteristic function for some subset Si0 of the image of Xi , then X1 2 S10 ; : : : ; Xn 2 Sn0 form a mutually independent family of events. The next theorem is quite handy in proving the independence of random variables in a variety of algebraic settings. Theorem 8.13. Suppose that G be a finite abelian group, and that W is a random variable uniformly distributed over G. Let Z be another random variable, taking values in some finite set U , and suppose that W and Z are independent. Let W U ! G be some function, and define Y WD W C .Z /. Then Y is uniformly distributed over G, and Y and Z are independent. Proof. Consider any fixed values t 2 G and u 2 U . Evidently, the events .Y D t / \ .Z D u/ and .W D t .u// \ .Z D u/ are the same, and therefore, because W and Z are independent, we have PŒ.Y D t/ \ .Z D u/ D PŒW D t
.u/PŒZ D u D
1 PŒZ D u: (8.16) jGj
Since this holds for every u 2 U , making use of total probability (8.9), we have X 1 1 X : PŒY D t D PŒ.Y D t / \ .Z D u/ D PŒZ D u D jGj jGj u2U
u2U
Thus, Y is uniformly distributed over G, and by (8.16), Y and Z are independent. (This conclusion could also have been deduced directly from (8.16) using Theorem 8.6—we have repeated the argument here.) Note that in the above theorem, we make no assumption about the distribution of Z , or any properties of the function . Example 8.25. Theorem 8.13 may be used to justify the security of the one-time pad encryption scheme. Here, the variable W represents a random, secret key — the “pad”—that is shared between Alice and Bob; U represents a space of possible messages; Z represents a “message source,” from which Alice draws her message according to some distribution; finally, the function W U ! G represents some invertible “encoding transformation” that maps messages into group elements.
8.3 Random variables
229
To encrypt a message drawn from the message source, Alice encodes the message as a group element, and then adds the pad. The variable Y WD W C .Z / represents the resulting ciphertext. Since Z D 1 .Y W /, when Bob receives the ciphertext, he decrypts it by subtracting the pad, and converting the resulting group element back into a message. Because the message source Z and ciphertext Y are independent, an eavesdropping adversary who learns the value of Y does not learn anything about Alice’s message: for any particular ciphertext t , the conditional distribution of Z given Y D t is the same as the distribution of Z . The term “one time” comes the fact that a given encryption key should be used only once; otherwise, security may be compromised. Indeed, suppose the key is used a second time, encrypting a message drawn from a second source Z 0 . The second ciphertext is represented by the random variable Y 0 WD W C .Z 0 /. In general, the random variables .Z ; Z 0 / and .Y ; Y 0 / will not be independent, since Y Y 0 D .Z / .Z 0 /. To illustrate this more concretely, suppose Z is uniformly distributed over a set of 1000 messages, Z 0 is uniformly distributed over a set of two messages, say, fu01 ; u02 g, and that Z and Z 0 are independent. Now, without any further information about Z , and adversary would have at best a 1-in-a-1000 chance of guessing its value. However, if he sees that Y D t and Y 0 D t 0 , for particular values t; t 0 2 G, then he has a 1-in-2-chance, since the value of Z is equally likely to be one of just two messages, namely, u1 WD 1 .t t 0 C .u01 // and u2 WD 1 .t t 0 C .u02 //; more formally, the conditional distribution of Z given .Y D t/ \ .Y 0 D t 0 / is essentially the uniform distribution on fu1 ; u2 g. In practice, it is convenient to define the group G to be the group of all bit strings of some fixed length, with bit-wise exclusive-or as the group operation. The encoding function simply “serializes” a message as a bit string. Example 8.26. Theorem 8.13 may also be used to justify a very simple type of secret sharing. A colorful, if militaristic, motivating scenario is the following. To launch a nuclear missile, two officers who carry special keys must insert their keys simultaneously into the “authorization device” (at least, that is how it works in Hollywood). In the digital version of this scenario, an authorization device contains a secret, digital “launch code,” and each officer holds a digital “share” of this code, so that (i) individually, each share reveals no information about the launch code, but (ii) collectively, the two shares may be combined in a simple way to derive the launch code. Thus, to launch the missile, both officers must input their shares into the authorization device; hardware in the authorization device combines the two shares, and compares the resulting code against the launch code it stores — if they match, the missile flies. In the language of Theorem 8.13, the launch code is represented by the random variable Z , and the two shares by W and Y WD W C .Z /, where (as in the previ-
230
Finite and discrete probability distributions
ous example) W U ! G is some simple, invertible encoding function. Because W and Z are independent, information about the share W leaks no information about the launch code Z ; likewise, since Y and Z are independent, information, about Y leaks no information about Z . However, by combining both shares, the launch code is easily constructed as Z D 1 .Y W /. Example 8.27. Let k be a positive integer. This example shows how we can take a mutually independent family of k random variables, and from it, construct a much larger, k-wise independent family of random variables. Let p be a prime, with p k. Let fHi gikD01 be a mutually independent family of random variables, each of which is uniformly distributed over Zp . Let us set H WD .H0 ; : : : ; Hk 1 /, which by assumption, is uniformly distributed over Zpk . For each s 2 Zp , we define the function s W Zpk ! Zp as follows: for r D P .r0 ; : : : ; rk 1 / 2 Zpk , s .r/ WD ki D01 ri s i ; that is, s .r/ is the value obtained by evaluating the polynomial r0 C r1 X C C rk 1 Xk 1 2 Zp ŒX at the point s. Each s 2 Zp defines a random variable s .H/ D H0 C H1 s C C Hk 1 s k 1 . We claim that the family of random variables fs .H/gs2Zp is k-wise independent, with each individual s .H/ uniformly distributed over Zp . By Theorem 8.10, it suffices to show the following: for all distinct points s1 ; : : : ; sk 2 Zp , the random variable W WD .s1 .H/; : : : ; sk .H// is uniformly distributed over Zpk . So let s1 ; : : : ; sk be fixed, distinct elements of Zp , and define the function W Zpk ! Zpk r 7! .s1 .r/; : : : ; sk .r//:
(8.17)
Thus, W D .H/, and by Lagrange interpolation (Theorem 7.15), the function is a bijection; moreover, since H is uniformly distributed over Zpk , so is W . Of course, the field Zp may be replaced by an arbitrary finite field. Example 8.28. Consider again the secret sharing scenario of Example 8.26. Suppose at the critical moment, one of the officers is missing in action. The military planners would perhaps like a more flexible secret sharing scheme; for example, perhaps shares of the launch code should be distributed to three officers, in such a way that no single officer can authorize a launch, but any two can. More generally, for positive integers k and `, with ` k C 1, the scheme should distribute shares among ` officers, so that no coalition of k (or fewer) officers can authorize a launch, yet any coalition of k C 1 officers can. Using the construction of the previous example, this is easily achieved, as follows. Let us model the secret launch code as a random variable Z , taking values in a finite set U . Assume that p is prime, with p k C 1, and that W U ! Zp is a simple, invertible encoding function. To construct the shares, we make use of
231
8.3 Random variables
random variables H0 ; : : : ; Hk 1 , where each Hi is uniformly distributed over Zp , and the family of random variables H0 ; : : : ; Hk 1 ; Z is mutually independent. For each s 2 Zp , we define the random variable Ys WD H0 C H1 s C C Hk
1s
k 1
C .Z /s k :
We can pick any subset S Zp of size ` that we wish, so that for each s 2 S , an officer gets the secret share Ys (along with the public value s). First, we show how any coalition of k C 1 officers can reconstruct the launch code from their collection of shares, say, Ys1 ; : : : ; YskC1 . This is easily done by means of the Lagrange interpolation formula (again, Theorem 7.15). Indeed, we only need to recover the high-order coefficient, .Z /, which we can obtain via the formula kC1 X Ysi Q .Z / D : sj / j ¤i .si i D1
Second, we show that no coalition of k officers learn anything about the launch code, even if they pool their shares. Formally, this means that if s1 ; : : : ; sk are fixed, distinct points, then Ys1 ; : : : ; Ysk ; Z form a mutually independent family of random variables. This is easily seen, as follows. Define H WD .H0 ; : : : ; Hk 1 /, and W WD .H/, where W Zpk ! Zpk is as defined in (8.17), and set Y WD .Ys1 ; : : : ; Ysk /. Now, by hypothesis, H and Z are independent, and H is uniformly distributed over Zpk . As we noted in Example 8.27, is a bijection, and hence, W is uniformly distributed over Zpk ; moreover (by Theorem 8.12), W and Z are independent. Observe that Y D W C 0 .Z /, where 0 maps u 2 U to . .u/s1k ; : : : ; .u/skk / 2 Zpk , and so applying Theorem 8.13 (with the group Zpk , the random variables W and Z , and the function 0 ), we see that Y and Z are independent, where Y is uniformly distributed over Zpk . From this, it follows (using Theorems 8.9 and Theorem 8.10) that the family of random variables Ys1 ; : : : ; Ysk ; Z is mutually independent, with each Ysi uniformly distributed over Zp . Finally, we note that when k D 1, ` D 2, and S D f0; 1g, this construction degenerates to the construction in Example 8.26. E XERCISE 8.11. Suppose X and X 0 are random variables that take values in a set S and that have essentially the same distribution. Show that if f W S ! T is a function, then f .X / and f .X 0 / have essentially the same distribution. E XERCISE 8.12. Let fXi gniD1 be a family of random variables, and let Si be the image of Xi for i D 1; : : : ; n. Show that fXi gniD1 is mutually independent if and
232
Finite and discrete probability distributions
only if for each i D 2; : : : ; n, and for all s1 2 S1 ; : : : ; si 2 Si , we have PŒXi D si j .X1 D s1 / \ \ .Xi
1
D si
1 /
D PŒXi D si :
E XERCISE 8.13. Suppose that W G ! G 0 is a surjective group homomorphism, where G and G 0 are finite abelian groups. Show that if g 0 ; h0 2 G 0 , and X and Y are independent random variables, where X is uniformly distributed over 1 .fg 0 g/, and Y takes values in 1 .fh0 g/, then X C Y is uniformly distributed over 1 .fg 0 C h0 g/. E XERCISE 8.14. Suppose X and Y are random variables, where X takes values in S , and Y takes values in T . Further suppose that Y 0 is uniformly distributed over T , and that .X ; Y / and Y 0 are independent. Let be a predicate on S T . Show that PŒ .X ; Y / \ .Y D Y 0 / D PŒ .X ; Y /=jT j. E XERCISE 8.15. Let X and Y be independent random variables, where X is uniformly distributed a set S , and Y is uniformly distributed over a set T S. Define a third random variable Z as follows: if X 2 T , then Z WD X ; otherwise, Z WD Y . Show that Z is uniformly distributed over T . E XERCISE 8.16. Let n be a positive integer, and let X be a random variable, uniformly distributed over f0; : : : ; n 1g. For each positive divisor d of n, let us define the random variable Xd WD X mod d . Show that: (a) if d is a divisor of n, then the variable Xd is uniformly distributed over f0; : : : ; d 1g; (b) if d1 ; : : : ; dk are divisors of n, then fXdi gkiD1 is mutually independent if and only if fdi gkiD1 is pairwise relatively prime. E XERCISE 8.17. Suppose X and Y are random variables, each uniformly distributed over Z2 , but not necessarily independent. Show that the distribution of .X ; Y / is the same as the distribution of .X C 1; Y C 1/. E XERCISE 8.18. Let I WD f1; : : : ; ng, where n 2, let B WD f0; 1g, and let G be a finite abelian group, with jGj > 1. Suppose that fXi b g.i;b/2I B is a mutually independent family of random variables, each uniformly distributed over G. For each ˇ D .b1 ; : : : ; bn / 2 B n , let us define the random variable Yˇ WD X1b1 C C Xnbn . Show that each Yˇ uniformly distributed over G, and that fYˇ gˇ 2B n is 3-wise independent, but not 4-wise independent.
8.4 Expectation and variance
233
8.4 Expectation and variance Let P be a probability distribution on a sample space ˝. If X is a real-valued random variable, then its expected value, or expectation, is X EŒX WD X .!/ P.!/: (8.18) !2˝
If S is the image of X , and if for each s 2 S we group together the terms in (8.18) with X .!/ D s, then we see that X s PŒX D s: (8.19) EŒX D s2S
From (8.19), it is clear that EŒX depends only on the distribution of X : if X 0 is another random variable with the same (or essentially the same) distribution as X , then EŒX D EŒX 0 . More generally, suppose X is an arbitrary random variable (not necessarily real valued) whose image is S , and f is a real-valued function on S . Then again, if for each s 2 S we group together the terms in (8.18) with X .!/ D s, we see that X f .s/PŒX D s: (8.20) EŒf .X / D s2S
We make a few trivial observations about expectation, which the reader may easily verify. First, if X is equal to a constant c (i.e., X .!/ D c for every ! 2 ˝), then EŒX D EŒc D c. Second, if X and Y are random variables such that X Y (i.e., X .!/ Y .!/ for every ! 2 ˝), then EŒX EŒY . Similarly, if X > Y , then EŒX > EŒY . In calculating expectations, one rarely makes direct use of (8.18), (8.19), or (8.20), except in rather trivial situations. The next two theorems develop tools that are often quite effective in calculating expectations. Theorem 8.14 (Linearity of expectation). If X and Y are real-valued random variables, and a is a real number, then EŒX C Y D EŒX C EŒY and EŒaX D aEŒX :
Proof. It is easiest to prove this using the defining equation (8.18) for expectation. For ! 2 ˝, the value of the random variable X C Y at ! is by definition X .!/ C
234
Finite and discrete probability distributions
Y .!/, and so we have EŒX C Y D
X
.X .!/ C Y .!//P.!/
!
D
X
X .!/P.!/ C
X
!
Y .!/P.!/
!
D EŒX C EŒY : For the second part of the theorem, by a similar calculation, we have X X EŒaX D .aX .!//P.!/ D a X .!/P.!/ D aEŒX : !
!
More generally, the above theorem implies (using a simple induction argument) that if fXi gi 2I is a finite family of real-valued random variables, then we have hX i X EŒXi : (8.21) E Xi D i 2I
i 2I
So we see that expectation is linear; however, expectation is not in general multiplicative, except in the case of independent random variables: Theorem 8.15. If X and Y are independent, real-valued random variables, then EŒXY D EŒX EŒY :
Proof. It is easiest to prove this using (8.20), with the function f .s; t / WD st applied to the random variable .X ; Y /. We have X EŒXY D st PŒ.X D s/ \ .Y D t / s;t
D
X
st PŒX D sPŒY D t
s;t
D
X
s PŒX D s
X
s
t PŒY D t
t
D EŒX EŒY :
More generally, the above theorem implies (using a simple induction argument) that if fXi gi 2I is a finite, mutually independent family of real-valued random variables, then hY i Y E Xi D EŒXi : (8.22) i 2I
i 2I
The following simple facts are also sometimes quite useful in calculating expectations:
235
8.4 Expectation and variance
Theorem 8.16. Let X be a 0=1-valued random variable. Then EŒX D PŒX D 1. Proof. EŒX D 0 PŒX D 0 C 1 PŒX D 1 D PŒX D 1. Theorem 8.17. If X is a random variable that takes only non-negative integer values, then X EŒX D PŒX i : i 1
Note that since X has a finite image, the sum appearing above is finite. Proof. Suppose that the image of X is contained in f0; : : : ; ng, and for i D 1; : : : ; n, let Xi be the indicator variable for the event X i . Then X D X1 C C Xn , and by linearity of expectation and Theorem 8.16, we have EŒX D
n X
EŒXi D
i D1
n X
PŒX i :
i D1
Let X be a real-valued random variable with WD EŒX . The variance of X is VarŒX WD EŒ.X /2 . The variance provides a measure of the spread or dispersion of the distribution of X around its expected value. Note that since .X /2 takes only non-negative values, variance is always non-negative. Theorem 8.18. Let X be a real-valued random variable, with WD EŒX , and let a and b be real numbers. Then we have (i) VarŒX D EŒX 2
2 ,
(ii) VarŒaX D a2 VarŒX , and (iii) VarŒX C b D VarŒX . Proof. For part (i), observe that VarŒX D EŒ.X
/2 D EŒX 2
2X C 2
D EŒX 2
2EŒX C EŒ2 D EŒX 2
D EŒX 2
2 ;
22 C 2
where in the third equality, we used the fact that expectation is linear, and in the fourth equality, we used the fact that EŒc D c for constant c (in this case, c D 2 ). For part (ii), observe that VarŒaX D EŒa2 X 2
D a2 .EŒX 2
EŒaX 2 D a2 EŒX 2
.a/2
2 / D a2 VarŒX ;
where we used part (i) in the first and fourth equality, and the linearity of expectation in the second.
236
Finite and discrete probability distributions
Part (iii) follows by a similar calculation (verify): VarŒX C b D EŒ.X C b/2
. C b/2
D .EŒX 2 C 2b C b 2 / D EŒX 2
.2 C 2b C b 2 /
2 D VarŒX :
The following is an immediate consequence of part (i) of Theorem 8.18, and the fact that variance is always non-negative: Theorem 8.19. If X is a real-valued random variable, then EŒX 2 EŒX 2 . Unlike expectation, the variance of a sum of random variables is not equal to the sum of the variances, unless the variables are pairwise independent: Theorem 8.20. If fXi gi 2I is a finite, pairwise independent family of real-valued random variables, then hX i X Var Xi D VarŒXi : i 2I
Proof. We have hX i hX 2 i Var Xi D E Xi i 2I
i 2I
X 2 Xi EΠi 2I
i 2I
D
X
EŒXi2
i 2I
C
X
EŒXi Xj
EŒXi EŒXj
i;j 2I i ¤j
X
EŒXi 2
i 2I
(by linearity of expectation and rearranging terms) X X D EŒXi2 EŒXi 2 i 2I
i 2I
(by pairwise independence and Theorem 8.15) X D VarŒXi : i 2I
Corresponding to Theorem 8.16, we have: Theorem 8.21. Let X be a 0=1-valued random variable, with p WD PŒX D 1 and q WD PŒX D 0 D 1 p. Then VarŒX D pq. Proof. We have EŒX D p and EŒX 2 D PŒX 2 D 1 D PŒX D 1 D p. Therefore, VarŒX D EŒX 2
EŒX 2 D p
p 2 D p.1
p/ D pq:
Let B be an event with PŒB ¤ 0, and let X be a real-valued random variable.
237
8.4 Expectation and variance
We define the conditional expectation of X given B, denoted EŒX j B, to be the expected value of the X relative to the conditional distribution P. j B/, so that X X EŒX j B D X .!/P.! j B/ D PŒB 1 X .!/P.!/: !2B
!2˝
Analogous to (8.19), if S is the image of X , we have X s PŒX D s j B: EŒX j B D
(8.23)
s2S
Furthermore, suppose I is a finite index set, and fBi gi 2I is a partition of the sample space, where each Bi occurs with non-zero probability. If for each i 2 I we group together the terms in (8.18) with ! 2 Bi , we obtain a law of total probability for expectation: X EŒX j Bi PŒBi : (8.24) EŒX D i 2I
Example 8.29. Let X be uniformly distributed over f1; : : : ; mg. Let us compute EŒX and VarŒX . We have EŒX D
m X sD1
s
1 m.m C 1/ 1 mC1 D D : m 2 m 2
We also have 2
EŒX D
m X sD1
s2
1 m.m C 1/.2m C 1/ 1 .m C 1/.2m C 1/ D D : m 6 m 6
Therefore, VarŒX D EŒX 2
EŒX 2 D
m2 1 : 12
Example 8.30. Let X denote the value of a roll of a die. Let A be the event that X is even. Then the conditional distribution of X given A is essentially the uniform distribution on f2; 4; 6g, and hence 2C4C6 D 4: 3 Similarly, the conditional distribution of X given Ax is essentially the uniform disEŒX j A D
tribution on f1; 3; 5g, and so x D EŒX j A
1C3C5 D 3: 3
238
Finite and discrete probability distributions
We can compute the expected value of X using these conditional expectations; indeed, we have 1 7 1 C3 D ; 2 2 2 which agrees with the calculation in previous example. x PŒA x D4 EŒX D EŒX j APŒA C EŒX j A
Example 8.31. Suppose that fXi gniD1 is a mutually independent family of random variables, where each Xi takes the value 1 with probability p and 0 with probability q WD 1 p. Let us set X WD X1 C C Xn . Note that the distribution of each Xi is that of a Bernoulli trial, as in Example 8.3, and the distribution of X is a binomial distribution, as in Example 8.18. By Theorems 8.16 and 8.21, we have EŒXi D p and VarŒXi D pq for i D 1; : : : ; n. Let us compute EŒX and VarŒX . By linearity of expectation, we have n X EŒX D EŒXi D np; i D1
and by Theorem 8.20, and the fact that fXi gniD1 is mutually independent (see Example 8.24), we have VarŒX D
n X
VarŒXi D npq:
i D1
Example 8.32. Our proof of Theorem 8.1 could be elegantly recast in terms of indicator variables. For B ˝, let XB be the indicator variable for B, so that XB .!/ D ı! ŒB for each ! 2 ˝. Equation (8.8) then becomes X XA D . 1/jJ j 1 XAJ ; ;¨J I
and by Theorem 8.16 and linearity of expectation, we have X X PŒA D EŒA D . 1/jJ j 1 EŒXAJ D . 1/jJ j ;¨J I
1
PŒXAJ :
;¨J I
E XERCISE 8.19. Let B be an event, with PŒB ¤ 0, and let fBi gi 2I be a finite, pairwise disjoint family of events whose union is B. Generalizing (8.24), show that for every real-valued random variable X , we have X EŒX j BPŒB D EŒX j Bi PŒBi ; i 2I
where I WD fi 2 I W PŒBi ¤ 0g. Also show that if EŒX j Bi ˛ for each i 2 I , then EŒX j B ˛.
8.4 Expectation and variance
239
E XERCISE 8.20. This exercise makes use of the notion of convexity (see §A8). (a) Prove Jensen’s inequality: if f is convex on an interval, and X is a random variable taking values in that interval, then EŒf .X / f .EŒX /. Hint: use induction on the size of the image of X . (Note that Theorem 8.19 is a special case of this, with f .s/ WD s 2 .) (b) Using part (a), show that if X takes non-negative real values, and ˛ is a positive number, then EŒX ˛ EŒX ˛ if ˛ 1, and EŒX ˛ EŒX ˛ if ˛ 1. (c) Using part (a), show that if X takes positive real values, then EŒX e EŒlog X : (d) Using part (c), derive the arithmetic/geometric mean inequality: for positive numbers x1 ; : : : ; xn , we have .x1 C C xn /=n .x1 xn /1=n : E XERCISE 8.21. Show that if fXi gniD1 is a pairwise independent family of 0/1valued random variables, and X WD X1 C C Xn , then VarŒX EŒX and VarŒX n=4. E XERCISE 8.22. For real-valued random variables X and Y , their covariance is defined as CovŒX ; Y WD EŒXY EŒX EŒY . Show that: (a) if X , Y , and Z are real-valued random variables, and a is a real number, then CovŒX C Y ; Z D CovŒX ; Z C CovŒY ; Z and CovŒaX ; Z D aCovŒX ; Z ; (b) if fXi gi 2I is a finite family of real-valued random variables, then hX i X X Var Xi D VarŒXi C CovŒXi ; Xj : i 2I
i 2I
i;j 2I i ¤j
E XERCISE 8.23. Consider again the game played between Alice and Bob in Example 8.11. Suppose that to play the game, Bob must place a one dollar bet. However, after Alice reveals the sum of the two dice, Bob may elect to double his bet. If Bob’s guess is correct, Alice pays him his bet, and otherwise, Bob pays Alice his bet. Describe an optimal playing strategy for Bob, and calculate his expected winnings. E XERCISE 8.24. A die is rolled repeatedly until it comes up “1,” or until it is rolled n times (whichever comes first). What is the expected number of rolls of the die?
240
Finite and discrete probability distributions
8.5 Some useful bounds In this section, we present several theorems that can be used to bound the probability that a random variable deviates from its expected value by some specified amount. Theorem 8.22 (Markov’s inequality). Let X be a random variable that takes only non-negative real values. Then for every ˛ > 0, we have PŒX ˛ EŒX =˛:
Proof. We have EŒX D
X
s PŒX D s D
s
X
s PŒX D s C
s<˛
X
s PŒX D s;
s˛
where the summations are over elements s in the image of X . Since X takes only non-negative values, all of the terms are non-negative. Therefore, X X EŒX s PŒX D s ˛ PŒX D s D ˛ PŒX ˛: s˛
s˛
Markov’s inequality may be the only game in town when nothing more about the distribution of X is known besides its expected value. However, if the variance of X is also known, then one can get a better bound. Theorem 8.23 (Chebyshev’s inequality). Let X be a real-valued random variable, with WD EŒX and WD VarŒX . Then for every ˛ > 0, we have PŒjX
j ˛ =˛ 2 :
Proof. Let Y WD .X /2 . Then Y is always non-negative, and EŒY D . Applying Markov’s inequality to Y , we have PŒjX
j ˛ D PŒY ˛ 2 =˛ 2 :
An important special case of Chebyshev’s inequality is the following. Suppose that fXi gi 2I is a finite, non-empty, pairwise independent family of real-valued random variables, each with the same distribution. Let be the common value of EŒXi , be the common value of VarŒXi , and n WD jI j. Set X x WD 1 Xi : X n i 2I
x is called the sample mean of fXi gi2I . By the linearity of expecThe variable X x D , and since fXi gi 2I is pairwise independent, it follows tation, we have EŒX
8.5 Some useful bounds
241
x D =n. from Theorem 8.20 (along with part (ii) of Theorem 8.18) that VarŒX Applying Chebyshev’s inequality, for every > 0, we have x j : PŒjX (8.25) n 2 The inequality (8.25) says that for all > 0, and for all ı > 0, there exists n0 (depending on and ı, as well as the variance ) such that n n0 implies x PŒjX
j ı:
(8.26)
In words: As n gets large, the sample mean closely approximates the expected value with high probability. This fact, known as the law of large numbers, justifies the usual intuitive interpretation given to expectation. Let us now examine an even more specialized case of the above situation, and assume that each Xi is a 0=1-valued random variable, taking the value 1 with probability p, and 0 with probability q WD 1 p. By Theorems 8.16 and 8.21, the Xi ’s have a common expected value p and variance pq. Therefore, by (8.25), for every > 0, we have x pj pq : PŒjX (8.27) n 2 The bound on the right-hand side of (8.27) decreases linearly in n. If one makes the stronger assumption that the family fXi gi 2I is mutually independent (so that P X WD i Xi has a binomial distribution), one can obtain a much better bound that decreases exponentially in n: Theorem 8.24 (Chernoff bound). Let fXi gi 2I be a finite, non-empty, and mutually independent family of random variables, such that each Xi is 1 with probability p and 0 with probability q WD 1 p. Assume that 0 < p < 1. Also, let n WD jI j x be the sample mean of fXi gi 2I . Then for every > 0, we have: and X x p e n2 =2q I (i) PŒX x (ii) PŒX x (iii) PŒjX
p
e
pj 2e
n 2 =2p n 2 =2
I
:
Proof. First, we observe that (ii) follows directly from (i) by replacing Xi by 1 Xi and exchanging the roles of p and q. Second, we observe that (iii) follows directly from (i) and (ii). Thus, it suffices to prove (i). Let ˛ > 0 be a parameter, whose value will be determined later. Define the ranx dom variable Z WD e ˛ n.X p/ . Since the function x 7! e ˛ nx is strictly increasing,
242
x we have X that
Finite and discrete probability distributions
p if and only if Z e ˛ n . By Markov’s inequality, it follows p D PŒZ e ˛ n EŒZ e
x PŒX
˛ n
:
(8.28)
So our goal is to bound EŒZ from above. For each i 2 I , define the random variable Zi WD e ˛.Xi p/ . Note that Z D Q i 2I Zi , that fZi gi 2I is a mutually independent family of random variables (see Theorem 8.12), and that for each i 2 I , we have EŒZi D e ˛.1
p/
p C e ˛.0
p/
q D pe ˛q C qe
˛p
:
It follows that hY
EŒZ D E
i Y Zi D EŒZi D .pe ˛q C qe
i2I
˛p n
/ :
i 2I
We will prove below that pe ˛q C qe
˛p
e˛
EŒZ e ˛
2 q n=2
2 q=2
:
(8.29)
From this, it follows that :
(8.30)
Combining (8.30) with (8.28), we obtain p e ˛
x PŒX
2 q n=2
˛ n
:
(8.31)
Now we choose the parameter ˛ so as to minimize the quantity ˛ 2 q n=2 ˛ n. The optimal value of ˛ is easily seen to be ˛ D =q, and substituting this value of ˛ into (8.31) yields (i). To finish the proof of the theorem, it remains to prove the inequality (8.29). Let ˇ WD pe ˛q C qe We want to show that ˇ e ˛
2 q=2
˛p
:
, or equivalently, that log ˇ ˛ 2 q=2. We have
ˇ D e ˛q .p C qe
˛
/ D e ˛q .1
q.1
e
˛
//;
and taking logarithms and applying parts (i) and (ii) of §A1, we obtain log ˇ D ˛q Clog.1 q.1 e
˛
// ˛q q.1 e
˛
/ D q.e
˛
C˛ 1/ q˛ 2 =2:
This establishes (8.29) and completes the proof of the theorem. Thus, the Chernoff bound is a quanti