TCP..IP Sockets in Java : Practical Guide for Programmers Morgan Kaufmann Practical Guides Series

Preface

For years, college courses in computer networking were taught with little or no hands on experience. For various reasons, including some good ones, instructors approached the principles of computer networking primarily through equations, analyses, and abstract descriptions of protocol stacks. Textbooks might have included code, but it would have been unconnected to anything students could get their hands on. We believe, however, that students learn better when they can see (and then build) concrete examples of the principles at work. And, fortunately, things have changed. The Internet has become a part of everyday life, and access to its services is readily available to most students (and their programs). Moreover, copious e x a m p l e s u g o o d and b a d - - o f nontrivial software are freely available. We wrote this book for the same reason we wrote TCP/IP Sockets in C: we needed a resource to support learning networking through p r o g r a m m i n g exercises in our courses. Our goal is to provide a sufficient introduction so that students can get their hands on real network services without too m u c h hand-holding. After grasping the basics, students can then move on to more advanced assignments, which support learning about routing algorithms, multimedia protocols, m e d i u m access control, and so on. We have tried to make this book equivalent to our earlier book to enable instructors to allow students to choose the language they use and still ensure that all students will come away with the same skills and understanding. Of course, it is not clear that this goal is achievable, but in any case the scope, price, and presentation level of the book are intended to be similar.

Intended Audience This book is aimed primarily at students in upper-division u n d e r g r a d u a t e or graduate courses in computer networks. It is intended as a supplement to a traditional textbook that explains the problems and principles of computer networks. At the same time, we have tried to make the

ix

X

Preface

m

book reasonably self-contained (except for the assumed programming background), so that it can also be used, for example, in courses on operating systems or distributed computing. For uses outside the context of a networking course, it will be helpful if the students have some acquaintance with the basic concepts of networking and TCP/IP. This book's other target audience consists of practitioners who know Java and want to learn about writing Java applications that use TCP/IP. This book should take such users far enough that they can start experimenting and learning on their own. Readers are a s s u m e d to have access to a computer equipped with Java. This book is based on Version 1.3 of Java and the Java Virtual Machine (JVM); however, the code should work with earlier versions of Java, with the exception of a few new Java methods. Java is about portability, so the particular hardware and operating system (OS) on which you run should not matter.

Approach Chapter 1 provides a general overview of networking concepts. It is not, by any means, a complete introduction, but rather is intended to allow readers to synchronize with the concepts and terminology used throughout the book. Chapter 2 introduces the mechanics of simple clients and servers; the code in this chapter can serve as a starting point for a variety of exercises. Chapter 3 covers the basics of message construction and parsing. The reader who digests the first three chapters should in principle be able to implement a client and server for a given (simple) application protocol. Chapter 4 then deals with techniques that are necessary when building more sophisticated and robust clients and servers. Finally, in keeping with our goal of illustrating principles through programming, Chapter 5 discusses the relationship between the p r o g r a m m i n g constructs and the underlying protocol implementations in somewhat more detail. Our general approach introduces programming concepts through simple p r o g r a m examples accompanied by line-by-line commentary that describes the purpose of every part of the program. This lets you see the important objects and methods as they are used in context. As you look at the code, you should be able to u n d e r s t a n d the purpose of each and every line. Java makes m a n y things easier, but it does not support some functionality that is commonly associated with the C/UNIX sockets interface (asynchronous I/O, select( )-style multiplexing). In C and C++, the socket interface is a generic application programming interface (API) for all types of protocols, not just TCP/IP. Java's socket classes, on the other hand, by default work exclusively with TCP and UDP over IPv4. Ironically, there does not seem to be anything in the Java specification or documentation that requires that an instance of the Socket class use TCP, or that a DatagramSoeket instance use UDP. Nevertheless, this book assumes this to be the case, as is true of current implementations. Our examples do not take advantage of all library facilities in Java. Some of these facilities, in particular serialization, effectively require that all communicating peers be i m p l e m e n t e d in Java. Also, to introduce examples as soon as possible, we wanted to avoid bringing in a thicket of m e t h o d s and classes that have to be sorted out later. We have tried to keep it simple, especially in the early chapters.

•

What This Book Is Not

xi

What This Book Is Not To keep the price of this b o o k within a reasonable range for a s u p p l e m e n t a r y text, we have had to limit its scope and maintain a tight focus on the goals outlined above. We omitted m a n y topics and directions, so it is p r o b a b l y worth mentioning some of the things this b o o k is not: • It is not an introduction to Java. We focus specifically on TCP/IP socket p r o g r a m m i n g using the Java language. We expect that the reader is already acquainted with the language and basic Java libraries (especially I/O), and knows how to develop p r o g r a m s in Java. • It is not a b o o k on protocols. Reading this b o o k will not m a k e you an expert on IP, TCP, FTP, HTTP, or any other existing protocol (except m a y b e the echo protocol). Our focus is on the interface to the TCP/IP services provided b y the socket abstraction. (It will help if you start with some idea a b o u t tl~e general workings of TCP and IP, b u t Chapter 1 m a y be an adequate substitute.) • It is not a guide to all of Java's rich collection of libraries that are designed to hide communication details (e.g., HTTPConnection) and make the p r o g r a m m e r ' s life easier. Since we are teaching the f u n d a m e n t a l s of how to do, not how to avoid doing, protocol development, we do not cover these parts of the API. We want readers to u n d e r s t a n d protocols in t e r m s of what goes on the wire, so we mostly use simple byte s t r e a m s and deal with character encodings explicitly. As a consequence, this text does not deal with URL, URLConnection, and so on. We believe that once you u n d e r s t a n d the principles, using these convenience classes will be straightforward. The network-relevant classes that we do cover include InetAddress, Socket, ServerSocket, DatagramPacket, DatagramSoeket, and UulticastSocket. • It is not a b o o k on object-oriented design. Our focus is on the i m p o r t a n t principles of TCP/IP socket p r o g r a m m i n g , and our examples are intended to illustrate t h e m concisely. As far as possible, we try to adhere to object-oriented design principles; however, w h e n doing so adds complexity that obfuscates the socket principles or bloats the code, we sacrifice design for clarity. This text does not cover design patterns for networking. (Though we would like to think that it provides s o m e of the b a c k g r o u n d necessary for understanding such patterns!) •

It is not a b o o k on writing production-quality code. Again, t h o u g h w e strive for robustness, the p r i m a r y goal of our code examples is education. In order to avoid obscuring the principles with large a m o u n t s of error-handling code, we have sacrificed s o m e r o b u s t n e s s for brevity and clarity.

• It is not a b o o k on doing your own native sockets i m p l e m e n t a t i o n in Java. We focus exclusively on TCP/IP sockets as provided by the s t a n d a r d Java distribution and do not cover the various socket i m p l e m e n t a t i o n w r a p p e r classes (e.g., Socketlmpl). • To avoid cluttering the examples with extraneous (nonsocket-related p r o g r a m m i n g ) code, we have made t h e m command-line based. While the b o o k ' s Web site, www.mkp.com/ practical/javasockets, contains a few examples of GUI-enhanced network applications, we do not include or explain t h e m in this text.

Preface

Xll

•

u

It is not a b o o k on Java applets. Applets use the same Java networking API so the communication code should be very similar; however, there are severe security restrictions on the kinds of c o m m u n i c a t i o n an applet can perform. We provide a very limited discussion of these restrictions and a single applet/application example on the Web site; however, a complete description of applet networking is beyond the scope of this text.

This b o o k will not make you an e x p e r t - - t h a t takes years of experience. However, we hope it will be useful as a resource, even to those who already know quite a bit about using sockets in Java. Both of us enjoyed writing it and learned quite a bit along the way.

Acknowledgments We would like to thank all the people who helped m a k e this b o o k a reality. Despite the book's brevity, m a n y h o u r s went into reviewing the original p r o p o s a l and the draft, and the reviewers' input has significantly shaped the final result. First, thanks to those who meticulously reviewed the draft of the text and m a d e suggestions for improvement. These include Michel Barbeau, Carlton University; Chris EdmondsonYurkanan, University of Texas at Austin, Ted Herman, University of Iowa; Dave Hollinger, Rensselaer Polytecnic Institute; Jim Leone, Rochester Institute of Technology; Dan Schmidt, Texas A&M University; Erick Wagner, EDS; and CSI4321, Spring 2001. Any errors that remain are, of course, our responsibility. We are very interested in weeding out such errors in future printings so if you find one, please email either of us. We will maintain an errata list on the b o o k ' s Web page. Finally, we are grateful to the folks at Morgan Kaufmarm. They care about quality and we appreciate that. We especially appreciate the efforts of Karyn Johnson, our editor, and Mei Levenson, our p r o d u c t i o n coordinator.

Feedback We invite your suggestions for the i m p r o v e m e n t of any aspect of this book. You can send feedback via the b o o k ' s Web page, www.mkp.com/practical/javasockets, or you can email us at the addresses below: Kenneth L. Calvert Michael J. Donahoo

[email protected] [email protected]

chapter

1

Introduction

M i l l i o n s of c o m p u t e r s all over the world are n o w c o n n e c t e d to the worldwide n e t w o r k k n o w n as the Internet. The Internet enables p r o g r a m s r u n n i n g on c o m p u t e r s t h o u s a n d s of miles apart to c o m m u n i c a t e and exchange information. If you have a c o m p u t e r c o n n e c t e d to a network, you m a y have u s e d a Web b r o w s e r - - a typical p r o g r a m that m a k e s use of the Internet. What does such a p r o g r a m do to c o m m u n i c a t e with others over a network? The answer varies with the application and the o p e r a t i n g s y s t e m (OS), b u t a great m a n y p r o g r a m s get access to n e t w o r k c o m m u n i c a t i o n services t h r o u g h the sockets application p r o g r a m m i n g interface (API). The goal of this b o o k is to get you s t a r t e d writing Java p r o g r a m s that use the sockets API. Before delving into the details of the API, it is w o r t h taking a brief look at the big picture of n e t w o r k s and p r o t o c o l s to see how an API for T r a n s m i s s i o n Control P r o t o c o l / I n t e r n e t Protocol fits in. Our goal here is not to teach you how n e t w o r k s and TCP/IP w o r k - - m a n y fine texts are available for that p u r p o s e [2, 4, 11, 16, 22J--but r a t h e r to i n t r o d u c e some basic concepts a n d terminology.

1.1

Networks, Packets, and Protocols

A c o m p u t e r n e t w o r k consists of m a c h i n e s i n t e r c o n n e c t e d by c o m m u n i c a t i o n channels. We call these m a c h i n e s hosts and routers. Hosts are c o m p u t e r s that r u n applications such as your Web browser. The application p r o g r a m s r u n n i n g on h o s t s are really the u s e r s of the network. Routers are m a c h i n e s w h o s e job is to relay, or forward, i n f o r m a t i o n f r o m one c o m m u n i c a t i o n channel to another. They m a y r u n p r o g r a m s b u t typically do not r u n application p r o g r a m s . For our p u r p o s e s , a communication channel is a m e a n s of conveying s e q u e n c e s of bytes f r o m one host to another; it m a y be a b r o a d c a s t t e c h n o l o g y like Ethernet, a dial-up m o d e m connection, or s o m e t h i n g m o r e sophisticated. Routers are i m p o r t a n t simply b e c a u s e it is not practical to connect every h o s t directly to every other host. Instead, a few h o s t s connect to a router, which c o n n e c t s to other routers, and so on to f o r m the network. This a r r a n g e m e n t lets each m a c h i n e get by with a relatively

2

Chapter 1: Introduction

I,,L]

A W

[]

Channel

(e.g., Ethernet) Host F i g u r e 1.1 :

1

" I ! ( IP ] ' I L

I

Router

~

d,p]

Channel "~ I

Host

A TCP/IP network.

small number of communication channels; most hosts need only one. Programs that exchange information over the network, however, do not interact directly with routers and generally remain blissfully unaware of their existence. By information we mean sequences of bytes that are constructed and interpreted by programs. In the context of computer networks, these byte sequences are generally called packets. A packet contains control information that the network uses to do its job and sometimes also includes user data. An example is information identifying the packet's destination. Routers use such control information to figure out how to forward each packet. A protocol is an agreement about the packets exchanged by communicating programs and what they mean. A protocol tells how packets are structured--for example, where the destination information is located in the packet and how big it ismas well as how the information is to be interpreted. A protocol is usually designed to solve a specific problem using given capabilities. For example, the HyperText Transfer Protocol (HTTP) solves the problem of transferring hypertext objects between servers, where they are stored, and Web browsers that make them available to h u m a n users. Implementing a useful network requires that a large number of different problems be solved. To keep things manageable and modular, different protocols are designed to solve different sets of problems. TCP/IP is one such collection of solutions, sometimes called a protocol suite. It happens to be the suite of protocols used in the Internet, but it can be used in stand-alone private networks as well. Henceforth when we talk about the "network," we mean any network that uses the TCP/IP protocol suite. The main protocols in the TCP/IP suite are the Internet Protocol (IP), the Transmission Control Protocol (TCP), and the User Datagram Protocol (UDP). It turns out to be useful to organize protocols into layers; TCP/IP and virtually all other protocol suites are organized this way. Figure 1.1 shows the relationships among the protocols, applications, and the sockets API in the hosts and routers, as well as the flow of data from one application (using TCP) to another. The boxes labeled TCP, UDP, and IP represent implementations of those protocols. Such implementations typically reside in the

[]

1.2 About Addresses

operating system of a host. Applications access the services provided by UDP and TCP through the sockets API. The arrow depicts the flow of data from the application, through the TCP and IP implementations, through the network, and back up through the IP and TCP implementations at the other end. In TCP/IP, the b o t t o m layer consists of the underlying communication c h a n n e l s n f o r example, Ethernet or dial-up m o d e m connections. Those channels are used by the network layer, which deals with the problem of forwarding packets toward their destination (i.e., what routers do). The single network layer protocol in the TCP/IP suite is the Internet Protocol; it solves the problem of making the sequence of channels and routers between any two hosts look like a single host-to-host channel. The Internet Protocol provides a datagram service: every packet is handled and delivered by the network independently, like letters or parcels sent via the postal system. To make this work, each IP packet has to contain the address of its destination, just as every package that you mail is addressed to somebody. (We'll say more about addresses shortly.) Although most delivery companies guarantee delivery of a package, IP is only a best-effort protocol: it attempts to deliver each packet, but it can (and occasionally does) lose, reorder, or duplicate packets in transit through the network. The layer above IP is called the transport layer. It offers a choice between two protocols: TCP and UDP. Each builds on the service provided by IP, but they do so in different ways to provide different kinds of transport, which are used by application protocols with different needs. TCP and UDP have one function in common: addressing. Recall that IP delivers packets to hosts; clearly, a finer granularity of addressing is needed to get a packet to a particular application, perhaps one of many using the network on the same host. Both TCP and UDP use addresses, called port numbers, to identify applications within hosts. They are called endto-end transport protocols because they carry data all the way from one program to another (whereas IP only carries data from one host to another). TCP is designed to detect and recover from the losses, duplications, and other errors that may occur in the host-to-host channel provided by IP. TCP provides a reliable byte-stream channel, so that applications do not have to deal with these problems. It is a connectionoriented protocol: before using it to communicate, two programs must first establish a TCP connection, which involves completing an exchange of handshake messages between the TCP implementations on the two communicating computers. Using TCP is also similar in many ways to file i n p u t / o u t p u t (I/O). In fact, a file that is written by one program and read by another is a reasonable model of communication over a TCP connection. UDP, on the other hand, does not attempt to recover from errors experienced by IP; it simply extends the IP best-effort datagram service so that it works between application programs instead of between hosts. Thus, applications that use UDP must be prepared to deal with losses, reordering, and so on.

1.2

About Addresses

When you mail a letter, you provide the address of the recipient in a form that the postal service can understand. Before you can talk to someone on the phone, you must supply their number to the telephone system. In a similar way, before a program can communicate with

4

Chapter 1: Introduction

I

another program, it m u s t tell the n e t w o r k where to find the other program. In TCP/IP, it takes two pieces of i n f o r m a t i o n to identify a particular program: an Internet address, u s e d by IP, and a port number, the additional address i n t e r p r e t e d by the t r a n s p o r t protocol (TCP or UDP). Internet a d d r e s s e s are 32-bit binary n u m b e r s . 1 In writing down Internet a d d r e s s e s for h u m a n c o n s u m p t i o n (as o p p o s e d to using t h e m inside applications), we typically show t h e m as a string of four decimal n u m b e r s s e p a r a t e d by periods (e.g., 10.1.2.3); this is called the dotted-quad notation. The four n u m b e r s in a dotted-quad string r e p r e s e n t the contents of the four bytes of the Internet a d d r e s s - - t h u s , each is a n u m b e r b e t w e e n 0 and 255. One special IP address w o r t h knowing is the loopback address, 127.0.0.1. This a d d r e s s is always assigned to a special loopback interface, which simply echoes t r a n s m i t t e d packets right back to the sender. The loopback interface is very useful for testing; it can be u s e d even w h e n a c o m p u t e r is not connected to the network. Technically, each Internet address refers to the connection b e t w e e n a host and an underlying c o m m u n i c a t i o n channel, such as a dial-up m o d e m or Ethernet card. Because each such n e t w o r k connection belongs to a single host, an Internet a d d r e s s identifies a host as well as its connection to the network. However, because a host can have multiple physical connections to the network, one host can have multiple Internet addresses. The port n u m b e r in TCP or UDP is always i n t e r p r e t e d relative to an Internet address. Returning to our earlier analogies, a port n u m b e r c o r r e s p o n d s to a r o o m n u m b e r at a given street address, say, that of a large building. The postal service uses the street a d d r e s s to get the letter to a mailbox; whoever empties the mailbox is then responsible for getting the letter to the p r o p e r r o o m within the building. Or consider a c o m p a n y with an internal telephone system: to speak to an individual in the company, you first dial the c o m p a n y ' s m a i n phone n u m b e r to connect to the internal telephone s y s t e m and then dial the extension of the particular telephone of the individual that you wish to speak with. In these analogies, the Internet a d d r e s s is the street a d d r e s s or the c o m p a n y ' s m a i n number, whereas the port c o r r e s p o n d s to the r o o m n u m b e r or telephone extension. Port n u m b e r s are 16-bit u n s i g n e d binary n u m b e r s , so each one is in the range 1 to 65,535 (0 is reserved).

1.3

A b o u t Names

Most likely you are a c c u s t o m e d to referring to hosts by name (e.g., host.example.com). However, the Internet protocols deal with numerical addresses, not names. You should u n d e r s t a n d that the use of n a m e s instead of a d d r e s s e s is a convenience feature that is i n d e p e n d e n t of the basic service provided by TCP/IP--you can write and use TCP/IP applications without ever

1Throughout this book the term Internet address refers to the addresses used with the current version of IP, which is version 4 [12]. Because it is expected that a 32-bit address space will be inadequate for future needs, a new version of IP has been defined [5]; it provides the same service but has much bigger Internet addresses (128 bits). IPv6, as the new version is known, has not been widely deployed; the sockets API will require some changes to deal with its much larger addresses [6].

[]

1.4 Clients and Servers

using a name. When you use a n a m e to identify a c o m m u n i c a t i o n endpoint, the s y s t e m has to do some extra w o r k to resolve the n a m e into an address. This extra step is o f t e n w o r t h it, for a couple of reasons. First, n a m e s are generally easier for h u m a n s to r e m e m b e r t h a n d o t t e d - q u a d s . Second, n a m e s provide a level of indirection, which insulates u s e r s f r o m IP a d d r e s s changes. During the writing of this book, the Web server for the p u b l i s h e r of this text, Morgan Kaufmann, c h a n g e d Internet a d d r e s s e s f r o m 208.164.121.48 to 216.200.143.124. However, b e c a u s e we refer to that Web server as www.mkp.com (clearly m u c h easier to r e m e m b e r t h a n 208.164.121.48) and b e c a u s e the change is reflected in the s y s t e m that m a p s n a m e s to a d d r e s s e s (www.mkp.com n o w resolves to the new Internet a d d r e s s i n s t e a d of 208.164.121.48), the change is t r a n s p a r e n t to p r o g r a m s that use the n a m e to access the Web server. The n a m e - r e s o l u t i o n service can access i n f o r m a t i o n f r o m a wide variety of sources. Two of the p r i m a r y sources are the Domain Name System (DNS) and local configuration databases. The DNS [9] is a d i s t r i b u t e d d a t a b a s e that m a p s domain names such as www.mkp.com to Internet a d d r e s s e s and o t h e r information; the DNS p r o t o c o l [10] allows h o s t s c o n n e c t e d to the Internet to retrieve i n f o r m a t i o n f r o m that d a t a b a s e using TCP or UDP. Local configuration d a t a b a s e s are generally OS-specific m e c h a n i s m s for local n a m e - t o - I n t e r n e t a d d r e s s m a p p i n g s .

1.4

Clients and Servers

In our postal and t e l e p h o n e analogies, each c o m m u n i c a t i o n is initiated by one party, who sends a letter or m a k e s the t e l e p h o n e call, while the other p a r t y r e s p o n d s to the initiator's contact by sending a r e t u r n letter or picking u p the p h o n e and talking. Internet c o m m u n i c a t i o n is similar. The t e r m s client and server refer to these roles: The client p r o g r a m initiates c o m m u n i c a t i o n , while the server p r o g r a m waits passively for and t h e n r e s p o n d s to clients that contact it. Together, the client and server c o m p o s e the application. The t e r m s client and server are descriptive of the typical situation in which the server m a k e s a particular capability--for example, a d a t a b a s e service--available to any client that is able to c o m m u n i c a t e with it. Whether a p r o g r a m is acting as a client or server d e t e r m i n e s the general f o r m of its use of the sockets API to establish c o m m u n i c a t i o n with its peer. (The client is the peer of the server and vice versa.) Beyond that, the client-server distinction is i m p o r t a n t b e c a u s e the client n e e d s to k n o w the server's a d d r e s s a n d p o r t initially, b u t not vice versa. With the sockets API, the server can, if necessary, learn the client's a d d r e s s i n f o r m a t i o n w h e n it receives the initial c o m m u n i c a t i o n f r o m the client. This is analogous to a t e l e p h o n e call--in o r d e r to be called, a p e r s o n does not n e e d to k n o w the t e l e p h o n e n u m b e r of the caller. As with a t e l e p h o n e call, once the c o n n e c t i o n is established, the distinction b e t w e e n server and client disappears. How does a client find out a server's IP a d d r e s s and p o r t n u m b e r ? Usually, the client knows the n a m e of the server it w a n t s m f o r example, f r o m a Universal Resource Locator (URL) such as http://www.mkp.com--and uses the n a m e - r e s o l u t i o n service to learn the c o r r e s p o n d i n g Internet address. Finding a server's p o r t n u m b e r is a different story. In principle, servers can use any port, but the client m u s t be able to learn what it is. In the Internet, there is a c o n v e n t i o n of assigning well-known p o r t n u m b e r s to certain applications. The Internet Assigned N u m b e r Authority

6

Chapter 1: Introduction

u

(IANA) oversees this assignment. For example, port number 21 has been assigned to the File Transfer Protocol (FTP). When you run an FTP client application, it tries to contact the FTP server on that port by default. A list of all the assigned port numbers is maintained by the numbering authority of the Internet (see http://www.iana.org/assignments/port-numbers).

1.5

W h a t Is a Socket?

A socket is an abstraction through which an application may send and receive data, in much the same way as an open file handle allows an application to read and write data to stable storage. A socket allows an application to plug in to the network and communicate with other applications that are plugged in to the same network. Information written to the socket by an application on one machine can be read by an application on a different machine and vice versa. Different types of sockets correspond to different underlying protocol suites and different stacks of protocols within a suite. This book deals only with the TCP/IP protocol suite. The main types of sockets in TCP/IP today are stream sockets and datagram sockets. Stream sockets use TCP as the end-to-end protocol (with IP underneath) and thus provide a reliable bytestream service. A TCP/IP stream socket represents one end of a TCP connection. Datagram sockets use UDP (again, with IP underneath) and thus provide a best-effort datagram service that applications can use to send individual messages up to about 65,500 bytes in length. Stream and datagram sockets are also supported by other protocol suites, but this book deals only with TCP stream sockets and UDP datagram sockets. A TCP/IP socket is uniquely identified by an Internet address, an end-to-end protocol (TCP or UDP), and a port number. As you proceed, you will encounter several ways for a socket to become b o u n d to an address. Figure 1.2 depicts the logical relationships among applications, socket abstractions, protocols, and port numbers within a single host. Note that a single socket abstraction can be referenced by multiple application programs. Each program that has a reference to a particular socket can communicate through that socket. Earlier we said that a port identifies an application on a host. Actually, a port identifies a socket on a host. From Figure 1.2, we see that multiple programs on a host can access the same socket. In practice, separate programs that access the same socket would usually belong to the same application (e.g., multiple copies of a Web server program), although in principle they could belong to different applications.

1.6

Exercises

1. Can you think of a real-life example of communication that does not fit the client-server model? 2. To how many different kinds of networks is your home connected? How many support two-way transport?

[]

1.6 Exercises

Applications

,-

.....

'~ . . . .-~. .

uDpSOCketsocketsreferences

TCP sockets . . . . . . . TCP ports

Sockets bound to ports

1

5535 UDPports UDP

("

IP

")

Figure 1.2: Sockets, protocols, and ports.

3. IP is a best-effort protocol, requiring that information be b r o k e n down into datagrams, which may be lost, duplicated, or reordered. TCP hides all of this, providing a reliable service that takes and delivers an u n b r o k e n stream of bytes. How might you go about providing TCP service on top of IP? Why would anybody use UDP when TCP is available?

chapter2

Basic Sockets

Y o u are now ready to learn about writing your own socket applications. We begin by demonstrating how Java applications identify network hosts. Then, we describe the creation of TCP and UDP clients and servers. Java provides a clear distinction between using TCP and UDP, defining a separate set of classes for both protocols, so we treat each separately.

2.1

Socket Addresses

IP uses 32-bit binary addresses to identify communicating hosts. A client must specify the IP address of the host running the server program when it initiates communication; the network infrastructure uses the 32-bit destination address to route the client's information to the proper machine. Addresses can be specified in Java using a string that contains either the dotted-quad representation of the numeric address (e.g., 169.1.1.1) or a name (e.g., server.example.corn). Java encapsulates the IP addresses abstraction in the InetAddress class which provides three static methods for creating lnetAddress instances, getByName() and getAllByName () take a name or IP address and return the corresponding InetAddress instance(s). For example, InetAddress.getByName("192.168.75.13") returns an instance identifying the IP address 192.168.75.13. The third method, getLocalHost (), returns an InetAddres s instance containing the local host address. Our first program example, InetAddressExample. java, demonstrates the use of InetAddress. The program takes a list of names or IP addresses as commandline parameters and prints the name and an IP address of the local host, followed by names and IP addresses of the hosts specified on the c o m m a n d line.

InetAdd ressExam pie.java

0

import java.net.*;

// for InetAddress

1

2 public class InetAddressExample { 3

9

!O

Chapter 2: Basic Sockets

[]

public static void main(String[] args) {

7 8

9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 3O

// Get name and IP address of the local host try { InetAddress address = InetAddress.getLocalHost(); System.out.println("Local Host:"); System.out.println("\t" + address.getHostName()); System.out.println("\t" + address.getHostAddress()); } catch (UnknownHostException e) { System.out.println("Unable to determine this host's address");

} for (int i = O; i < args.length; i++) { // Get name(s)/address(es) of hosts given on command line try { InetAddress[] addressList = InetAddress.getAllByName(args[i]); System.out.println(args[i] + ":"); // Print the first name. Assume array contains at least one entry. System.out.println("\t" + addressList[O].getHostName()); for (int j = O; j < addressList.length; j++) System.out.println("\t" + addressList[j].getHostAddress()); } catch (UnknownHostException e) { System.out.println("Unable to find address for " + args[i]);

}

InetAddressExample.java 1. Print i n f o r m a t i o n a b o u t the local host: lines 6-14 9 Create a n InetAddress i n s t a n c e for the local host: line 8 9 Print the local h o s t i n f o r m a t i o n : lines 9-11 getH0stName() and getH0stAddress() r e t u r n a string for the host n a m e and IP address, respectively. 2. R e q u e s t i n f o r m a t i o n for e a c h h o s t specified on the c o m m a n d line: lines 16-28 9 Create a n a r r a y of InetAddress i n s t a n c e s for the specified host: line 19 TnetAddress.getAllByName() r e t u r n s an array of InetAddress instances, one for each of the specified host's addresses. 9 Print the h o s t i n f o r m a t i o n : lines 22-24

To use this application to find information about the local host and the publisher's Web server (www.mkp.com), do the following:

[]

2.1 Socket Addresses

! !

% java InetAddressExample www.mkp.com Local Host: tractor.farm.com 169.1.1.2 www.mkp.com: www.mkp.com 216.200.143.124 If we k n o w the IP a d d r e s s of a h o s t (e.g., 169.1.1.1), we find the n a m e of the h o s t b y % java InetAddressExample 169. i. i. 1 Local Host: tractor, farm. com 169.1.1.2 169.1.1.1: base. farm. com 169.1.I.i W h e n the n a m e service is not available for s o m e reason--say, the p r o g r a m is running on a m a c h i n e t h a t is n o t c o n n e c t e d to a n y n e t w o r k - - a t t e m p t i n g to i d e n t i f y a h o s t b y n a m e m a y fail. Moreover, it m a y take a significant a m o u n t of time to do so, as the s y s t e m tries v a r i o u s ways to r e s o l v e the n a m e to an IP a d d r e s s . It is t h e r e f o r e g o o d to k n o w t h a t y o u can always refer to a h o s t u s i n g the IP a d d r e s s in d o t t e d - q u a d n o t a t i o n . In a n y of o u r e x a m p l e s , if a r e m o t e h o s t is specified by n a m e , the h o s t r u n n i n g the e x a m p l e m u s t be c o n f i g u r e d to c o n v e r t n a m e s to a d d r e s s e s , or the e x a m p l e w o n ' t work. If y o u can p i n g a h o s t u s i n g one of its n a m e s (e.g., r u n the c o m m a n d "ping server.example.corn"), t h e n the e x a m p l e s s h o u l d w o r k with n a m e s . If y o u r ping test fails or the e x a m p l e h a n g s , try specifying the h o s t b y IP a d d r e s s , w h i c h avoids the n a m e - t o - a d d r e s s c o n v e r s i o n a l t o g e t h e r .

InetAddress

1

Creators static I n e t A d d r e s s [ ] getAllByName(String host) R e t u r n s the list of a d d r e s s e s for the specified host.

host

H o s t n a m e or a d d r e s s

1For each Java networking class described in this text, we present only the primary methods and omit methods that are deprecated or whose use is beyond the scope of this text. As with everything in Java, the specification is a moving target. This information is included to provide an overall picture of the Java socket interface, not as a final authority. We encourage the reader to refer to the API specifications from java.sun.com as the current and definitive source.

!2

Chapter 2: Basic Sockets

[]

static InetAddress getByName(String host) static InetAddress getLocalHost0 Returns an IP address for the specified/local host.

host

Host name or IP address

Accessors byte[ ] getAddress0 Returns the 4 bytes of the 32-bit IP address in big-endian order. String getHostAddress() Returns the IP address in dotted-quad notation (e.g., "169.1.1.2"). String getHostName() Returns the canonical name of the host associated with the address. boolean isMulticastAddress() Returns true if the address is a multicast address (see Section 4.3.2).

Operators boolean equals(Object address) Returns true if address is non-null and represents the same address as this $netAddress instance.

address

2.2

Address to compare

TCP S o c k e t s

Java provides two classes for TCP: Socket and ServerSocket. An instance of Socket represents one end of a TCP connection. A TCP connection is an abstract two-way channel whose ends are each identified by an IP address and port number. Before being used for communication, a TCP connection must go through a setup phase, which starts with the client's TCP sending a connection request to the server's TCP. An instance of ServerSocket listens for TCP connection requests and creates a new Socket instance to handle each incoming connection.

2.2.1

TCPClient

The client initiates communication with a server that is passively waiting to be contacted. The typical TCP client goes through three steps: 1. Construct an instance of Socket: The constructor establishes a TCP connection to the specified remote host and port.

[]

2.2 TCP Sockets

13

2. C o m m u n i c a t e using the socket's I/O streams: A c o n n e c t e d instance of Socket contains an InputStream a n d 0utputStream that can be u s e d j u s t like any other Java I/O s t r e a m (see C h a p t e r 3). 3. Close the c o n n e c t i o n using the c l o s e ( ) m e t h o d of Socket. Our first TCP application, called TCPEchoClient.java, is a client that c o m m u n i c a t e s with an e c h o s e r v e r using TCP. An echo server simply r e p e a t s w h a t e v e r it receives back to the client. The string to be e c h o e d is p r o v i d e d as a c o m m a n d - l i n e a r g u m e n t to our client. Many s y s t e m s include an echo server for d e b u g g i n g and testing p u r p o s e s . To test if the s t a n d a r d echo server is running, try telnetting to p o r t 7 (the default echo port) on the server (e.g., at c o m m a n d line " t e l n e t s e r v e r , example, corn 7" or use your basic telnet application).

TCPEchoClient.java 0 import java.net.*; // for Socket 1 import java.io.*; // for lOException and Input/OutputStream 2 3 public class rCPEchoClient { 4 public static void main(String[] args) throws IOException { 5 6 7 if ((args.length < 2) II (args.length > 3)) // Test for correct # of args throw new lllegalArgumentException("Parameter(s): <Server> <Word> []"); 8 9 10 String server = args[0]; // Server name or IP address 11 // Convert input String to bytes using the default character encoding 12 byte[] byteBuffer = args[l].getBytes(); 13 14 int servPort = (args.length == 3) ? Integer.parselnt(args[2]) : 7; 15 16 // Create socket that is connected to server on specified port 17 Socket socket = new Socket(server, servPort) ; 18 System.out.println("Connected to server...sending echo string"); 19 20 InputStream in = socket, getlnputStream() ; 21 OutputStream out = socket, getOutputStream() ; 22 23 out.write(byteBuffer); // Send the encoded string to the server 24 25 // Receive the same string back from the server 26 int totalBytesRcvd = 0; // Total bytes received so far 27 int bytesRcvd; // Bytes received in last read 28 while (totalBytesRcvd < byteBuffer.length) { 29 if ((bytesRcvd = in.read(byteBuffer, totalBytesRcvd, byteBuffer.length - totalBytesRcvd)) == -I) 3O 31 throw new SocketException("Connection closed prematurely");

14 32 33 34 35 36 37 38 39

Chapter 2: Basic Sockets

m

totalBytesRcvd += bytesRcvd; } System.out.println("Received: socket.close();

" + new String(byteBuffer)) ;

// Close the socket and its streams

} }

TCP Ec hoCI ie nt.ja va

1. Application setup and parameter parsing: lines 0-14 [] Convert the echo string: line 12 TCP sockets send and receive sequences of bytes. The getBytes() m e t h o d of String returns a byte array representation of the string. (See Section 3.1 for a discussion of character encodings.) [] Determine the port of the echo server: line 14 The default echo port is 7. If we specify a third parameter, I n t e g e r . p a r s e I n t ( ) takes the string and returns the equivalent integer value. 2. TCP socket creation: line 17 The Socket constructor creates a socket and establishes a connection to the specified server, identified either by name or IP address. Note that the underlying TCP deals only with IP addresses. If a name is given, the implementation resolves it to the corresponding address. If the connection attempt fails for any reason, the constructor throws an lOBxception. 3. Get socket input and output streams: lines 20-21 Associated with each connected Socket instance is an InputStream and 0utputStream. We send data over the socket by writing bytes to the 0utputStream just as we would any other stream, and we receive by reading from the InputStream. 4. Send the string to echo server: line 23 The w r i t e ( ) m e t h o d of 0utputStream transmits the given byte array over the connection to the server.

5. Receive the reply from the echo server: lines 25-33 Since we know the number of bytes to expect from the echo server, we can repeatedly receive bytes until we have received the same number of bytes we sent. This particular form of read() takes three parameters: 1) buffer to receive into, 2) byte offset into the buffer where the first byte received should be placed, and 3) the m a x i m u m n u m b e r of bytes to be placed in the buffer, read() blocks until some data is available, reads up to the specified m a x i m u m n u m b e r of bytes, and returns the n u m b e r of bytes actually placed in the buffer (which may be less than the given maximum). The loop simply fills

[]

2.2 TCP Sockets

15

up byteBuffer until we receive as many bytes as we sent. If the TCP connection is closed by the other end, read() returns -1. For the client, this indicates that the server prematurely closed the socket. Why not just a single read? TCP does not preserve read() and w r i t e ( ) message boundaries. That is, even though we sent the echo string with a single w r i t e ( ) , the echo server may receive it in multiple chunks. Even if the echo string is handled in one chunk by the echo server, the reply may still be broken into pieces by TCP. One of the most c o m m o n errors for beginners is the assumption that data sent by a single w r i t e ( ) will always be received in a single read (). 6. Print e c h o e d string: line 35 To print the server's response, we must convert the byte array to a string using the default character encoding. 7. Close socket: line 37 When the client has finished receiving all of the echoed data, it closes the socket. We can communicate with an echo server named server.example.com with IP address 169.1.1.1 in either of the following ways: % java TCPEchoClient server.example.com "Echo this!" Received: Echo this! % java TCPEchoClient 169. i. i. 1 "Echo this!" Received: Echo this! See TCPEchoClientGUI. java on the book's W e b site for an implementation of the T C P echo client with a graphical interface.

Socket Constructors Socket(InetAddress remoteAddr, int remotePort) Socket(String remoteHost, int remotePort) Socket(InetAddress remoteAddr, int remotePort, InetAddress localAddr, int localPort) Socket(String remoteHost, int remotePort, InetAddress localAddr, int localPort) Constructs a TCP socket connected to the specified remote address and port. The first two forms of the constructor do not specify the local address and port, so a default local address and some available port are chosen. Specifying the local address may be useful on a host with multiple interfaces.

remoteAddr remoteHost remotePort

Remote host address Remote host name or IP address (in dotted-quad form) Remote port

16

Chapter 2: Basic Sockets

[]

localAddr

Local address; use null to specify using the default local address

localPort

Local port; a localPort of 0 allows the constructor to pick any available port

Operators void close() Closes the TCP socket and its I/O streams.

void shutdownTnput() Closes the input side of a TCP stream. Any unread data is silently discarded, including data buffered by the socket, data in transit, and data arriving in the future. Any subsequent a t t e m p t to read from the socket will return end-of-stream (-1); any subsequent call to getlnputStream() will cause an lOException to be thrown (see Section 4.5). void shutdown0utput() Closes the output side of a TCP stream. The implementation will attempt to deliver any data already written to the socket's output stream to the other end. Any subsequent attempt to write to the socket's output stream or to call get0utputStream() will cause an IOException to be thrown (see Section 4.5).

Accessors/Mutators InetAddress getlnetAddress() int getPort() Returns the remote socket address/port.

InputStream getlnputStream0 OutputStream get0utputStream0 Returns a stream for reading/writing bytes f r o m / t o the socket.

boolean getKeepAlive() void setKeepAlive(boolean on) R e t u r n s / s e t s keepalive message behavior. If keepalive is enabled, TCP sends a probe to the other end of the connection when no data has been exchanged for a systemdependent amount of time (usually two hours). If the remote socket is still alive, it will acknowledge the probe (invisible to the application). However, if the other end fails to acknowledge several probes in a row, the local TCP closes the connection, and subsequent operations on it will throw an exception. Keepalive is disabled by default. on

If true (false), enable (disable) keepalive.

[]

2.2 TCP Sockets

|

InetAddress getLocalAddress()

int getLocalPort() Returns the local socket address/port. int getReceiveBufferSize() int getSendBufferSize()

void setReceiveBufferSize(int size) void setSendBufferSize(int size) Returns/sets the size of the send/receive buffer for the socket (see Section 4.4). size

Number of bytes to allocate for the socket send/receive buffer

int getSoLinger() void setSoLinger(boolean on, int linger) Returns/sets the m a x i m u m amount of time (in milliseconds) that close() will block waiting for all data to be delivered, getSoLinger() returns -1 if lingering is disabled (see Section 5.4). Lingering is off by default. on

If true, the socket lingers on close(), up to the m a x i m u m specified time.

linger

The m a x i m u m amount of time (milliseconds) a socket lingers

on close() int getSoTimeout() void setSoTimeout(int timeout) Returns/sets the m a x i m u m amount of time that a read() on this socket will block. If the specified n u m b e r of milliseconds elapses before any data is available, an I n t e r ruptedIOException is thrown (see Section 4.2). timeout

The m a x i m u m time (milliseconds) to wait for data on a read(). The value 0 (the default) indicates that there is no time limit, meaning that a read will not return until data is available.

boolean getTcpNoDelay() void setTcpNoDelay(boolean on) Returns/sets whether the Nagle algorithm to coalesce TCP packets is disabled. To avoid small TCP packets, which make inefficient use of network resources, Nagle's algorithm (enabled by default) delays packet transmission under certain conditions to improve the opportunities to coalesce bytes from several writes into a single TCP packet. This delay is unacceptable to some types of interactive applications. on

If true (false), disable (enable) Nagle's algorithm.

18

Chapter 2: Basic Sockets

m

Caveat: By default, Socket is implemented on top of a TCP connection; however, in Java, you can actually change the underlying implementation of Socket. This book is about TCP/IP, so for simplicity we assume that the underlying implementation for all of the these networking classes is the default.

2.2.2

TCP Server

We now turn our attention to constructing a server. The server's job is to set up a communication endpoint and passively wait for connections from clients. The typical TCP server goes through two steps: 1. Construct a ServerSocket instance, specifying the local port. This socket listens for incoming connections to the specified port. 2. Repeatedly: 9 Call the accept () m e t h o d of ServerSocket to get the next incoming client connection. Upon establishment of a new client connection, an instance of Socket for the new connection is created and returned by accept (). 9 Communicate with the client using the returned Socket's InputStream and OutputStream. 9 Close the new client socket connection using the close() m e t h o d of Socket. Our next example, TCPEchoServer. java, implements the echo service used by our client program. The server is very simple. It runs forever, repeatedly accepting a connection, receiving and echoing bytes until the connection is closed by the client, and then closing the client socket.

TCPEchoServer.java 0 import java.net.* ; / / for Socket, ServerSocket, and InetAddress 1 import java.io.*; / / for IOException and Input/0utputStream 2 3 public class TCPEchoServer { 4 private static final int BUFSIZE = 32; // Size of receive buffer 5 6 public static void main(String[] args) throws lOException { 7 8 if (args.length != i) // Test for correct # of args 9 throw new lllegalArgumentException("Parameter(s): "); 10 11 int servPort = Integer. parselnt (args [0 ] ) ; 12 13 // Create a server socket to accept client connection requests 14 ServerSocket servSock = new ServerSocket(servPort) ; 15 16 int recvMsgSize; // Size of received message 17 byte[] byteBuffer = new byte[BUFSlZE]; // Receive buffer 18

m

19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38

2.2 TCP Sockets

19

for (;;) { // Run forever, accepting and servicing connections Socket clntSock = servSock.accept() ; // Get client connection System.out.println("Handling client at " + clntSock.getInetAddress().getHostAddress() clntSock, getPort ()) ;

+ " on port " +

InputStream in = clntSock, getlnputStream() ; OutputStream out = clntSock.getOutputStream() ; // Receive until client closes connection, indicated b y - i return while ((recvMsgSize = in.read(byteBuffer)) != -I) out.write(byteBuffer, O, recvMsgSize) ; clntSock, close () ; // Close the socket. } /* NOT REACHED */

We are done with this client!

} }

TCPEchoServer.java

1. Application setup and p a r a m e t e r parsing: lines 0-12 2. Server socket creation: line 15 servSock listens for client connection requests on the port specified in the constructor. 3. Loop forever, iteratively handling i n c o m i n g connections" lines 20-35 9 Accept an i n c o m i n g connection: line 21 The sole purpose of a ServerSocket instance is to supply a new, connected Socket instance for each new TCP connection. When the server is ready to handle a client, it calls accept (), which blocks until an incoming connection is made to the ServerSocket's port. accept() then returns an instance of Socket that is already connected to the remote socket and ready for reading and writing. 9 Report c o n n e c t e d client: lines 23-25 We can query the newly created Socket instance for the address and port of the connecting client. The getlnetAddress() m e t h o d of Socket returns an instance of InetAddress containing the address of the client. We call getHostAddress() to return the IP address as a dotted-quad String. The ge tPort() m e t h o d of Socket returns the port of the client. 9 Get socket input and output streams: lines 27-28 Bytes written to this socket's 0utputStream will be read from the client's socket's InputStream, and bytes written to the client's 0utputStream will be read from this socket's InputStream.

20

Chapter 2: Basic Sockets

9

[]

Receive and repeat data until the client closes: lines 30-32 The while loop repeatedly reads bytes (when available) from the input stream and immediately writes the same bytes back to the output stream until the client closes the connection. The read() m e t h o d of InputStream reads up to the m a x i m u m n u m b e r of bytes the array can hold (in this case, BUFSIZE bytes) into the byte array (byteBuffer) and returns the n u m b e r of bytes read. read () blocks until data is available and returns -1 if there is no data available, indicating that the client closed its socket. In the echo protocol, the client closes the connection when it has received the n u m b e r of bytes back that it sent, so in the server we expect to receive a -1 from read(). Recall that in the client, receiving a -1 from read() indicates an error because it indicates that the server prematurely closed the connection. As previously mentioned, read() does not have to fill the entire byte array to return. In fact, it can return after having read only a single byte. The w r i t e ( ) m e t h o d of 0utputStream writes recvMsgSize bytes from byteBuffer to the socket. The second parameter indicates the offset into the byte array of the first byte to send. In this case, 0 indicates to take bytes starting from the front of byteBuffer. If we had used the form of w r i t e ( ) that takes only the buffer argument, all the bytes in the buffer array would have been transmitted, possibly including bytes that were not received from the client!

9 Close client socket: line 34

ServerSocket Constructors

ServerSocket(int localPort) ServerSocket(int localPort, int queueLimit) ServerSocket(int localPort, int queueLimit, InetAddress localAddr) Construct a TCP socket that is ready to accept incoming connections to the specified local port. Optionally, the size of the connection queue and the local address can be set.

localPort

Local port. A port of 0 allows the constructor to pick any available port.

queueLimit

The m a x i m u m size of the queue of incomplete connections and sockets waiting to be accept()ed. If a client connection request arrives when the queue is full, the connection is refused. Note that this may not necessarily be a hard lirrdt. For most platforms, it cannot be used to precisely control client population.

m 2.2 TCP Sockets

localAddr

21

The IP address to which connections to this socket should be addressed (must be one of the local interface addresses). If the address is not specified, the socket will accept connections to any of the host's IP addresses. This may be useful for hosts with multiple interfaces where the server socket should only accept connections on one of its interfaces.

Operators Socket accept() Returns a connected Socket instance for the next n e w incoming connection to the server socket. If no established connection is waiting, accept() blocks until one is established or a timeout occurs (see setSoTimeout()). void close() Closes the underlying TCP socket. After invoking this method, incoming client connection requests for this socket are rejected.

Accessors/Mutators InetAddress getlnetAddress () int getLocalPort() Returns the local a d d r e s s / p o r t of the server socket. int getSoTimeoutO void setSoTimeout(int timeout) Returns/sets the m a x i m u m amount of time (in milliseconds) that an accept() will block for this socket. If the timer expires before a connection request arrives, an InterruptedlOException is thrown. A timeout value of 0 indicates no timeout: calls to accept () will not return until a new connection is available, regardless of how much time passes (see Section 4.2).

2.2.3

Input and O u t p u t Streams

As illustrated by the examples above, the primary paradigm for I/O in Java is the stream abstraction. A stream is simply an ordered sequence of bytes. Java input streams support reading bytes, and output streams support writing bytes. In our TCP client and server, each Socket instance holds an InputStream and an 0utputStream instance. When we write to the output stream of a Socket, the bytes can (eventually) be read from the input stream of the Socket at the other end of the connection. 0utputStream is the abstract superclass of all output streams in Java. Using an OutputStream, we can write bytes to, flush, and close the output stream.

22

Chapter 2: Basic Sockets

II

OutputStream abstract v o i d write(int data)

Writes a single byte to the output stream.

data

Byte (low-order 8 bits) to write to output stream

v o i d write(byte[] data)

Writes entire array of bytes to the output stream.

data

Bytes to write to output stream

v o i d write(byte[ ] data, int offset, int length)

Writes length bytes from data starting from byte offset.

data offset length

Bytes from which to write to output stream Starting byte to send in data Number of bytes to send

v o i d flush()

Pushes any buffered data out to the stream. v o i d close()

Terminates the stream.

InputStream is the abstract superclass of all input streams. Using an InputStream, we can read bytes from and close the input stream.

InputStream a b s t r a c t int read() Read and return a single byte from the input stream. The byte read is in the least significant byte of the returned integer. This method returns -1 on end-of-stream. int read(byte[] data)

Reads up to data.length bytes (or until the end-of-stream) from the input stream into data and returns the number of bytes read. If no data is available, read () blocks until at least I byte can be read or the end-of-stream is detected, indicated by a return of -1.

data

Buffer to receive data from input stream

int read(byte[ ] data, int offset, int length) Reads up to length bytes (or until the end-of-stream) from the input stream into data, starting at position offset, and returns the n u m b e r of bytes read. If no data

[]

2.3 UDP Sockets

23

is available, read() blocks until at least 1 byte can be read or the end-of-stream is detected, indicated by a return of -1.

data offset length

Buffer to receive data from input stream Starting byte of data in which to write Maximum number of bytes to read

int available()

Returns the number of bytes available for input. void close() Terminates the stream.

2.3

UDP Sockets

UDP provides an end-to-end service different from that of TCP. In fact, UDP performs only two functions: 1) it adds another layer of addressing (ports) to that of IP, and 2) it detects data corruption that may occur in transit and discards any corrupted messages. Because of this simplicity, UDP sockets have some different characteristics from the TCP sockets we saw earlier. For example, UDP sockets do not have to be connected before being used. Where TCP is analogous to telephone communication, UDP is analogous to communicating by mail: you do not have to "connect" before you send a package or letter, but you do have to specify the destination address for each one. Similarly, each message--called a datagram--carries its own address information and is independent of all others. In receiving, a UDP socket is like a mailbox into which letters or packages from many different sources can be placed. As soon as it is created, a UDP socket can be used to send/receive messages t o / f r o m any address and t o / f r o m many different addresses in succession. Another difference between UDP sockets and TCP sockets is the way that they deal with message boundaries: UDP sockets preserve them. This makes receiving an application message simpler, in some ways, than it is with TCP sockets. (This is discussed further in Section 2.3.4.) A final difference is that the end-to-end transport service UDP provides is best-effort: there is no guarantee that a message sent via a UDP socket will arrive at its destination, and messages can be delivered in a different order than they were sent Oust like letters sent through the mail). A program using UDP sockets must therefore be prepared to deal with loss and reordering. (We'll provide an example of this later.) Given this additional burden, why would an application use UDP instead of TCP? One reason is efficiency: if the application exchanges only a small amount of data--say, a single request message from client to server and a single response message in the other direction-TCP's connection establishment phase at least doubles the number of messages (and the number of round-trip delays) required for the communication. Another reason is flexibility: when something other than a reliable byte-stream service is required, UDP provides a minimaloverhead platform on which to implement whatever is needed. Java programmers use UDP sockets via the classes DatagramPacket and DatagramSocket. Both clients and servers use DatagramSockets to send and receive DatagramPackets.

24

Chapter 2: Basic Sockets

2.3.1

DatagramPacket

[]

Instead of sending and receiving streams of bytes as with TCP, UDP endpoints exchange self-contained messages, called datagrams, which are represented in Java as instances of DatagramPacket. To send, a Java program constructs a DatagramPacket instance and passes it as an argument to the send() method of a DatagramSocket. To receive, a Java program constructs a DatagramPacket instance with preallocated space (a byte[ ]), into which the contents of a received message can be copied (if/when one arrives), and then passes the instance to the receive () method of a DatagramSocket. In addition to the data, each instance of DatagramPacket also contains address and port information, the semantics of which depend on whether the datagram is being sent or received. When a DatagramPacket is sent, the address and port identify the destination; for a received DatagramPacket, they identify the source of the received message. Thus, a server can receive into a DatagramPacket instance, modify its buffer contents, then send the same instance, and the modified message will go back to its origin. Internally, a DatagramPacket also has length and offset fields, which describe the location and number of bytes of message data inside the associated buffer. See the following reference and Section 2.3.4 for some pitfalls to avoid when using DatagramPackets.

Datag ram Packet Constructors DatagramPacket(byte[ ] DatagramPacket(byte[ ] DatagramPacket(byte[ ] DatagramPacket(byte[]

buffer, int length) buffer, int offset, int length) buffer, int length, InetAddress remoteAddr, int remotePort) buffer, int offset, int length, InetAddress remoteAddr, int re-

motePort) Constructs a datagram and makes the given byte array its data buffer. The first two forms are typically used to construct DatagramPackets for receiving because the destination address is not specified (although it could be specified later with setAddress() and setPort ()). The second two forms are typically used to construct DatagramPackets for sending.

buffer length

offset

Datagram payload Number of bytes of the buffer that will actually be used. If the datagram is sent, length bytes will be transmitted. If receiving into this datagram, length specifies the maximum number of bytes to be placed in the buffer. Location in the buffer array of the first byte of message data to be sent/received; defaults to 0 if unspecified.

[]

remoteAddr remotePort

2.3 UDP Sockets

25

Address (typically destination) of the datagram Port (typically destination) of the datagram

Accessors/Mutators InetAddress getAddress() void setAddress(InetAddress address)

Returns/sets the datagram address. There are other ways to set the address: 1) the address of a DatagramPacket instance can also be set by the constructor, and 2) the receive() method of DatagramSocket sets the address to the datagram sender's address.

address

Datagram address

int getPort() void setPort(int port)

Returns/sets the datagram port. There are other ways to set the address: 1) the port can be explicitly set by the constructor or the setPort() method, and 2) the receive() method of DatagramSocket sets the port to the datagram sender's port.

port

Datagram port

int getT.ength() void setLength(int length)

Returns/sets the internal length of the datagram. The internal datagram length can be set explicitly by the constructor or by the setLength() method. Attempting to make it larger than the length of the associated buffer results in an IllegalArgumentException. The receive() method of DatagramSocket uses the internal length in two ways: 1) on input, it specifies the maximum number of bytes of a received message that will be copied into the buffer, and 2) on return, it indicates the number of bytes actually placed in the buffer.

length

Length in bytes of the usable portion of the buffer

int get0ffset()

Returns the location in the buffer of the first byte of data to be sent/received. There is no s e t 0 f f s e t ( ) method; however, it can be set with setData().

byte[] getDataO Returns the buffer associated with the datagram. The returned object is a reference to the byte array that was most recently associated with this DatagramPacket, either by the constructor or by setData(). The length of the returned buffer may be greater than the internal datagram length, so the internal length and offset values should be used to determine the actual received data.

26

Chapter 2: Basic Sockets

[]

void setData(byte[] buffer) void setData(byte[] buffer, int offset, int length) Makes the given byte array the datagram buffer. The first form makes the entire byte array the buffer; the second form makes bytes offset through offset + length- 1 the buffer. The first form never updates the internal offset and only updates the internal length if the given buffer's length is less than the current internal length. The second form always updates the internal offset and length.

buffer offset length

2.3.2

Preallocated byte array for datagram packet data Location in buffer where first byte is to be accessed Number of bytes to be read from/written into buffer

UDPClient

A UDP client begins by sending a datagram to a server that is passively waiting to be contacted. The typical UDP client goes through three steps: 1. Construct an instance of DatagramSocket, optionally specifying the local address and port. 2. Communicate by sending and receiving instances of DatagramPacket using the send() and receive() methods of DatagramSocket. 3. When finished, deallocate the socket using the close() method of DatagramSocket. Unlike a Socket, a DatagramSocket is not constructed with a specific destination address. This illustrates one of the major differences between TCP and UDP. A TCP socket is required to establish a connection with another TCP socket on a specific host and port before any data can be exchanged, and, thereafter, it only communicates with that socket until it is closed. A UDP socket, on the other hand, is not required to establish a connection before communication, and each datagram can be sent to or received from a different destination. (The connect () method of DatagramSocket does allow the specification of the remote address and port, but its use is optional.) Our UDP echo client, UDPEchoClientTimeout. java, sends a datagram containing the string to be echoed and prints whatever it receives back from the server. A UDP echo server simply repeats each datagram that it receives back to the client. Of course, a UDP client only communicates with a UDP server. Many systems include a UDP echo server for debugging and testing purposes. One consequence of using UDP is that datagrams can be lost. In the case of our echo protocol, either the echo request from the client or the echo reply from the server may be lost in the network. Recall that our TCP echo client sends an echo string and then blocks on read () waiting for a reply. If we try the same strategy with our UDP echo client and the echo request datagram is lost, our client will block forever on receive (). To avoid this problem, our client specifies a maximum amount of time to block on receive(), after which it tries again by resending the echo request datagram. Our echo client performs the following steps:

[]

2.3 UDP Sockets

27

1. Send the echo string to the server. 2. Block o n r e c e i v e ( ) for u p to t h r e e s e c o n d s , s t a r t i n g over (up to five times) if the reply is n o t received b e f o r e the t i m e o u t . 3. T e r m i n a t e the client.

UDPEchoClientTimeout.java O 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39

import java.net.*; import java.io.*;

/ / for DatagramSocket, DatagramPacket, and InetAddress

// for lOBxception

public class UDPEchoClientTimeout { private static final int TIMEOUT = 3000; private static final int MAXTRIBS = 5;

// Resend timeout (milliseconds) // Maximum retransmissions

public static void main(String[] args) throws lOException { if ((args.length < 2) I I (args.length > 3)) // Test for correct # of args throw new lllegalArgumentException("Parameter(s)' <Server> <Word> []"); InetAddress serverAddress = InetAddress.getByName(args[0]); // Server address // Convert the argument String to bytes using the default encoding byte[] bytesZoSend = args[l].getBytes(); int servPort = (args.length == 3) ? Integer.parselnt(args[2])

9 7;

DatagramSocket socket = new DatagramSocket() ;

socket.setSoTimeout(TIMEOUT);

// Maximum receive blocking time (milliseconds)

DatagramPacket sendPacket = new DatagramPacket(bytesToSend, bytesToSend.length, serverAddress, servPort) ;

DatagramPacket receivePacket = new DatagramPacket(new byte[bytesToSend.length],

// Sending packet

// Receiving packet bytesToSend.length);

int tries = 0; // Packets may be lost, so we have to keep trying boolean receivedResponse = false; do { socket.send(sendPacket); // Send the echo string try { socket.receive(receivePacket); // Attempt echo reply reception if (!receivePacket.getAddress().equals(serverAddress)) // Check source throw new lOException("Received packet from an unknown source"); receivedResponse = true;

28 40 41 42 43 44 45 46 47 48 49 50 51 52 53

Chapter 2: Basic Sockets

[]

} catch (InterruptedlOException e) { // We did not get anything tries += I; System.out.println("Timed out, " + ( ~ T R I E S - tries) + " more tries...") ; } } while ((!receivedResponse)

&& (tries < ~s

;

if (receivedResponse) System.out.println("Received: " + new String(receivePacket.getData())); else System. out.println("No response -- giving up.") ; socket.close(); } }

UDPEchoClientTimeout.java 1. Application setup and parameter parsing: lines 0-17 Convert argument to bytes: line 15 2. UDP socket creation: line 19 This instance of DatagramSocket can send datagrams to any UDP socket. We do not specify a local address or port so some local address and available port will be selected. We can explicitly set t h e m with the setLocalAddress() and setLocalPort() m e t h o d s or in the constructor.

3. Set the socket timeout: line 21 The timeout for a datagram socket controls the m a x i m u m a m o u n t of time (milliseconds) a call to r e c e i v e ( ) will block. Here we set the timeout to three seconds. Note that timeouts are not precise: the call may block for more than the specified time (but not less). 4. Create d a t a g r a m to send: lines 23-24 To create a datagram for sending, we need to specify three things: data, destination address, and destination port. For the destination address, we may identify the echo server either by name or IP address. If we specify a name, it is converted to the actual IP address in the constructor. 5. Create d a t a g r a m to receive: lines 26-27 To create a datagram for receiving, we only need to specify a byte array to hold the d a t a g r a m data. The address and port of the datagram source will be filled in by receive (). 6. Send the datagram: lines 29-44 Since datagrams may be lost, we m u s t be prepared to retransmit the datagram. We loop sending and attempting a receive of the echo reply up to five times. 9 Send the datagram: line 32 send() transmits the datagram to the address and port specified in the datagram.

II

2.3 UDP Sockets

29

9 Handle d a t a g r a m reception: lines 33-43 r e c e i v e ( ) blocks until it either receives a d a t a g r a m or the timer expires. Timer expiration is indicated by an InterruptedlOException. If the timer expires, we increment the send a ttemp t count (tries) and start over. After the m a x i m u m n u m b e r of tries, the while loop exits without receiving a datagram. If r e c e i v e ( ) succeeds, we set the loop flag receivedResponse to true, causing the loop to exit. Since packets may come from anywhere, we check the source address of the recieved d a t a g r a m to verify that it matches the address of the specified echo server. 7. Print r e c e p t i o n results: lines 46-49 If we received a datagram, receivedResponse is true, and we can print the datagram data. 8. Close the socket: line 51 We invoke the UDP client using the same parameters as used in the TCP client.

DatagramSocket Constructors DatagramSocket() DatagramSocket(int localPort) DatagramSocket(int localPort, I n e t A d d r e s s localAddr) Constructs a UDP socket. Either or both the local port and address may be specified. If the local port is not specified, the socket is bound to any available local port. If the local address is not specified, one of the local addresses is chosen.

localPort

Local port; a localPort of 0 allows the constructor to pick any available port.

localAddr

Local address

Operators void close() After closing, datagrams may no longer be sent or received using this socket.

void connect(InetAddress remoteAddr, int remotePort) Sets the remote address and port of the socket. Attempting to send datagrams with a different address will cause an exception to be thrown. The socket will only receive datagrams from the specified port and address. Datagrams from any other port or address are ignored. This is strictly a local operation because there is no end-to-end connection. Caveat: A socket that is connected to a multicast or broadcast address can

30

Chapter 2: Basic Sockets

[]

only send datagrams, because a datagram source address is always a unicast address (see Section 4.3).

remoteAddr remotePort

Remote address Remote port

void disconnect() Removes the remote address and port specification of the socket (see connect()). void receive(DatagramPacket packet) Places data from the next received message into the given DatagramPacket.

packet

Receptacle for received information, including source address and port as well as message data. (See the DatagramPacket reference for details of semantics.)

void send(DatagramPacket packet) Sends a datagram from this socket.

packet

Specifies the data to send and the destination address and port. If packet does not specify a destination address, the DatagramSocket must be "connected" to a remote address and port (see connect ()).

Accessors/Mutatots InetAddress getlnetAddressO int getPort0 Returns the remote socket address/port. InetAddress getLocalAddress0 int getLocalPort0 Returns the local socket address/port. int getReceiveBufferSize0 int getSendBufferSize0 void setReceiveBufferSize(int size) void setSendBufferSize(int size) The DatagramSocket has limits on the maximum datagram size that can be sent/ received through this socket. The receive limit also determines the amount of message data that can be queued waiting to be returned via receive(). That is, when the amount of buffered data exceeds the limit, arriving packets are quietly discarded. Setting the size is only a hint to the underlying implementation. Also, the semantics of the limit may vary from system to system: it may be a hard limit on some and soft on others.

size

Desired limit on packet and/or queue size (bytes)

II

2.3 UDP Sockets

31

int getSoTimeout() void setSoTimeout(int timeout) Returns/sets the m a x i m u m amount of time that a receive () will block for this socket. If the specified time elapses before data is available, an InterruptedlOException is thrown.

timeout

2.3.3

The m a x i m u m amount of time (milliseconds) that r e c e i v e ( ) will block for the socket. A timeout of 0 indicates that a receive will block until data is available.

UDPServer

Like a TCP server, a UDP server's job is to set up a communication endpoint and passively wait for the client to initiate the communication; however, since UDP is connectionless, UDP communication is initiated by a datagram from the client, without going through a connection setup as in TCP. The typical UDP server goes through four steps: 1. Construct an instance of DatagramSocket, specifying the local port and, optionally, the local address. The server is now ready to receive datagrams from any client. 2. Receive an instance of DatagramPacket using the r e c e i v e ( ) m e t h o d of DatagramSocket. When receive() returns, the datagram contains the client's address so we know where to send the reply. 3. Communicate by sending and receiving DatagramPackets using the send() and receive() methods of DatagramSocket. 4. When finished, deallocate the socket using the close() m e t h o d of DatagramSocket. Our next program example, UDPEchoServer. java, implements the UDP version of the echo server. The server is very simple: it loops forever, receiving datagrams and then sending the same datagrams back to the client. Actually, our server only receives and sends back the first 255 (ECHOMAX) characters of the datagram; any excess is silently discarded by the socket implementation (see Section 2.3.4).

U D P Ec h oSe r y e r .ja v a

O import java.net.*; / / for 1 import java.io.*; / / for 2 3 public class UDPEchoServer 4 5 private s t a t i c final int

DatagramSocket, DatagramPacket, and InetAddress IOException { ECHO~X = 255;

/ / Maximum size of echo datagram

32 6 7

Chapter 2: Basic Sockets

I

public static void main(String[] args) throws lOF,xception {

8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26

if (args.length != i) // Test for correct argument list throw new lllegalArgumentException("Parameter(s) : ") ; int servPort = Integer.parselnt(args[O]); DatagramSocket socket = new DatagramSocket (servPort) ; DatagramPacket packet = new DatagramPacket(new byte[ECHOMAX], ECHOMAX); for (;;) { // Run forever, receiving and echoing datagrams socket.receive(packet); // Receive packet from client System.out.println("Handling client at " + packet.getAddress().getHostAddress() + " on port " + packet.getPort()); socket.send(packet); // Send the same packet back to client packet.setLength(ECHOMAX); // Reset length to avoid shrinking buffer } /* NOT REACHED */

U D P Ec h oSe r y e r .ja v a

1. Application setup and parameter parsing: lines 0-12 UDPEchoServer takes a single p a r a m e t e r , the local p o r t of the echo server socket. 2. Create and set up datagram socket: line 14 Unlike our UDP client, a UDP server m u s t explicitly set its local p o r t to a n u m b e r k n o w n by the client; otherwise, the client will not know the d e s t i n a t i o n p o r t for its echo r e q u e s t datagram. When the server receives the echo d a t a g r a m f r o m the client, it can find out the client's a d d r e s s and port f r o m the datagram. 3. Create datagram: line 15 UDP m e s s a g e s are c o n t a i n e d in datagrams. We c o n s t r u c t an instance of DatagramPacket with a buffer of ECHOMAX (255) bytes. This d a t a g r a m will be u s e d b o t h to receive the echo r e q u e s t and to s e n d the echo reply. 4. Iteratively handle incoming echo requests: lines 17-23 The UDP server uses a single socket for all c o m m u n i c a t i o n , unlike the TCP server, which creates a new socket with every successful accept ().

9 Receive an echo request datagram: lines 18-20 The r e c e i v e ( ) m e t h o d of DatagramSocket blocks until a d a t a g r a m is received f r o m a client (unless a t i m e o u t is set). There is no connection, so each d a t a g r a m m a y come

[]

2.3 UDP Sockets

~

from a different sender. The datagram itself contains the sender's (client's) source address and port. 9 Send echo reply: line 21 packet already contains the echo string and echo reply destination address and port, so the send() m e t h o d of DatagramSocket can simply transmit the d a t a g r a m previously received. Note that when we receive the datagram, we interpret the datagram address and port as the source address and port, and when we send a datagram, we interpret the datagram's address and port as the destination address and port. 9 Reset buffer size: line 22 The internal length of packet was set to the length of the message just processed, which may have been smaller than the original buffer size. If we do not reset the internal length before receiving again, the next message will be truncated if it is longer than the one just received.

2.3.4

Sending and Receiving with UDP Sockets

A subtle but important difference between TCP and UDP is that UDP preserves message boundaries. Each call to r e c e i v e ( ) returns data from at m o s t one call to send() Moreover, different calls to r e c e i v e ( ) will never return data from the same call to send(). When a call to w r i t e ( ) on a TCP socket's output stream returns, all the caller knows is that the data has been copied into a buffer for transmission; the data may or may not have actually been t r a n sm itted yet. (This is covered in more detail in Chapter 5.) UDP, however, does not provide recovery from network errors and, therefore, does not buffer data for possible retransmission. This means that by the time a call to send() returns, the message has been passed to the underlying channel for transmission and is (or soon will be) on its way out the door. Between the time a message arrives from the network and the time its data is returned via read() or r e c e i v e ( ) , the data is stored in a first-in, first-out (FIFO) queue of received data. With a connected TCP socket, all received-but-not-yet-delivered bytes are treated as one continuous sequence of bytes (see Chapter 5). For a UDP socket, however, the received data may have come from different senders. A UDP socket's received data is kept in a queue of messages, each with associated information identifying its source. A call to r e c e i v e ( ) will never return more than one message. However, if r e c e i v e ( ) is called with a DatagramPacket containing a buffer of size n, and the size of the first message in the receive queue exceeds n, only the first n bytes of the message are returned. The remaining bytes are quietly discarded, with no indication to the receiving p r o g r a m that information has been lost! For this reason, a receiver should always supply a DatagramPacket with a buffer big enough to hold the largest message allowed by its application protocol at the time it calls r e c e i v e ( ) . This technique will guarantee that no data will be lost. The m a x i m u m a m o u n t of data that can be transmitted in a DatagramPacket is 65,507 b y t e s - - t h e largest payload that can be carried in a UDP datagram. It is important to r e m e m b e r here that each instance of DatagramPacket has an internal notion of message length that may be changed whenever a message is received into

34

Chapter 2: Basic Sockets

[]

that instance (to reflect the number of bytes in the received message). Applications that call receive() more than once with the same instance of DatagramPacket should explicitly reset the internal length to the actual buffer length before each subsequent call to receive(). Another potential source of problems for beginners is the getData() method of DatagramPacket, which always returns the entire original buffer, ignoring the internal offset and length values. Receiving a message into the DatagramPacket only modifies those locations of the buffer into which message data was placed. For example, suppose buf is a byte array of size 20, which has been initialized so that each byte contains its index in the array: 0 ] 1 I 2 I 3 I 4 I 5101

~

I~I9

I10111112113114115116117118119

I

Suppose also that dg is a DatagramPacket, and that we set dg's buffer to be the middle 10 bytes of buf : dg. setData(buf, 5,10) ; Now suppose that dgsocket is a DatagramSocket, and that somebody sends an 8-byte message containing

I 41 I 42 143 I 44 I 45 I 46 I 4 7 1 4 8 I to dgsocket. The message is received into dg: dgsocket .receive(dg) ; Now, calling dg. getData() returns a reference to the original byte array buf, whose contents are now 0 I 11 2 I 3 I 4 I 411 421 431 441 451 461 471 481 131 141 151 161 171 181 19 I Note that only bytes 5-12 of buf have been modified and that, in general, the application needs to use get0ffset() and getData() to access just the received data. One possibility is to copy the received data into a separate byte array, like this: byte[] destBuf = new byte[dg.getLength()]; System.arraycopy(dg.getData(), dg.getOffset(), destSuf, O, destSuf.length);

2.4

Exercises

1. For TCPF.choServer. java, we explicitly specify the port to the socket in the constructor. We said that a socket must have a port for communication, yet we do not specify a port in TCPY.choClient. java. How is the echo client's socket assigned a port? 2. When you make a phone call, it is usually the callee that answers with "Hello." What changes to our client and server examples would be needed to implement this? 3. What happens if a TCP server never calls accept()? What happens if a TCP client sends data on a socket that has not yet been accept ()ed at the server?

[]

2.4 Exercises

~

4. Servers are supposed to run for a long time without stopping--therefore, they m u s t be designed to provide good service no matter what their clients do. Examine the server examples (TCPEchoServer. java and UDPEchoServer. java) and list anything you can think of that a client might do to cause it to give poor service to other clients. Suggest improvements to fix the problems that you find. 5. Modify TCPEchoServer. java to read and write only a single byte at a time, sleeping one second between each byte. Verify that TCPEchoClient.java requires multiple reads to successfully receive the entire echo string, even though it sent the echo string with one write(). 6. Modify TCPEchoServer. java to read and write a single byte and then close the socket. What happens when the TCPEchoClient sends a multibyte string to this server? What is happening? (Note that the response could vary by OS.) 7. Modify UDPEchoServer. java so that it only echoes every other datagram it receives. Verify that UDPEchoClientTimeout. java retransmits datagrams until it either receives a reply or exceeds the n u m b e r of retries. 8. Modify UDPEchoServer. java so that ECHOMAX is m u c h shorter (say, 5 bytes). Then use UDPEchoClientTimeout. java to send an echo string that is too long. What happens? 9. Verify experimentally the size of the largest message you can send and receive using a DatagramPacket. 10. While UDPEchoServer. java explicitly specifies its local port in the constructor, we do not specify the local port in UDPEchoClientTimeout. java. How is the UDP echo client's socket given a port number? Hint: The answer is different for TCP.

chapter

3

Sending and Receiving Messages

W h e n writing programs to communicate via sockets, you will generally be implementing an application protocol of some sort. Typically you use sockets because your program needs to provide information to, or use information provided by, another program. There is no magic: sender and receiver m u s t agree on how this information will be encoded, who sends what information when, and how the communication will be terminated. In our echo example, the application protocol is trivial: neither the client's nor the server's behavior is affected by the contents of the bytes they exchange. Because most applications require that the behaviors of client and server depend u p o n the information they exchange, application protocols are usually more complicated. The TCP/IP protocols transport bytes of user data without examining or modifying them. This allows applications great flexibility in how they encode their information for transmission. For various reasons, most application protocols are defined in terms of discrete messages made up of sequences of fields. Each field contains a specific piece of information encoded as a sequence of bits. The application protocol specifies exactly how these sequences of bits are to be formatted by the sender and interpreted, or parsed, by the receiver so that the latter can extract the meaning of each field. About the only constraint imposed by TCP/IP is that information m u s t be sent and received in chunks whose length in bits is a multiple of eight. So from now on we consider messages to be sequences of bytes. Given this, it may be helpful to think of a transmitted message as a sequence of numbers, each between 0 and 255, inclusive (that being the range of binary values that can be encoded in 8 bits--1 byte). As a concrete example for this chapter, let's consider the problem of transferring price quote information between vendors and buyers. A simple quote for some quantity of a particular item might include the following information: Item n u m b e r : A large integer identifying the item Item description: A text string describing the item Unit price: The cost per item in cents Quantity: The number of units offered at that price

37

38

Chapter 3" Sending and Receiving Messages

u

Discounted?: Whether the price includes a discount In stock?: Whether the i t e m is in stock We collect this i n f o r m a t i o n in a class ItemQuote. java. For convenience in viewing the information in our p r o g r a m examples, we include a t o S t r i n g ( ) m e t h o d . T h r o u g h o u t this chapter, the variable item refers to an instance of ItemQuote.

ItemQuote.java 0

6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34

public class ItemQuote { public public public public public public

long itemNumber; String itemDescription; int quantity; int unitPrice; boolean discounted; boolean inStock;

// // // // // //

Item identification number String description of item Number of items in quote (always >= i) Price (in cents) per item Price reflect a discount? Item(s) ready to ship?

public ItemQuote(long itemNumber, String itemDescription, int quantity, int unitPrice, boolean discounted, boolean inStock) this itemNumber = itemNumber; this itemDescription = itemDescription; this quantity = quantity; this unitPrice = unitPrice; this discounted = discounted; this inStock = inStock;

public String toString() { final String EOLN = java.lang.System.getProperty("line.separator"); String value = "Item#=" + itemNumber + EOLN + "Description=" + itemDescription + EOLN + "Quantity=" + quantity + BOLN + "Price(each)=" + unitPrice + BOLN + "Total=" + (quantity * unitPrice); if (discounted) value += " (discounted)"; if (inStock) value += BOLN + "In Stock" + EOLN; else value += EOLN + "Out of Stock" + EOLN; return value;

ItemQuote.java

[]

3.1

3.1 Encoding Information

39

Encoding Information

What if a client program needs to obtain quote information from a vendor program? The two programs m u s t agree on how the information contained in the ItemQuote will be represented as a sequence of bytes "on the wire"--sent over a TCP connection or carried in a UDP datagram. (Note that everything in this chapter also applies if the "wire" is a file that is written by one program and read by another.) In our example, the information to be represented consists of primitive types (integers, booleans) and a character string. Transmitting information via the network in Java requires that it be written to an OutputStream (of a Socket) or encapsulated in a DatagramPacket (which is then sent via a DatagramSocket). However, the only data types to which these operations can be applied are b y t e s and arrays of bytes. As a strongly typed language, Java requires that other types--String, int, and so on--be explicitly converted to these transmittable types. Fortunately, the language has a number of built-in facilities that make such conversions more convenient. Before dealing with the specifics of our example, however, we focus on some basic concepts of representing information as sequences of bytes for transmission.

3.1.1

Text

Old-fashioned text--strings of printable (displayable) characters--is perhaps the most comm o n form of information representation. When the information to be transmitted is natural language, text is the most natural representation. Text is convenient for other forms of information because h u m a n s can easily deal with it when printed or displayed; numbers, for example, can be represented as strings of decimal digits. To send text, the string of characters is translated into a sequence of bytes according to a character set. The canonical example of a character encoding system is the venerable American Standard Code for Information Interchange (ASCII), which defines a one-to-one mapping between a set of the most commonly used printable characters in English, and binary values. For example, in ASCII the digit 0 is represented by the byte value 48, 1 by 49, and so on up to 9, which is represented by the byte value 57. ASCII is adequate for applications that only need to exchange English text. As the economy becomes increasingly globalized, however, applications need to deal with other languages, including many that use characters for which ASCII has no encoding, and even some (e.g., Chinese) that use more than 256 characters and thus require more than I byte per character to encode. Encodings for the world's languages are defined by companies and by standards bodies. Unicode is the most widely recognized such character encoding; it is standardized by the International Organization for Standardization

(ISO). Fortunately, Java provides good support for internationalization, in several ways. First, Java uses Unicode to represent characters internally. Unicode defines a 16-bit (2-byte) code for each character and thus supports a m u c h larger set of characters than ASCII. In fact, the Unicode standard currently defines codes for over 49,000 characters and covers "the principal written languages and symbol systems of the world." [21] Second, Java supports various other standard encodings and provides a clean separation between its internal representation and the encoding used when characters are input or output.

~O

Chapter 3: Sending and Receiving Messages

n

The getBytes() methods of class String implement this internal-to-external conversion, returning the sequence of bytes that represent the given string in some external encoding-either the default encoding or an explicitly named one. (This type of conversion may also happen implicitly--for example, by writing a string to an instance of 0utputStreamWriter.) Similarly, String provides constructors that take a byte array and the name of a particular encoding, and return a String instance containing the sequence of characters represented by the byte sequence according to the encoding. (If no encoding is explicitly requested, the default encoding for the platform is used.) Suppose the value of item.itemNumber is 123456. Using ASCII, that part of the string representation of item produced by toString() would be encoded as

1105111611011109L 1611491501511 21 31 4 I 'i' 't'

'e' 'm'

'#' '='

'i'

'2'

'3'

'4'

'5'

'6'

Using the "ISO8859_1" encoding would produce the same sequence of byte values, because the International Standard 8859-1 encoding (which is also known as ISO Latin 1) is an extension of ASCII--it maps the characters of the ASCII set to the same values as ASCII. However, if we used the North American version of IBM's Extended Binary Coded Decimal Interchange Code (EBCDIC), known in Java as the encoding "Cp037," the result would be rather different:

11371163L1331148112311261241124212431244124 12461 'i'

't'

'e'

'm'

'#' '='

'1'

'2'

'3'

'4'

'5'

'6'

If we used Unicode, the result would use 2 bytes per character, with i byte containing zero and the other byte containing the same value as with ASCII. Obviously the primary requirement in dealing with character encodings is that the sender and receiver must agree on the code to be used.

3.1.2

Binary Numbers

Transmitting large numbers as text strings is not very efficient: each character in the digit string has one of only 10 values, which can be represented using, on average, less than 4 bits per digit. Yet the standard character codes invariably use at least 8 bits per character. Moreover, it is inconvenient to perform arithmetic computation and comparisons with numbers encoded as strings. For example, a receiving program computing the total cost of a quote (quantity times unit price) will generally need to convert both amounts to the local computer's native (binary) integer representation before the computation can be performed. For a more compact and computation-friendly encoding, we can transmit the values of the integers in our data as binary values. To send binary integers as byte sequences, the sender and receiver need to agree on several things: 9 Integer size: How many bits are used to represent the integer? The sizes of Java's integer types are fixed by the language definition--shorts are 2 bytes, ints are 4, longs are 8--so a Java sender and receiver only need to agree on the primitive type to be used. (Communicating with a non-Java application may be more complex.) The size of an integer

i

3.1 Encoding Information

~ |

type, along with the encoding (signed/unsigned, see below), determines the m a x i m u m and m i n i m u m values that can be represented using that type. 9 Byte order: Are the bytes of the binary representation written to the stream (or placed in the byte array) from left to right or right to left? If the most significant byte is transmitted first and the least significant byte is transmitted last, that's the so-called big-endian order. Little-endian is, of course, just the opposite. 9 Signed or unsigned: Signed integers are usually transmitted in two's-complement representation. For k-bit numbers, the two's-complement encoding of the negative integer - n , 1 < n < 2 k-l, is the binary value of 2k - n; and the non-negative integer p, 0 < p < 2k-1 - 1, is encoded simply by the k-bit binary value of p. Thus, given k bits, two's complement can represent values in the range - 2 k-1 through 2k-1 - 1, and the most significant bit (msb) tells whether the value is positive (msb = 0) or negative (msb = 1). On the other hand, a k-bit unsigned integer can encode values in the range 0 through 2k - 1 directly. Consider again the itemNumber. It is a long, so its binary representation is 64 bits (8 bytes). If its value is 12345654321 and the encoding is big-endian, the 8 bytes sent would be (with the byte on the left transmitted first): 0

0

0

2

223 219 188 49

If, on the other hand, the value was sent in little-endian order, the transmitted byte values would be:

If the sender uses big-endian when the receiver is expecting little-endian, the receiver will end up with an itemNumber of 3583981154337816576! Most network protocols specify big-endian byte order; in fact it is sometimes called network byte order. Note that the most significant bit of the 64-bit binary value of 12345654321 is O, so its signed (two's-complement) and unsigned representations are the same. More generally, the distinction between k-bit signed and unsigned values is irrelevant for values that lie in the range 0 through 2k-1 - 1. Unfortunately, protocols often use unsigned integers; Java's lack of unsigned integer types means that some care is required in dealing with such values, especially in decoding. (See program ItemQuoteDecoderBin. java in Section 3.4.2 for an example.) As with strings, Java provides mechanisms to turn primitive integer types into sequences of bytes and vice versa. In particular, streams that support the Data0utput interface have methods writeShort (), w r i t e I n t (), and writeLong(), which allow those types to be written out directly. These methods all write out the bytes of integer primitive types in big-endian byte order using two's-complement representation. Similarly, implementations of the DataInput interface have m e t h o d s r e a d I n t ( ) , readShort(), and so on. The next section describes some ways to compose instances of these classes.

2

Chapter 3: Sending and Receiving Messages DataOutputStream

writeDouble(3.14)

BufferedOutputStream

--'--I 3.14 (8 bytes)

~~

343 (4 bytes)

~

343

800 (2 bytes)

~

800 I

writelnt(343) < ,I

writeShort(800)

--~

DatalnputStream readDouble ()=readlnt () ~____1 I 3.14 (8 bytes) ~ = I

3.2

3.14

1

I ~~

3.14

I

~

343 i=

Z

soo

OutputStream

L

~

BufferedlnputStream

343 (4 bytes) [ readShort() = I 800 (2 bytes) = I F i g u r e 3.1 :

[]

InputStream

~ !

I

Stream composition.

Composing I / 0 Streams

Java's stream classes can be composed to provide powerful encoding and decoding facilities. For example, we can wrap the 0utputStream of a Socket instance in a Buffered0utputStream instance to improve performance by buffering bytes temporarily and flushing them to the underlying channel all at once. We can then wrap that instance in a Data0utputStream to send primitive data types. We would code this composition as follows:

Socket socket = new Socket(server, port) ; DataOutputStream out = new DataOutputStream( new BufferedOutputStream(socket.getOutputStream())) ; Figure 3.1 demonstrates this composition. Here, we write our primitive data values, one by one, to Data0utputStream, which writes the binary data to Buffered0utputStream, which buffers the data from the three writes and then writes once to the socket 0utputStream, which controls writing to the network. We create a parallel composition for the InputStream on the other endpoint to efficiently receive primitive data types. A complete description of the Java I/O API is beyond the scope of this text; however, Table 3.1 provides a list of some of the relevant Java I/O classes as a starting point for exploiting its capabilities.

3.3

Framing and Parsing

Converting data to wire format is of course only half the story; the original information must be recovered at the receiver from the transmitted sequence of bytes. Application protocols typically deal with discrete messages, which are viewed as collections of fields. Framing refers to the problem of enabling the receiver to locate the beginning and end of the message in

m

3.3 Framing and Parsing

4 3

I/O Class

Function

Buffered[Input/Output ]Stream Checked[Input/Output ]Stream Data[Input/Output]Stream Digest[Input/Output]Stream GZIP[Input/Output]Stream Object[Input/Output ]Stream PushbackInputStream PrintOutputStream ZIP[Input/Output]Stream

Performs buffering for I/O optimization. Maintains a checksum on data. Handles read/write for primitive data types. Maintains a digest on data. De/compresses a byte stream in GZIP format. Handles read/write objects and primitive data types. Allows a byte or bytes to be "unread." Prints string representation of data type. De/compresses a byte stream in ZIP format.

Table 3.1 : Java I/O Classes

the stream, and of the fields within the message. W h e t h e r i n f o r m a t i o n is e n c o d e d as text, as multibyte binary n u m b e r s , or as some c o m b i n a t i o n of the two, the application p r o t o c o l m u s t enable the receiver of a m e s s a g e to d e t e r m i n e w h e n it has received all of the m e s s a g e and to parse it into fields. If the fields in a m e s s a g e all have fixed sizes and the m e s s a g e is m a d e u p of a fixed n u m b e r of fields, t h e n the size of the m e s s a g e is k n o w n in advance and the receiver can simply read the expected n u m b e r of bytes into a b y t e [ ] buffer. This technique was u s e d in TCPEchoClient. java, where we knew the n u m b e r of bytes to expect f r o m the server. However, w h e n some field ( a n d / o r the whole message) can vary in length, as with the itemDescription in our example, we do not k n o w b e f o r e h a n d how m a n y bytes to read. Marking the end of the m e s s a g e is easy in the special case of the last m e s s a g e to be sent on a TCP connection: the s e n d e r simply closes the sending side of the c o n n e c t i o n (using shutdown0utput() or c l o s e ( ) ) after sending the message. After the receiver r e a d s the last byte of the message, it receives an e n d - o f - s t r e a m indication (i.e., r e a d ( ) r e t u r n s -1), and thus can tell that it has as m u c h of the m e s s a g e as there will ever be. The same principle applies to the last field in a m e s s a g e sent as a DatagramPacket. In all o t h e r cases, the m e s s a g e itself m u s t contain additional framing i n f o r m a t i o n enabling the receiver to p a r s e the field/message. This i n f o r m a t i o n typically takes one of the following forms:

9 Delimiter: The end of the variable-length field or m e s s a g e is indicated by a unique marker, an explicit byte s e q u e n c e that i m m e d i a t e l y follows, b u t does not occur in, the data.

9 Explicit length: The variable-length field or m e s s a g e is p r e c e d e d by a (fixed-size) length field that tells how m a n y bytes it contains. The delimiter-based a p p r o a c h is o f t e n u s e d with variable-length text: A particular character or s e q u e n c e of characters is defined to m a r k the end of the field. If the entire m e s s a g e consists of text, it is s t r a i g h t f o r w a r d to read in characters using an instance of Reader (which

44

Chapter 3: Sending and Receiving Messages

[]

handles the byte-to-character translation), looking for the delimiter sequence, and returning the character string preceding it. Unfortunately, the Reader classes do not support reading binary data. Moreover, the relationship between the number of bytes read from the underlying InputStream and the number of characters read from the Reader is unspecified, especially with multibyte encodings. When a message uses a combination of the two framing methods mentioned above, with some explicit-length-delimited fields and others using character markers, this can create problems. The class Framer, defined below, allows an InputStream to be parsed as a sequence of fields delimited by specific byte patterns. The static method Framer.nextToken() reads bytes from the given InputStream until it encounters the given sequence of bytes or the stream ends. All bytes read up to that point are then returned in a new byte array. If the end of the stream is encountered before any data is read, null is returned. The delimiter can be different for each call to nextToken(), and the method is completely independent of any encoding. A couple of words of caution are in order here. First, nextToken() is terribly inefficient; for real applications, a more efficient pattern-matching algorithm should be used. Second, when using Framer.nextToken() with text-based message formats, the caller must convert the delimiter from a character string to a byte array and the returned byte array to a character string. In this case the character encoding needs to distribute over concatenation, so that it doesn't matter whether a string is converted to bytes all at once, or a little bit at a time. To make this precise, let E() represent an encoding--that is, a function that maps character sequences to byte sequences. Let a and b be sequences of characters, so E(a) denotes the sequence of bytes that is the result of encoding a. Let "+" denote concatenation of sequences, so a + b is the sequence consisting of a followed by b. This explicit-conversion approach (as opposed to parsing the message as a character stream) should only be used with encodings that have the property that E(a + b)= E(a)+ E(b); otherwise, the results may be unexpected. Although most encodings supported in Java have this property, some do not. In particular, UnicodeBig and UnicodeLittle encode a String by first outputting a byte-order indicator (the 2-byte sequence 254-255 for big-endian, and 255-254 for little-endian), followed by the 16-bit Unicode value of each character in the String, in the indicated byte order. Thus, the encoding of "Big fox" using UnicodeBig is as follows:

12 412 510 166 I o 11051 0 11031 0 1 321 0 11021 0 Ii111 0 11201 [mark]

'B'

'i'

'g'

' '

'f'

'o'

'x'

while the encoding of "Big" concatenated with the encoding of "fox", using the same encoding, is as follows"

12 4J2 1 o 166 I o j l051 0 11o312 4125 1 0 132 I o 11o21 o i Xlll 0 112ol [mark ]

'B'

'i'

'g'

[mark ]

' '

'f'

'o'

'x'

Using either of these encodings to convert the delimiter results in a byte sequence that begins with the byte-order marker. Moreover, if the byte array returned by nextToken() does not begin with one of the markers, any attempt to convert it to a String using one of these encodings

[]

3.3 Framing and Parsing

4

will throw an exception. The encodings UnicodeBigUnmarked and UnicodeLittleUnmarked (supported in JDK as of 1.3) omit the byte-order marker, so they are suitable for use with Framer. nextToken().

Framerjava 0

import java.io.*;

// for InputStream and ByteArrayOutputStream

1

2 3 4 5 6 7

public class Framer { public static byte[] nextToken(InputStream in, byte[] delimiter) throws lOException { int nextByte ; // If the stream has already ended, return null if ((nextByte = in.read()) == -i) return null;

8

9 i0 Ii 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36

ByteArrayOutputStream tokenBuffer = new ByteArrayOutputStream(); do { tokenBuffer.write(nextByte); byte[] currentToken = tokenBuffer.toByteArray(); if (endsWith(currentToken, delimiter)) { int tokenLength = currentToken.length - delimiter.length; byte[] token = new byte[tokenLength]; System.arraycopy(currentToken, O, token, O, tokenLength); return token;

} } while ((nextByte = in.read()) != -i); return tokenBuffer.toByteArray();

// Stop on end-of-stream // Received at least 1 byte

}

// Returns true if value ends with the bytes in suffix private static boolean endsWith(byte[] value, byte[] suffix) { if (value.length < suffix.length) return false; for (int offset = i; offset <= suffix.length; offset++) if (value[value.length- offset] != suffix[suffix.lengthreturn false; return true;

offset])

} }

Framerjava

46

Chapter 3: Sending and Receiving Messages

!

1. nextToken(): lines 4-24 Read f r o m i n p u t s t r e a m until delimiter or end-of-stream. 9 T e s t for e n d - o f - s t r e a m : lines 8-10 If the i n p u t s t r e a m is already at end-of-stream, r e t u r n null. 9 Create a b u f f e r to h o l d t h e b y t e s of the token: line 12 We use a ByteArray0utputStream to collect the data byte by byte. The ByteArray[Input I Output ]Stream classes allow a byte array to be h a n d l e d like a s t r e a m of bytes. 9 Put the last b y t e r e a d into the b u f f e r 9 line 14 9 Get a b y t e a r r a y c o n t a i n i n g the i n p u t so far: line 15 It is very inefficient to create a new byte array on each iteration, b u t it is simple. 9 C h e c k w h e t h e r the d e l i m i t e r is a suffix of the c u r r e n t token: lines 16-21 If so, create a new byte array containing the bytes read so far, m i n u s the delimiter suffix, and r e t u r n it. 9 Get n e x t byte: line 22 9 R e t u r n t h e c u r r e n t t o k e n o n e n d - o f - s t r e a m : line 23 2. endswith(): lines 26-35 9 C o m p a r e lengths: lines 28-29 The candidate sequence m u s t be at least as long as the delimiter to be a match. 9 C o m p a r e b y t e s , r e t u r n f a l s e o n a n y difference: lines 31-33 C o m p a r e the last delim.length bytes of the t o k e n to the delimiter. 9

3.4

If n o difference, r e t u r n true: line 34

Implementing Wire Formats in Java

To e m p h a s i z e the fact that the same i n f o r m a t i o n can be r e p r e s e n t e d "on the wire" in different ways, we define an interface ItemQuoteEncoder, which has a single m e t h o d that takes an ItemQuote instance a n d converts it to a b y t e [ ] that can be written to an 0utputStream or e n c a p s u l a t e d in a DatagramPacket.

ItemQuoteEncoder.java 0 public interface ItemQuoteEncoder { 1 byte[] encode(ItemQuote item) throws Exception; 2 }

ItemQuoteEncoder.java The specification of the c o r r e s p o n d i n g decoding functionality is given by the ItemQuoteDecoder interface, which has m e t h o d s for parsing m e s s a g e s received via s t r e a m s or in Data-

I

3.4 Implementing Wire Formats in Java

4

gramPackets. Each m e t h o d performs the same function: extracting the information for one message and returning an ItemQuote instance containing the information.

ItemQuoteDecoder.java 0 import java.io.* ; // for InputStream and IOException 1 import java.net.*; // for DatagramPacket 2 3 public interface ItemQuoteDecoder { 4 ItemQuote decode(InputStream source) throws IOException; 5 ItemQuote decode(DatagramPacket packet) throws IOException; 6 }

ItemQuoteDecoder.java Sections 3.4.1 and 3.4.2 present two different implementations for these interfaces: one using a text representation, the other, a hybrid encoding.

3.4.1

Text-Oriented Representation

Clearly we can represent the ItemQuote information as text. One possibility is to simply transmit the output of the t o S t r i n g ( ) method using a suitable character encoding. To simplify parsing, the approach in this section uses a different representation, in which the values of itemNumber, itemDescription, and so on are transmitted as a sequence of delimited text fields. The sequence of fields is as follows: (Discount?) (In Stock?> The Item Number field (and the other integer-valued fields, Quantity and Price) contain a sequence of decimal digit characters followed by a space character (the delimiter). The Description field is just the description text. However, because the text itself may include the space character, we have to use a different delimiter; we choose the newline character, represented as \n in Java, as the delimiter for this field. Boolean values can be encoded in several different ways. One possibility is to include the string "true" or the string "false", according to the value of the variable. A more compact approach (and the one used here) is to encode both values (discounted and inStock) in a single field; the field contains the character '8' if discounted is true, indicating that the item is discounted, and the c h a r a c t e r ' s ' if inStock is true, indicating that the item is in stock. The absence of a character indicates that the corresponding boolean is false, so this field may be empty. Again, a different delimiter (\n) is used for this final field, to make it slightly easier to recognize the end of the message even when this field is empty. A quote for 23 units of item number 12345, which has the description "AAA Battery" and price $14.45, and which is both in stock and discounted, would be represented as

12345 AAA Battery\n23 1445 ds\n

48

Chapter 3" Sending and Receiving Messages

m

Constants needed by both the encoder and the decoder are defined in the ItemQuoteTextConst interface, which defines "ISO8859_1" as the default encoding (we could just as easily have used any other encoding as the default) and 1024 as the m a x i m u m length (in bytes) of an encoded message. Limiting the length of an encoded message limits the flexibility of the protocol, but it also provides for sanity checks by the receiver. I te m Q u o t e T e x t C o

n s t.ja va

O 1 2

public interface ItemOuoteTextConst { public static final String DEFAULT_ENCODING = "IS0_8859_I"; public static final int MAX_WIRE_LENGTH = 1024;

3

} Ite m Qu oteTextCo

n s t.ja va

ItemQuoteEncoderText implements the text encoding. ItemQuoteEncoderText.java

O 1 2

import java.io.*;

// for ByteArrayOutputStream and OutputStreamWriter

public class ItemQuoteEncoderText implements ItemOuoteEncoder, ItemOuoteTextConst { private String encoding;

// Character encoding

7

public ItemQuoteEncoderText() { encoding = DEFAULT_ENCODING;

8

}

9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

public ItemQuoteEncoderText(String encoding) { this.encoding = encoding;

} public byte[] encode(ItemQuote item) throws Exception { ByteArrayOutputStream buf = new ByteArrayOutputStream(); OutputStreamWriter out = new OutputStreamWriter(buf, encoding); out.write(item.itemNumber + ....); if (item.itemDescription.indexOf('\n') != -i) throw new lOException("Invalid description (contains newline)"); out.write(item.itemDescription + "\n" + item.quantity + .... + item.unitPrice + ....); if (item.discounted) out.write('d'); // Only include 'd' if discounted if (item.inStock) out.write('s'); // Only include 's' if in stock

1

26 27 28 29 30 31 32

3.4 Implementing Wire Formats in Java

49

out.write( '\n'); out. flush() ; if (buf.size() > MAX_WIRE_LENGTH) throw new lOException("Bncoded length too long"); return buf. toByteArray() ;

} }

ItemQuoteEncoderText.java 1. Constructors: lines 6-12 If no encoding is explicitly specified, we use the default encoding specified in the constant interface. 2. encode() method: lines 14-31 9

Create an o u t p u t buffer: lines 15-16 A ByteArray0utputStream collects the bytes to be returned. Wrapping it in an OutputWriter allows us to take advantage of the latter's methods for converting strings to bytes.

9 Write the first integer, followed b y a space delimiter: line 17 9 C h e c k for delimiter: lines 18-19 Make sure that the field delimiter is not contained in the field itself. If it is, throw an exception. 9 Output itemDescription and other integers: lines 20-21 9 Write the flag characters if the b o o l e a n s are true: lines 22-25

9 Write the delimiter for the flag field: line 26 9 Flush the o u t p u t stream: lines 27-29 Flush everything to the underlying stream, and call size() to check that the resulting byte sequence is not too long. The length restriction allows the receiver to know how big a buffer is needed to receive into a DatagramPacket. (For stream communication, this is not necessary, but it is still convenient.)

9 Return the b y t e array f r o m the o u t p u t stream: line 30 The decoding class ItemQuoteDecoderText simply inverts the encoding process.

ItemQuoteDecoderText.java 0 import java.io.*; // for InputStream, ByteArrayInputStream, and IOException 1 import java.net.*; // for DatagramPacket 2 3 public class ItemQuoteDecoderText implements ItemQuoteDecoder, ItemQuoteTextConst { 4 5 private String encoding; // Character encoding

50 6 7

Chapter 3: Sending and Receiving Messages

[]

public ItemOuoteDecoderText() { encoding = DEFAULT_ENCODING;

8

9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 3O 31 32 33 34 35 36

} public ItemOuoteDecoderText(String this.encoding = encoding;

encoding) {

} public ItemQuote decode(InputStream wire) throws lOException { String itemNo, description, quant, price, flags; byte[] space = " ".getBytes(encoding); byte[] newline = "\n".getBytes(encoding); itemNo = new String(Framer.nextToken(wire, space), encoding); description = new String(Framer.nextToken(wire, newline), encoding); quant = new String(Framer.nextToken(wire, space), encoding); price = new String(Framer.nextToken(wire, space), encoding); flags = new String(Framer.nextToken(wire, newline), encoding); return new ItemQuote(Long.parseLong(itemNo), description, Integer.parselnt(quant), Integer.parselnt(price), (flags.indexOf('d') != -i), (flags.indexOf('s') != -i));

public ItemQuote decode(DatagramPacket p) throws lOException { ByteArraylnputStream payload = new ByteArraylnputStream(p.getData(), p.getOffset(), p.getLength()); return decode(payload);

}

ItemQuoteDecoderText.java

1. V a r i a b l e s a n d c o n s t r u c t o r s : lines 5-13 9 Encoding: line 5 The encoding u s e d in the d e c o d e r m u s t be the same as in the encoder! 9 C o n s t r u c t o r s : lines 7-13 If no encoding is given at c o n s t r u c t i o n time, the default defined in TtemQuoteDecoderTextConst is used. 2. S t r e a m decode(): lines 15-29 9 C o n v e r t delimiters" lines 17-18 We get the e n c o d e d f o r m of the delimiters a h e a d of time, for efficiency.

I

3.4 Implementing Wire Formats in Java

5 |

9 Call the nextToken() m e t h o d for each field: lines 19-23

For each field, we call Framer. nextToken() with the appropriate delimiter and convert the result according to the specified encoding. 9 Construct ItemQuote: lines 24-28 Convert to native types using the wrapper conversion methods and test for the presence of the flag characters in the last field. 3. Packet decode(): lines 31-35 Extract the data, convert to a stream, and call the stream decode() method.

3.4.2

Combined Data Representation

Our next encoding represents the integers of the ItemQuote as fixed-size, binary numbers: itemNumber as 64 bits, and quantity and unitPrice as 32 bits. It encodes the boolean values as flag bits, which occupy the smallest possible space in an encoded message. Also, the variablelength string itemDescription is encoded in a field with an explicit length indication. The binary encoding and decoding share coding constants in the ItemQuoteBinConst interface. Ite m Q u o t e Bi nCon s t .ja va

O

public interface ItemQuoteBinConst { public static final String DEFAULT_ENCODING = "IS0_8859_i"; public static final int DISCOUNT_FLAG = 1 << 7; public static final int IN_STOCK_FLAG = 1 << 0; public static final int MAX_DESC_LEN = 255; public static final int MAX_WIRE_LENGTH = 1024;

ItemQuoteBinConst.java

ItemQuoteEncoderBin implements the binary encoding. Ite m Q u o t e E ncode r Bi n .java

0

import java.io.*;

// for ByteArrayOutputStream and DataOutputStream

I

2 3 4 5 6 7 8

9

public class ItemQuoteEncoderBin implements ItemQuoteEncoder, ItemQuoteBinConst { private String encoding;

// Character encoding

public ItemQuoteEncoderBin() { encoding = DEFAULT_ENCODING;

}

52 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 3O 31 32 33 34 35

Chapter 3: Sending and Receiving Messages

[]

public ItemQuoteEncoderBin(String encoding) { this.encoding = encoding;

} public byte[] encode(ItemQuote item) throws Exception { ByteArrayOutputStream buf = new ByteArrayOutputStream(); DataOutputStream out = new DataOutputStream(buf); out.writeLong(item.itemNumber); out.writeInt(item.quantity); out.writeInt(item.unitPrice); byte flags = O; i f (item.discounted) flags ]= DISCOUNT_FLAG; i f (item.inStock) flags I= IN_STOCK_FLAG;

out.writeByte(flags); byte[] encodedDesc = item.itemDescription.getBytes(encoding); if (encodedDesc.length > MAX_DESC_LEN) throw new IOException("Item Description exceeds encoded length limit"); out.writeByte(encodedDesc.length); out.write(encodedDesc); out.flush(); return buf.toByteArray();

Ite m Q u o t e E n c o d e r Bi n .ja v a

1. Constants, variables, and constructors: lines 4-12 2. encode(): lines 14-34 9 Set u p Output: lines 16-17 Again, a ByteArray0utputStream collects the bytes of the encoded message. Encapsulating the ByteArray0utputStream in a Data0utputStream allows use of its methods for writing binary integers. 9 Write integers" lines 18-20 The writeLong() method writes the long's 8 bytes to the stream in big-endian order. Similarly, w r i t e I n t () outputs 4 bytes. 9 Write booleans as flags: lines 21-26 Encode each boolean using a single bit in a flag byte. Initialize the flag byte to O, then set the appropriate bits to 1, if either discounted or inStock is true. (The bits are defined in the ItemQuoteBinConst interface to be the most and least significant bits of the byte, respectively.) Write the byte to the stream. 9 C o n v e r t description string to bytes: line 2 7 Although Data0utDutStream nrovides methods for writing Strings, it SUDDOrts only one

!

3.4 Implementing Wire Formats in Java

5

fixed encoding, namely, UTF-8. Because we want to support alternative encodings, we convert the string to bytes explicitly. 9 Check description length: lines 28-29 We are going to use an explicit length encoding for the string, with a single byte giving the length. The biggest value that byte can contain is 255 bytes, so the length of the encoded string must not exceed 255 bytes. If it does, we throw an exception. 9 Write e n c o d e d string: lines 30-31 Write the length of the encoded string, followed by the bytes in the buffer. 9 Flush output stream, return bytes: line 32 Ensure that all bytes are flushed from the Data0utputStream to the underlying byte buffer. ItemQuoteDecoderBin implements the corresponding decoder function. I t e m Q u o t e D e c o d e r Bi n .ja v a

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

import java.io.*; // for ByteArrayInputStream import java.net.*; // for DatagramPacket public class ItemQuoteDecoderBin implements ItemQuoteDecoder, ItemQuoteBinConst { private String encoding;

// Character encoding

public ItemQuoteDecoderBin() { encoding = DEFAULT_ENCODING;

} public ItemQuoteDecoderBin(String encoding) { this.encoding = encoding;

} public ItemQuote decode(InputStream wire) throws lOException { boolean discounted, inStock; DatalnputStream src = new DataInputStream(wire); long itemNumber = src.readLong(); int quantity = src.readInt(); int unitPrice = src.readlnt(); byte flags = src.readByte(); int stringLength = src.read(); // Returns an unsigned byte as an int if (stringLength == -i) throw new EOFException(); byte[] stringBuf = new byte[stringLength]; src.readFully(stringBuf); String itemDesc = new String(stringBuf,encoding); return new ItemQuote(itemNumber,itemDesc, quantity, unitPrice, ((flags & DISCOUNT_FLAG) == DISCOUNT_FLAG), ((flags & IN_STOCK_FLAG) == IN_STOCK_FLAG));

54 32 33 34 35 36 37 38

Chapter 3: Sending and Receiving Messages

1

public ItemQuote decode(DatagramPacket p) throws lOException { ByteArrayInputStream payload = new ByteArraylnputStream(p.getData(), p.getOffset(), p.getLength()); return decode(payload); }

ItemQuoteDecoderBin.java 1. Constants, variables, and constructors: lines 5-13

2. Stream decode: lines 15-31 9 Wrap the InputStream: line 17 Using the given InputStream, construct a DataInputStream so we can make use of the m e t h o d s readLong() and r e a d I n t ( ) for reading binary data types from the input.

9 Read integers: lines 18-20 Read the integers back in the same order they were written out. The readLong() m e t h o d reads 8 bytes and constructs a (signed) long using big-endian byte ordering. The r e a d I n t ( ) m e t h o d reads 4 bytes and does the same thing. Either will throw an EOFException if the stream ends before the requisite number of bytes is read. 9 Read flag byte: line 21 The flag byte is next; the values of the individual bits will be checked later. 9 Read string length: lines 22-24 The next byte contains the length of the encoded string. Note that we use the read() method, which returns the contents of the next byte read as an integer between 0 and 255 (or -1), and that we read it into an int. If we read it into a byte (which is signed), we would not be able to distinguish between the case where the length is 255 and the case where the stream ends p r e m a t u r e l y - - b o t h would return -1, since the signed interpretation of the 8-bit binary representation of 255 is -1. 9 Allocate buffer and read encoded string: lines 25-26 Once we know how long the encoded string is, we allocate a buffer and call readFully(), which does not return until enough bytes have been read to fill the given buffer. readFully() will throw an F,0FException if the stream ends before the buffer is filled. Note the advantage of the length-prefixed String representation: bytes do not have to be interpreted as characters until you have them all.

9 Check flags: lines 29-30 The expressions used as parameters in the call to the constructor illustrate the standard m e t h o d of checking whether a particular bit is set (equal to 1) in an integer type. 3. Packet decode:dines 33-37 Simply wrap the packer's data in a ByteArrayInputStream and pass to the stream-decoding method.

[]

3.4.3

3.4 Implementing Wire Formats in Java

5

Sendingand Receiving

The encodings p r e s e n t e d above can be u s e d with b o t h Sockets a n d DatagramSockets. We show the TCP u s a g e first.

SendTCP.java 0 1 2 3 4 5 6 7

import j a v a . i o . * ; import j a v a . n e t . * ;

public class SendTCP { public s t a t i c void main(String args[]) throws Exception { i f (args.length != 2) / / Test for correct # of args throw new IllegalArgumentException("Parameter(s): ");

8

9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34

/ / for Input/0utputStream / / for Socket

InetAddress destAddr = InetAddress. getByName(args[O]) ; // Destination address int destPort = Integer.parselnt(args[l]); // Destination port

Socket sock = new Socket(destAddr, destPort) ; ItemQuote quote = new ItemQuote(1234567890987654L, "5mmSuper Widgets", 1000, 12999, true, false); // Send text-encoded quote ItemQuoteEncoder coder = new ItemQuoteEncoderText(); byte[] codedQuote = coder.encode(quote); System.out.println("Sending Text-Encoded Quote (" + codedQuote.length + " bytes): "); System.out.println(quote);

sock.getOutputStream().write(codedQuote); // Receive binary-encoded quote ItemQuoteDecoder decoder = new ItemQuoteDecoderBin(); ItemQuote receivedQuote = decoder.decode(sock.getlnputStream()); System. out. println("Received Binary-Encode Quote :") ; System. out. println(receivedQuote) ; sock.close();

}

SendTCP.java

56

Chapter 3: Sending and Receiving Messages

1

1. Socket setup: line 13 2. Send using text encoding: lines 18-24 3. Receive using binary encoding: lines 26-30

RecvTCP.java 0 1 2 3 4 5 6 7 8 9 i0 ii 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

import java.io.*; import java.net.*;

// for Input/OutputStream // for Socket and ServerSocket

public class RecvTCP { public static void main(String args[]) throws Exception { if (args.length != i) // Test for correct # of args throw new IllegalArgumentException("Parameter(s): "); int port = Integer.parseInt(args[O]);

// Receiving Port

ServerSocket servSock = new ServerSocket(port) ; Socket clntSock = servSock, accept () ; // Receive text-encoded quote ItemQuoteDecoder decoder = new ItemQuoteDecoderText(); ItemQuote quote = decoder.decode(clntSock.getInputStream()) ; System.out.println("Received Text-Encoded Quote:"); System.out.println(quote);

/ / Repeat quote with binary encoding, adding 10 cents to the price ItemQuoteEncoder encoder = new ItemQuoteEncoderBin(); quote.unitPrice +: 10; / / Add 10 cents to unit price System.out.println("Sending ( b i n a r y ) . . . "); clntSock, getOutputStream(), write (encoder. encode (quote)) ; clntSock, close() ; servSock, close() ; } }

RecvTCP.java 1. Socket setup: line 12

2. Accept client connection: line 13

[]

3.4 Implementing Wire Formats in Java

5

3. Receive and print out a t e x t - e n c o d e d m e s s a g e : lines 15-19 4. Send a b i n a r y - e n c o d e d m e s s a g e : lines 21-25 Note that before sending, we add 10 cents to the unit price given in the original m e s s a g e . To d e m o n s t r a t e the use of the encoding and decoding classes with d a t a g r a m s , we include a simple UDP s e n d e r and receiver. Since this is very similar to the TCP code, we do not include any code description.

SendUDP.java 0 1 2 3 4 5 6 7 8 9 I0 ii 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

import java.net.*; import java.io.*;

/ / for DatagramSocket, DatagramPacket, and InetAddress / / for IOF.xception

public class SendUDP { public static void main(String args[]) throws Exception { if (args.length != 2 && args.length != 3) // Test for correct # of args throw new lllegalArgumentException("Parameter(s)' " + " [<encoding]") ; InetAddress destAddr = InetAddress. getByName (args [0 ] ) ; // Destination address // Destination port int destPort = Integer. parseInt (args [i] ) ;

ItemQuote quote = new ItemQuote(1234567890987654L, "5mm Super Widgets", 1000, 12999, true, f a l s e ) ; DatagramSocket sock = new DatagramSocket(); // UDP socket for sending

ItemQuoteEncoder encoder = ( a r g s . l e n g t h == 3 ? new ItemQuoteEncoderText(args[2]) ' new ItemQuoteEncoderText()); byte[] codedQuote = encoder.encode(quote);

DatagramPacket message = new DatagramPacket(codedQuote, codedQuote.length, destAddr, destPort) ; sock. send(message) ; sock. close() ; } } i

SendUDP.java

58

Chapter 3: Sending and Receiving Messages

[]

RecvUDP.java O import java.net.*; / / for DatagramSocket and DatagramPacket 1 import java.io.*; / / for 10Exception 2 3 public class RecvUDP implements ItemQuoteTextConst { 4 5 public static void main(String[] args) throws Exception { 6 7 i f (args.length != 1 && args.length != 2) / / Test for correct # of args 8 throw new lllegalArgumentException("Parameter(s): [<encoding>]"); 9 10 int port = Integer.parselnt(args[O]); // Receiving Port 11 12 DatagramSocket sock = new DatagramSocket(port); / / UDP socket for receiving 13 ItemQuoteDecoder decoder = (args.length == 2 ? / / Which encoding 14 new ItemquoteDecoderText(args[1]) : 15 new ItemQuoteDecoderText() ); 16 17 DatagramPacket packet = new DatagramPacket( 18 new byte [MAX_WIRE_LENGTH], MAX_WIRE_LENGTH) ; 19 sock.receive(packet) ; 20 21 ItemQuote quote = decoder.decode(packet); 22 System.out.println(quote); 23 24 sock.close(); 25 26

RecvUDP.java

3.5 Wrapping Up We have seen how Java data types can be encoded in different ways, and how messages can be constructed from various types of information. You may be aware that recent versions of Java include serialization capabilities--the S e r i a l i z a b l e and Externalizable interfaces--which provide for instances of supporting Java classes to be converted to and from byte sequences very easily. It might seem that having these interfaces available would eliminate the need for what has been described above, and that is to some extent true. However, it is not always the case, for a couple of reasons. First, the encoded forms produced by S e r i a l i z a b l e may not be very efficient. They may include information that is meaningless outside the context of the Java Virtual Machine (JVM), and may also incur overhead to provide flexibility that may not be needed. Second, S e r i a l i z a b l e and Externalizable cannot be used when a different wire format has already been specified--

m 3.6 Exercises

59

for example, by a standardized protocol. And finally, custom-designed classes have to provide their own implementations of the serialization interfaces anyway. A basic tenet of good protocol design is that the protocol should constrain the implementor as little as possible and should minimize assumptions about the platform on which the protocol will be implemented. We therefore avoid the use of S e r i a l i z a b l e and Externalizable in this book, and instead use more direct encoding and decoding methods.

3.6

Exercises

1. What happens if the Encoder uses a different encoding than the Decoder? 2. Positive integers larger than 231- 1 cannot be represented as ints in Java, yet they can be represented as 32-bit binary numbers. Write a m e t h o d to write such an integer to a stream. It should take a long and an O u t p u t S t r e a m as parameters. 3. Rewrite the binary encoder so that the Item Description is terminated by " \ r \ n " instead of being length encoded. Use Send/RecvTCP to test this new encoding. 4. The nextToken() m e t h o d of DelimitedInputStream assumes that either the delimiter or an end-of-stream (EoS) terminates a token; however, finding the EoS may be an error in some protocols. Rewrite nextToken() to include a second, boolean parameter. If the parameter value is true, then the EoS terminates a token without error; otherwise, the EoS generates an error.

chapter4

Beyond the Basics The

client and server examples in Chapter 2 demonstrate the basic model for programming with sockets in Java. The next step is to apply these concepts in various programming models, such as multitasking, nonblocking I/O, and broadcasting.

4.1 Multitasking Our basic TCP echo server from Chapter 2 handles one client at a time. If a client connects while another is already being serviced, the server will not echo the new client's data until it has finished with the current client, although the new client will be able to send data as soon as it connects. This type of server is known as an iterative server. Iterative servers handle clients sequentially, finishing with one client before servicing the next. They work best for applications where each client requires a small, b o u n d e d a m o u n t of server connection time; however, if the time to handle a client can be long, the wait experienced by subsequent clients may be unacceptable. To d e m o n st r a t e the problem, add a 10-second sleep using Thread. sleep( ) after the Socket constructor call in TCPEchoClient. java and experiment with several clients simultaneously accessing the TCP echo server. Here the sleep operation simulates an operation that takes significant time, such as slow file or network I/O. Note that a new client m u s t wait for all already-connected clients to complete before it gets service. What we need is some way for each connection to proceed independently, without interfering with other connections. Java threads provide exactly that: a convenient m e c h a n i s m allowing servers to handle many clients simultaneously. Using threads, a single application can work on several tasks concurrently. In our echo server, we can give responsibility for each client to an independently executing thread. All of the examples we have seen so far consist of a single thread, which simply executes the main() method. In this section we describe two approaches to coding c o n c u r r e n t servers, namely, threadper-client, where a new thread is spawned to handle each client connection, and t h r e a d pool, where a fixed, p r e s p a w n e d set of threads work together to handle client connections.

61

62

Chapter 4: Beyond the Basics

4.1.1

[]

Java Threads

Java provides two approaches for performing a task in a new thread: 1) defimng a subclass of the Thread class with a run() m e t h o d that performs the task, and instantiating it; or 2) defining a class that implements the Runnable interface with a run() m e t h o d that performs the task, and passing an instance of that class to the Thread constructor. In either case, the new thread does not begin execution until its s t a r t ( ) m e t h o d is invoked. The first approach can only be used for classes that do not already extend some other class; therefore, we focus on the second approach, which is always applicable. The Runnable interface contains a single m e t h o d prototype:

public void run(); When the s t a r t () m e t h o d of an instance of Thread is invoked, the JVM causes the instance's run() m e t h o d to be executed in a new thread, concurrently with all others. Meanwhile, the original thread returns from its call to s t a r t () and continues its execution independently. (Note that directly calling the run() m e t h o d of a Thread or Runnable instance has the normal procedure-call semantics: the m e t h o d is executed in the caller's thread.) The exact interleaving of thread execution is determined by several factors, including the implementation of the JVM, the load, the underlying OS, and the host configuration. For example, on a uniprocessor system, threads share the processor sequentially; on a multiprocessor system, multiple threads from the same application can run simultaneously on different processors. In the following example, ThreadExample. java implements the Runnable interface with a run() m e t h o d that repeatedly prints a greeting to the system output stream.

ThreadExample.java 0 public class ThreadExample implements Runnable { 1 2 private String greeting; / / Message to print to console 3 4 public ThreadExample(String greeting) { 5 this.greeting = greeting; } 6 7 public void run() { 8 for (;;) { 9 10 System.out.println(Thread.currentThread().getName() + ": " + greeting); 11 try { 12 Thread.sleep((long) (Math.random() * i00)); // Sleep 0 to i00 milliseconds 13 } catch (InterruptedException e) {} // Will not happen } 14 15 16 17 public static void main(String[] args) { new Thread (new ThreadExample ("Hello")). start () ; 18

n

19 20 21 22

4.1 Multitasking

63

new Thread(new ThreadExample("Aloha")).start(); new Thread(new ThreadExample("Ciao")).start();

} } Th read Exam pie.java

1. Declaration of i m p l e m e n t a t i o n of the Runnable interface: line 0 Since ThreadExample implements the Runnable interface, it can be passed to the constructor of Thread. If ThreadExample fails to provide a run() method, the compiler will complain. 2. Member variables a n d constructor: lines 2-6 Each instance of ThreadExample contains its own greeting string. 3. run ( ): lines 8-15 Loop forever performing: 9 Print the t h r e a d n a m e a n d instance greeting: line 10 The static m e t h o d Thread.currentThread() returns a reference to the thread from which it is called, and getName() returns a string containing the name of that thread. 9 Suspend thread: lines 11-13 After printing its instance's greeting message, each thread sleeps for a r a n d o m amount of time (between 0 and 100 milliseconds) by calling the static m e t h o d Thread. sleep(), which takes the number of milliseconds to sleep as a parameter. Math. random() returns a r a n d o m double between 0.0 and 1.0. Thread. sleep() can be interrupted by another thread, in which case an InterruptedException is thrown. Our example does not include an interrupt call, so the exception will not happen in this application. 4. main(): lines 17-21 Each of the three statements in main() does the following: 1) creates a new instance of ThreadExample with a different greeting string, 2) passes this new instance to the constructor of Thread, and 3) calls the new Thread instance's s t a r t () method. Each thread independently executes the run() m e t h o d of ThreadExample, while the main() thread terminates. Note that the JVM does not terminate until all n o n d a e m o n (see Threads API) threads terminate. Upon execution, an interleaving of the three greeting messages is printed to the console. The exact interleaving of the numbers depends u p o n the factors mentioned earlier.

4.1.2

Server Protocol

Since the two server approaches we are going to describe (thread-per-client and thread pool) are independent of the particular client-server protocol, we want to be able to use the same protocol code for both. The code for the echo protocol is given in the class EchoProtocol, which encapsulates the implementation of the server side of the echo protocol. The idea is that the server creates a separate instance of EchoProtocol for each connection, and protocol

64

Chapter 4: Beyond the Basics

m

e x e c u t i o n begins w h e n run() is called on an instance. The code in run() is a l m o s t identical to the c o n n e c t i o n h a n d l i n g code in TCPEchoServer. java, except that a logging capability (described shortly) has b e e n added. The class i m p l e m e n t s the Runnable interface, so we can create a t h r e a d t h a t i n d e p e n d e n t l y executes run(), or we can invoke run() directly.

EchoProtocol.java 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 3O 31 32 33 34 35 36 37 38 39

import java.net. *; import java. io. * ; import java.util. *;

// for Socket // for lOException and Input/OutputStream // for ArrayList

class EchoProtocol implements Runnable { static public final int BUFSIZE = 32; private Socket clntSock; private Logger logger;

// Size (in bytes) of I/O buffer

// Connection socket // Logging facility

public EchoProtocol(Socket clntSock, Logger logger) { this.clntSock = clntSock; this.logger = logger;

} public void run() { ArrayList entry = new ArrayList(); entry.add("Client address and port = " + clntSock.getlnetAddress().getHostAddress() + ":" + clntSock.getPort()); entry.add("Thread = " + Thread.currentThread().getName());

try { // Get the input and output I/O streams from socket InputStream in = clntSock, getInputStream() ; OutputStream out = clntSock.getOutputStream() ; int recvUsgSize; // int totalBytesEchoed = 0; // byte[] echoBuffer = new byte[BUFSIZE]; // // Receive until client closes connection, while ((recvUsgSize = in.read(echoBuffer)) out.write(echoBuffer, O, recvMsgSize); totalBytesEchoed += recvUsgSize;

Size of received message Bytes received from client Receive Buffer indicated by -i != -i) {

} entry.add("Client finished; echoed " + totalBytesEchoed + " bytes."); } catch (IOException e) { entry.add("Exception = " + e.getMessage());

}

m 4.1 Multitasking

40 41 42 43 44 45 46 47 48 49

try { // Close socket clntSock.close(); } catch (IOException e) { entry.add("Exception = " + }

65

e.getMessage());

logger, writeEntry(entry) ;

} }

EchoProtocol.java

1. Declaration of i m p l e m e n t a t i o n of the Runnable interface: line 4 2. Member variables and constructor: lines 7-13 Each instance of EchoProtocol contains a socket for the connection and a reference to the logger. 3. run(): lines 15-48 Implement the echo protocol: 9 Write the client and t h r e a d i n f o r m a t i o n to a buffer: lines 16-20 ArrayList is a dynamically sized container of Objects. The add() m e t h o d of ArrayList inserts the specified object at the end of the list. In this case, the inserted object is a String. Each element of the ArrayList represents a line of output to the logger. 9 Execute the echo protocol: lines 22-45 9

Write the elements (one per line) of the ArrayList instance to the logger: line 47

The logger allows for synchronized reporting of thread creation and client completion, so that entries from different threads are not interleaved. This facility is defined by the Logger interface, which has methods for writing strings or object collections.

Logger.java 0

import java.util.*;

// for Collection

1

2 3 4 5

public interface Logger { public void writeEntry(Collection entry); // Write list of lines public void writeEntry(String entry); // Write single line }

Logger.java

66

Chapter 4: Beyond the Basics

1

w r i t e E n t r y ( ) logs the given string or object collection. How it is logged d e p e n d s on the implementation. One possibility is to send the log m e s s a g e s to the console. Con s o l e L o g g e r ja va

0

import java.util.*;

// for Collection and Iterator

1

2 3 4 5 6 7

class ConsoleLogger implements Logger { public synchronized void writeEntry(Collection entry) { for (Iterator line = entry.iterator(); line.hasNext();) System.out.println(line.next()); System.out.println();

}

8

9 10 11 12 13

public synchronized void writeEntry(String entry) { System. out. println(entry) ; System. out. println() ;

}

Con sole Log ge r .ja va

Another possibility is to write the log m e s s a g e s to a file specified in the constructor, as in the following: FileLogger.java

0 1 2 3 4 5 6 7

8 9 10 ii 12 13 14 15 16

import java.io.*; import java.util.*;

// for PrintWriter and FileWriter // for Collection and Iterator

class FileLogger implements Logger { PrintWriter out;

// Log file

public FileLogger(String filename) throws lOException { out = new PrintWriter(new FileWriter(filename), true) ; // Create log file

} public synchronized void writeEntry(Collection entry) { for (Iterator line = entry.iterator() ; line.hasNext();) out. println(line, next ()) ; out. println() ;

}

[]

17 18 19 20 21

4.1 Multitasking

67

public synchronized void writeEntry(String entry) { out. println(entry) ; out. println() ;

}

FileLogger.java

We are now r e a d y to i n t r o d u c e s o m e different a p p r o a c h e s to c o n c u r r e n t servers.

4.1.3

Thread-per-Client

In a thread-per-client server, a new t h r e a d is c r e a t e d to handle each connection. The server executes a loop that r u n s forever, listening for connections on a specified p o r t and r e p e a t e d l y accepting an incoming c o n n e c t i o n f r o m a client and t h e n spawning a new t h r e a d to handle that connection. TCPEchoServerThread. j ava i m p l e m e n t s the thread-per-client architecture. It is very similar to the iterative server, using a single indefinite loop to receive and p r o c e s s client requests. The m a i n difference is that it creates a t h r e a d to handle the c o n n e c t i o n i n s t e a d of handling it directly. (This is possible b e c a u s e EchoProtoeol i m p l e m e n t s the Runnable interface.) TCP Ec hoSe rye rTh read .ja va

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

import java.net.*; import java.io.*;

// for Socket, ServerSocket, and InetAddress // for IOException and Input/0utputStream

public class TCPEchoServerThread { public s t a t i c void main(String[] args) throws 10Exception { if (args.length != I) // Test for correct # of args throw new IllegalArgumentException("Parameter(s) : ") ; int echoServPort = Integer.parselnt(args[O]);

// Server port

// Create a server socket to accept client connection requests ServerSocket servSock = new ServerSocket (echoServPort) ; Logger logger = new ConsoleLogger() ;

// Log messages to console

// Run forever, accepting and spawning threads to service each connection for (; ;) { try { Socket clntSock = servSock.accept(); // Block waiting for connection

68 21 22 23 24 25 26 27 28 29 30 31

Chapter 4: Beyond the Basics

[]

EchoProtocol protocol = new EchoProtocol(clntSock, logger); Thread thread = new Thread(protocol); thread.start(); logger.writeEntry("Created and started Thread = " + thread.getName()); } catch (lOException e) { logger.writeEntry("Exception = " + e.getMessage());

} } /* NOT REACHED */

} } TCP EchoSe r v e r T h r e a d .ja v a

1. P a r a m e t e r p a r s i n g and s e r v e r s o c k e t / l o g g e r creation: lines 7-15 2. Loop f o r e v e r , h a n d l i n g i n c o m i n g c o n n e c t i o n s : lines 17-28 9 A c c e p t a n i n c o m i n g connection" line 20 9 Create a p r o t o c o l i n s t a n c e to handle n e w connection: line 21 Each c o n n e c t i o n gets its own instance of EchoProtocol. Each instance m a i n t a i n s the state of its particular connection. The echo p r o t o c o l has little internal state, b u t m o r e s o p h i s t i c a t e d protocols m a y require substantial a m o u n t s of state. 9 Create, start, a n d log a n e w thread for the c o n n e c t i o n : lines 22-24 Since EchoProtocol i m p l e m e n t s the Runnable interface, we can give our new instance to the Thread constructor, and the new t h r e a d will execute the run() m e t h o d of EchoProtocol w h e n s t a r t ( ) is invoked. The getName() m e t h o d of Thread r e t u r n s a S t r i n g containing a n a m e for the new thread. 9 H a n d l e e x c e p t i o n f r o m accept(): lines 25-27 If some I/O error occurs, accept () throws an IOException. In our earlier iterative echo server (TCPEchoServer. java), the exception is not handled, and such an error t e r m i n a t e s the server. Here we handle the exception by logging the error and continuing execution.

4.1.4

Factoring the Server

Our t h r e a d e d server does what we want it to, but the code is not very reusable or extensible. First, the echo p r o t o c o l is h a r d - c o d e d in the server. What if we want an HTTP server instead? We could write an HTTPProtocol and replace the i n s t a n t i a t i o n of EchoProtocol in main(); however, we w o u l d have to revise main() and have a separate m a i n class for each different p r o t o c o l that we i m p l e m e n t . We want to be able to instantiate a p r o t o c o l instance of the a p p r o p r i a t e type for each c o n n e c t i o n w i t h o u t knowing any specifics about the protocol, including the n a m e of a constructor. This p r o b l e m - - i n s t a n t i a t i n g an object w i t h o u t knowing details about its t y p e - - a r i s e s f r e q u e n t l y in object-oriented p r o g r a m m i n g , and there is a s t a n d a r d solution: use a factory. A

II

4.1 Multitasking

69

factory object supplies instances of a particular class, hiding the details of how the instance is created, such as what constructor is used. For our protocol factory, we define the ProtocolFactory interface to have a single method, createProtocol(), which takes Socket and Logger instances as arguments and returns an instance implementing the desired protocol. Our protocols will all implement the Runnable interface, so that once we have an instance we can simply call run() (or s t a r t ( ) on a Thread constructed from the protocol instance) to execute the protocol for that connection. Thus, our protocol factory returns instances that implement the Runnable interface:

ProtocolFactory.java 0

import java.net.*;

// for Socket

1

2 public interface ProtocolFactory { 3 public Runnable createProtocol(Socket clntSock, Logger logger); 4 }

Protocol Factory.java We now need to implement a protocol factory for the echo protocol. The factory class is simple. All it does is return a new instance of EchoProtocol whenever createProtocol () is called.

EchoProtocolFactory.java 0

import java.net.*;

// for Socket

1

2 public class EchoProtocolFactory implements ProtocolFactory { 3 4 5 6

public Runnable createProtocol(Socket clntSock, Logger logger) { return new EchoProtocol(clntSock, logger); } }

EchoProtocolFactory.java We have factored out some of the details of protocol instance creation from our server, so that the various iterative and concurrent servers can reuse the protocol code. However, the server approach (iterative, thread-per-client, etc.) is still hard-coded in the main(). These server approaches deal with how to dispatch each connection to the appropriate handling mechanism. To provide greater extensibility, we want to factor out the dispatching model from the main() of TCPEchoServerThread. java so that we can use any dispatching model with any protocol. Since we have many potential dispatching models, we define the Dispatcher interface to hide the particulars of the threading strategy from the rest of the server code. It contains a

70

Chapter 4: Beyond the Basics []

single method, startDispatching(), which tells the dispatcher to start handling clients accepted via the given ServerSocket, creating protocol instances using the given ProtocolFactory, and logging via the given Logger. Dispatcher.java

0 1 2 3 4

import java.net.*;

// for ServerSocket

public interface Dispatcher { public void startDispatching(ServerSocket servSock, Logger logger, ProtocolFactory protoFactory) ;

5

} Dispatcher.java

To implement the thread-per-client dispatcher, we simply pull the for loop from main() in TCPEchoServerThread. java into the startDispatching() method of the new dispatcher. The only other change we need to make is to use the protocol factory instead of instantiating a particular protocol. Th read PerDi s patc he r .ja va

0 1 2 3 4 5 6 7

import java.net.*; import java.io.*;

class ThreadPerDispatcher implements Dispatcher { public void startDispatching(ServerSocket servSock, Logger logger, ProtocolFactory protoFactory) { // Run forever, accepting and spawning threads to service each connection for (;;) { try { Socket clntSock = servSock.accept(); // Block waiting for connection Runnable protocol = protoFactory.createProtocol(clntSock, logger); Thread thread = new Thread(protocol); thread.start(); logger.writeEntry("Created and started Thread = " + thread.getName()); } catch (IOException e) { logger.writeEntry("Exception = " + e.getMessage());

8

9 10 11 12 13 14 15 16 17 18 19 20 21

// for Socket and ServerSocket // for IOException

} } /* NOT REACHED */

} } ThreadPerDispatcher.java

[]

4.1 Multitasking

71

We demonstrate the use of this dispatcher and protocol factory in ThreadMain. java, which we introduce after discussing the thread-pool approach to dispatching.

4.1.5

Thread Pool

Every new thread consumes system resources: spawning a thread takes CPU cycles and each thread has its own data structures (e.g., stacks) that consume system memory. In addition, the scheduling and context switching among threads creates extra work. As the number of threads increases, more and more system resources are consumed by thread overhead. Eventually, the system is spending more time dealing with thread management than with servicing connections. At that point, adding an additional thread may actually increase client service time. We can avoid this problem by limiting the total number of threads and reusing threads. Instead of spawning a new thread for each connection, the server creates a thread pool on startup by spawning a fixed number of threads. When a new client connection arrives at the server, it is assigned to a thread from the pool. When the thread finishes with the client, it returns to the pool, ready to handle another request. Connection requests that arrive when all threads in the pool are busy are queued to be serviced by the next available thread. Like the thread-per-client server, a thread-pool server begins by creating a ServerSocket. Then it spawns N threads, each of which loops forever, accepting connections from the (shared) ServerSocket instance. When multiple threads simultaneously call accept() on the same ServerSocket instance, they all block until a connection is established. Then the system selects one thread, and the Socket instance for the new connection is returned only in that thread. The other threads remain blocked until the next connection is established and another lucky winner is chosen. Since each thread in the pool loops forever, processing connections one by one, a threadpool server is really a set of iterative servers. Unlike the thread-per-client server, a thread-pool thread does not terminate when it finishes with a client. Instead, it starts over again, blocking on accept (). A thread pool is simply a different model for dispatching connection requests, so all we really need to do is write another dispatcher. PoolDispatcher. java implements our thread-pool dispatcher. To see h o w the thread-pool server would be implemented without dispatchers and protocol factories, see TCPEchoServerPool. java on the book's W e b site.

Pool Di s patc he r .ja v a

import java.net. *; import java. io. * ;

// for Socket and ServerSocket // for lOException

class PoolDispatcher implements Dispatcher { static final String NUMTHREADS = "8"; static final String THREADPROP = "Threads"; private int numThreads;

// Default thread-pool size // Name of thread property

// Number of threads in pool

72 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 3O 31 32 33 34 35 36 37 38 39 4O 41 42 43 44 45 46 47

Chapter 4: Beyond the Basics

m

public PoolDispatcher() { // Get the number of threads from the System properties or take the default numThreads = Integer. parselnt (System. getProperty(THREADPROP, NUMTHREADS)) ;

} public void startDispatching(final ServerSocket servSock, final Logger logger, final ProtocolYactory protoFactory) { // Create N-I threads, each running an iterative server for (int i = O; i < (numThreads - i); i++) { Thread thread = new Thread() { public void run() { dispatchLoop(servSock, logger, protoFactory);

} }; thread.start(); logger.writeEntry("Created

and started Thread = " + thread.getName());

logger.writeEntry("Iterative server starting in main thread " + Thread. currentThread(), getName ()) ; // Use main thread as Nth iterative server dispatchLoop(servSock, logger, protoFactory) ; /* NOT REACHED */

private void dispatchLoop(ServerSocket servSock, Logger logger, ProtocolFactory protoFactory) { // Run forever, accepting and handling each connection for (;;) { try { Socket clntSock = servSock.accept(); / / Block waiting for connection Runnable protocol = protoFactory.createProtocol(clntSock, logger); protocol.run(); } catch (lOException e) { logger.writeEntry("Exception = " + e.getMessage());

}

}

PoolDispatcher.java 1. PoolDispatcher(): lines 10-13 The t h r e a d - p o o l solution n e e d s an additional piece of information: the n u m b e r of t h r e a d s in the pool. We n e e d to provide this i n f o r m a t i o n to the instance before the t h r e a d pool is c o n s t r u c t e d . We could pass the n u m b e r of t h r e a d s to the constructor, b u t this limits our o p t i o n s b e c a u s e the c o n s t r u c t o r interface varies by dispatcher. We use s y s t e m p r o p e r t i e s

[]

4.1 Multitasking

~

to specify the number of threads to PoolDispatcher. The call to System.getProperty() returns a String containing the value of the "Threads" property or the default value if the property is not defined. The string is then converted to an integer. (See the discussion of system properties in the text below.) 2. startDispatching(): lines 15-32 9 Spawn N - 1 t h r e a d s to execute dispatchLoop(): lines 17-26 For each loop iteration, an instance of an anonymous class that extends Thread is created. When the s t a r t () m e t h o d is called, the thread executes the run() m e t h o d of this anonymous class. The run() m e t h o d simply calls dispatchLoop(), which implements an iterative server. 9 Execute dispatchLoop() in the main thread: lines 27-30 The original calling thread serves as the N t h thread of the pool. 3. dispatchLoop(): lines 34-46 9 Accept an incoming connection: line 39 Since there are N threads executing dispatchLoop(), up to N threads can be blocked on servSock's accept (), waiting for an incoming connection. The system ensures that only one thread gets a Socket for any particular connection. If no threads are blocked on accept() when a client connection is established, the new connection is queued until the next call to accept() (see Section 5.4.1). 9 Create a protocol instance to handle n e w connection: line 40 9 Run the protocol for the connection: line 41 9 Handle exception f r o m accept(): lines 42-44 Since threads are reused, the thread-pool solution only pays the overhead of thread creation N times, irrespective of the total number of client connections. Since we control the m a x i m u m number of simultaneously executing threads, we can control scheduling overhead. Of course, if we spawn too few threads, we can still have clients waiting a long time for service; therefore, the size of the thread pool should be tuned so that client connection time is minimized. In PoolDispatcher. java, we used system properties to specify the number of threads in the pool. Here, we give a brief description of how system properties work.. The System class contains a Properties instance that holds a set of externally defined property/value pairs (e.g., class path and JVM version). We can also define our own properties. For example, we might want to know a user's favorite color. We could place this information in the "user.favoritecolor" property. The following code demonstrates how to fetch and print out all system properties, using the g e t P r o p e r t i e s ( ) m e t h o d of System, and how to find a particular property value, using System.getProperty(). The second parameter to getProperty() specifies a value ("None"), to be used if the property is not found. (See L i s t P r o p e r t i e s . java on the book's Web site for a complete example.)

System.getProperties().list(System.out); // Print all System properties System.out.println("\nFavorite Color: " + // Print favorite color property System. getProperty("user, favoritecolor", "None") ) ;

74

Chapter 4: Beyond the Basics

[]

When running Java programs from the c o m m a n d line, we simply use the -D option to set a property value. For example, to set the property "user.favoritecolor" to "blue," we would try

% java-Duser.favoritecolor=blue

ListProperties

Note that properties are typically defined with hierarchical (general to specific) names, such as For brevity's sake, we did not use such a name, but in a production application hierarchical names should be used to avoid collisions. The main() of ThreadUain. java d emo n s trates how to use either the thread-per-client or thread-pool server. This application takes three parameters: 1) the port n u m b e r for the server, 2) the protocol name (use "Echo" for the echo protocol), and 3) the dispatcher name (use "ThreadPer" or "Pool" for the thread-per-client and thread-pool servers, respectively). The n u m b e r of threads for the thread pool defaults to 8. However, this can be changed to 4, for example, by setting a system property using the -DThreads=4 option to the JVM.

java.class.path.

% java-DThreads=4 ThreadMain 5000 Echo Pool Note that you must compile EchoProtocolFactory. java, ThreadPerDispatcher. java, and PoolDispatcher.java explicitly before running ThreadMain.java. Failure to do so will result in a ClassNotFoundException. Those classes are not referenced by n a m e in ThreadMain (that's the idea!), so they will not be automatically compiled with ThreadMain.

ThreadMain.java 0 import java.net. *; // for ServerSocket 1 import java. io. * ; // for IOException 2 3 public class ThreadMain { 4 public static void main(String[] args) throws Exception { 5 6 7 if (args.length != 3) // Test for correct # of args throw new lllegalArgumentException("Parameter(s) : []" 8 9 + " "); 10 11 int servPort = Integer.parselnt(args[0]); // Server Port 12 String protocolName = args[l] ; // Protocol name 13 String dispatcherName = args[2]; // Dispatcher name 14 15 ServerSocket servSock = new ServerSocket(servPort); 16 Logger logger = new ConsoleLogger(); // Log messages to console 17 ProtocolYactory protoFactory = (ProtocolFactory) // Get protocol factory 18 Class.forName(protocolName + "ProtocolFactory").newlnstance(); 19 Dispatcher dispatcher = (Dispatcher) // Get dispatcher 20 Class.forName(dispatcherName + "Dispatcher").newlnstance(); 21

m 4.2 Nonblocking I/O

22 23 24 25

75

dispatcher, startDispatching(servSock, logger, protoFactory) ; /* NOT REACHED */

} }

ThreadMain.java 1. Application setup and parameter parsing: lines O-13 2. Create s e r v e r socket and logger: lines 15-16

3. Instantiate a protocol factory: lines 17-18 The protocol name is passed as the second parameter. We adopt the naming convention of ProtocolFactory for the class name of the factory for the protocol name . For example, if the second parameter is "Echo," the corresponding protocol factory is EchoProtocolFactory. The static m e t h o d Class.forName() takes the name of a class and returns a Class object. The newInstance() m e t h o d of Class creates a new instance of the class using the parameterless constructor, protoFactory refers to this new instance of the specified protocol factory. 4. Instantiate a dispatcher: lines 19-20 The dispatcher name is passed as the third parameter. We adopt the naming convention of Dispatcher for the class name of the dispatcher of type . For example, if the third parameter is "ThreadPer," the corresponding dispatcher is ThreadPerDispatcher. dispatcher refers to the new instance of the specified dispatcher. 5. Start dispatching clients: line 22 ThreadMain. java makes it easy to use other protocols and dispatchers. The book's Web site contains some additional examples. For example, see TimeProtocolFactory.java for an implementation of the time protocol where clients can get the server time by simply connecting to the server on the time port. See GUIThreadUain. j ava on the book's Web site for an example of server integration with a GUI. This application lists the currently connected client. You will need a protocol-specific GUI implementation (see GUIEchoProtocolFactory. java). 1 The parameters to this application are the same as for ThreadMain. For this application to work, you must specify the GUI version of the protocol factory on the c o m m a n d line (e.g., GUIEcho instead of Echo).

4.2

Nonblocking I/O

Socket I/O calls may block for several reasons. Data input methods read() and receive() block if data is not available. A w r i t e ( ) on a TCP socket may block if there is not sufficient space to buffer the transmitted data. The accept () m e t h o d of ServerSocket and the Socket constructor

1Clearly, more decomposition is possible, but it is beyond the scope of this book.

76

Chapter 4: Beyond the Basics

m

both block until a connection has been established (see Section 5.4). Meanwhile, long roundtrip times, high error rate connections, and slow (or deceased) servers may cause connection establishment to take a long time. In all of these cases, the method returns only after the request has been satisfied. Of course, a blocking method call halts the execution of the application. What about a program that has other tasks to perform while waiting for call completion (e.g., updating the "busy" cursor or responding to user requests)? These programs may have no time to wait on a blocked method call. Or what about lost UDP datagrams? In UDPEchoClientTimeout. java, the client sends a datagram to the server and then waits to receive a response. If a datagram is not received before the timer expires, receive () unblocks to allow the client to handle the datagram loss. Here we describe the general nonblocking approaches (where they exist) for various I/O methods. (Note: As this book goes to press, additional nonblocking I/O features have been proposed for version 1.4 of the JDK. Because these features are still under development, we do not cover them here.)

4.2.1 accept(), read(), and receive() For these methods, we can set a b o u n d on the m a x i m u m time (in milliseconds) to block, using the setSoTimeout() m e t h o d of Socket, ServerSocket, and DatagramSocket. If the specified time elapses before the m e t h o d returns, an InterruptedIOException is thrown. For Socket instances, we can also use the a v a i l a b l e ( ) method of the socket's InputStream to check for available data before calling read ().

4.2.2

Connecting and Writing

The Socket constructor attempts to establish a connection to the host and port supplied as arguments, blocking until either the connection is established or a system-imposed timeout occurs. Unfortunately, the system-imposed timeout is long (on the order of minutes), and Java does not provide any means of shortening it. A w r i t e ( ) call blocks until the last byte written is copied into the TCP implementation's local buffer; if the available buffer space is smaller than the size of the write, some data must be successfully transferred to the other end of the connection before the call to w r i t e ( ) will return (see Section 5.1 for details). Thus, the amount of time that a w r i t e ( ) may block is controlled by the receiving application. Unfortunately, Java currently does not provide any way to cause a w r i t e ( ) to time out, nor can it be interrupted by another thread. Therefore, any protocol that sends a large enough amount of data over a Socket instance can block for an u n b o u n d e d amount of time. (See Section 5.2 for further discussion on the consequences.)

4.2.3

Limiting Per-Client Time

Suppose we want to implement the Echo protocol with a limit on the amount of time taken to service each client. That is, we define a target, TIMELIMIT, and implement the protocol in such a way that after TIMELIMIT milliseconds, the protocol instance is terminated. One approach

II

4.2 Nonblocking I/O

77

simply has the p r o t o c o l instance keep track of the a m o u n t of the r e m a i n i n g time, and use setSoTimeout () to e n s u r e that no r e a d ( ) call blocks for longer t h a n that time. Unfortunately, there is no way to b o u n d the d u r a t i o n of a w r i t e ( ) call, so we c a n n o t really g u a r a n t e e that the time limit will hold. TimelimitEchoProtocolFactory. java i m p l e m e n t s this approach.

TimelimitEchoProtocolFactory.java 0 1 2 3 4 5 6 7

import j a v a . n e t . * ; import j a v a . i o . * ; import j a v a . u t i l . * ;

public class TimelimitEchoProtocolFactory

implements ProtocolFactory {

public Runnable createProtocol(Socket clntSock, Logger logger) { return new TimelimitEchoProtocol(clntSock, logger);

}

8

9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38

/ / for Socket / / for IOException and Input/OutputStream / / for ArrayList

} class TimelimitEchoProtocol implements Runnable { private static final int BUFSIZE = 32; // Size (in bytes) of receive buffer private static final String TIMELIMIT = "i0000"; // Default time limit (ms) private static final String TIMELIMITPROP = "Timelimit"; // Thread property private int timelimit; private Socket clntSock; private Logger logger; public TimelimitEchoProtocol(Socket clntSock, Logger logger) { this.clntSock = clntSock; this.logger = logger; / / Get the time limit from the System properties or take the default timelimit = Integer.parseInt(System.getProperty(TIMELIMITPROP, TIMELIMIT));

} public void run() { ArrayList entry = new ArrayList(); entry.add("Client address and port = " + clntSock.getInetAddress().getHostAddress() + ":" + clntSock.getPort()); entry.add("Thread = " + Thread'currentThread()'getName()); try { / / Get the input and output I/O streams from socket InputStream in = clntSock, getInputStream() ; OutputStream out = clntSock, getOutputStream() ;

78 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 6O 61 62 63 64 65 66 67 68 69 7O 71

Chapter 4: Beyond the Basics

II

int recvMsgSize; // Size of received message int totalBytesEchoed = O; // Bytes received from client byte[] echoBuffer = new byte[BUFSIZE]; // Receive buffer long endTime = System.currentTimeMillis() + timelimit; int timeBoundMillis = timelimit; clntSock, setSoTimeout (timeBoundMillis) ; // Receive until client closes connection, indicated by -i while ((timeBoundMillis > O) && // catch zero values ((recvMsgSize = in.read(echoBuffer)) != -i)) { out.write(echoBuffer, O, recvMsgSize); totalBytesEchoed += recvMsgSize; timeBoundMillis = (int) (endTime - System.currentTimeMillis()) clntSock.setSoTimeout(timeBoundMillis);

;

} entry.add("Client finished; echoed " + totalBytesEchoed + " bytes,"); } catch (InterruptedlOException dummy) { entry.add("Read timed out"); } catch (lOException e) { entry,add("Exception = " + e,getMessage());

} try { // Close socket clntSock.close(); } catch (IOException e) { entry.add("Exception = " +

e.getMessage());

} logger, writeEntry(entry) ;

} }

TimelimitEchoProtocolFactory.java

TimelimitF, choProtocolFactory.java contains b o t h the factory and protocol instance classes. The factory is exactly like EchoProtocolFactory, with the exception that it instantiates TimelimitEehoProtoeol instead of EchoProtocol. The TimelimitEchoProtoeol class is similar to the EchoProtocol class, except that it a t t e m p t s to b o u n d the total time an echo connection can exist. The default time is 10 seconds; the total n u m b e r of milliseconds per connection can be set using the "Timelimit" property. Another a p p r o a c h to limiting client service time involves starting two threads per client: one that executes the protocol and another that acts as a "watchdog," sleeping until TIMELIMIT milliseconds pass or the other (protocol) t h r e a d finishes and interrupts it, whichever comes

[]

4.3 Multiple Recipients

79

first. If the watchdog awakens and the protocol thread has not finished, the watchdog terminates the protocol thread. Unfortunately, threads killing other threads is deprecated in Java, because the victim thread's abrupt termination might leave some objects in an inconsistent or unrecoverable state. Since there is no other way to interrupt a blocking w r i t e ( ) , this solution usually will not work. Finally, note that we could attem p t to use nonblocking I/O instead of threads. Be warned, however, that these solutions typically involve polling loops employing busy-waiting. While adding threads does consume extra CPU and m e m o r y resources, the overhead is generally small, especially compared to that of busy-waiting.

4.3

Multiple Recipients

So far all of our sockets have dealt with communication between exactly two entities, usually a server and a client. Such one-to-one communication is sometimes called unicast. Some information is of interest to multiple recipients. In such cases, we could unicast a copy of the data to each recipient, but this may be very inefficient. Unicasting multiple copies over a single network connection wastes bandwidth by sending the same information multiple times. In fact, if we want to send data at a fixed rate, the bandwidth of our network connection defines a hard limit on the n u m b e r of receivers we can support. For example, if our video server sends 1Mbps streams and its network connection is only 3Mbps (a healthy connection rate), we can only support three simultaneous users. Fortunately, networks provide a way to use bandwidth more efficiently. Instead of making the sender responsible for duplicating packets, we can give this job to the network. In our video server example, we send a single copy of the stream across the server's connection to the network, which then duplicates the data only when appropriate. With this model of duplication, the server uses only 1Mbps across its connection to the network, irrespective of the n u m b e r of clients. There are two types of one-to-many service: broadcast and multicast. With broadcast, all hosts on the (local) network receive a copy of the message. With multicast, the message is sent to a multicast address, and the network delivers it only to those hosts that have indicated that they want to receive messages sent to that address. In general, only UDP sockets are allowed to broadcast or multicast.

4.3.1 Broadcast Broadcasting UDP datagrams is similar to unicasting datagrams, except that a broadcast address is used instead of a regular (unicast) IP address. The local broadcast address (255.255.255 .255) sends the message to every host on the same broadcast network. Local broadcast messages are never forwarded by routers. A host on an Ethernet network can send a message to all other hosts on that same Ethernet, but the message will not be forwarded by a router. IP also specifies directed broadcast addresses, which allow broadcasts to all hosts on a specified network; however, since most Internet routers do not forward directed broadcasts, we do not deal with t h e m here.

80

Chapter 4: Beyond the Basics

[]

There is no networkwide broadcast address that can be used to send a message to all hosts. To see why, consider the impact of a broadcast to every host on the Internet. Sending a single datagram would result in a very, very large number of packet duplications by the routers, and bandwidth would be consumed on each and every network. The consequences of misuse (malicious or accidental) are too great, so the designers of IP left such an Internetwide broadcast facility out on purpose. Even so, local broadcast can be very useful. Often, it is used in state exchange for network games where the players are all on the same local (broadcast) network. In Java, the code for unicasting and broadcasting is the same. To play with broadcasting applications, simply run SendUDP. java using a broadcast destination address. Run RecvUDP. java as you did before (except that you can run several receivers at one time). Caveat: Some operating systems do not give regular users permission to broadcast, in which case this will not work.

4.3.2

Multicast

As with broadcast, the main difference between multicast and unicast is the form of the address. A multicast address identifies a set of receivers. The designers of IP allocated a range of the address space (from 224.0.0.0 to 239.255.255.255) dedicated to multicast. With the exception of a few reserved multicast addresses, a sender can send datagrams addressed to any address in this range. In Java, multicast applications generally communicate using an instance of MulticastSocket, a subclass of DatagramSocket. It is important to u n d e r s t a n d that a MulticastSocket is actually a UDP socket (DatagramSocket), with some extra multicastspecific attributes that can be controlled. Our next example implements the multicast version of SendUDP. java (see page 57).

SendUDPMulticast.c

0 import java.net.*; / / for MulticastSocket, DatagramPacket, and InetAddress 1 import java.io.*; / / for 10Exception 2 3 public class SendUDPMulticast { 4 public static void main(String args[]) throws Exception { 5 6 if ((args.length < 2) II (args.length > 3)) // Test for correct # of args 7 throw new lllegalArgumentException( 8 "Parameter(s): <Multicast Addr> []"); 9 10 InetAddress destAddr = InetAddress.getByName(args[0]); // Destination address 11 if (!destAddr.isMulticastAddress()) // Test if multicast address 12 throw new lllegalArgumentException("Not a multicast address"); 13 14 int destPort = Integer.parselnt(args[l]); // Destination port 15

m 4.3 Multiple Recipients

16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39

8 |

int TTL; // Time To Live for datagram if (args.length == 3) TTL = Integer. parselnt (args [2 ] ) ; else TTL = i; // Default TTL ItemQuote quote = new ItemQuote(1234567890987654L,

"5mm Super Widgets", i000, 12999, true, false);

MulticastSocket sock = new MulticastSocket(); // Multicast socket to sending sock.setTimeToLive(TTL); // Set TTL for all datagrams ItemQuoteEncoder encoder = new ItemQuoteEncoderText() ; byte[] codedQuote = encoder.encode(quote) ;

/ / Text encoding

// Create and send a datagram DatagramPacket message = new DatagramPacket(codedQuote, codedQuote.length, destAddr, destPort) ; sock. send (message) ; sock.close();

} } SendUDPMulticast.c

The only significant differences b e t w e e n our unicast a n d multicast s e n d e r s are that 1) we verify that the given a d d r e s s is multicast, and 2) we set the initial Time To Live (TTL) value for the multicast datagram. Every IP d a t a g r a m contains a TTL, initialized to s o m e default value a n d d e c r e m e n t e d (usually by one) by each r o u t e r that f o r w a r d s the packet. When the TTL reaches zero, the packet is discarded. By setting the initial value of the TTL, we limit the distance a packet can travel f r o m the sender. 2 Unlike b r o a d c a s t , n e t w o r k multicast duplicates the m e s s a g e only to a specific set of receivers. This set of receivers, called a m u l t i c a s t group, is identified by a s h a r e d multicast (or group) address. Receivers n e e d some m e c h a n i s m to notify the n e t w o r k of their interest in receiving data sent to a particular multicast address, so that the n e t w o r k can f o r w a r d packets to them. This notification, called j o i n i n g a group, is a c c o m p l i s h e d with the joinGroup() m e t h o d of MulticastSocket. Our multicast receiver joins a specified group, receives a n d prints a single multicast m e s s a g e f r o m that group, and exits.

2The rules for multicast TTL are actually not quite so simple. It is not necessarily the case that a packet with TTL = 4 can travel four hops from the sender; however, it will not travel more than four hops.

82

Chapter 4: Beyond the Basics

[]

RecvU DPM u Iticas t.java

0 1 2 3 4 5 6 7

import java.net.*; import java.io.*;

public class RecvUDPMulticast implements ItemQuoteTextConst { public static void main(String[] args) throws Exception { if (args.length != 2) // Test for correct # of args throw new lllegalArgumentException("Parameter(s): <Multicast Addr> ");

8

9 lO ii 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

// for MulticastSocket, DatagramPacket, and InetAddress // for lOException

InetAddress address = InetAddress.getByName(args[0]); // Multicast address if (!address.isMulticastAddress()) // Test if multicast address throw new lllegalArgumentException("Not a multicast address"); int port = Integer.parselnt(args[l]);

// Multicast port

MulticastSocket sock = new MulticastSocket(port) ; / / Multicast receiving socket sock.joinGroup(address); / / Join the multicast group // Create and receive a datagram

DatagramPacket packet = new DatagramPacket( new byte[MAX_WIRE_LENGTH], MAX_WIRE_LENGTH); sock.receive(packet) ; ItemQuoteDecoder decoder = new ItemQuoteDecoderText(); ItemQuote quote = decoder.decode(packet); System. out. println(quote) ;

// Text decoding

sock.close(); } }

RecvUDPMulticast.java

The only significant difference between our multicast and unicast receiver is that the multicast receiver m u s t join the multicast group by supplying the desired multicast address. The book's Web site also contains another example of a sender and receiver multicast pair. UultieastImageSender. java transmits a set of images (JPEG or GIF) specified on the c o m m a n d line, in three-second intervals. UulticastImageReceiver. java receives each image and displays it in a window. Multicast datagrams can, in fact, be sent from a DatagramSoeket by simply using a multicast address. You can test this by using SendUDP. java (see page 57) to send to the multicast receiver. However, a MulticastSocket has a few capabilities that a DatagramSocket does not,

I

4.3 Multiple Recipients

83

including 1) allowing specification of the datagram TTL, and 2) allowing the interface through which datagrams are sent to the group to be specified/changed (an interface is identified by its Internet address). A multicast receiver, on the other hand, must use a MulticastSocket because it needs the ability to join a group. MulticastSocket is a subclass of DatagramSocket, so it provides all of the DatagramSocket methods. We only present methods specific to or adapted for MulticastSocket.

MulticastSocket Constructors MulticastSocketO MulticastSocket(int localPort) Constructs a datagram socket that can perform some additional multicast operations. The second form of the constructor specifies the local port. If the local port is not specified, the socket is bound to any available local port.

localPort

Local port. A localPort of 0 allows the constructor to pick any available port.

Operators void joinGroup(InetAddress groupAddress) void leaveGroup(InetAddress groupAddress) Join/leave a multicast group. A socket may be a member of multiple groups simultaneously. Joining a group of which this socket is already a member or leaving a group of which this socket is not a member may generate an exception.

groupAddress

Multicast address identifying group

void send(DatagramPacket packet, byte TTL) Send a datagram from this socket with the specified TTL.

packet

Packet to transmit. Either the packet must specify a destination address or the UDP socket must have a specified remote address and port (see connect()).

TTL

Time to live for this packet

Accessors InetAddress getInterface() void s e t I n t e r f a c e ( I n e t A d d r e s s interface) Returns/sets the interface to use for multicast operations on this socket. This is primarily used on hosts with multiple interfaces. Join/leave requests and datagrams

84

Chapter 4: Beyond the Basics

II

will be sent and d a t a g r a m s will be received using this interface. The default multicast interface is p l a t f o r m dependent.

interface

Address of one of host's multicast interfaces

int getTimeToLive0

void setTimeToLive(int TTL) Returns/sets the Time To Live for all datagrams sent on this socket. This can be overridden on a per-datagram basis using the send() method that takes the TTL as a parameter.

TTL

Time To Live for this packet

The decision to use b r o a d c a s t or multicast d e p e n d s on several factors, including the n e t w o r k location of receivers and the knowledge of the c o m m u n i c a t i n g parties. The scope of a b r o a d c a s t on the Internet is restricted to a local b r o a d c a s t network, placing severe restrictions on the location of the b r o a d c a s t receivers. Multicast c o m m u n i c a t i o n m a y include receivers anywhere in the network, 3 so multicast has the advantage that it can cover a distributed set of receivers. The disadvantage of IP multicast is that receivers m u s t know the a d d r e s s of a multicast group to join. Knowledge of an a d d r e s s is not required to receive broadcast. In some contexts, this m a k e s b r o a d c a s t a better m e c h a n i s m than multicast for discovery. All hosts can receive b r o a d c a s t by default, so it is simple to ask all hosts on a single n e t w o r k a question like "Where's the printer?" UDP unicast, multicast, and b r o a d c a s t are all i m p l e m e n t e d using an underlying UDP socket. The semantics of m o s t i m p l e m e n t a t i o n s are such that a UDP d a t a g r a m will be delivered to all sockets b o u n d to the destination port of the packet. That is, a DatagramSoeket or MultieastSoeket instance b o u n d to a local port X (with local address not specified, i.e., a wild card), on a host with address Y will receive any UDP d a t a g r a m destined for port X that is 1) unicast with destination address Y, 2) multicast to a group that any application on Y has joined, or 3) b r o a d c a s t where it can reach host Y. A receiver can use connect() to limit the d a t a g r a m source a d d r e s s and port. Also, a DatagramSoeket can specify the local unicast address, which prevents delivery of multicast and b r o a d c a s t packets. See UDPEehoClientTimeout. java for an example of destination a d d r e s s verification and Section 5.5 for details on d a t a g r a m demultiplexing.

4.4

Socket Options

The TCP/IP protocol developers spent a good deal of time thinking about the default behaviors that would satisfy m o s t applications. (If you doubt this, read RFCs 1122 and 1123, which describe in excruciating detail the r e c o m m e n d e d b e h a v i o r s - - b a s e d on years of experience--

3At the time of writing of this book, there are severe limitations on who can receive multicast traffic on the Internet; however, multicast availability should improve over time. Multicast should work if the sender and receivers are on the same LAN.

[]

4.5 Closing Connections

85

for i m p l e m e n t a t i o n s of the TCP/IP protocols.) For m o s t applications, the designers did a good job; however, it is s e l d o m the case that "one size fits all" really fits all. We have already seen an example in our UDP echo client. By default, the r e c e i v e ( ) m e t h o d of DatagramSocket blocks indefinitely waiting on a datagram. In our example, we change that behavior by specifying a timeout for receives on the UDP socket using setSoTimeout(). In socket parlance, each type of behavior we can change is called a socket option. In Java, the socket type (e.g., Socket, ServerSocket, DatagramSocket, and MulticastSocket) determines the applicable socket options, which are typically queried and controlled using accessor methods like getSoTimeout() and setSoTimeout(). Unfortunately, the Java API allows access to only a subset of the options in the underlying sockets API. This is at ]east partly because options tend to vary in availability from platform to platform, and Java is all about portability. However, as the versions wear on, access to more and more socket options is being added in Java. Check the latest official documentation for the various socket types to see the available options.

4.5

Closing Connections

You've probably never given m u c h thought to who closes a connection. In phone conversations, either side can start the process of terminating the call. It typically goes s o m e t h i n g like this: "Well, I guess I'd better go." "OK. Bye." "Bye." Network protocols, on the other hand, are typically very specific about who "closes" first. In the echo protocol, Figure 4.1(a), the server dutifully echoes everything the client sends. When the client is finished, it calls c l o s e ( ) . After the server has received and echoed all of the data sent before the client's call to c l o s e ( ) , its read operation r e t u r n s a -1, indicating that the client is finished. The server then calls close ( ) on its socket. The close is a critical part of the protocol because without it the server doesn't know w h e n the client is finished sending characters to echo. In HTTP, Figure 4.1(b), it's the server that initiates the connection close. Here, the client sends a r e q u e s t ("GET") tO the server, and the server r e s p o n d s by sending a h e a d e r (normally

"To Be.

"To Be.

.

. .

. .

Get/Guide.html ..." .

200 OK ...

"Or Not To Be"

...

"Or Not To Be"

... "

Closed

Closed

Closed

Closed

(a)

(b)

Figure 4.1: Echo (a) and HTTP (b) protocol termination.

86

Chapter 4: Beyond the Basics

m

starting with "200 OK"), followed by the requested file. Since the client does not know the size of the file, the server m u s t indicate the end-of-file by closing the socket. Calling close() on a Socket terminates both directions (input and output) of data flow. (Subsection 5.4.2 provides a more detailed description of TCP connection termination.) Once an endpoint (client or server) closes the socket, it can no longer send or receive data. This means that close() can only be used to signal the other end when the caller is completely finished communicating. In the echo protocol, once the server receives the close from the client, it immediately closes. In effect, the client close indicates that the communication is completed. HTTP works the same way, except that the server is the terminator. Let's consider a different protocol. Suppose you want a compression server that takes a s t r e a m of bytes, compresses them, and sends the compressed stream back to the client. Which endpoint should close the connection? Since the stream of bytes from the client is arbitrarily long, the client needs to close the connection so that the server knows when the stream of bytes to be compressed ends. When should the client call close()? If the client calls close() on the socket immediately after it sends the last byte of data, it will not be able to receive the last bytes of compressed data. Perhaps the client could wait until it receives all of the compressed data before it closes, as the echo protocol does. Unfortunately, neither the server nor the client knows how m a n y bytes to expect, so this will not work either. What is needed is a way to tell the other end of the connection "I am through sending," without losing the ability to receive. Fortunately, sockets provide a way to do this. The shutdownTnput () and shutdown0utput () m e t h o d s of Socket allow the I/O streams to be closed independently. After shutdownlnput(), the socket's input stream can no longer be used. Any undelivered data is silently discarded, and any a t t e m p t to read from the socket's input stream will return -1. After shutdown0utput () is called on a Socket, no more data may be sent on the socket's output stream. Attempts to write to the stream throw an IOException. Any data written before the call to shutdown0utput () may be read by the remote socket. After this, a read on the input stream of the remote socket will return -1. An application calling shutdown0utput() can continue to read from the socket and, similarly, data can be written after calling shutdownInput (). In the compression protocol (see Figure 4.2), the client writes the bytes to be compressed, closing the output stream using shutdown0utput() when finished sending, and reads the compressed byte stream from the server. The server repeatedly reads the u n c o m p r e s s e d data


Closed

Figure 4.2: Compression protocol termination.

m 4.5 Closing Connections

87

and writes the c o m p r e s s e d data until the client p e r f o r m s a shutdown, causing the server read to r e t u r n -1, indicating an end-of-stream. The server then closes the connection and exits. After the client calls shutdown0utput (), it needs to read any remaining c o m p r e s s e d bytes f r o m the server. Our c o m p r e s s i o n client, CompressClient. java, i m p l e m e n t s the client side of the compression protocol. The u n c o m p r e s s e d bytes are read f r o m the file specified on the c o m m a n d line, and the c o m p r e s s e d bytes are written to a new file. If the u n c o m p r e s s e d filename is "data", the c o m p r e s s e d filename is "data. gz". Note that this i m p l e m e n t a t i o n works for small files, but that there is a flaw that causes deadlock for large files. (We discuss and correct this shortcoming in Section 5.2.)

CompressClient.java 0 import java.net.*; // for Socket 1 import java.io.*; // for lOException and [File]Input/0utputStream 2 3 public class CompressClient { 4 public static final int BUFSIZE = 256; // Size of read buffer 5 6 public static void main(String[] args) throws 10Exception { 7 8 if (args.length != 3) // Test for correct # of args 9 throw new lllegalArgumentException("Parameter(s): <Server> "); 10 11 // Server name or IP address 12 String server = args [0] ; int port = I n t e g e r . p a r s e l n t ( a r g s [ 1 ] ) ; // Server port 13 // File to read data from String filename = args[2]; 14 15 // Open input and output file (named input.gz) 16 FilelnputStream fileln = new FilelnputStream(filename); 17 FileOutputStream fileOut = new FileOutputStream(filename + " .gz"); 18 19 // Create socket connected to server on specified port 20 Socket sock = new Socket(server, port) ; 21 22 // Send uncompressed byte stream to server 23 sendBytes(sock, fileln); 24 25 // Receive compressed byte stream from server 26 InputStream sockIn = sock.getInputStream(); 27 int bytesRead; // Number of bytes read 28 byte[] buffer = new byte[BUFSIZE]; // Byte buffer 29 while ((bytesRead = sockIn.read(buffer)) != -i) { 3O fileOut.write(buffer, O, bytesRead); 31

88 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 5O 51 52

Chapter 4: Beyond the Basics

System. out. print ("R") ;

[]

// Reading progress indicator

} System. out. println() ; sock.close(); fileln.close(); fileOut.close();

// End progress indicator line

// Close the socket and its streams // Close file streams

private static void sendBytes(Socket sock, InputStream fileIn) throws IOException { OutputStream sockOut = sock.getOutputStream(); int bytesRead; // Number of bytes read byte[] buffer = new byte[BUYSIZE]; // Byte buffer while ((bytesRead = fileIn.read(buffer)) != -1) { sockOut.write(buffer, O, bytesRead); System.out.print("W"); // Writing progress indicator

} sock.shutdownOutput();

// Finished sending

}

CompressClient.java

1. Application setup and p a r a m e t e r parsing: lines 0-14 2. Create socket and o p e n files: lines 16-21 3. Invoke sendBytes() to t r a n s m i t bytes: lines 23-24 4. Receive the c o m p r e s s e d data stream: lines 26-34 The while loop receives the compressed data stream and writes the bytes to the output file until an end-of-stream is signaled by a -1 from read (). 5. Close socket a n d file streams: lines 36-38 6. sendBytes(): lines 41-51 Given a socket connected to a compression server and the file input stream, read all of the u n c o m p r e s s e d bytes from the file and write them to the socket output stream. 9 Get socket o u t p u t stream: line 43 9 Send u n c o m p r e s s e d b y t e s to c o m p r e s s i o n server: lines 44-49 The while loop reads from the input stream (in this case from a file) and repeats the bytes to the socket output stream until end-of-file, indicated by -1 from read(). Each write is indicated by a "W" printed to the console.

m 4.5 Closing Connections

89

9 Shut d o w n the socket output stream: line 50

After reading and sending all of the bytes from the input file, shut down the output stream, notifying the server that the client is finished sending. The close will cause a -1 return from read() on the server. To implement the compression server, we simply write a protocol and factory for our threaded server architecture. Our protocol implementation, CompressProtocolFactory.java, implements the server-side compression protocol using the GZIP compression algorithm. The server receives the uncompressed bytes from the client and writes them to a GZIP0utputStream, which wraps the socket's output stream.

C o m pres s Protocol F a c t o r y . j a v a

0 1 2 3 4 5 6 7

import import import import

java.net.*; java.io.* ; java.util.*; java.util.zip.*;

// // // //

for for for for

Socket lOBxception and Input/OutputStream ArrayList GZIPOutputStream

public class CompressProtocolFactory implements ProtocolFactory { public static final int BUFSIZE = 1024;

// Size of receive buffer

8

9 lO ll 12 13 14 15 16 17 18 19 2O 21 22 23 24 25 26 27 28 29 30 31

public Runnable createProtocol(final Socket clntSock, final Logger logger) { return new Runnable() { public void run() { CompressProtocolFactory.handleClient(clntSock, logger);

} }; } public static void handleClient(Socket clntSock, Logger logger) { ArrayList entry = new ArrayList(); entry.add("Client address and port = " + clntSock.getlnetAddress().getHostAddress() + ":"+ clntSock.getPort()); entry.add("Thread = " + Thread.currentThread().getName()); try { // Get the input and output streams from socket InputStream in = clntSock, getInputStream() ; GZIPOutputStream out = new GZIPOutputStream(clntSock. getOutputStream()) ; byte[] buffer = new byte[BUFSIZE]; // Allocate read/write buffer int bytesRead; // Number of bytes read // Receive until client closes connection, indicated by-i return

90 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48

Chapter 4: Beyond the Basics

II

while ((bytesRead = in.read(buffer)) out.write(buffer, O, bytesRead) ;

!= -i)

out.finish(); // Flush bytes from GZIPOutputStream } catch (lOException e) { logger.writeEntry("Exception = " + e.getMessage());

try { // Close socket clntSock.close(); } catch (lOException e) { entry, add("Exception = " +

e. getMessage()) ;

logger, writeEntry(entry) ;

}

CompressProtocolFactory.java 1. Factory m e t h o d for c o m p r e s s i o n protocol: lines 9-15 c r e a t e P r o t o c o l ( ) returns an anonymous class instance that implements the Runnable interface. The run() m e t h o d of this instance simply calls the static m e t h o d CompressProt o c o l F a c t o r y . h a n d l e C l i e n t ( ) , which implements the server-side compression protocol. Note that we do n o t need a separate CompressProtocol class, because c r e a t e P r o t o c o l ( ) returns the type of instance (one that implements Runnable) that we need. 2. handleClient(): lines 17-38 Given a socket connected to the compression client, read the u n c o m p r e s s e d bytes from the client and write the compressed bytes back. 9 Get socket I/O streams: lines 26-27 The socket's output stream is wrapped in a GZlP0utputStream. The sequence of bytes written to this stream is compressed, using the GZIP algorithm, before being written to the underlying output stream. 9 Read u n c o m p r e s s e d and write c o m p r e s s e d bytes: lines 29-33 The while loop reads from the socket input stream and writes to the GZlP0utputStream, which in turn writes to the socket output stream, until the end-of-stream indication is received. 9 Flush and close: lines 35-44 Calling f i n i s h ( ) on the GZlP0utputStream is necessary to flush any bytes that may be buffered by the compression algorithm. A simple iterative version of the server can be found in CompressServer. java on the book's Web site.

n

4.6

4.8 Exercises

91

Applets

Applets can perform network communication using TCP/IP sockets, but there are severe restrictions on how and with w h o m they can converse. Without such restrictions, unsuspecting Web browsers might execute malicious applets that could, for example, send fake email, attempt to hack other systems while the browser user gets the blame, and so on. These security restrictions are enforced by the Java security manager, and violations by the applet result in a SecurityException. Typically, browsers only allow applets to communicate with the host that served the applet. This means that applets are usually restricted to communicating with applications executing on that host, usually a Web server originating the applet. The list of security restrictions and general applet programming is beyond the scope of this book. It is worth noting, however, that the default security restrictions can be altered, if allowed by the browser user. Suppose that you wanted to implement an applet that allowed users to type and save notes to themselves on their browser. Browser security restrictions prevent applets from saving data directly on the local file system, so you would need some other means besides local disk I/O to save the notes. FileClientApplet. java (available from the book's Web site) is an applet that allows the user to type text into an editor window and, by clicking the "Save" button, copy the text over the network to a server (running on port 5000). The server, YCPFileServer. java (also on the book's Web site), saves the data to a file. It takes a port (use 5000 to work with the applet) and the name of the file. The server must execute on the Web server that serves the applet to the browser. Note that there is nothing applet specific about the server. FileClientApplet .html on the Web site demonstrates how to integrate the applet into a Web page. Be warned that the applet is based on Swing, and most browsers don't have the Swing library. The HTML file should download the necessary file to make this work, but it is not guaranteed.

4.7

Wrapping Up

We have discussed some of the ways Java provides access to advanced features of the sockets API, and how built-in features such as threads can be used with socket programs. In addition to these facilities, Java provides several mechanisms that operate on top of TCP or UDP and attempt to hide the complexity of protocol development. For example, Java Remote Method Invocation (RMI) allows Java objects on different hosts to invoke one another's methods as if the objects all reside locally. The URL class and associated classes provide a framework for developing Web-related programs. Many other standard Java library mechanisms exist, providing an amazing range of services. These mechanisms are beyond the scope of this book; however, we encourage you to look at the book's Web site for descriptions and code examples for some of these libraries.

4.8

Exercises

1. State precisely the conditions under which an iterative server is preferable to a multiprocessing server.

~

Chapter 4: Beyond the Basics

III

2. Would you ever need to i m p l e m e n t a timeout in a client or server that uses TCP? 3. How can you determine the m i n i m u m and m a x i m u m allowable sizes for a socket's send and receive buffers? Determine the m i n i m u m s for your system. 4. Write an iterative dispatcher using the dispatching f r a m e w o r k f r o m this chapter. 5. Write the server side of a r a n d o m - n u m b e r server using the protocol factory f r a m e w o r k f r o m this chapter. The client will connect and send the u p p e r bound, B, on the r a n d o m n u m b e r to the server. The server should r e t u r n a r a n d o m n u m b e r b e t w e e n 1 and B, inclusive. All n u m b e r s should be specified in binary f o r m a t as 4-byte, two's-complement, big-endian integers. 6. Modify TCPEehoClient.java so that it closes its o u t p u t side of the connection before a t t e m p t i n g to receive any echoed data.

chapter 5

Under the. Hood

S o m e of the subtleties of network programming are difficult to grasp without some understanding of the data structures associated with the socket implementation and certain details of how the underlying protocols work. This is especially true of TCP sockets (i.e., instances of Socket). This chapter describes some of what goes on under the hood when you create and use an instance of Socket or ServerSocket. (The initial discussion and Section 5.5 apply as well to DatagramSocket and UultieastSocket. However, m o s t of this chapter focuses on TCP sockets, that is, Socket and ServerSocket.) Please note that this description covers only the normal sequence of events and glosses over m a n y details. Nevertheless, we believe that even this basic level of understanding is helpful. Readers who want the full story are referred to the TCP specification [13] or to one of the more comprehensive treatises on the subject [3, 22]. Figure 5.1 is a simplified view of some of the information associated with a Socket instance. The classes are supported by an underlying implementation that is provided by the JVM a n d / o r the platform on which it is running (i.e., the "socket layer" of the host's OS). Operations on the Java objects are translated into manipulations of this underlying abstraction. In this chapter, "Socket" refers generically to one of the classes in Figure 5.1, while "socket" refers to the underlying abstraction, whether it is provided by an underlying OS or the JVM implementation itself (e.g., in an e m b e d d e d system). It is important to note that other (possibly non-Java) programs running on the same host may be using the network via the underlying socket abstraction, and thus competing with Java Socket instances for resources such as ports. By "socket structure" here we mean the collection of data structures in the underlying implementation (of both the JVM and TCP/IP, but primarily the latter) that contain the information associated with a particular Socket instance. For example, the socket structure contains, among other information 9 The local and remote Internet addresses and port numbers associated with the socket. The local Internet address (labeled "Local IP" in the figure) is one of those assigned to the local host; the local port is set at Socket creation time. The remote address and port identify the remote socket, if any, to which the local socket is connected. We will say more

93

94

Chapter 5: Under the Hood

II

cu

o

+~ 4-J ~iD

4-J

H

O

o

Socket, DatagramSocket, MulticastSocket, or ServerSocket instance

q2

C~

<

SendQ [ To network o

I RecvQ Closed

Local port Underlying socket structure Local IP Remote port Remote IP Figure 5.1: Data structures associated with a socket.

about how and when these values are determined shortly (Section 5.5 contains a concise summary). 9 A FIFO queue of received data waiting to be delivered and a queue for data waiting to be transmitted. 9 For a TCP socket, additional protocol state information relevant to the opening and closing TCP handshakes. In Figure 5.1, the state is "Closed"; all sockets start out in the Closed state. Knowing that these data structures exist and how they are affected by the underlying protocols is useful because they control various aspects of the behavior of the various Socket objects. For example, because TCP provides a reliable byte-stream service, a copy of any data written to a Socket's 0utputStream must be kept until it has been successfully received at the other end of the connection. Writing data to the output stream does not imply that the data has actually been sent--only that it has been copied into the local buffer. Even f l u s h ( ) i n g a Socket's 0utputStream doesn't guarantee that anything goes over the wire immediately. Moreover, the nature of the byte-stream service means that message boundaries are not preserved in the input stream. As we saw in Section 3.3, this complicates the process of receiving and parsing for some protocols. On the other hand, with a DatagramSoeket, packets are not buffered for retransmission, and by the time a call to the send () method returns, the data has been given to

[]

5.1 Buffering and TCP

95

the network subsystem for transmission. If the network subsystem cannot handle the message for some reason, the packet is silently dropped (but this is rare). The next three sections deal with some of the subtleties of sending and receiving with TCP's byte-stream service. Then, Section 5.4 considers the connection establishment and termination of the TCP protocol. Finally, Section 5.5 discusses the process of matching incoming packets to sockets and the rules about binding to port numbers.

5.1

Buffering and TCP

As a programmer, the most important thing to remember when using a TCP socket is this: You cannot a s s u m e a n y correspondence between writes to the output stream at one end o f the connection a n d reads from the input stream at the other end.

In particular, data passed in a single invocation of the output stream's w r i t e ( ) method at the sender can be spread across multiple invocations of the input stream's read() method at the other end; and a single read() may return data passed in multiple write()s. To see this, consider a program that does the following: byte [] buffer0 = new byte [I000 ] ; byte[] bufferl = new byte[2000]; byte[] buffer2 = new byte[S000]; ,

Socket s = new Socket(destAddr, destPort) ; 0utputStream out = s. get0utputStream() ;

out .write(buffer0) ; ,

out .write(bufferl) ;

out. write (buffer2) ;

s.close() ; where the ellipses represent code that sets up the data in the buffers but contains no other calls to out. write(). Throughout this discussion, "in" refers to the InputStream of the receiver's Socket, and "out" refers to the 0utputStream of the sender's Socket. This T C P connection transfers 8000 bytes to the receiver. The w a y these 8000 bytes are grouped for delivery at the receiving end of the connection depends on the timing between the out.write()s and in.read()s at the two ends of the connection--as well as the size of the buffers provided to the in. read() calls.

9~

Chapter 5: Under the Hood

Receiving implementation

Sending implementation

I

I

I RecvQ

SendQ send()

Receiving program

I

I Delivered

TCP protocol 6500 bytes

1500 bytes

0 bytes First write (1000 bytes) Second write (2000 bytes) :!jiiii~Third write (5000 bytes)

Figure 5.2: State of the three queues after three writes.

We can think of the sequence of all bytes sent (in one direction) on a TCP connection up to a particular instant in time as being divided into three FIFO queues:

1. SendQ: Bytes buffered in the underlying implementation at the sender that have been written to the output stream but not yet successfully transmitted to the receiving host.

2. RecvQ: Bytes buffered in the underlying implementation at the receiver waiting to be delivered to the receiving program--that is, read from the input stream.

3. Delivered: Bytes already read from the input stream by the receiver. A call to out.write() at the sender appends bytes to SendQ. The TCP protocol is responsible for moving bytes--in order--from SendQ to RecvQ. It is important to realize that this transfer cannot be controlled or directly observed by the user program, and that it occurs in chunks whose sizes are more or less independent of the size of the buffers passed in write()s. Bytes are moved from RecvQ to Delivered as they are read from the Socket's InputStream by the receiving program; the size of the transferred chunks depends on the amount of data in RecvQ and the size of the buffer given to read(). Figure 5.2 shows one possible state of the three queues after the three o u t . w r i t e ( ) s in the example above, but before any in. reads ()s at the other end. The different shading patterns denote bytes passed in the three different invocations of write() shown above. Now suppose the receiver calls read() with a byte array of size 2000. The read() call will move all of the 1500 bytes present in the waiting-for-delivery (RecvQ) queue into the byte array and return the value 1500. Note that this data includes bytes passed in both the first and second calls to write(). At some time later, after TCP has completed transfer of more data, the three partitions might be in the state shown in Figure 5.3. If the receiver now calls read () with a buffer of size 4000, that many bytes will be moved from the waiting-for-delivery (RecvQ) queue to the already-delivered (Delivered) queue; this includes the remaining 1500 bytes from the second write(), plus the first 2500 bytes from the third write(). The resulting state of the queues is shown in Figure 5.4. The number of bytes returned by the next call to read() depends on the size of the buffer and the timing of the transfer of data over the network from the send-side socket/TCP

II Receiving implementation

Sending implementation

97

Receiving program

Delivered

RecvQ

SendQ

5.2 Buffer Deadlock

m_ 1500 bytes

6000 bytes

500 bytes

n

First write (1000 bytes)

B

Second write (2000 bytes)

,i:::i~: Third write (5000 bytes) Figure 5.3: After first read().

I

Receiving program

Receiving implementation

Sending implementation

I SendQ

500 bytes

I

I

I Delivered

RecvQ

In

l

5500 bytes

2000 bytes

n

First write (1000 bytes) Second write (2000 bytes) ,, Third write (5000 bytes)

Figure 5.4: After another read().

implementation to the receive-side implementation. The movement of data from the SendQ to the RecvQ buffer has important implications for the design of application protocols. We have already encountered the need to parse messages as they are received via a Socket when inband delimiters are used for framing (see Section 3.3). In the following sections, we consider two more subtle ramifications.

5.2

Buffer Deadlock

Application protocols have to be designed with some care to avoid deadlock--that is, a state in which each peer is blocked waiting for the other to do something. For example, it is pretty obvious that if both client and server try to receive immediately after a connection is established, deadlock will result. Deadlock can also occur in less immediate ways. The buffers SendQ and RecvQ in the implementation have limits on their capacity. Although the actual amount of m e m o r y they use may grow and shrink dynamically, a hard limit is necessary to prevent all of the system's m e m o r y from being gobbled up by a single

98

Chapter 5: Under the Hood

II

TCP connection u n d e r control of a misbehaving program. Because these buffers are finite, they can fill up, and it is this fact, coupled with TCP's flow control mechanism, that leads to the possibility of another f o r m of deadlock. Once RecvQ is full, the TCP flow control m e c h a n i s m kicks in and prevents the transfer of any bytes f r o m the sending host's SendQ, until space b e c o m e s available in RecvQ as a result of the receiver calling the input stream's read() method. (The p u r p o s e of the flow control m e c h a n i s m is to ensure that the sender does not t r a n s m i t more data t h a n the receiving s y s t e m can handle.) A sending p r o g r a m can continue to call send until SendQ is full; however, once SendQ is full, a call to o u t . w r i t e ( ) will block until space b e c o m e s available, that is, until some bytes are t r a n s f e r r e d to the receiving socket's RecvQ. If RecvQ is also full, everything stops until the receiving p r o g r a m calls in. read () and some bytes are t r a n s f e r r e d to Delivered. Let's a s s u m e the sizes of SendQ and RecvQ are SQS and RQS, respectively. A w r i t e ( ) call with a byte array of size n such that n > SQS will not r e t u r n until at least n - SQS bytes have b e e n t r a n s f e r r e d to RecvQ at the receiving host. If n exceeds (SQS + RQS), w r i t e ( ) cannot r e t u r n until after the receiving p r o g r a m has read at least n - (SQS + RQS) bytes f r o m the input stream. If the receiving p r o g r a m does not call read(), a large send() m a y not complete successfully. In particular, if b o t h ends of the connection invoke their respective o u t p u t s t r e a m s ' w r i t e ( ) m e t h o d simultaneously with buffers greater than SQS + RQS, deadlock will result: neither write will ever complete, and b o t h p r o g r a m s will r e m a i n blocked forever. As a concrete example, consider a connection b e t w e e n a p r o g r a m on Host A and a p r o g r a m on Host B. A s s u m e SQS and RQS are 500 at b o t h A and B. Figure 5.5 shows what h a p p e n s w h e n b o t h p r o g r a m s try to send 1500 bytes at the same time. The first 500 bytes of data in the p r o g r a m at Host A have been t r a n s f e r r e d to the other end; another 500 bytes have b e e n copied into SendQ at Host A. The remaining 500 bytes cannot be s e n t - - a n d therefore out . w r i t e ( ) will not r e t u r n - - u n t i l space frees up in RecvQ at Host B. Unfortunately, the same situation holds in the p r o g r a m at Host B. Therefore, neither p r o g r a m ' s w r i t e ( ) call will ever complete.

send(s,buffer ,1500,0) ;

send(s,buffer,1500,O); To be sent

SendQ

RecvQ

Delivered

Delivered

RecvQ

SendQ

To be sent

I

I

I

I

Program

I

I

I

Implementation I

Host A

I

I

Implementation

I

!

I I

I

Program I

Host B

Figure 5.5: Deadlock due to simultaneous write()s to output streams at opposite ends of the connection.

m 5.2 Buffer Deadlock

99

The moral of the story: Design the protocol carefully to avoid sending large quantities of data simultaneously in both directions. Can this really happen? Let's review the compression protocol example in Section 4.5. Try running the compression client with a large file that is still large after compression. The precise definition of "large" here depends on your system, but a file that is already compressed and exceeds 2MB should do nicely. For each read/write, the compression client prints an "R"/"W" to the console. If both the u n c o m p r e s s e d and compressed versions of the file are large enough, your client will print a series of Ws and then stop without terminating or printing any Rs. Why does this happen? The program CompressClient. java sends all of the u n c o m p r e s s e d data to the compression server before it attempts to read anything from the compressed stream. The server, on the other hand, simply reads the u n c o m p r e s s e d byte sequence and writes the compressed sequence back to the client. (The n u m b e r of bytes the server reads before it writes some compressed data depends on the compression algorithm it uses.) Consider the case where SendQ and RecvQ for both client and server hold 500 bytes each and the client sends a 10,000-byte (uncompressed) file. Suppose also that for this file the server reads about 1000 bytes and then writes 500 bytes, for a 2:1 compression ratio. After the client sends 2000 bytes, the server will eventually have read t h e m all and sent back 1000 bytes, and the client's RecvQ and the server's SendQ will both be full. After the client sends another 1000 bytes and the server reads them, the server's subsequent attempt to write will block. When the client sends the next 1000 bytes, the client's SendQ and the server's RecvQ will both fill up. The next client write will block, creating deadlock. How do we solve this problem? The easiest solution is to execute the client writing and reading loop in separate threads. One thread repeatedly reads a buffer of u n c o m p r e s s e d bytes from a file and sends t h e m to the server until the end of the file is reached, whereupon it calls shutdown0utput() on the socket. The other thread repeatedly reads a buffer of compressed bytes from the server and writes them to the output file, until the input stream ends (i.e., the server closes the socket). When one thread blocks, the other thread can proceed independently. We can easily modify our client to follow this approach by putting the call to SendBytes() in CompressClient. java inside a thread as follows: Thread thread = new Thread() { public void run() { try { SendBytes(sock, fileIn); } catch (Exception ignored) {}

} }; thread.start(); See CompressClientNoDeadlock.java on the book's W e b site for the complete example. Can we solve this problem without using threads? To guarantee deadlock avoidance in a single threaded solution, we need nonblocking writes, which are not available in the current version of Java (see Section 4.2).

! OO

5.3

Chapter 5: Under the Hood

m

Performance Implications

The TCP implementation's need to copy user data into SendQ for potential retransrnission also has implications for performance. In particular, the sizes of the SendQ and RecvQ buffers affect the t h r o u g h p u t achievable over a TCP connection. Throughput refers to the rate at which bytes of user data from the sender are made available to the receiving program; in programs that transfer a large amount of data, we want to maximize this rate. In the absence of network capacity or other limitations, bigger buffers generally result in higher throughput. The reason for this has to do with the cost of transferring data into and out of the buffers in the underlying implementation. If you want to transfer n bytes of data (where n is large), it is generally much more efficient to call w r i t e ( ) once with a buffer of size n than it is to call it n times with a single byte. 1 However, if you call w r i t e ( ) with a size parameter that is much larger than SQS, the system has to transfer the data from the user address space in SQSsized chunks. That is, the socket implementation fills up the SendQ buffer, waits for data to be transferred out of it by the TCP protocol, refills SendQ, waits some more, and so on. Each time the socket implementation has to wait for data to be removed from SendQ, some time is wasted in the form of overhead (a context switch occurs). This overhead is comparable to that incurred by a completely new call to write(). Thus, the effective size of a call to w r i t e ( ) is limited by the actual SQS. For reading from the InputStream, the same principle applies: however large the buffer we give to read(), it will be copied out in chunks no larger than RQS, with overhead incurred between chunks. If you are writing a program for which throughput is an important performance metric, you will want to change the send and receive buffer sizes using the setSendBufferSize() and setReceiveBufferSize() methods of Socket. Although there is always a system-imposed m a x i m u m size for each buffer, it is typically significantly larger than the default on m o d e r n systems. Remember that these considerations apply only if your program needs to send an amount of data significantly larger than the buffer size, all at once. Note also that these factors may make little difference if the program deals with some higher-level stream derived from the Socket's basic input stream (say, by using it to create an instance of Filter0utputStream or PrintWriter), which may perform its own internal buffering or add other overhead.

5.4

TCP Socket Life Cycle

When a new instance of the Socket class is created--either via one of the public constructors or by calling the accept () method of a ServerSocket--it can immediately be used for sending and receiving data. That is, when the instance is returned, it is already connected to a remote peer and the opening TCP message exchange, or handshake, has been completed by the implementation.

1The same thing generally applies to reading data from the Socket's InputStream, although calling read() with a larger buffer does not guarantee that more data will be returned.

m

5.4 TCP S o c k e t Life Cycle

101

Call Socket (W.X.Y.Z, Q) Returns instance

Blocks

<

Closed 9

OJO .~-4 >-

Create structure

Local port

Fill in local a n d remote address

Established

Connecting Local port

P

Handshake completes

Local port

P

v

Local IP Remote port Remote IP

Local IP A.B.C.D Send connection Remote port Q r e q u e s t to server Remote IP W.X.Y.Z

Local IP A.B.C.D Remote port

Q

Remote IP W.X.Y.Z

Figure 5.6: Client-side connection establishment.

Let us therefore consider in more detail how the underlying structure gets to and from the connected, or "Established," state; as you'll see later (see Section 5.4.2), these details affect the definition of reliability and the ability to create a Socket or ServerSocket b o u n d to a particular port.

5.4.1 Connecting The relationship between an invocation of the Socket constructor and the protocol events associated with connection establishment at the client are illustrated in Figure 5.6. In this and the remaining figures of this section, the large arrows depict external events that cause the underlying socket structures to change state. Events that occur in the application p r o g r a m - that is, m e t h o d calls and returns--are shown in the upper part of the figure; events such as message arrivals are shown in the lower part of the figure. Time proceeds left to right in these figures. The client's Internet address is depicted as A.B.C.D, while the server's is W.X.Y.Z; the server's port number is Q. When the client calls the Socket constructor with the server's Internet address, W.X.Y.Z, and port, Q, the underlying implementation creates a socket instance; it is initially in the Closed state. If the client did not specify the local a d d r e s s / p o r t in the constructor call, a local port number (P), not already in use by another TCP socket, is chosen by the implementation. The local Internet address is also assigned; if not explicitly specified, the address of the network interface through which packets will be sent to the server is used. The implementation copies the local and remote addresses and ports into the underlying socket structure, and initiates the TCP connection establishment handshake. The TCP opening handshake is known as a 3-way handshake because it typically involves three messages: a connection request from client to server, an acknowledgment from server to client, and another acknowledgment from client back to server. The client TCP considers the connection to be established as soon as it receives the acknowledgment from the server.

| 02

Chapter 5: Under the Hood

III

Call ServerS0cket (Q)

r

I)

L~

~o

Returns instance

<

Listening

Closed o

Create structure

.~--4

Local port

Local port

Q

Local IP

*

Remote port

Remote port

*

Remote IP

Remote IP

*

Local IP

Fill in local port, set state

Figure 5.7: Server-side socket setup.

In the n o r m a l case, this h a p p e n s quickly. However, the Internet is a best-effort network, and either the client's initial m e s s a g e or the server's r e s p o n s e can get lost. For this reason, the TCP i m p l e m e n t a t i o n r e t r a n s m i t s h a n d s h a k e m e s s a g e s multiple times, at increasing intervals. If the client TCP does not receive a r e s p o n s e f r o m the server after some time, it times out and gives up. In this case the constructor throws an IOExeeption. The connection timeout is generally long, and thus it can take on the order of m i n u t e s for a Socket ( ) constructor to fail. If the server is not accepting connections--say, if there is no p r o g r a m associated with the given port at the d e s t i n a t i o n - - t h e server-side TCP will send a rejection m e s s a g e instead of an acknowledgment, and the c o n s t r u c t o r will throw an IOException almost immediately. The sequence of events at the server side is rather different; we describe it in Figures 5.7, 5.8, and 5.9. The server first creates an instance of ServerSocket associated with its well-known port (here, Q). The socket i m p l e m e n t a t i o n creates an underlying socket structure for the new ServerSocket instance, and fills in Q as the local port and the special wildcard address C*" in the figures) for the local IP address. (The server m a y also specify a local IP a d d r e s s in the constructor, b u t typically it does not. In case the server host has more than one IP address, not specifying the local a d d r e s s allows the socket to receive connections a d d r e s s e d to any of the server host's addresses.) The state of the socket is set to "Listening", indicating that it is ready to accept incoming connection requests a d d r e s s e d to its port. This sequence is depicted in Figure 5.7. The server can now call the ServerSocket's accept () method, which blocks until the TCP opening h a n d s h a k e has been c o m p l e t e d with some client and a new connection has b e e n established. We therefore focus in Figure 5.8 on the events that occur in the TCP i m p l e m e n t a t i o n w h e n a client connection request arrives. Note that everything depicted in this figure h a p p e n s "under the covers," in the TCP implementation.

[]

Listening Local port

o

Incoming connection request from A.B.C.D/P

5.4 TCP Socket Life Cycle

Listening

Listening Q

Local port

1 O3

Local port

Q

Local IP

Local IP

Local IP

*

Remote port

Remote port

Remote port

*

Remote IP

Remote IP

Remote IP

*

[ C!e _te_ ew_ stTytyf _ .................. and continue handshake

Established

Connecting Local port

Q

Local IP W.X.Y.Z Remote port

P

Remote IP A.B.C.D

Handshake completes

Local port

Q

Local IP W.X.Y.Z Remote port

P

Remote IP A.B.C.D

Figure 5.8: Incoming connection request processing.

When the r e q u e s t for a c o n n e c t i o n arrives f r o m the client, a new socket structure is created for the connection. The new socket's a d d r e s s e s are filled in b a s e d on the arriving packet: the p a c k e t ' s d e s t i n a t i o n Internet a d d r e s s a n d p o r t (W.X.Y.Z a n d Q, respectively) b e c o m e the local Internet a d d r e s s a n d port; the p a c k e t ' s source a d d r e s s a n d p o r t (A.B.C.D a n d P) b e c o m e the r e m o t e Internet a d d r e s s a n d port. Note that the local p o r t n u m b e r of the new socket is always the same as that of the ServerSocket. The new socket's state is set to "Connecting", and it is a d d e d to a list of n o t - q u i t e - c o n n e c t e d sockets a s s o c i a t e d with the socket s t r u c t u r e of the ServerSocket. Note that the ServerSocket itself does not change state, nor does any of its a d d r e s s i n f o r m a t i o n change. In a d d i t i o n to creating a new u n d e r l y i n g socket structure, the server-side TCP implementation sends an acknowledging TCP h a n d s h a k e m e s s a g e back to the client. However, the server TCP does not consider the h a n d s h a k e c o m p l e t e until the third m e s s a g e of the 3-way h a n d s h a k e is received f r o m the client. When that m e s s a g e eventually arrives, the new s t r u c t u r e ' s state is set to "Established", and it is t h e n (and only then) m o v e d to a list of socket s t r u c t u r e s associated with the ServerSocket structure, which r e p r e s e n t e s t a b l i s h e d c o n n e c t i o n s ready to be accept ( ) e d via the ServerSoeket. (If the third h a n d s h a k e m e s s a g e fails to arrive, eventually the "Connecting" s t r u c t u r e is deleted.)

104

Chapter 5: Under the Hood

II

Call accept () 4-a e~

Returns Socket instance for this structure

Blocks until new connection is established

<

Listening

Listening

o

Local port

Q

Local IP

Events of Figure 5.8

Listening

Local port

Q

Local port

Q

*

Local IP

*

Local IP

*

Remote port

*

Remote port

*

Remote port

*

Remote IP

*

Remote IP

*

Remote IP

*

4-a

/

Established Local port

Q

Local IP W.X.Y.Z Remote port

P

Remote IP A.B.C.D

Figure 5.9: accept () processing.

Now we can c o n s i d e r (in Figure 5.9) w h a t h a p p e n s w h e n the server p r o g r a m calls the S e r v e r S o e k e t ' s accept () m e t h o d . The call u n b l o c k s as s o o n as t h e r e is s o m e t h i n g in its associa t e d list of socket s t r u c t u r e s for n e w c o n n e c t i o n s . (Note t h a t this list m a y already be n o n - e m p t y w h e n accept () is called.) At t h a t time, one of the n e w c o n n e c t i o n s t r u c t u r e s is r e m o v e d f r o m the list, a n d an i n s t a n c e of Socket is c r e a t e d for it a n d r e t u r n e d as the r e s u l t of the accept (). It is i m p o r t a n t to n o t e t h a t each s t r u c t u r e in the ServerSocket's a s s o c i a t e d list r e p r e s e n t s a fully e s t a b l i s h e d TCP c o n n e c t i o n with a client at the o t h e r end. Indeed, the client can s e n d d a t a as s o o n as it receives the s e c o n d m e s s a g e of the o p e n i n g h a n d s h a k e - - w h i c h m a y be long b e f o r e the server calls accept () to get a Socket i n s t a n c e for it.

5.4.2

Closing aTCP Connection

TCP h a s a graceful close m e c h a n i s m t h a t allows applications to t e r m i n a t e a c o n n e c t i o n w i t h o u t h a v i n g to w o r r y a b o u t loss of d a t a t h a t m i g h t still be in transit. The m e c h a n i s m is also d e s i g n e d to allow d a t a t r a n s f e r s in each d i r e c t i o n to be t e r m i n a t e d i n d e p e n d e n t l y , as in the

II

| O5

5.4 TCP Socket Life Cycle

Call close()/shutdownOutput() Returns immediately

Established

Closing

o

Local port Local IP

Remote port

P A.B.C.D Q

Remote IP W.X.Y.Z

Start close handshake

Local port

Half closed P

Local IP A.B.C.D Remote port

Q

-- R e m o t e IP W.X.Y.Z

Close handshake completes

Time-Wait

Close handshake Local port P Local port 9 "i n i t i a t e d b y Local IP A.B.C.D r e m o t e Local IP completes Remote port Q Remote port

Remote IP W.X.Y.Z

P A.B.C.D Q

Remote IP W.X.Y.Z

Figure 5. ! O: Closing a TCP connection first.

compression example of Section 4.5. It works like this: the application indicates that it is finished sending data on a connected socket by calling c l o s e ( ) or by calling shutdown0utput (). At that point, the underlying TCP implementation first transmits any data remaining in SendQ (subject to available space in RecvQ at the other end), and then sends a closing TCP handshake message to the other end. This closing handshake message can be thought of as an end-oftransmission marker: it tells the receiving TCP that no more bytes will be placed in RecvQ. (Note that the closing handshake message itself is not passed to the receiving application, but that its position in the byte stream is indicated by read() returning -1.) The closing TCP waits for an acknowledgment of its closing handshake message, which indicates that all data sent on the connection made it safely to RecvQ. Once that acknowledgment is received, the connection is "Half closed." It is not completely closed until a symmetric handshake happens in the other direction--that is, until both ends have indicated that they have no more data to send. The closing event sequence in TCP can happen in two ways: either one application calls c l o s e ( ) (or shutdown0utput()) and completes its closing handshake before the other calls c l o s e ( ) , or both call c l o s e ( ) simultaneously, so that their closing handshake messages cross in the network. Figure 5.10 shows the sequence of events in the implementation when the application invokes c l o s e ( ) before the other end closes. The closing handshake message is sent, the state of the socket structure is set to "Closing", and the call returns. After this point, further reads and writes on the Socket are disallowed (they throw an exception). When the acknowledgment for the close handshake is received, the state changes to "Half closed", where it remains until the other end's close handshake message is received. Note that if the remote endpoint goes away while the connection is in this state, the local underlying structure will stay around indefinitely. When the other end's close handshake message arrives, an acknowledgment is sent and the state is changed to "Time-Wait". Although the corresponding Socket instance in the application program may have long since vanished, the associated underlying structure continues to exist in the implementation for a minute or more; the reasons for this are discussed on page 107.

| 06

Chapter 5: Under the Hood

[]

Call close()

.,-.~

.,--4

Returns immediately

< Established Local port Local IP A.B.C.D .F..~

Remote port

Close-Wait

Close

o

Q

handshake initiated by remote completes

Local port

P Finish close handshake,

Local IP A.B.C.D delete structure Remote port

Remote IP W.X.Y.Z

Q

y

Remote IP W.X.Y.Z

Figure 5.1 !: Closing after the other end closes.

Figure 5.11 shows the simpler sequence of events at the endpoint that does not close first. When the closing handshake message arrives, an acknowledgment is sent immediately, and the connection state becomes "Close-Wait." At this point, we are just waiting for the application to invoke the Socket's close() method. When it does, the final close handshake is initiated and the underlying socket structure is deallocated, although references to its original Socket instance may persist in the Java program. In view of the fact that both close() and shutdown0utput() return without waiting for the closing handshake to complete, you may wonder how the sender can be assured that sent data has actually made it to the receiving program (i.e., to Delivered). In fact, it is possible for an application to call close() or shutdown0utput() and have it complete successfully (i.e., not throw an Exception) while there is still data in SendQ. If either end of the connection then crashes before the data makes it to RecvQ, data may be lost without the sending application knowing about it. The best solution is to design the application protocol so that the side that calls close() first does so only after receiving application-level assurance that its data was received. For example, when our TCPEehoClient program receives the echoed copy of the data it sent, there should be nothing more in transit in either direction, so it is safe to close the connection. Java does provide a way to modify the behavior of the Socket's close() method, namely, the setSoLinger() method, setSoLinger() controls whether close() waits for the closing handshake to complete before returning. It takes two parameters, a boolean that indicates whether to wait, and an integer specifying the number of seconds to wait before giving up. That is, when a timeout is specified via setSoLinger(), close() blocks until the closing handshake is completed, or until the specified amount of time passes. At the time of this writing, however, close() provides no indication that the closing handshake failed to complete, even if the time limit set by setSoLinger() expires before the closing sequence completes. In other words, setSoLinger() does not provide any additional assurance to the application in current implementations.

m 5.5 Demultiplexing Demystified

! O7

The final subtlety of closing a TCP connection revolves around the need for the TimeWait state. The TCP specification requires that when a connection terminates, at least one of the sockets persists in the Time-Wait state for a period of time after both closing handshakes complete. This requirement is motivated by the possibility of messages being delayed in the network. If both ends' underlying structures go away as soon as both closing handshakes complete, and a n e w connection is immediately established between the same pair of socket addresses, a message from the previous connection, which h a p p e n e d to be delayed in the network, could arrive just after the new connection is established. Because it would contain the same source and destination addresses, the old message could be mistaken for a message belonging to the new connection, and its data might (incorrectly) be delivered to the application. Unlikely though this scenario may be, TCP employs multiple mechanisms to prevent it, including the Time-Wait state. The Time-Wait state ensures that every TCP connection ends with a quiet time, during which no data is sent. The quiet time is supposed to be equal to twice the m a x i m u m a m o u n t of time a packet can remain in the network. Thus, by the time a connection goes away completely (i.e., the socket structure leaves the Time-Wait state and is deallocated) and clears the way for a new connection between the same pair of addresses, no messages from the old instance can still be in the network. In practice, the length of the quiet time is implementation dependent, because there is no real m e c h a n i s m that limits how long a packet can be delayed by the network. Values in use range from 4 minutes down to 30 seconds or even shorter. The m o s t important consequence of Time-Wait is that as long as the underlying socket structure exists, no other socket is permitted to be associated with the same local port. In particular, any attempt to create a Socket instance using that port will throw an TOException.

5.5

Demultiplexing Demystihed

The fact that different sockets on the same machine can have the same local address and port n u m b e r is implicit in the discussions above. For example, on a machine with only one IP address, every new Socket instance accept ()ed via a ServerSocket will have the same local port n u m b e r as the ServerSocket. Clearly the process of deciding to which socket an incoming packet should be delivered--that is, the d e m u l t i p l e x i n g process--involves looking at more than just the packet's destination address and port. Otherwise there could be ambiguity about which socket an incoming packet is intended for. The process of matching an incoming packet to a socket is actually the same for both TCP and UDP, and can be s u m m a r i z e d by the following points: 9 The local port in the socket structure m u s t ma tc h the destination port n u m b e r in the incoming packet. 9 Any address fields in the socket structure that contain the wildcard value (*) are considered to m a t c h a n y value in the corresponding field in the packet. 9 If there is more than one socket structure that matches an incoming packet for all four address fields, the one that matches using the fewest wildcards gets the packet.

| O~

Chapter 5" Under the Hood

Listening

Local port

II

Listening

99

Local port

Established

99 10.1.2.3

Local port

Local IP

Local IP

Remote port

Remote port

Remote port

Remote IP

Remote IP

Remote IP

0

1

2

Established

99

Local IP 192.168.3.2 30001 172.16.1.9

Local port Local IP Remote port Remote IP

1025 10.1.2.3 25 10.5.5.8

Figure 5.12: Demultiplexing with multiple matching sockets.

For example, consider a host with two IP addresses, 10.1.2.3 and 192.168.3.2, and with a subset of its active TCP socket structures, as shown in Figure 5.12. The structure labeled 0 is associated with a ServerSocket and has port 99 with a wildcard local address. Socket structure 1 is also for a ServerSocket on the same port, but with the local IP address 10.1.2.3 specified (so it will only accept connection requests to that address). Structure 2 is for a connection that was accepted via the ServerSocket for structure 0, and thus has the same local port number, but also has its local and remote Internet addresses filled in. Other sockets belong to other active connections. Now consider a packet with source IP address 172.16.1.10, source port 56789, destination IP address 10.1.2.3, and destination port 99. It will be delivered to the socket associated with structure 1, because that one matches with the fewest wildcards. When a program attempts to create a socket with a particular local port number, the existing sockets are checked to make sure that no socket is already using that local port. A Socket () constructor will throw an exception if a n y socket matches the local port and local IP address (if any) specified in the constructor. This can cause problems in the following scenario: 1. A client p r o g r a m creates a Socket with a specific local port number, say, P, and uses it to communicate with a server. 2. The client closes the Socket, and the underlying structure goes into the Time-Wait state. 3. The client program terminates and is immediately restarted. If the new incarnation of the client attempts to use the same local port number, the Socket constructor will throw an IOException, because of the other structure in the Time-Wait state. As of this writing, the only way around this is to wait until the underlying structure leaves the Time-Wait state. So what determines the local/foreign address/port? For a ServerSocket, all constructors require the local port. The local address may be specified to the constructor; otherwise, the local address is the wildcard (*) address. The foreign address and port for a ServerSocket are always wildcards. For a Socket, all constructors require specification of the foreign address and port. The local address a n d / o r port may be specified to the constructor. Otherwise, the local address is the address of the network interface through which the connection to the server is established, and the local port is a randomly selected, unused port number greater

I

5.6 Exercises

109

than 1023. For a Socket instance returned by accept(), the local address is the destination address from the initial handshake message from the client, the local port is the local port of the ServerSocket, and the foreign a d d r e s s / p o r t is the local a d d r e s s / p o r t of the client. For a DatagramSocket, the local address a n d / o r port may be specified to the constructor. Otherwise the local address is the wildcard address, and the local port is a randomly selected, unused port number greater than 1023. The foreign address and port are initially both wildcards, and remain that way unless the connect () m e t h o d is invoked to specify particular values.

5.6

Exercises

1. The TCP protocol is designed so that simultaneous connection attempts will succeed. That is, if an application using port P and Internet address W.X.Y.Z attempts to connect to address A.B.C.D, port Q, at the same time as an application using the same address and port tries to connect to W.X.Y.Z, port P, they will end up connected to each other. Can this be made to happen when the programs use the sockets API? 2. The first example of "buffer deadlock" in this chapter involves the programs on both ends of a connection trying to send large messages. However, this is not necessary for deadlock. How could the TCPEchoClient from Chapter 2 be made to deadlock when it connects to the TCPEchoServer from that chapter?

Bibliography [1] Case, J. D., Fedor, M., and Schoffstall, M. L., "Simple Network Management Protocol (SNMP)." Internet Request for Comments 1157, May 1990. [2] Comer, Douglas E., Internetworking with TCP/IP, Volume k Principles, Protocols, and Architecture (third edition). Prentice Hall, 1995. [3] Comer, Douglas E., and Stevens, David L., Internetworking with TCP/IP, Volume II: Design, Implementation, and Internals (third edition). Prentice Hall, 1999. [4] Comer, Douglas E., and Stevens, David L., Internetworking with TCP/IP, Volume III: ClientServer Programming and Applications (BSD version, second edition). Prentice Hall, 1996. [5] Deering, S., and Hinden, R., "Internet Protocol, Version 6 (IPv6) Specification." Internet Request for Comments 2460, December 1998. [6] Gilligan, R., Thomson, S., Bound, J., and Stevens, W., "Basic Socket Interface Extensions for IPv6." Internet Request for Comments 2553, March 1999. [7] Hughes, M., Shoffner, M., Hamner, D., and Bellus, U., Java Network Programming (second edition). Manning, 1999. [8] International Organization for Standardization, Information Processing SystemsmOpen Systems InterconnectionmSpecification of Abstract Syntax Notation One (ASN.1). International Standard 8824, December 1987. [9] Mockapetris, P., "Domain Names--Concepts and Facilities." Internet Request for Comments 1034, November 1987. [10] Mockapetris, P., "Domain Names--Implementation and Specification." Internet Request for Comments 1035, November 1987. [11] Peterson, L. L., and Davie, B. S., Computer Networks: A Systems Approach (second edition). Morgan Kaufmann, 2000. [12] Postel, J., "Internet Protocol." Internet Request for Comments 791, September 1981.

III

| | 2

Bibliography m

[13] Postel, J., "Transmission Control Protocol." Internet Request for Comments 793, September 1981. [14] Postel, J., "User Datagram Protocol." Internet Request for Comments 768, August 1980. [15] Steedman, D., Abstract Syntax Notation One (ASN.1)--The Tutorial and Reference. Technology Appraisals, U.K., 1990. [16] Stevens, W. R., TCP/IP Illustrated, Volume 1: The Protocols. Addison-Wesley, 1994. [17] Stevens, W. R., UNIX Network Programming: Networking APIs: Sockets and XTI (second edition). Prentice Hall, 1997. [18] Sun Microsystems Incorporated, "External Data Representation Standard." Internet Request for Comments 1014, June 1987. [19] Sun Microsystems Incorporated, "Network File System Protocol Specification." Internet Request for Comments 1094, March 1989. [20] Sun Microsystems Incorporated, "Network File System Protocol Version 3 Specification." Internet Request for Comments 1813, June 1995. [21] The Unicode Consortium, The Unicode Standard, Version 3.0. Addison-Wesley Longman, 2O0O. [22] Wright, G. R., and Stevens, W. R., TCP/IP Illustrated, Volume 2: The Implementation. Addison-Wesley, 1995.

Index

3-way handshakes, 101-102 accept () method of ServerSocket blocking by, 75-76 connection establishment events, 103-104 described, 21 returning Socket class, 18-19 in thread-per-client server, 70 in thread pool server, 71-73 addresses broadcast, 79 connection establishment, 101 defined, 3 demultiplexing, 107-109 destination addresses, 9 dotted-quad notation, 4 IP (Internet), 9-12, 93-94 multicast, 80 socket structures, 93-94 sockets, 9-12 types of, 4 American Standard Code for Information Exchange (ASCII), 39 applets, 91 application protocols, 37, 42-43 applications, 5 architecture of TCP/IP networks, 2 big-endian byte order, 41 binary numbers, 40-42 blocking socket I/O calls, 75-79 boolean values, encoding, 47, 51 broadcast addresses, 79 broadcasting, 79-80, 84

browser applet security, 91 BufferedInputStream, 42 BufferedOutputStream, 42 buffering closing connections, 105106 of datagrams, 24-25 deadlock, 97-99 FIFO queues, 96-97 flow control mechanism, 98 I/O streams, 42 memory limitations, 97-98 performance, effects on, 100 reliable service protocol, 94-97 setting size, 17 TCP and, 94-99 write() method with, 20, 76 byte order, 41 bytes required for transmission, 39 character sets, 39 client closing connections, 85-90 defined, 5 handshake events, 101-102 TCP, 12-18 TCPEchoClient. java, 13-15 UDP, 26-31 UDPEchoClientTimeout. java, 27-28 close() method closing connections, 85-90, 105-107 of DatagramSocket class, 26, 29,31 setSoLinger() method, 106

of Socket class, 15, 16 Closed state, 94, 101 Close-Wait state, 106 closing connections, 85-90, 104-107 TCP streams, 16 UDP streams, 29 Closing state, 105 communication channels, 1 composing I/O streams, 42 CompressClient. java, 87-89 CompressClientNoDeadlock. java, 99 compression protocol closing connections, 86-90 deadlock, 99 CompressProtocolFactory. java, 89-90 concurrent servers. See multitasking connect () method UDP, 26 in DatagramSocket, 29 Connecting state, 103 connection-oriented protocols, 3 connections Close-Wait state, 106 closing, 85-90, 104-107 Closing state, 105 Connecting state, 103 creating, 14, 18, 28-30 disconnecting, 30 dispatching, 69-75 Established state, 103 establishment events, 101-104 Half closed state, 105 memory limitations, 97-98 Time-Wait state, 105,107

113

1 I 4

Index

m

connections (continued) write/read relationship, 95-96 ConsoleLogger. java, 66 corruption of data, UDP treatment of, 23 createProtocol(), 69 creating TCP sockets, 13-21 UDP sockets, 26-33 datagram service, 3, 23, 24-26 datagram sockets, 6. See also DatagramSocket class DatagramPacket class, 24-26, 33-34, 39, 43, 46, 49 datagrams creation, 28, 32 encoding information for, 57-58 lost, 26 maximum size, 33 multicasting, 80-84 sending, 28-29 TTL (Time To Live) values, 81, 83 DatagramSocket class accessors/mutators, 30-31 close() method, 26, 29, 31 constructors, 29 instancing by clients, 26 methods, 29-30 MulticastSocket subclass, 80-84 receive() method, 24, 26, 29, 31-34, 75-76 send() method, 24, 26, 28, 31-34, 98 DataInput interface, 41 DataInputStream, 42 Data0utput interface, 41 Data0utputStream, 42 deadlocked buffers, 97-99 defaults, socket, 84-85 delimiters, 43-44, 47 Delivered, 96-97 demultiplexing algorithm, 107-109 destination addresses, 9 directed broadcast addresses, 79 disconnect () method, 30 Dispatcher. java, 69-70 dispatching, 69-75 Domain Name System (DNS), 5 dotted-quad notation, 4 echo servers EchoProtocolYactory. java, 69-70

EchoProtocol. java, 63-65 TCP version, 13-14 UDP version, 31-32 encode() method, 49 encoding of information, 39-59 application protocols, 42-43 boolean values, 47, 51 character sets, 39-40 combined data representation, 51-54 composing I/O streams, 42 delimiters, 43-44, 47 framing, 42-46 integer types, 40-41 serialization capabilities, 58-59 TCP implementation, 55- 57 text data, 39-40, 47-51 UDP socket implementation, 57-58 end-of-message markers, 43 end-of-stream, 22, 43, 45, 46, 59,87 end-to-end transport protocols, 3 Established state, 103 exceptions handshake timeouts, 102 multiple sockets with same address, 108 security, 91 thread errors, 68 explicit-length fields, 43 factories, 68-71, 75, 89-90 factoring servers, 68-71 fields defined, 37 explicit-length, 43 YileClientApplet. java, 91 FileLogger. java, 66-67 File Transfer Protocol (FTP), 6 flow control mechanism, TCP, 98 flush() method, 22 Framer. java, 45-46 framing, 42-46 getByName() method, 9-12 getBytes() methods, 40 getData() method, 25, 33-34 getInetAddress() method, 19 getLocalHost() method, 9-12 getPort () method, 19 getProperties() method, 73 graceful close mechanism, 104-107 GZIPOutputStream, 43, 89, 90 Half closed state, 105

handshake messages closing, 105-107 defined, 3 establishing connections, 100-104 hosts, 1 Hypertext Transfer Protocol (HTTP) closing connections, 85-86 purpose of, 2 images, multicasting, 82 information definition of, 2 encoding of. See encoding of information InetAddress class, 9-12 InetAddressExample. java, 9-11 input/output (I/O) buffering, 42 closing. See close() method input. See input streams nonblocking, 75-79 output. See output streams shutdown methods, 86-89, 105-106 input streams closing. See close() method composing, 42 creating, 21-23 framing, 42-46 Java classes, table of, 43 shutdownInput (), 86-89 TCP implementation, 55-56 write/read relationship, 96 InputStream, 13-14, 19, 21-23, 42 integer types, 40-41 internationalization, Java support for, 39 Internet addresses, 4, 9-12, 93-94 Internet Protocol (IP), 2-3 IP addresses, 9-12, 93-94 ISO Latin I, 40 ItemQuoteSinConst. java, 51 ItemQuoteDecoderSin. java, 53-54 ItemQuoteDecoder. java, 47 ItemQuoteDecoderText. java, 49-51 ItemQuoteEncoderBin. java, 51-53 TtemQuoteEncoder. java, 46 ItemQuot eEnc oderText, java, 48-49 ItemQuote. java, 38 ItemQuoteTextConst. java, 48 iterative servers, 61

III Index

joinGroup() method, 81.83 joining a group, 81-82 keepalive message behavior, 16 layers of TCP/IP, 2-3 leaveGroup() method, 83 length of datagrams, setting, 25 lengths of messages, 43 lingering, 17 little-endian byte order, 41, 44 local broadcast addresses, 79 local host IP addresses, obtaining, 9-10 Logger. j ava, 65-66 logging, 64-67 loopback address, 4 memory limitations, deadlocks from, 97-98 message boundaries not preserved by TCP, 15 preserved by UDP, 23 messages defined, 3 7 delimiters, 43-44, 47 encoding. See encoding of information framing, 42-43 multicast groups, 81-82 multicasting, 80-84 MulticastSoeket class, 80-84 multiplexing, demultiplexing process, 107-109 multitasking, 61-75 factoring servers, 68-71, 74-75 nonblocking I/O, 75-79 pooled threads, 61, 71-75 server protocol, 63-67 thread-per-client, 67-68 threads. See threads Nagle's algorithm, 17 names of Internet hosts, 4-5, 10-11 network byte order, 41 network layer, 3 network protocols. See protocols; Transmission Control Protocol (TCP); User Datagram Protocol (UDP) networks, 1 nextToken() method, 44-46 nonblocking I/O, 75-79 numbers, transmitting, 40-42 options, socket, 84-85

output streams composing, 42 defined, 21-23 flushing, 94 framing, 42-46 Java classes, table of, 43 shutdown0utput (), 86-89 TCP implementation, 55-56 write/read relationship, 95-96 0utputStream, 13-14, 19, 21-23, 42 OutputStreamWriter, 40 packets addresses, 3 defined, 2 message boundaries, 15, 23 Time To Live (TTL) values, 81, 83 parsing, 37, 43-45 peers, 5 PoolDispatcher. java, 71-73 port numbers defined, 3-4 finding by client, 5-6 getPort () method, 19 multiple sockets with, 107-109 socket structures, 93-94 price quote information example, 37-38 PrintWriter, 1 O0 properties, 73 ProtocolFactory. java, 69, 77-79

protocols closing connections, 85-90 compression. See compression protocol defined, 2 factoring, 68-71, 74-75 HTTP, 2, 85-86 IP, 2-3 TCP. See Transmission Control Protocol (TCP) in TCP/IP suite, 2 timing out, 76-77 UDP. See User Datagram Protocol (UDP) queues, data, 94-99 read () method blocking by, 75-76 correspondence with write(), 96 data at server, 20 end-of-stream indication, 43

115

grouping data with TCP socket, 96 maximum timeout, setting, 17 message boundaries, 15 performance vs. buffer size, 100 syntax, 22 Reader class delimiters, 43-44 receive() method, 24, 26, 29-34, 75-76 receiving data. See input stream RecvQ, 96-99, 105-106 RecvTCP. java, 56-57 RecvUDP.java, 58 RecvUDPMulticast. java, 82-83 reliable byte-stream channels, 3, 94-97, 100 Remote Method Invocation (RMI), 91 retransmission of packets. See reliable byte-stream channels routers, 1-2 run() method, 62-65 Runnable interface, 62-65, 69 security of applets, 91 send () method buffer limits, 98 DatagramSocket class, 24, 26, 28, 31-34 sending data. See output streams SendQ, 96-99, 105-106 SendTCP. java, 55-56 SendUPD. j ava, 57 SendUDPMulticast. java, 80-81 Serializable interface, 58-59 serialization capabilities, 58-59 servers closing connections, 85-90 compression, 86-90 concurrent. See multitasking defined, 5 factoring, 68-71, 74-75 handshake events, 102-104 iterative, 61 loggers, 64-67 port numbers, 5-6 TCP. See TCP servers; TCP sockets text-encoded messages, receiving, 56-57 thread-pool, 71-75 UDP, 31-33. See also UDP sockets ServerSocket class accept() method, 18-19, 75-76, 103-104, 109

I I6

Index

[]

ServerSocket class (continued) constructing server with, 18-21 constructors, 20-21 demultiplexing, 107-109 establishment of connections, 103 methods, 21 purpose of, 12 setReceiveBufferSize() method, 100 setSendBufferSize() method, 100 setSoLinger() method, 106 setSoTimeout() method, 76-77 shutdownlnput () method, 86 shutdown0utput () method, 86-89, 105-106 signed integers, 41 Socket class, 12-18 accept () method creating, 18-19 accessors, 16-17 blocked I/O, 76 buffer size, methods for setting, 100 connection establishment events, 101-104 constructors, 15-16, 108 creating socket instances, 13-14, 18-19 demultiplexing, 107-109 getlnetAddress() method, 19 getPort() method, 19 implementation, underlying, 18 instantiation by accept () at server, 18 methods, 16 shutdown methods, 86-89 socket options, 84-85 sockets addresses, 9-12 creating, 13-14, 18-19 defaults, 84-85 defined, 6 identification, 6 servers. See ServerSocket class UDP. See UDP sockets stream sockets, 6 streams composition, 42 decoding, 54 encoding. See encoding of information end-of-stream indication, 43 input. See input streams

interface integer methods, 41 output. See output streams structures, socket connection establishment events affecting, 101-104 fields, multiplexing, 107-108 Half closed state, 105 Internet addresses of sockets, 93-94 ports of sockets, 93-94 protocol state information, 94 queues of data, 94 Socket instances, 93-94 wildcard values, 107-109 system properties, 73-74 TCP clients, 12-18 TCP connections closing, 15, 85-90, 104-10 7 defined, 12 end-of-stream indication, 43 establishing, 12-13 TCPEchoClient. java, 13-15 TCP servers, 18-21 TCP sockets buffering, 94-97 closing connections, 104-107 data structures, 94 input/output streams, 21-23 reliable service requirements, 3, 94-97, 100 ServerSocket class, 20-21 socket class, 15-18 TCP clients, 12-18 TCP servers, 18-21 write/read relationship, 96 TCPEchoClient. java, 13-15 TCPEchoServer. java, 18-20 TCPEchoServerThread. java, 67-68 TfiPFileServer. java, 91 text data, 39-40, 47-51 thread pools, 61, 71-75 ThreadExample. java, 62-63 ThreadMain. java, 74-75 thread-per-client, 61, 67-68 ThreadPerDispatcher. java, 70 threads, 61- 75 creating, 62, 68 deadlock, avoiding with, 99 exceptions, 68 nonblocking I/O, 75-79 resources consumed, 71 reusing. See thread pools Runnable interface, 62-65, 69 server protocol, 63-67 suspending, 63

thread-per-client, 61, 67-68 time, maximum blocking, 76-79 watchdog, 78-79 TimeLimitEchoProtocol, 77-78 Time Limi tEcho Pro toc oIF actory. java, 77-79

time, m a x i m u m blocking, 76-79 Time To Live (TTL) values, 8 i, 83 Time-Wait state, 105,107 timing out of handshakes, 102 Transmission Control Protocol (TCP), 2-3. See also TCP sockets transport layer, 3 two's-complement representation, 41

UDP clients, 26-31 UDP sockets, 23-35 creating, 28 DatagramPacket class, 24-26 encoding information for, 57-58 getData() method, 25, 33-34 I/O with, 33-34 lost datagrams, 26 multicasting, 80-84 sending datagrams, 28-29 UDP clients, 26-31 UDP servers, 31-33 UDPEchoClientTimeout. java, 27-29 UDPEchoServer. java, 31-33 vs. TCP sockets, 23 UDPEchoClientTimeout. java, 27-29 UDPEchoServer. java, 31-33 unicast, 79 Unicode encodings, 39, 44-45 unsigned integers, 41 User Datagram Protocol (UDP) datagram sockets, 6 functions of, 23 part of TCP/IP, 2 purpose of, 3 wildcard values, 107-109 write() method blocking by, 75-77, 98 buffer parameters, 20 correspondence with read (), 96 message boundaries, 15 performance vs. buffer size, 100 syntax, 22