www.acm.org/crossroads
Spring 2010 • Issue 16.3
Spring 2010—Issue 16.3
CROSSROADS STAFF EDITOR-IN-CHIEF Chris Harrison, Carnegie Mellon University DEPARTMENTS CHIEF Tom Bartindale, University of Newcastle EDITORS Ryan K.L. Ko, Nanyang Technological University James Stanier, University of Sussex Malay Bhattacharyya, Indian Statistical Institute Inbal Talgam, Weizmann Institute of Science Sumit Narayan, University of Connecticut DEPARTMENT EDITORS Daniel Gooch, University of Bath David Chiu, Ohio State University Rob Simmons, Carnegie Mellon University Michael Ashley-Rollman, Carnegie Mellon University Dima Batenkov, Weizmann Institute of Science COPY CHIEF Erin Claire Carson, University of California, Berkeley COPY EDITORS Leslie Sandoval, University of New Mexico Scott Duvall, University of Utah Andrew David, University of Minnesota ONLINE EDITORS Gabriel Saldaña, Instituto de Estudios Superiores de Tamaulipas, Mexico Srinwantu Dey, University of Florida MANAGING EDITOR AND PROFESSIONAL ADVISOR Jill Duffy, ACM Headquarters INSTITUTIONAL REVIEWERS Ernest Ackermann, Mary Washington College Peter Chalk, London Metropolitan University Nitesh Chawla, University of Notre Dame José Creissac Campos, University of Minho Ashoke Deb, Memorial University of Newfoundland Steve Engels, University of Toronto João Fernandes, University of Minho Chris Hinde, Loughborough University Michal Krupka, Palacky University Piero Maestrini, ISTI-CNR, Pisa José Carlos Ramalho, University of Minho Suzanne Shontz, Pennsylvania State University Roy Turner, University of Maine Ping-Sing Tsai, University of Texas—Pan American Andy Twigg, University of Cambridge Joost Visser, Software Improvement Group Tingkai Wang, London Metropolitan University Charles Won, California State University, Fresno OFFERING #XRDS0163 ISSN#: 1528-4981 (PRINT) 1528-4982 (ELECTRONIC) Front cover image courtesy of Opte Project.
COLUMNS & DEPARTMENTS LETTER FROM THE EDITOR: PLUGGING INTO THE CLOUD . . . . . . . . . . . . . . . . . . . . 2 by Chris Harrison, Editor-in-Chief
ELASTICITY IN THE CLOUD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 by David Chiu
CLOUD COMPUTING IN PLAIN ENGLISH . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 by Ryan K. L. Ko
FEATURES
7
VOLUNTEER COMPUTING: THE ULTIMATE CLOUD by David P. Anderson As a collective whole, the resource pool of all the privately-owned PCs in the world dwarfs all others. It’s also self-financing, self-updating, and self-maintaining. In short, it’s a dream come true for volunteer computing, and the cloud makes it possible.
CLOUDS AT THE CROSSROADS: RESEARCH PERSPECTIVES by Ymir Vigfusson and Gregory Chockler
10
Despite its ability to cater to business needs, cloud computing is also a first-class research subject, according to two researchers from IBM Haifa Labs.
SCIENTIFIC WORKFLOWS AND CLOUDS by Gideon Juve and Ewa Deelman
14
How is the cloud affecting scientific workflows? Two minds from the University of Southern California explain.
THE CLOUD AT WORK: INTERVIEWS WITH PETE BECKMAN OF ARGONNE NATIONAL LAB AND BRADLEY HOROWITZ OF GOOGLE
19
by Sumit Narayan and Chris Heiden Two leaders in the computing world explain how they view cloud computing from the research and industry perspectives.
STATE OF SECURITY READINESS by Ramaswamy Chandramouli and Peter Mell
24
Fears about the security readiness of the cloud are preventing organizations from leveraging it, and it’s up to computing professionals and researchers to start closing that gap.
THE BUSINESS OF CLOUDS by Guy Rosen
26
Businesses are flocking to cloud computing-based solutions for their business needs. The best way to understand the magnitude of this mass movement is to look at the hard data.
Contact ACM and Order Today! Phone: 1.800.342.6626 (USA/Canada) Postal Address: ACM Member Services +1.212.626.0500 (outside USA/Canada) P.O. Box 11405 Fax: +1.212.944.1318 New York, NY 10286-1405 USA Internet: http://store.acm.org/acmstore Please note the offering numbers for fulfilling claims or single order purchase below. Copyright 2010 by the Association for Computing Machinery, Inc. Permission to make digital or hard copies of part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page or initial screen of the document. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers, or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from Publications Dept., ACM Inc., fax +1 (212) 869-0481, or
[email protected]. Crossroads is distributed free of charge over the internet. It is available from: http://www.acm.org/crossroads/. Articles, letters, and suggestions are welcomed. For submission information, please contact
[email protected].
Letter from the Editor
Plugging into the Cloud
A
fter reading this issue, I had to seriously reevaluate my perception and definition of cloud computing. Unsurprisingly, given the wide array of computing models it encompasses, agreement among even experts is somewhat elusive.
For end users, cloud computing’s inherent intangibility makes it tough to get a good grip on what it is and isn’t, where it begins and ends. However, one thing is for sure: Cloud computing is hot and will soon have a big presence on your PC. Google, already a big player in the consumer space with services like Gmail and Google Docs, is readying ChromeOS, a thin operating system that boots right to a browser. With Chrome OS, document storage and heavy computation (like web searches) will all occur in the cloud. Is this a taste of things to come? Fortunately for programmers and students, Google has opened up its “app engine” back end, joining other powerful services like Amazon’s EC2 and Yahoo!’s BOSS. If you’ve been thinking about getting your feet wet in the cloud, there really isn’t a better time to start tinkering!
Open Hack Day In fact, I’m already guilty. As part of Yahoo!’s Open Hack day this past October, Bryan Pendleton, Julia Schwarz, and I (all Carnegie Mellon University students) built a cloud-based application in Python we call The Inhabited Web. In the 24 “hacking” hours permitted by the contest, we built the back end on the Google App Engine (appengine. google.com), making it massively parallel and distributed. The idea, briefly, is to embed a simple visualization into web pages, next to the browser's scroll bar. Small triangles are used to represent users’ positions on the current page (scroll position). Collectively, this allows you to see where people are congregating on a web page, perhaps next to a great shopping bargain, interesting news story, or funny video. You can check it out and sign up your web site for the service at wwww.inhabitedweb.com. Speaking of the web, we invite you to join our Facebook group (ACM Crossroads) and also to let us know what you think via email (
[email protected]) and Twitter (hashtag #xrds). I hope you find the current issue stimulating. The whole Crossroads team has been hard at work for three months on this cloud-centric edition of the magazine, and we are very excited about the amazing lineup of feature articles, covering topics from security and entrepre-
2
Spring 2010/ Vol. 16, No. 3
neurship, all the way to volunteer computing. You’ll also find interviews with people working on the biggest and best cloud computing systems (see page 19).
Presenting XRDS This issue also marks the last Crossroads that will arrive in the present format. We’re very excited to announce Crossroads will be relaunching as of the next issue with an all-new look and tons of fresh content for students. We’ve placed special emphasis on recurring columns headed up by our new editorial team. Expect everything from code snippets and school advice, to historical factoids and lab highlights, to event listings and puzzles. Heading up these departments is a talented team from all over the globe: Daniel Gooch (University of Bath), David Chiu (Ohio State University), Rob Simmons (Carnegie Mellon), Dima Batenkov (Weizmann Institute of Science, Israel), Michael Ashley-Rollman (Carnegie Mellon), Erin Claire Carson (University of California-Berkeley). I am also very pleased to announce James Stanier (University of Sussex) is now part of the senior editorial team, responsible for soliciting and magazine feature articles, joining Ryan K. L. Ko (Nanyang Technical University, Singapore), Inbal Talgam (Weizmann Institute of Science, Israel), Sumit Narayan (University of Connecticut), and Tom Bartindale (Newcastle University). —Chris Harrison, Editor-in-Chief
Biography Editor-in-chief Chris Harrison is a PhD student in the Human-Computer Interaction Institute at Carnegie Mellon University. His research interests primarily focus on novel input methods and interaction technologies, especially those that leverage hardware and the environment in new and compelling ways. Over the past four years, he has worked on several projects in the area of social computing and input methods at IBM Research, AT&T Labs, and most recently, Microsoft Research.
www.acm.org/crossroads
Crossroads
Elasticity in the Cloud By David Chiu
T
ake a second to consider all the essential services and utilities we consume and pay for on a usage basis: water, gas, electricity. In the distant past, some people have suggested that computing be treated under the same model as most other utility providers. The case could certainly be made. For instance, a company that supports its own computing infrastructure may suffer from the costs of equipment, labor, maintenance, and mounting energy bills. It would be more cost-effective if the company paid some third-party provider for its storage and processing requirements based on time and usage. While it made perfect sense from the client’s perspective, the overhead of becoming a computingas-a-utility provider was prohibitive until recently. Through advancements in virtualization and the ability to leverage existing supercomputing capacities, utility computing is finally becoming realized. Known to most as cloud computing, leaders, such as Amazon Elastic Compute Cloud (EC2), Azure, Cloudera, and Google’s App Engine, have already begun offering utility computing to the mainstream. A simple, but interesting property in utility models is elasticity, that is, the ability to stretch and contract services directly according to the consumer’s needs. Elasticity has become an essential expectation of all utility providers. When’s the last time you plugged in a toaster oven and worried about it not working because the power company might have run out of power? Sure, it’s one more device that sucks up power, but you’re willing to eat the cost. Likewise, if you switch to using a more efficient refrigerator, you would expect the provider to charge you less on your next billing cycle. What elasticity means to cloud users is that they should design their applications to scale their resource requirements up and down whenever possible. However, this is not as easy as plugging or unplugging a toaster oven.
A Departure from Fixed Provisioning Consider an imaginary application provided by my university, Ohio State. Over the period of a day, this application requires 100 servers during peak time, but only a small fraction of that during down time. Without elasticity, Ohio State has two options: either provision a fixed amount of 100 servers, or less than 100 servers. While the former case, known as over-provisioning, is capable of handling peak loads, it also wastes servers during down time. The latter case of under-provisioning might address, to some extent, the presence of idle machines. However, its inability to handle peak loads may cause users to leave its service. By designing our applications to scale servers accordingly to the load, the cloud offers a departure from the fixed provisioning scheme. To provide an elastic model of computing, providers must be able to support the sense of having an unlimited number of resources. Because computing resources are unequivocally finite, is elasticity a reality? Sharing Resources In the past several years, we have experienced a new trend in processor development. CPUs are now being shipped with multi- and many-cores on each chip in an effort to continue the speed-up, as predicted by Moore’s Law. However, the superfluous cores (even a single core) are underutilized or left completely idle. Crossroads
www.acm.org/crossroads
Spring 2010/ Vol. 16, No. 3
3
David Chiu
System engineers, as a result, turn to statistical multiplexing for maximizing the utilization of today’s CPUs. Informally, statistical multiplexing allows a single resource to be shared by splitting it into variable chunks and allocating each to a consumer. In the meantime, virtualization technology, which allows several instances of operating systems to be run on a single host machine, has matured to a point of production. Virtualization has since become the de-facto means toward enabling CPU multiplexing, which allows cloud providers to not only maximize the usage of their own physical resources, but also multiplex their resources among multiple users. From the consumers’ perspective, they are afforded a way to allo cate ondemand, independent, and more im portant, fullycontrollable systems. But even with virtualization, the question persists: What if the physical resources run out? If that ever occurred, the provider would simply have to refuse service, which is not what users want to hear. Currently, for most users, EC2 only allows 20 simultaneous machine instances to be allocated at any time. Another option might be to preempt currently running processes. Although both are unpopular choices, they certainly leave room for the provider to offer flexible pricing options. For instance, a provider can charge a normal price
for low-grade users, who might be fine with having their service interrupted very infrequently. High-grade users, on the other hand, can pay a surplus for having the privilege to preempt services and also to prevent from being preempted.
Looking Forward With the realization of cloud computing, many stakeholders are afforded on-demand access to utilize any amount of computing power to satisfy their relative needs. The elastic paradigm brings with it exciting new development in the computing community. Certainly, scaling applications to handle peak loads has been a long-studied issue. While downscaling has received far less attention in the past, the cloud invokes a novel incentive for applications to contract, which offers a new dimension for cost optimization problems. As clouds gain pace in industry and academia, they identify new opportunities and may potentially transform computing, as we know it.
elasticity means to cloud users is ❝What that they should design their applications to scale their resource requirements up and down whenever possible.
4
Spring 2010/ Vol. 16, No. 3
❞
Biography David Chiu is a student at The Ohio State University and an editor for Crossroads.
www.acm.org/crossroads
Crossroads
Cloud Computing in Plain English By Ryan K. L. Ko
I
am not an evangelist of cloud computing, and I must admit that, like many, I was once a skeptic. As a fledgling researcher, I was also quite appalled at how many seasoned researchers were able to recently claim that their research “has always been” cloud computing. I suppose many people believe cloud computing is just a buzzword and are also quite turned off by the ever-growing list of acronyms plaguing the computer science world. But is “cloud computing” really just another buzzword brought forward by the software giants, or is there something more? Significance of the Cloud Computing Era Fundamentally, cloud computing is a concept that aims to enable end-users to easily create and use software without a need to worry about the technical implementations and nitty-gritties such as the software’s physical hosting location, hardware specifications, efficiency of data processing, and so forth. This concept is already evident in many current technologies that are not explicitly labeled as cloud computing. For example, end-users no longer need to learn a new language or worry about the program’s memory requirements to create a Facebook or MySpace application. A small- to medium-sized enterprise no longer needs to own and maintain actual physical servers to host Web applications but are instead able to lease virtual private servers (VPS) for a monthly subscription fee. With cloud computing, end-users and businesses can simply store and work on data in a “cloud,” which is a virtual environment that embodies data centers, services, applications, and the hardworking folks at the IT companies. The key difference between this and other similar-sounding approaches, such as grid computing or utility computing, is in the concept of abstracting services from products. This is done by virtualizing the products (for example, the complex network of computers, servers, and applications that are used in the back end) so that computing is now accessible to anyone with a computing need of any size. By accessible, we mean that it is easy for a non-technical person to use this software and even create his or her own. This marks the change from the focus on full implementation of computing infrastructures before the year 2000 to the abstraction of the high-level, value-driven activities from the low-level, technical activities and details in the present and near future. In the words of those advocating cloud computing, it means that we are now moving toward services instead of focusing on selling products, and practically anyone can utilize computing to the max. (More technical information on these services can be found in “The Business of Clouds,” page 26.) So, what does all this mean for common folks like you and me? It means that we are freed from the need to upgrade hardware and the need to spend more than half of the time trying to make a product work, but are now able to focus on the real essence of our activities—the valueadding activities (cf. Michael Porter’s Competitive Advantage). With cloud computing, a startup company would no longer need to worry about the RAID configurations and the number of scheduled backup jobs, but instead could focus on more important details, such as the actual web content, the number of emails to set up for its employees, and the file structure and permissions to be granted for its content management structure. Crossroads
www.acm.org/crossroads
Spring 2010/ Vol. 16, No. 3
5
Ryan K. L. Ko
Now, when we look beneath the sales talk and big promises of cloud computing and observe the shifts in trends in our computing approaches, we start to realize that cloud computing is not just another buzzword, but something that embodies this innate attempt by humans to make computing easier. The evolution of computing languages from the first generation (assembly languages) to the more human-readable fourth-generation languages (4GLs, SQL), and the evolution from structural/modular programming to object-oriented programming are both earlier evidences of this trend. Cloud computing’s focus is on empowering Internet users with the ability to focus on value-adding activities and services and outsource the worries of hardware upgrades and technical configurations to the “experts residing” in the virtual cloud. In today’s context, cloud computing loosely means that software you use does not reside on your own computer, but rather on a host computer, accessed via the Internet, run by someone else. Given this fact, there are bound to be many problems and loopholes. Hence, it is not rare to find researchers claiming that they are working in a research area that contributes to cloud computing. With so much at stake, experts from computer security, service computing, computer networking, software engineering and many other related areas are crucial people in this turn of a new era.
While it is my greatest wish for you have a better understanding of cloud computing through this article, I hope that I have also opened up your mind to witnessing the increasing influence of cloud computing in our daily lives.
Biography Ryan K. L. Ko is a final year PhD candidate at Nanyang Technological University, Singapore, specializing in the semantic web and business process management. He is also an editor for Crossroads.
Cloud Computing Starter Kit While there are plenty of sites and articles describing cloud computing, not many have an objective view of this highpotential but controversial topic. The following resources have been selected by Crossroads’ editors in an attempt to help other students understand the meaning, concerns, and latest trends of cloud computing.
“Like it or not, cloud computing is the wave of the future.” By Therese Poletti, MarketWatch.
“www.marketwatch.com/story/like-not-cloudcomputing-wave
Imminent Issues If we are evolving into a cloud-oriented environment and way of doing business, we will need to urgently address both data privacy and data security concerns. Researchers need to find the right balance between convenience and security. It’s a balancing act: when convenience increases, security decreases, and vice versa. As cloud computing is a highly trust-based system, many researchers are now geared toward creating better trust evaluation mechanisms and authentication procedures, while the industry is busy figuring out scalability solutions, data integrity, and security issues. Once a hacker or malicious attack successfully penetrates the security boundaries of the cloud, or an employee of a cloud vendor betrays the trust of the public, our data and critical information is at the com-
“A layman’s summary of the recent cloud computing trend. “Microsoft to battle in the clouds.” By Rory Cellan-Jones, BBC News.
“http://news.bbc.co.uk/2/hi/technology/7693993.stm “See in particular the short video clip on Microsoft Azure in this piece from the BBC.
“Storm warning for cloud computing.” By Bill Thompson, BBC News.
“http://news.bbc.co.uk/2/hi/technology/7421099.stm “Highlighting concerns surrounding cloud computing. “Cloud computing is a trap, warns GNU founder Richard Stallman.”
❝
Cloud computing loosely means that software you use does not reside on your own computer, but rather on a host computer, accessed via the Internet, run by someone else.
By Bobbie Johnson, Guardian.co.uk
“www.guardian.co.uk/technology/2008/sep/29/ cloud.computing.richard.stallman
“Richard Stallman on why he’s against cloud computing.
❞
“Click’s Favourite Cloud Links.”
plete mercy of these criminals. To further increase the security, we would need legislation and laws to catch up with the nature of cloud computing, as it will be a borderless and large-impact problem.
From Click’s BBC News
“http://news.bbc.co.uk/2/hi/programmes/click_online/ 7464153.stm
“See in particular G.ho.st, a global virtual computer hosting site.
How Can Graduates Approach Cloud Computing? The best way to approach this field is to have a good balance between the quest of knowledge and discernment. Do not bounce on the latest buzzwords you hear. Take a step back and try to see how things fit together. A good way to do this is to organize and draw what you have learned into mind maps. Crossroads has prepared a starter kit (see sidebar), introducing some non-technical links to interesting articles and videos to kickstart your journey.
6
Spring 2010/ Vol. 16, No. 3
“Dell attempts to copyright ‘cloud computing.’” By Agam Shah, for IDG News Service, published on TechWorld “www.techworld.com/opsys/news/index.cfm?newsid=102279 “Just for fun, Dell tries to beat other computing companies to the punchline.
www.acm.org/crossroads
Crossroads
Volunteer Computing The Ultimate Cloud By David P. Anderson
C
omputers continue to get faster exponentially, but the computational demands of science are growing even faster. Extreme requirements arise in at least three areas.
1) Physical simulation: Scientists use computers to simulate physical reality at many levels of scale: molecule, organism, ecosystem, planet, galaxy, universe. The models are typically chaotic, and studying the distribution of outcomes requires many simulation runs with perturbed initial conditions. 2) Compute-intensive analysis of large data: Modern instruments (optical and radio telescopes, gene sequencers, gravitational wave detectors, particle colliders) produce huge amounts of data, which in many cases requires compute-intensive analysis. 3) Biology-inspired algorithms such as genetic and flocking algorithms for function optimization. These areas engender computational tasks that would take hundreds or thousands of years to complete on a single PC. Reducing this to a feasible interval—days or weeks—requires high-performance computing (HPC). One approach is to build an extremely fast computer— a supercomputer. However, in the areas listed above, the rate of job completion, rather than the turnaround time of individual jobs, is the important performance metric. This subset of HPC is called highthroughput computing. To achieve high throughput, the use of distributed computing, in which jobs are run on networked computers, is often more costeffective than supercomputing. There are many approaches to distributed computing:
•
cluster computing, which uses dedicated computers in a single location.
•
desktop grid computing, in which desktop PCs within an organization (such as a department or university) are used as a computing resource. Jobs are run at low priority, or while the PCs are not being otherwise used.
•
grid computing, in which separate organizations agree to share their computing resources (supercomputers, clusters, and/or desktop grids).
•
cloud computing, in which a company sells access to computers on a pay-as-you-go basis.
•
volunteer computing, which is similar to desktop grid computing except that the computing resources are volunteered by the public.
Each of these paradigms has an associated resource pool: the computers in a machine room, the computers owned by a university, the computers owned by a cloud provider. In the case of volunteer comCrossroads
puting, the resource pool is the set of all privately-owned PCs in the world. This pool is interesting for several reasons. For starters, it dwarfs the other pools. The number of privatelyowned PCs is currently 1 billion and is projected to grow to 2 billion by 2015. Second, the pool is self-financing, self-updating and self-maintaining. People buy new PCs, upgrade system software, maintain their computers, and pay their electric bills. Third, consumer PCs, not special-purpose computers, are state of the art. Consumer markets drive research and development. For example, the fastest processors today are GPUs developed for computer games. Traditional HPC is scrambling to use GPUs, but there are already 100 million GPUs in the public pool, and tens of thousands are already being used for volunteer computing.
History of Volunteer Computing In the mid-1990s, as consumer PCs became powerful and millions of them were connected to the Internet, the idea of using them for distributed computing arose. The first two projects, GIMPS and distributed.net, were launched in 1996 and 1997. GIMPS finds prime numbers of a particular type, and distributed.net breaks cryptosystems via brute-force search of the key space. Both projects attracted tens of thousands of volunteers and demonstrated the feasibility of volunteer computing. In 1999 two new projects were launched, SETI@home and Folding@home. SETI@home from University of California-Berkeley analyzes data from the Arecibo radio telescope, looking for synthetic signals from space. Folding@home, from Stanford, studies how proteins are formed from gene sequences. These projects received significant media coverage and moved volunteer computing into the awareness of the global public. These projects all developed their own middleware, the application-independent machinery for distributing jobs to volunteer computers and for running jobs unobtrusively on these computers, as well as web interfaces by which volunteers could register, communicate with other volunteers, and track their progress. Few scientists had the resources or skills to develop such software, and so for several years there were no new projects. In 2002, with funding from the National Science Foundation, the BOINC project was established to develop general-purpose middleware for volunteer computing, making it easier and cheaper for scientists to use. The first BOINC-based projects launched in 2004, and today there about 60 such projects, in a wide range of scientific areas. Some of the larger projects include Milkyway@home (from Rensselaer Polytechnic Institute; studies galactic structure), Einstein@home (from University of Wisconsin and Max Planck Institute; searches for gravitational
www.acm.org/crossroads
Spring 2010/ Vol. 16, No. 3
7
David P. Anderson
waves), Rosetta@home (from University of Washington; studies proteins of biomedical importance), ClimatePrediction.net (from Oxford University; studies long-term climate change), and IBM World Community Grid (operated by IBM; hosts 5-10 humanitarian applications from various academic institutions).
ClimatePrediction.net, from Oxford University, simulates the Earth’s climate change during the next 100 years.
being used for. As a result, public awareness of science is increased, and research projects that are outside of the current academic mainstream can potentially get significant computing resources. Scientific adoption. Volunteer computing has not yet been widely adopted. Sixty research groups are currently using volunteer computing, while perhaps a hundred times that many could benefit from it. Cluster and grid computing are much more widely used by scientists. The HPC community, on whom scientists rely for guidance, has ignored volunteer computing, perhaps because it offers neither control nor funding. In addition, although BOINC has reduced the barrier to entry, few research groups have the resources and skills needed to operate a project. The most promising solution to this is to create umbrella projects serving multiple scientists and operated at a higher organizational level (for example, at the level of a university). Energy efficiency. The FLOP/Watt ratio of a PC is lower than that of a supercomputer, and it is tempting to conclude that volunteer computing is less energy-efficient than supercomputing. However, this is not necessarily the case. In cold climates, for example, energy used by a PC may replace energy used by a space heater, to which the PC is thermodynamically equivalent. No study has been done taking such factors into account.
Evaluating Volunteer Computing Volunteer computing can be compared with other high-performance computing paradigms in several dimensions. Performance. About 900,000 computers are actively participating in volunteer computing. Together they supply about 10 PetaFLOPS (trillion floating-point operations per second) of computing power; the fraction supplied by GPUs is about 70 percent and growing. As a comparison, the fastest supercomputer supplies about 1.4 PetaFLOPS, and the largest grids number in the tens of thousands of hosts. So in terms of throughput, volunteer computing is competitive with other paradigms, and it has the near-term potential to greatly surpass them: if participation increases to 4 million computers, each with a 1 TeraFLOPS GPU (the speed of current high-end models) and computing 25 percent of the time, the result will be 1 ExaFLOPS of computing power. Other paradigms are projected to reach this level only in a decade or more. Actually, since 4 million PCs is only 0.4 percent of the resource pool, the near-term potential of volunteer computing goes well beyond Exa-scale. Cost effectiveness. For scientists, volunteer computing is cheaper than other paradigms—often dramatically so. A medium-scale project (10,000 computers, 100 TeraFLOPS) can be run using a single server computer and one or two staff for roughly $200,000 per year. An equivalent CPU cluster costs at least an order of magnitude more. Cloud computing is even more expensive. For example, Amazon Elastic Computing Cloud (EC2) instances provide 2 GigaFLOPS and cost $2.40 per day. To attain 100 TeraFLOPS, 50,000 instances would be needed, costing $43.8 million per year. However, studies suggest that cloud computing is cost-effective for hosting volunteer computing project servers. Resource allocation policy and public outreach. In traditional HPC paradigms, resources are allocated by bureaucracies: funding agencies, institutions, and committees. The public, although it pays for the resources, has no direct voice in their allocation, and doesn’t know how they’re being used. In volunteer computing, the public has direct control over how resources are allocated, and knows what they’re
8
Spring 2010/ Vol. 16, No. 3
The BOINC Project The BOINC software consists of two parts: server software that is used by to create projects and client software. Anyone—academic researchers, hobbyists, malicious hackers—can create a project. Projects are independent. Each one operates its own server and provides its own web site. BOINC has no centralized component other than a web site from which its software can be downloaded. And in terms of software, volunteers install and run client software on their computers. The client software is available for all major platforms, including Windows, Linux, and Mac OS X. Having installed the client program, volunteers can then attach it to any set of projects, and for each project can assign a resource share that determines how the computer’s resources are divided among the projects. The choice of projects is up to the volunteer. Attaching to a project allows it to run arbitrary executables on one’s computer, and BOINC provides only limited (account-based) sandboxing. So the volunteer must assess the project’s authenticity, its technical competence, and its scientific merit. The ownership of intellectual property resulting from the project may also be a factor. BOINC encourages volunteers to participate in multiple projects simultaneously. By doing so, they avoid having their computer go idle if one project is down. Multiple attachment also helps projects whose supply of work is sporadic. More generally, by making it easy to join and leave projects, BOINC encourages volunteers to occasionally evaluate the set of available projects, and to devote their computing resources to that projects that, in their view, are doing the most important and best research. BOINC does accounting of credit, a numerical measure of a volunteer’s contribution to a project. The accumulation of a large amount of credit in a particular project can be a disincentive to try other projects. To combat this, BOINC provides a cross-project notion of identity (based on the volunteer’s email address). Each project exports its credit statistics as XML files, and various third-party credit statistics
www.acm.org/crossroads
Crossroads
Volunteer Computing: The Ultimate Cloud
sites import these files and display cross-project credit, that is, the volunteer’s total credit across all projects. Even with the modest number (60) of current projects, the process of locating them, reading their web sites, and attaching to a chosen set is a tedious process, and will become infeasible if the number of projects grows to hundreds or thousands. BOINC provides a framework for dealing with this problem. A level of indirection can be placed between client and projects. Instead of being attached directly to projects, the client can be attached to a web service called an account manager. The client periodically communicates with the account manager, passing it account credentials and receiving a list of projects to attach to. This framework has been used by third-party developers to create “one-stop shopping” web sites, where volunteers can read summaries of all existing BOINC projects and can attach to a set of them by checking boxes. The framework could also be used for delegation of project selection, analogous to mutual funds. For example, volunteers wanting to support cancer research could attach to an American Cancer Society account manager. American Cancer Society experts would then select a dynamic weighted “portfolio” of meritorious cancer-related volunteer projects.
The BOINC client software lets volunteers attach to projects and monitor the progress of jobs.
Human Factors All HPC paradigms involve human factors, but in volunteer computing these factors are particularly crucial and complex. To begin with, why do people volunteer? This question is currently being studied rigorously. Evidence suggests that there are several motivational factors. One such factor is to support scientific goals, such as curing diseases, finding extraterrestrial life, or predicting climate change. Another factor is community. Some volunteers enjoy participating in the online communities and social networks that form, through message boards and other web features, around volunteer computing projects. Yet another reason people volunteer is because of the credit incentive. Some volunteers are interested in the performance of computer systems, and they use volunteer computing to quantify and publicize the performance of their computers. There have been attempts to commercialize volunteer computing by paying participants, directly or via a lottery, and reselling the computing power. These efforts have failed because the potential buyers, such as pharmaceutical companies, are unwilling to have their data on computers outside of their control.
Crossroads
To attract and retain volunteers, a project must perform a variety of human functions. It must develop web content describing its research goals, methods, and credentials. It must provide volunteers with periodic updates (via web or email) on its scientific progress. It must manage the moderation of its web site’s message boards to ensure that they remain positive and useful. It must publicize itself by whatever media are available—mass media, alumni magazines, blogs, social networking sites, and so on. Volunteers must trust projects, but projects cannot trust volunteers. From a project’s perspective, volunteers are effectively anonymous. If a volunteer behaves maliciously, for example by intentionally falsifying computational results, the project has no way to identify and punish the offender. In other HPC paradigms, such offenders can be identified and disciplined or fired.
Technical Factors Volunteer computing poses a number of technical problems. For the most part, these problems are addressed by BOINC, and scientists need not be concerned with them. Heterogeneity. The volunteer computer population is extremely diverse in terms of hardware (processor type and speed, RAM, disk space), software (operating system and version) and networking (bandwidth, proxies, firewalls). BOINC provides scheduling mechanisms that assign jobs to the hosts that can best handle them. However, projects still generally need to compile applications for several platforms (Windows 32 and 64 bit, Mac OS X, Linux 32 and 64 bit, various GPU platforms). This difficulty may soon be reduced by running applications in virtual machines. Sporadic availability and churn. Volunteer computers are not dedicated. The time intervals when a computer is on, and when BOINC is allowed to compute, are sporadic and generally unpredictable. BOINC tracks these factors and uses them in estimating job completion times. In addition, computers are constantly joining and leaving the pool of a given project. BOINC must address the fact that computers with many jobs in progress may disappear forever. Result validation. Because volunteer computers are anonymous and untrusted, BOINC cannot assume that job results are correct, or that the claimed credit is accurate. One general way of dealing with this is replication: that is, send a copy of each job to multiple computers; compare the results; accept the result if the replicas agree; otherwise issue additional replicas. This is complicated by the fact that different computers often do floating-point calculations differently, so that there is no unique correct result. BOINC addresses this with a mechanism called homogeneous redundancy that sends instances of a given job to numerically identical computers. In addition, redundancy has the drawback that it reduces throughput by at least 50 percent. To address this, BOINC has a mechanism called adaptive replication that identifies trustworthy hosts and replicates their jobs only occasionally. Scalability. Large volunteer projects can involve a million hosts and millions of jobs processed per day. This is beyond the capabilities of grid and cluster systems. BOINC addresses this using an efficient server architecture that can be distributed across multiple machines. The server is based on a relational database, so BOINC leverages advances in scalability and availability of database systems. The communication architecture uses exponential backoff after failures, so that
www.acm.org/crossroads
Spring 2010/ Vol. 16, No. 3
9
David P. Anderson
the rate of client requests remains bounded even when a server comes up after a long outage. Security. Volunteer computing poses a variety of security challenges. What if hackers break into a project server and use it to distribute malware to the attached computers? BOINC prevents this by requiring that executables be digitally signed using a secure, offline signing computer. What if hackers create a fraudulent project that poses as academic research while in fact stealing volunteers’ private data? This is partly addressed by account-based sandboxing: applications are run under an unprivileged user account and typically have no access to files other than their own input and outputs. In the future, stronger sandboxing may be possible using virtual machine technology.
Future of Volunteer Computing Volunteer computing has demonstrated its potential for highthroughput scientific computing. However, only a small fraction of this potential has been realized. Moving forward will require progress in three areas. 1. Increased participation: The volunteer population has remained around 500,000 for several years. Can it be grown by an order of magnitude or two? A dramatic scientific breakthrough, such as the discovery of a cancer treatment or a new astronomical phenomenon, would certainly help it popularity. Or, the effective use of social networks like Facebook could spur more people to volunteer. Another way to increase participation might be to have computer manufacturers or software vendors bundle BOINC with other products.
Currently, Folding@Home is bundled with the Sony Playstation 3 and with ATI GPU drivers. 2. Increased scientific adoption: The set of volunteer projects is small and fairly stagnant. It would help if more universities and institutions created umbrella projects, or if there were more support for higher-level computing models, such as workflow management systems and MapReduce. Two other factors that would increase scientific adoption are the promotion of volunteer computing by scientific funding agencies and and increased acceptance of volunteer computing by the HPC and computer science communities. 3. Tracking technology: Today, the bulk of the world’s computing power is in desktop and laptop PCs, but in a decade or two it may shift to energy-efficient mobile devices. Such devices, while docked, could be used for volunteer computing. If these challenges are addressed, and volunteer computing experiences explosive growth, there will be thousands of projects. At this point volunteers can no longer be expected to evaluate all projects, and new allocation mechanisms will be needed. For example, the “mutual fund” idea mentioned above, or something analogous to decision markets, in which individuals are rewarded for participating in new projects that later produce significant results. Such “expert investors” would steer the market as a whole.
Biography David P. Anderson is a research scientist at the Space Sciences Laboratory at the University of California-Berkeley.
Clouds at the Crossroads Research Perspectives By Ymir Vigfusson and Gregory Chockler
D
espite its promise, most cloud computing innovations have been almost exclusively driven by a few industry leaders, such as Google, Amazon, Yahoo!, Microsoft, and IBM. The involvement of a wider research community, both in academia and industrial labs, has so far been patchy without a clear agenda. In our opinion, the limited participation stems from the prevalent view that clouds are mostly an engineering and business-oriented phenomenon based on stitching together existing technologies and tools. Here, we take a different stance and claim that clouds are now mature enough to become first-class research subjects, posing a range of unique and exciting challenges deserving collective attention from the research community. For example, the realization of privacy in clouds is a cross-cutting interdisciplinary challenge, permeating the entire stack of any imaginable cloud architecture. The goal of this article is to present some of the research directions that are fundamental for cloud computing. We pose various challenges that span multiple domains and disciplines. We hope these questions will provoke interest from a larger group of researchers and academics who wish to help shape the course of the new technology.
10
Spring 2010/ Vol. 16, No. 3
An Architectural View The physical resources of a typical cloud are simply a collection of machines, storage, and networking resources collectively representing the physical infrastructure of the data center(s) hosting the cloud computing system. Large clouds may contain some hundreds of thousands of computers. The distributed computing infrastructure offers a collection of core services that simplify the development of robust and scalable services on top of a widely distributed, failure-prone, physical platform. The services supported by this layer typically include communication (for example, multicast and publish-subscribe), failure detection, resource
www.acm.org/crossroads
Crossroads
Clouds at the Crossroads: Research Perspectives
usage monitoring, group membership, data storage (such as distributed file systems and key-value lookup services), distributed agreement (consensus), and locking. The application resource management layer manages the allocation of physical resources to the actual applications and platforms including higher-level service abstractions (virtual machines) offered to end-users. The management layer deals with problems related to the application placement, load balancing, task scheduling, servicelevel agreements, and others. Finally, we enumerate some cross-cutting concerns that dissect the entire cloud infrastructure. We will focus on these issues: energy, privacy and consistency, the lack of standards, benchmarks, and test beds for conducting cloud related research.
Energy Large cloud providers are natural power hogs. To reduce the carbon footprint, data centers are frequently deployed in proximity to hydroelectric plants and other clean energy sources. Microsoft, Sun, and Dell have advocated putting data centers in shipping containers consisting of several thousand nodes at a time, thus making deployment easier. Although multi-tenancy and the use of virtualization improves resource utilization over traditional data centers, the growth of cloud provider services has been rapid, and power consumption is a major operating expense for the large industry leaders. Fundamental questions exist of how, where, and at what cost can we reduce power consumption in the cloud. Here we examine three examples to illustrate potential directions. Solid-state disks (SSDs) have substantially faster access times and draw less power than regular mechanical disks. The downside is that SSDs are more expensive and lack durability because blocks can become corrupted after 100,000 to 1,000,000 write-erase cycles. SSDs have made their way into the laptop market—the next question is whether cloud data centers will follow [14]. Can we engineer mechanisms to store read-intensive data on SSDs instead of disks? Google has taken steps to revamp energy use in hardware by producing custom power supplies for computers which have more than double the efficiency of regular ones [12]. They even patented a “water-based” data center on a boat that harnesses energy from ocean tides to power the nodes and also uses the sea for cooling. How can we better design future hardware and infrastructure for improved energy efficiency? How can we minimize energy loss in the commodity machines currently deployed in data centers? In the same fashion that laptop processors adapt the CPU frequency to the workload being performed, data center nodes can be powered up or down to adapt to variable access patterns, for example, due to diurnal cycles or flash crowds. Some CPUs and disk arrays have more flexible power management controls than simple on/off switches, thus permitting intermediate levels of power consumption [13]. File systems spanning multiple disks could, for instance, bundle infrequently accessed objects together on “sleeper” disks [9]. More generally, how should data and computation be organized on nodes to permit software to decrease energy use without reducing performance?
Privacy Concerns Storing personal information in the cloud clearly raises privacy and security concerns. Sensitive data are no longer barred by physical obscurity or obstructions. Instead, exact copies can be made in an instant. Crossroads
Technological advances have reduced the ability of an individual to exercise personal control over his or her personal information, making it elusive to define privacy within clouds [5]. The companies that gather information to deliver targeted advertisements are working toward their ultimate product: you. The amount of information known by large cloud providers about individuals is staggering, and the lack of transparent knowledge about how this information is used has provoked concerns. Are there reasonable notions of privacy that would still allow businesses to collect and store personal information about their customers in a trustworthy fashion? How much are users willing to pay for additional privacy? We could trust the cloud partially, while implementing mechanisms for auditing and accountability. If privacy leaks have serious legal repercussions, then cloud providers would have incentives to deploy secure information flow techniques (even if they are heavyhanded) to limit access to sensitive data and to devise tools to locate the responsible culprits if a breach is detected [17]. How can such mechanisms be made practical? Is the threat of penalty to those individuals who are caught compromising privacy satisfactory, or should the cloud be considered an untrusted entity altogether? If we choose not to trust the cloud, then one avenue of research is to abstract it as a storage and computing device for encrypted information. We could use a recent invention in cryptography called fully homomorphic encryption [10]; a scheme allowing the sum and multiplication (and hence arbitrary Boolean circuits) to be performed on encrypted data without needing to decrypt it first. Unfortunately, the first implementations are entirely impractical, but beg the question whether homomorphic encryption can be made practical. Another approach is to sacrifice the generality of homomorphic encryption. We can identify the most important functions that need to be computed on the private data and devise a practical encryption scheme to support these functions—think MapReduce [7] on encrypted data. As a high-level example, if all emails in Gmail were encrypted by the user’s public key and decrypted by the user’s web browser, then Gmail could not produce a search index for the mailbox. However, if each individual word in the email were encrypted, Gmail could produce an index (the encrypted words would just look like a foreign language) but would not understand the message contents. The latter case implies that Gmail could not serve targeted ads to the user. What are the practical points on the privacy versus functionality spectrum with respect to computational complexity and a feasible cloud business model? Secure multiparty computation (SMC) allows mutually distrusting agents to compute a function on their collective inputs without revealing their inputs to other agents [19]. Could we partition sensitive information across clouds, perhaps including a trusted third-party service, and perform SMC on the sensitive data? Is SMC the right model?
Consistency In a broad sense, consistency governs the semantics of accessing the cloud-based services as perceived by both the developers and end users. The consistency issues are particularly relevant to the distributed computing infrastructure services (see Figure 1), such as data storage. The most stringent consistency semantics, known as serializability or strong consistency [11], globally orders the service requests and
www.acm.org/crossroads
Spring 2010/ Vol. 16, No. 3
11
Ymir Vigfusson and Gregory Chockler
presents them as occurring in an imaginary global sequence. For example, suppose Alice deposits $5 to a bank account with the initial balance of $0 concurrently with Bob’s deposit of $10 to the same account. If Carol checks the account balance twice and discovers it first to be $10 and then $15, then no user would ever see $5 as the valid balance of that account (since in this case, Bob’s deposit gets sequenced before Alice’s). In the database community, this type of semantics is typically implied by ACID (atomicity, consistency, isolation, and durability). Intuitively, supporting serializability requires the participants to maintain global agreement about the command ordering. Since cloud services are typically massively distributed and replicated (for scalability and availability), reaching global agreement may be infeasible. Brewer’s celebrated CAP theorem [2] asserts that it is impossible in a large distributed system to simultaneously maintain (strong) consistency, availability, and to tolerate partitions—that is, network connectivity losses. Researchers have looked for practical ways of circumventing the CAP theorem. Most work has so far focused on relaxing the consistency semantics; basically substituting serializability or (some of ) the ACID properties with weaker guarantees. For instance, it does not matter if Carol and Bob in the example above would see either $5 or $10 as the intermediate balances, as long as both of them will eventually see $15 as the final balance. This observation underlies the notion of eventual consistency [18], which allows states of the concurrently updated objects to diverge provided that eventually the differences are reconciled, for example, when the network connectivity is restored. Apart from eventual consistency, other ways of weakening consistency semantics have looked into replacing single global ordering with multiple orderings. For instance, causal consistency [1] allows different clients to observe different request sequences as long as each observed sequence is consistent with the partial cause-effect order. Weaker consistency semantics work well only for specific types of applications, such as cooperative editing, but do not easily generalize to arbitrary services. (Just imagine what would happen if withdrawals were allowed in the bank account example above.) Moreover, semantics that are weaker than serializability (or ACID) tend to be difficult to explain to users and developers lacking the necessary technical background. Yet another problem is that for certain types of data, such as the meta-data of a distributed file system, it might be inherently impossible to compromise on strong consistency without risking catastrophic data losses at a massive scale. The possible research questions here would have to address questions such as can we produce a comprehensive and rigorous framework to define and reason about the diverse consistency guarantees. The framework should unify both weaker and stronger models and could serve as a basis for rigorous study of various consistency semantics of cloud services and their relative power. It should be expressive enough to allow new properties to be both easily introduced, for example by composing the existing basic properties, and understood by both developers and consumers of the cloud services. It should also help to bridge diverse perspectives on consistency that exist today within different research communities like the database and distributed systems communities. Although it is well understood that a cloud architecture should accommodate both strongly and weakly consistent services, it is unclear how the two can be meaningfully combined within a single system. How should they interact, or what implications would such a model have on performance and scalability?
12
Spring 2010/ Vol. 16, No. 3
Current approaches to supporting strong consistency primarily focus on isolating the problem into “islands” of server replicas. While beneficial for scalability, such an approach creates an extra dependency on a set of servers that have to be carefully configured and maintained. Can we make strong consistency services that are more dynamic and easier to reconfigure, providing a simpler and more robust solution?
Standards, Benchmarks, Test Beds Technical innovations are often followed by standards wars, and cloud computing is no exception. There is a plethora of cloud interoperability alliances and consortia (for example, Open Cloud Manifesto, DTMF Open Cloud Standards Incubator, Open Group’s Cloud Work Group). The largest incumbents in the market are nevertheless reluctant to follow suit and have chosen to define their own standards. Whereas the strategy is understandable, the lack of interoperability may have adverse effect on consumers who become locked-in on a single vendor. The worry is that clouds become natural monopolies. The Internet was built on open standards. The question is whether clouds will be as well. Making cloud services open and interoperable may stimulate competition and allow new entrants to enter the cloud market. Customers would be free to migrate their data from a stagnant provider to a new or promising one without difficulty when they so choose. Can the smaller players leverage their collective power to lobby for an open and flexible cloud computing standard that fosters competition while still allowing businesses to profit? Or can this be accomplished by the larger companies or governments? What business models are suitable for an open cloud? On the technical side, could users switch between providers without needing their support, for instance by using a third-party service? Different cloud providers often adopt similar APIs for physical resources and the distributed computing infrastructure. For instance, MapReduce and Hadoop expose a similar API, as do the various keyvalue lookup services (Amazon’s Dynamo [8], Yahoo!’s PNUTS [6], memcached [4]). Other components have more diverse APIs, for instance locking services like Google’s Chubby [3], Yahoo!’s Zookeeper [16], and real-time event dissemination services. The broad question asks what components and interfaces are the “right” way to provide the cloud properties mentioned previously. A more specific question is how we can compare and contrast different implementations of similar components. For instance, how can we evaluate the properties of key-value stores like PNUTS and Facebook’s Cassandra [15]? The most appealing approach is to compare well-defined metrics on benchmark traces, such as the TPC benchmark for databases (www.tpc.org). How can we obtain such traces, or perhaps synthetically generate them until real ones are produced? Also, consensus benchmarks enable researchers outside the major incumbent companies to advance the core cloud technologies. Developing distributed computing infrastructure layers or data storage systems is a hard task, but evaluating them for the massive scale imposed by clouds without access to real nodes is next to impossible. Academics who work on peer-to-peer systems (P2P), for example, rely heavily on the PlanetLab (www.planet-lab.org) test bed for deployment. PlanetLab constitutes more than 1,000 nodes distributed across nearly 500 sites, making it ideal an ideal resource for experimental validation of geographically networked systems which sustain heavy churn (peer arrivals and departures). The nodes in the data centers underlying the cloud tend to be numerous, hierarchically structured with respect to networking equipment, and face limited random churn but occasionally suffer from large-scale correlated failures.
www.acm.org/crossroads
Crossroads
Clouds at the Crossroads: Research Perspectives
PlanetLab’s focus on wide-area networks is suboptimal for cloud platform research, unfortunately, and the same holds true for other similar resources. A handful of test beds appropriate for cloud research have made their debut recently, including Open Cirrus from HP, Intel and Yahoo!, and the Open Cloud Testbed. We encourage other players to participate and contribute resources to cloud research, with the goal of providing a standard test bed with openaccess, at least for academia, including researchers from underrepresented universities. Who will create the future “CloudLab”?
How to Get Involved Students and researchers who are interested in shaping cloud computing should consider participating in the LADIS (www.cs.cornell.edu/ projects/ladis2010) or HotCloud (www.usenix.org/events/hotcloud10) workshops, or the upcoming Symposium on Cloud Computing (SoCC: http://research.microsoft.com/en-us/um/redmond/events/socc2010). Large industry players are currently driving the research bandwagon for cloud computing, but the journey is only beginning. A concerted multi-disciplinary effort is needed to turn the cloud computing promise into a success.
Biographies Dr. Ymir Vigfusson is a postdoctoral researcher with the Distributed Middleware group at the IBM Research Haifa Labs. His research is focused around distributed systems, specifically, real-world problems that embody deep trade-offs. He holds a PhD from Cornell University. Dr. Gregory Chockler is a research staff member in the Distributed Middleware group at the IBM Research Haifa Labs. His research interests span a wide range of topics in the area of large-scale distributed computing and cloud computing. He is one of the founders and organizers of the ACM/SIGOPS Workshop on Large-Scale Distributed Systems and Middleware (LADIS). He holds a PhD from the Hebrew University of Jerusalem.
10. Gentry, C. 2009. Fully homomorphic encryption using ideal lattices. In Proceedings of the ACM Symposium on Theory of Computing (STOC’09). 11. Gray, J., and Reuter, A. 1993. Isolation concepts. In Transaction Processing: Concepts and Techniques, chap. 7. Morgan Kaufmann. 12. Hoelzle, U. and Weihl, B. 2006. High-efficiency power supplies for home computers and servers. Google Inc. http://services. google. com/blog_resources/PSU_white_paper.pdf. 13. Khuller, S., Li, J., and Saha, B. 2010. Energy efficient scheduling via partial shutdown. In Proceedings of ACM-SIAM Symposium on Discrete Algorithms (SODA). 14. Narayanan, D., Donnelly, A., Thereska, E., Elnikety, S., and Rowstron, A. 2009. Migrating server storage to SSDs: Analysis of tradeoffs. In Proceedings of EuroSys. 15. Ramakrishnan, R. 2009. Data management challenges in the cloud. In Proceedings of ACM SIGOPS LADIS. http://www.cs. cornell. edu/projects/ladis2009/talks/ramakrishnan-keynote-ladis2009.pdf. 16. Reed B. and Junqueira, F. P. 2008. A simple totally ordered broadcast protocol. In Proceedings of the 2nd Workshop on Large-Scale Distributed Systems and Middleware (LADIS’08). ACM. 17. Smith, G. 2007. Principles of secure information flow analysis. In Malware Detection, Christodorescu, M., et al. Eds., Springer-Verlag. Chap. 13, 291-307. 18. Vogels, W. 2008. Eventually consistent. ACM Queue 6, 6. 19. Wenliang, D. and Atallah, M. J. 2001. Secure multi-party computation problems and their applications: a review and open problems. In Proceedings of the Workshop on New Security Paradigms.
References 1. Ahamad, M., Hutto, P. W., Neiger, G., Burns, J. E., and Kohli, P. 1995. Causal memory: Definitions, implementations and programming. Distributed Comput. 9. 37-49. 2. Brewer, E. 2000. Towards robust distributed systems. In Proceedings of Principles of Distributed Computing (PODC). 3. Burrows, M. 2006. The Chubby lock service for loosely-coupled distributed systems. In Proceedings of the 70th USENIX Symposium on Operating Systems Design and Implementation (OSDI’06). USENIX Association. 335-350 4. Danga Interactive. memcached: A distributed memory object caching system. http://www.danga.com/memcached/. 5. DeCandia, G., Hastorun, D., Jampani, et al. 2007. Dynamo: Amazon’s highly available key-value store. In Proceedings of the 21st ACM SIGOPS Symposium on Operating Systems Principles (SOSP’07). Association for Computing Machinery. 205-220. 6. Cavoukian, A. 2008. Privacy in the clouds. White Paper on Privacy and Digital Identity: Implications for the Internet. http://www. ipc. on. ca/images/Resources/privacyintheclouds.pdf. 7. Cooper, B., Ramakrishnan, R., et al. 2008. PNUTS: Yahoo!’s hosted data serving platform. Proc. VLDB Endow. 1, 2. 1,277-1,288. 8. Dean, J. and Ghemawat, S. 2008. MapReduce: Simplified data processing on large clusters. Comm. ACM 51, 1. 107-113. 9. Ganesh, L., Weatherspoon, H., Balakrishnan, M., and Birman K. 2007. Optimizing power consumption in large-scale storage systems. In Proceedings of HotOS. Crossroads
www.acm.org/crossroads
Spring 2010/ Vol. 16, No. 3
13
Scientific Workflows and Clouds By Gideon Juve and Ewa Deelman
I
n recent years, empirical science has been evolving from physical experimentation to computationbased research. In astronomy, researchers seldom spend time at a telescope, but instead access the large number of image databases that are created and curated by the community [42]. In bioinformatics, data repositories hosted by entities such as the National Institutes of Health [29] provide the data gathered by Genome-Wide Association Studies and enable researchers to link particular genotypes to a variety of diseases. Besides public data repositories, scientific collaborations maintain community-wide data resources. For example, in gravitational-wave physics, the Laser Interferometer Gravitational-Wave Observatory [3] maintains geographically distributed repositories holding time-series data collected by the instruments and their associated metadata. Along with the large increase in online data, the need to process these data is growing. In addition to traditional high performance computing (HPC) centers, a nation-wide cyberinfrastructure—a computational environment, usually distributed, that hosts a number of heterogeneous resources; cyberinfrastructure could refer to both grids and clouds or a mix of the two—is being provided to the scientific community, including the Open Science Grid (OSG) [36] and the TeraGrid [47]. These infrastructures, also known as grids [13], allow access to highperformance resources over wide area networks. For example, the TeraGrid is composed of computational and data resources at Indiana University, Louisiana University, University of Illinois, and others. These resources are accessible to users for storing data and performing parallel and sequential computations. They provide remote login access as well as remote data transfer and job scheduling capabilities. Scientific workflows are used to bring together these various data and compute resources and answer complex research questions. Workflows describe the relationship of the individual computational components and their input and output data in a declarative way. In astronomy, scientists are using workflows to generate science-grade mosaics of the sky [26], to examine the structure of galaxies [46], and, in general, to understand the structure of the universe. In bioinformatics, researchers are using workflows to understand the underpinnings of complex diseases [34, 44]. In earthquake science, workflows are used to predict the magnitude of earthquakes within a geographic area over a period of time [10]. In physics, workflows are used to search for gravitational waves [5] and model the structure of atoms [40]. In ecology, scientists use workflows to explore the issues of biodiversity [21]. Today, workflow applications are running on national and international cyberinfrastructures such as OSG, TeraGrid, and EGEE [11]. The broad spectrum of distributed computing provides unique opportunities for large-scale, complex scientific applications in terms of resource selection, performance optimization, and reliability. In addition to the large-scale cyberinfrastructure, applications can target campus clusters, or utility computing platforms such as commercial [1, 17] and academic clouds [31]. However, these opportunities also bring with them many challenges. It’s hard to decide which resources to use and how long they will
14
Spring 2010/ Vol. 16, No. 3
be needed. It’s hard to determine what the cost-benefit tradeoffs are when running in a particular environment. And it’s difficult to achieve good performance and reliability for an application on a given system. Clouds have recently appeared as an option for on-demand computing. Originating in the business sector, clouds can provide computational and storage capacity when needed, which can result in infrastructure savings for a business. One idea driving cloud computing is that businesses can plan only for a sustained level of capacity while reaching out to the cloud for resources in times of peak demand. When using the cloud, consumers pay only for what they use in terms of computational resources, storage, and data transfer in and out of the cloud. Although clouds were built primarily with business computing needs in mind, they are also being considered in science. In this article we focus primarily on workflow-based scientific applications and describe how they can benefit from the new computing paradigm.
Workflow Applications Scientific workflows are being used today in a number of disciplines. They stitch together computational tasks so that they can be executed automatically and reliably on behalf of the researcher. These workflows are composed of a number of image-processing applications that discover the geometry of the input images on the sky, calculate the geometry of the output mosaic on the sky, re-project the flux in the input images to conform to the geometry of the output mosaic, model the background radiation in the input images to achieve common flux scales and background levels across the mosaic, and rectify the background that makes all constituent images conform to a common background level. These normalized images are added together to form the final mosaic. Figure 1 shows a mosaic of the Rho Oph dark cloud created using this workflow. Montage mosaics can be constructed in different sizes, which dictate the number of images and computational tasks in the workflow. For example, a 4-degree square mosaic (the moon is 0.5 degrees square) corresponds to a workflow with approximately 5,000 tasks and 750 input images. Workflow management systems enable the efficient and reliable execution of these tasks and manage the data products they produce (both intermediate and final). Figure 2 shows a graphical representation of a small Montage workflow containing 1,200 computational tasks. Workflow management systems such as Pegasus [4, 9, 39] orchestrate the execution of these tasks on desktops, grids, and clouds. Another example is from the earthquake science domain, where researchers use workflows to generate earthquake hazard maps of
www.acm.org/crossroads
Crossroads
Scientific Workflows and Clouds
Figure 1: In this 75x90 arcmin view of the Rho Oph dark cloud as seen by 2MASS, the three-color composite is constructed using Montage. J band is shown as blue, H as green, and K as red. (Image courtesy of Bruce Berriman and J. Davy Kirkpatrick.) Southern California [38]. These maps show the maximum seismic shaking that can be expected to happen in a given region over a period of time (typically 50 years). Figure 3 shows a map constructed from individual computational points. Each point is obtained from a hazard curve (shown around the map) and each curve is generated by a workflow containing approximately 800,000 to 1,000,000 computational tasks [6]. This application requires large-scale computing capabilities such as those provided by the NSF TeraGrid [47]. In order to support such workflows, software systems need to 1) adapt the workflows to the execution environment (which, by necessity, is often heterogeneous and distributed), 2) optimize workflows for performance to provide a reasonable time to solution,
Figure 2: A graphical representation of the Montage workflow with 1,200 computational tasks represented as ovals. The lines connecting the tasks represent data dependencies.
Crossroads
Figure 3: In this shake map of Southern California, points on the map indicate geographic sites where the CyberShake calculations were performed. The curves show the results of the calculations. (Image courtesy of CyberShake Working Group, Southern California Earthquake Center including Scott Callaghan, Kevin Milner, Patrick Small, and Tom Jordan.) 3) provide reliability so that scientists do not have to manage the potentially large numbers of failures, and 4) manage data so that it can be easily found and accessed at the end of the execution.
Science Clouds Today, clouds are also emerging in academia, providing a limited number of computational platforms on demand: Cumulus [49], Eucalyptus [33], Nimbus [31], OpenNebula [43]. These science clouds provide a great opportunity for researchers to test out their ideas and harden codes before investing more significant resources and money into the potentially larger-scale commercial infrastructure. To support the needs of a large number of different users with different demands in the software environment, clouds are primarily built using resource virtualization technologies [2, 7, 50] that enable the hosting of a number of different operating systems and associated software and configurations on a single hardware host. Clouds that provide computational capacities (Amazon EC2 [1], Nimbus, Cumulus) are often referred to as an infrastructure as a service (IaaS) because they provide the basic computing resources needed to deploy applications and services. Platform as a service (PaaS) clouds such as Google App Engine [17] provide an entire application development environment including frameworks, libraries, and a deployment container. Finally, software as a service (SaaS) clouds provide complete end-user applications for tasks such as photo sharing, instant messaging [25], and many others. Commercial clouds were built with business users in mind, but scientific applications can benefit from them as well. Scientists, however, often have different requirements than enterprise customers. In particular, scientific codes often have parallel components and use MPI [18] or shared memory to manage message-based communication between processors. More coarse-grained parallel applications such as workflows rely on a shared file system to pass data between processes.
www.acm.org/crossroads
Spring 2010/ Vol. 16, No. 3
15
Gideon Juve and Ewa Deelman
Additionally, scientific applications are often composed of many interdependent tasks and consume and produce large amounts of data (often in the Terabyte range [5, 10]). Clouds are similar to grids, in that they can be configured (with additional work and tools) to look like a remote cluster, presenting interfaces for remote job submission and data transfer. As such, scientists can use existing grid software and tools to get their work done. Another interesting aspect of the cloud is that, by default, it includes resource provisioning as part of the usage mode. Unlike the grid, where jobs are often executed on a best-effort basis, when running on the cloud, a user requests a certain amount of resources and has them dedicated for a given duration of time. How many resources and how fast one can request them is an open question. Resource provisioning is particularly useful for workflow-based applications, where overheads of scheduling individual, inter-dependent tasks in isolation (as it is done by grid clusters) can be very costly. For example, if there are two dependent jobs in the workflow, the second job will not be released to a local resource manager on the cluster until the first job successfully completes. Thus the second job will incur additional queuing delays. In the provisioned case, as soon as the first job finishes, the second job is released to the local resource manager and since the resource is dedicated, it can be scheduled right away. Thus the overall workflow can be executed much more efficiently. Virtualization also opens up a greater number of resources to legacy applications. These applications are often very brittle and require a very specific software environment to execute successfully. Today, scientists struggle to make the codes that they rely on for weather prediction, ocean modeling, and many other computations work on different execution sites. No one wants to touch the codes that have been designed and validated many years ago in fear of breaking their scientific quality. Clouds and their use of virtualization technologies may make these legacy codes much easier to run. With virtualization, the environment can be customized with a given OS, libraries, software packages, and the like. The needed directory structure can be created to anchor the application in its preferred location without interfering with other users of the system. The downside is that the environment needs to be created and this may require more knowledge and effort on the part of the scientist than they are willing or able to spend.
Scientific Workflows The canonical example of a cloud is Amazon’s Elastic Compute Cloud (EC2), which is part of Amazon Web Services (AWS). AWS services provide computational, storage, and communication infrastructure on-demand via web-based APIs. AWS offers five major services. 1. Elastic Compute Cloud (EC2): a service for provisioning virtual machine instances from Amazon’s compute cluster, which allows users to deploy virtual machine (VM) images with customized operating systems, libraries, and application code on a variety of predefined hardware configurations (CPU, memory, disk). 2. Simple Storage Service (S3): an object-based storage system for the reliable storage of binary objects (typically files), which provides operations to “put” and “get” objects from a global object store that is accessible both inside and outside Amazon’s cloud.
16
Spring 2010/ Vol. 16, No. 3
3. Elastic Block Store: a block-based storage system that provides network attached storage volumes to EC2. Volumes can be attached to an EC2 instance as block device and formatted for use as reliable, unshared file system. 4. Simple Queue Service: a distributed queue service for sending messages between nodes in a distributed application, which allows messages queued by one node to be retrieved and processed by another. 5. SimpleDB: a structured key-value storage service, which enables database records to be stored, indexed and queried by key. In addition, Amazon’s cloud provides services for monitoring (CloudWatch), parallel computing (Elastic MapReduce), relational storage (RDS), and others. There are many ways to deploy a scientific workflow on a cloud, depending on the services offered by the cloud and the requirements of the workflow management system. Many of the existing workflows were developed for HPC systems such as clusters, grids and supercomputers. Porting these workflows to the cloud involves either adapting the workflow to the cloud or adapting the cloud to the workflow. Adapting the workflow to the cloud involves changing the workflow to take advantage of cloud-specific services. For example, rather than using a batch scheduler to distribute workflow tasks to cluster nodes, a workflow running on Amazon’s cloud could make use of the Simple Queue Service. Adapting the cloud to the workflow involves configuring the cloud to resemble the environment for which the application was created. For example, an HPC cluster can be emulated in Amazon EC2 by provisioning one VM instance to act as a head node running a batch scheduler, and several others to act as worker nodes. One of the great benefits of the cloud for workflow applications is that both adaptation approaches are possible. Scientific workflows require large quantities of compute cycles to process tasks. In the cloud, these cycles are provided by virtual machines such as those provided by Amazon EC2. Many virtual machine instances must be used simultaneously to achieve the performance required for large scale workflows. These collections of VMs, called “virtual clusters” [12], can be managed using existing offthe-shelf batch schedulers such as PBS [34, 48] or Condor [8, 24]. Setting up a virtual cluster in the cloud involves complex configuration steps that can be tedious and error-prone. To automate this process, software such as Nimbus Context Broker [22] can be used. This software gathers information about the virtual cluster and uses it to generate configuration files and start services on cluster VMs. In addition to compute cycles, scientific workflows rely on shared storage systems for communicating data between workflow tasks distributed across a group of nodes, and for storing input and output data. To achieve good performance, these storage systems must scale well to handle data from multiple workflow tasks running in parallel on separate nodes. When running on HPC systems, workflows can usually make use of a high-performance, parallel file system such as Lustre [45], GPFS [41], or Panasas [37]. In the cloud, workflows can either make use of a cloud storage service, or they can deploy their own shared file system. To use a cloud storage service, the workflow management system would likely need to change the way it manages data. For example, to use Amazon S3, a workflow task needs to fetch input data from S3 to
www.acm.org/crossroads
Crossroads
Scientific Workflows and Clouds
a local disk, perform its computation, then transfer output data from the local disk back to S3. Making multiple copies in this way can reduce workflow performance. Another alternative would be to deploy a file system in the cloud that could be used by the workflow. For example, in Amazon EC2, an extra VM can be started to host an NFS file system and worker VMs can mount that file system as a local partition. If better performance is needed then several VMs can be started to host a parallel file system such as PVFS [23, 52] or GlusterFS [16]. Although clouds like Amazon’s already provide several good alternatives to HPC systems for workflow computation, communication and storage, there are still challenges to overcome. Virtualization overhead. Although virtualization provides greater flexibility, it comes with a performance cost. This cost comes from intercepting and simulating certain low-level operating system calls while the VM is running. In addition, there is the overhead of deploying and unpacking VM images before the VM can start. These overheads are critical for scientific workflows because in many cases the entire point of using a workflow is to run a computation in parallel to improve performance. Current estimates put the overhead of existing virtualization software at around 10 percent [2, 15, 51] and VM startup time takes between 15 and 80 seconds depending on the size of the VM image [19, 32]. Fortunately, advances in virtualization technology, such as improved hardware-assisted virtualization, may reduce or eliminate runtime overheads in the future. Lack of shared or parallel file systems. Although clouds provide many different types of shared storage systems, they are not typically designed for use as file systems. For example, Amazon EBS does not allow volumes to be mounted on multiple instances, and Amazon S3 does not provide a standard file system interface. To run on a cloud like Amazon’s, a workflow application must either be modified to use these different storage systems, which takes time, or they must create their own file system using services available in the cloud, which is at least difficult and potentially impossible depending on the file system desired (for example, Lustre cannot be deployed on Amazon EC2 because it requires kernel modifications that EC2 does not allow). Relatively slow networks. In addition to fast storage systems, scientific workflows rely on high-performance networks to transfer data quickly between tasks running on different hosts. The HPC systems typically used for scientific workflows are built using high-bandwidth, lowlatency networks such as InfiniBand [20] and Myrinet [27]. In comparison, most existing commercial clouds are equipped with commodity gigabit Ethernet, which results in poor performance for demanding workflow applications. Fortunately, the use of commodity networking hardware is not a fundamental characteristic of clouds and it should be possible to build clouds with high-performance networks in the future.
Future Outlook While many scientists can make use of existing clouds that were designed with business users in mind, in the future we are likely to see a great proliferation of clouds that have been designed specifically for science applications. We already see science clouds being deployed at traditional academic computing centers [14, 28, 30]. One can imagine that these science clouds will be similar to existing clouds, but will come equipped with features and services that are even more useful to computational scientists. Like existing clouds, they will potentially Crossroads
come in a variety of flavors depending on the level of abstraction desired by the user. IaaS science clouds could provide access to the kinds of high-performance infrastructure found in HPC systems such as high-speed networks, and parallel storage systems. In addition they could come with science-oriented infrastructure services such as workflow services and batch scheduling services. PaaS science clouds could be similar to the science portals and gateways used today. They could provide tools for scientists to develop and deploy applications using domain-specific APIs and frameworks. Such systems could include access to collections of datasets used by the scientists, such as genome repositories and astronomical image archives. Finally, some commonly used science applications could be deployed using a SaaS model. These applications would allow scientists from around the world to upload their data for processing and analysis. Additionally, HPC centers are looking at expanding their own infrastructure by relying on cloud technologies to virtualize local clusters, which would allow them to provide customized environments to a wide variety of users in order to meet their specific requirements. At the same time, HPC centers can also make use of commercial clouds to supplement their local resources when user demand is high. Clearly, clouds can be directly beneficial to HPC centers where the staff is technically savvy. However, the adoption of clouds for domain scientists depends strongly on the availability of tools that would make it easy to leverage the cloud for scientific computations and data management.
Biographies Gideon Juve is a PhD student in computer science at the University of Southern California. His research interests include distributed and highperformance computing, scientific workflows, and computational science. Ewa Deelman is a research associate professor at the University of Southern California Computer Science Department and a project leader at the USC Information Sciences Institute, where she heads the Pegasus project, which designs and implements workflow mapping techniques for large-scale workflows running in distributed environments.
References 1. Amazon. Elastic compute cloud. http://aws.amazon.com/ec2/. 2. Barham, P., Dragovic, B., Fraser, K., Hand, S., Harris, T., Ho, A., Neugebauer, R., Pratt, I., and Warfield, A. 2003. Xen and the art of virtualization. In Proceedings of the 19th ACM Symposium on Operating Systems Principles. 164-177. 3. Barish, B. C. and Weiss, R. 1999. LIGO and the detection of gravitational Waves. Physics Today 52. 44. 4. Berriman, G. B., Deelman, E., Good, J., Jacob, J., Katz, D. S., Kesselman, C., Laity, A., Prince, T. A., Singh, G., and Su, M.-H. 2004. Montage: A grid enabled engine for delivering custom science-grade mosaics on demand. In SPIE Conference 5487: Astronomical Telescopes. 5. Brown, D. A., Brady, P. R., Dietz, A., Cao, J., Johnson, B., and McNabb, J. 2006. A case study on the use of workflow technologies for scientific analysis: Gravitational wave data analysis. In Workflows for e-Science, Taylor, I., Deelman, E., Gannon, D., and Shields, M., Eds., Springer. 6. Callaghan, S., Maechling, P., Deelman, E., Vahi, K., Mehta, G., Juve, G., Milner, K., Graves, R., Field, E., Okaya, D., Gunter, D., Beattie, K., and Jordan, T. 2008. Reducing time-to-solution using distributed highthroughput mega-workflows—Experiences from SCEC CyberShake.
www.acm.org/crossroads
Spring 2010/ Vol. 16, No. 3
17
Gideon Juve and Ewa Deelman
In Proceedings of the 4th IEEE International Conference on e-Science (e-SCIENCE’08).
29. NCBI. The database of genotypes and phenotypes (dbGaP). 2009. http://www.ncbi.nlm.nih.gov/gap.
7. Clark, B., Deshane, T., Dow, E., Evanchik, S., Finlayson, M., Herne, J., and Matthews, J. N. 2004. Xen and the art of repeated research. In Proceedings of the USENIX Annual Technical Conference, FREENIX Track. 135–144. 8. Condor. http://www.cs.wisc.edu/condor. 9. Deelman, E., Singh, G., Su, M.-H., Blythe, J., Gil, Y., Kesselman, C., Mehta, G., Vahi, K., Berriman, G. B., Good, J., Laity, A., Jacob, J. C., and Katz, D. S. 2005. Pegasus: A framework for mapping complex scientific workflows onto distributed systems. Scientific Program. J. 13. 219-237. 10. Deelman, E., Callaghan, S., Field, E., Francoeur, H., Graves, R., Gupta, N., Gupta, V., Jordan, T. H., Kesselman, C., Maechling, P., Mehringer, J., Mehta, G., Okaya, D., Vahi, K., and Zhao, L. 2006. Managing large-scale workflow execution from resource provisioning to provenance tracking: The CyberShake example. In Proceedings of the 2nd IEEE International Conference on e-Science and Grid Computing (e-SCIENCE’06). 14.
31. Nimbus Science Cloud. http://workspace.globus.org/clouds/ nimbus.html. 32. Nurmi, D., Wolski, R., Grzegorczyk, C., Obertelli, G., Soman, S., Youseff, L., and Zagorodnov, D. 2008. Eucalyptus: A technical report on an elastic utility computing architecture linking your programs to useful systems. Computer Science Tech. rep. 2008-10. University of California, Santa Barbara. 33. Nurmi, D., Wolski, R., Grzegorczyk, C., Obertelli, G., Soman, S., Youseff, L., and Zagorodnov, D. 2008. The Eucalyptus open-source cloud-computing system. In Cloud Computing and its Applications. 34. Oinn, T., Li, P., Kell, D. B., Goble, C., Goderis, A., Greenwood, M., Hull, D., Stevens, R., Turi, D., and Zhao, J. 2006. Taverna/myGrid: Aligning a workflow system with the life sciences community. In Workflows in e-Science, Taylor, I., Deelman,E., Gannon, D., and Shields, M., Eds., Springer.
11. EGEE Project. Enabling Grids for E-sciencE. http://www.eu-egee.org/.
35. OpenPBS. http://www.openpbs.org.
12. Foster, I., Freeman, T., Keahey, K., Scheftner, D., Sotomayer, B., and Zhang, X. 2006. Virtual Clusters for Grid Communities. In Proceedings of the 6th IEEE International Symposium on Cluster Computing and the Grid (CCGRID’06). 513-520.
36. Open Science Grid. http://www.opensciencegrid.org.
13. Foster, I., Kesselman, C., and Tuecke, S. 2001. The anatomy of the grid: Enabling scalable virtual organizations. Int. J. High Perform. Comput. Appl. 15. 200-222. 14. FutureGrid. http://futuregrid.org/. 15. Gilbert, L., Tseng, J., Newman, R., Iqbal, S., Pepper, R., Celebioglu, O., Hsieh, J., and Cobban, M. 2005. Performance implications of virtualization and hyper-threading on high energy physics applications in a grid environment. In Proceedings of the 19th IEEE International Parallel and Distributed Processing Symposium (IPDPS’05). 16. Gluster Inc. GlusterFS. http://www.gluster.org. 17. Google App Engine. http://code.google.com/appengine/. 18. Gropp, W., Lusk, E., and Skjellum, A. 1994. Using MPI: Portable Parallel Programming with the Message Passing Interface. MIT Press, Cambridge, MA. 19. Hyperic Inc. CloudStatus. http://www.cloudstatus.com. 20. InfiniBand Trade Association. InfiniBand. http://www.infinibandta. org/.
37. Panasas Inc. Panasas. http://www.panasas.com. 38. Paul, R. W. G., Somerville, G., Day, S. M., and Olsen, K. B. 2006. Ground motion environment of the Los Angeles region. Structural Design Tall Special Buildings 15. 483-494. 39. Pegasus. http://pegasus.isi.edu. 40. Piccoli, L. 2008. Lattice QCD workflows: A case study. In Challenging Issues in Workflow Applications (SWBES’08). 41. Schmuck, F. and Haskin, R. 2002. GPFS: A shared-disk file system for large computing clusters. In Proceedings of the 1st USENIX Conference on File and Storage Technologies. 42. Skrutskie, M. F., Schneider, S. E., Stiening, R., Strom, S. E., Weinberg, M. D., Beichman, C., Chester, T., Cutri, R., Lonsdale, C., and Elias, J. 1997. The Two Micron All Sky Survey (2MASS): Overview and status. In The Impact of Large Scale Near-IR Sky Surveys, F. Garzon, F., et al, Eds., Kluwer Academic Publishing Company, Dordrecht. 25. 43. Sotomayor, B., Montero, R., Llorente, I., and Foster, I. 2008. Capacity leasing in cloud systems using the opennebula engine. In Cloud Computing and Applications.
21. Jones, M., Ludascher, B., Pennington, D., and Rajasekar, A. 2005. Data integration and workflow solutions for ecology. In Data Integration in Life Sciences.
44. Stevens, R. D., Robinson, A. J., and Goble, C. A. 2003. myGrid: Personalised bioinformatics on the information grid. Bioinformatics 19.
22. Keahey, K. and Freeman, T. 2008. Contextualization: Providing one-click virtual clusters. In Proceedings of the 4th International Conference on eScience (e-SCIENCE’08).
46. Taylor, I., Shields, M., Wang, I., and Philp, R. 2003. Distributed P2P computing within Triana: A galaxy visualization test case. In Proceedings of the IEEE International Parallel and Distributed Processings Symposium (IPDPS’03).
23. Ligon, W. B. and Ross, R. B. 1996. Implementation and performance of a parallel file system for high performance distributed applications. In Proceedings of the 5th IEEE International Symposium on High Performance Distributed Computing. 471–480. 24. Litzkow, M. J., Livny, M., and Mutka, M. W. 1988. Condor: A hunter of idle workstations. In Proceedings of the 8th International Conference on Distributed Computing Systems. 104-111. 25. Microsoft. Software as a service. http://www.microsoft.com/ serviceproviders/saas/default.mspx. 26. Montage. http://montage.ipac.caltech.edu. 27. Myricom. Myrinet. http://www.myri.com/myrinet/. 28. NASA Ames Research Center. Nebula. http://nebula.nasa.gov.
18
30. NERSC. Magellan. http://www.nersc.gov/nusers/systems/magellan.
Spring 2010/ Vol. 16, No. 3
45. Sun Microsystems. Lustre. http://www.lustre.org.
47. TeraGrid. http://www.teragrid.org/. 48. Torque. http://supercluster.org/torque. 49. Wang, L., Tao, J., Kunze, M., Rattu, D., and Castellanos, A. C. 2008. The Cumulus Project: Build a scientific cloud for a data center. In Cloud Computing and its Applications. Chicago. 50. Xenidis, J. 2005. rHype: IBM research hypervisor. IBM Research. 51. Youseff, L., Wolski, R., Gorda, B., and Krintz, C. 2006. Paravirtualization for HPC systems. In Lecture Notes in Computer Science, vol. 4331, 474. 52. Yu, W. and Vetter, J. S. 2008. Xen-based HPC: A parallel I/O perspective. In Proceedings of the 8th IEEE International Symposium on Cluster Computing and the Grid (CCGrid’08).
www.acm.org/crossroads
Crossroads
The Cloud at Work, Interviews with Pete Beckman of Argonne National Lab and Bradley Horowitz of Google data, are located remotely, and scientists have to then work out protocols, policies, and security measures necessary to run stuffs remotely or to get to the data that is remote. So, grid computing was focused primarily on sharing the resources among the providers and the genesis of cloud computing came from some technologies that allowed folks to do this in a very clear and sandboxed way. They provide a very definitive and well-described interface and also allow you to run a piece of code or software remotely. Virtual machine technology in particular has made this much easier than in the past. Rather than the complicated nature of deciding which software can be run and which packages are available, now you are able to ship the entire virtual machine to other sites, thereby allowing for utility computing, another way we refer to cloud computing. The challenge that still remains is data, how to best share the data.
Pete Beckman, director of the Argonne Leadership Computing Facility, Argonne National Laboratory, interviewed by Sumit Narayan Pete Beckman is the director of the Argonne Leadership Computing Facility at Argonne National Laboratory (ANL) in Illinois. Argonne National Lab is the United State’s first science and engineering research laboratory as well as home to one of the world’s fastest supercomputers. Beckman explains cloud computing from a scientist’s perspective, and speculates where it might be headed next. (He also notes that Argonne has a well-developed student internship program, but not enough candidates!) —Sumit Narayan Sumit Narayan: “Cloud computing” is the new buzz word among computer scientists and technologists. It’s used in different ways to define a variety of things. Tell us a little about the origins of cloud computing. What does cloud computing mean to you? Pete Beckman: Distributed computing, which people have often referred as “computing out there,” as opposed to “computing on your local machine,” has been around for a very long time, somewhere around 20 years. We went from a period of distributed computing, to meta-computing, to grid computing, and now to cloud computing. They’re all a little different, but the notion is that the services, either compute or Crossroads
SN: What are they key new technologies behind cloud computing? Can you talk a little about the different services available through cloud computing? PB: From a technical standpoint, the only one that has a strong root in technology is the virtual machine-based “infrastructure as a service” (IaaS). That’s the only one that has a technological breakthrough. All the others are just a model breakthrough. In other words, the idea that I could store my data locally or remotely has been around for a long time. The idea that I can create a web application that makes it look like I’m running something locally when it is really remote—these are model differences in terms of APIs and providing a per-user capacity and so forth. The technology, the one that is really a technological breakthrough, is using and shipping around virtual machines. An example of model breakthrough is what people are doing when they say they are going to run their email on the cloud. To an organization, it looks like they have their emails present locally like they used to. This is because they never really had it close to them on a server down the hall. They were probably POP’ping, or IMAP’ping to a server that was within their infrastructure, but probably in another building. When people move their email to the cloud, they are now getting that as a service remotely and are being charged an incremental fee, like a per-user fee. There is no provisioning that the site has to do for hosting and running their own machines. To the user, it all looks the same, except that the IMAP server is now some other IMAP server. The people are doing the same for calendars, HR, travel, etc. So all these systems which used to reside locally in an IT organization on-site are getting cloud versions, which essentially, only requires a network connection. Now the virtual machine part, that’s really a technological breakthrough that allows me to run anything, not just what they provide in a package like POP or IMAP, but anything I want. That is unique and the new thing over the last couple of years.
www.acm.org/crossroads
Spring 2010/ Vol. 16, No. 3
19
SN: Do you think the era of personal computers is coming to a close? Will we increasingly rely on cloud services like Google for search, computation, on Dropbox/S3 for storage, or Skype for communication? PB: It is changing. Whether or not it comes to a close, probably not. But it is changing dramatically. Let me give you a couple of examples. There already are netbooks. People are migrating into using a really cheap portable device. We have had lightweight Linux for quite some time now. You may also know about Google’s Chrome OS. I have one of the initial versions of a netbook, and it is really cool. You can do simple student stuff: web, email, PDFs, Skype, basic spreadsheet, and Word documents. But now, a lot of people are using Google Docs for that. If you look at the future of computing, if we really get to a ubiquitous network and all these things rely on network, then these lightweight portal devices will become the way many people access their machine or their servers, or cloud services. Some things still need intense local hacking, for example, image or media editing. But, even those are making their way out into the cloud. There is a great Photoshop-as-a-service app, that lets you upload a picture and then using a web interface, change the color, scratch it, shrink it, crop it, etc. And again, you don’t really require a high-end notebook for that. An inexpensive Atom processor on your netbook is enough. With respect to media editing, Animoto is an example one that is happening in the cloud. However, for most home users, there will still be a gap. Uploading photos or videos of my kids to the cloud for editing is probably still out of reach. But not for long. SN: Cloud computing is well suited for business needs. As a scientist, why do you think cloud computing is important? What opportunities is cloud computing opening in the scientific community that weren’t supported by big computing clusters like Blue Gene? PB: Scientists have been using computers and hosting data remotely for a long time. They are used to this. It’s kind of funny when people who are not from the scientific community visit Argonne, because they imagine scientists here come to use our supercomputer. The fact is, they don’t. They are able to do [their computing] using SSH on their laptop from a coffee shop in Italy. The scientific community has been doing its work remotely for a long time, either on a supercomputer or mid-range machines. But now instead of the science community provisioning all these servers and mid-range computers, we will be able to allow the cloud to do that for us. Of course, supercomputers are still super. They are different from cloud resources that are primarily useful for mid-range computing. Argonne has a project for cloud computing, Magellan, funded by the U.S. Department of Energy for research into the scientific community and the cloud as a way to free up scientists so that they are not provisioning, or setting up two or three servers down the hall. The other thing that’s changing is, in the past, supercomputers had a very well-defined set of software stacks—the package is either in the stack or not, in terms of support for that package. But with IaaS cloud architecture, scientists can customize and make their own special stacks. We see this a lot in metagenomics and biology, where scientists have a very complex workflow of 10 different tools and they want to create a web interface for that. They want it all together in a package so that they can run their genomics application. Doing it in a virtual
20
Spring 2010/ Vol. 16, No. 3
machine means they don’t have to worry whether their package is supported, or if they have the right version of Perl, or if they added the Java bindings to MySQL, they can just put everything together in a virtual machine and ship it around and run it wherever. SN: Is there anything about cloud computing that worries you, like security, infrastructure? What are the risks? PB: Security is very complex in a virtualized environment. It becomes a very big challenge and there are a couple of things we want to be able to do. We really want to give people a total virtual machine. That means we would be giving them root access on a virtual machine. In the past, the language we have used to describe security has been to say that there is a “user,” and an “escalated privileged user,” like a root. All the documentation, discussion, cyber-security plans differentiate those two very clearly. Users and escalated privileged user— administrator, or a root. In a virtualized environment, when you give someone a virtual machine, they have the root access on that virtual machine, not on the complete infrastructure. As an analogy, you can think of it as someone being able to turn a mobile phone on and off. That’s administrator privileges on the phone. But you don’t get to control the cell towers. They’re still controlled by someone else: the mobile phone service providers. So this notion of security really has to change. We have a lot yet to explore and change, and there will be a lot of research in that space. Another thing of course will be, if you do something that you are not supposed to do when you are using a virtual machine, who is to blame? Is it the users to whom you handed the virtual machine? Are they responsible for all the security? If they upload their virtual machine that has a bunch of security holes in it and someone gets into that—how do we stop that? How do we manage that risk? How do we scan our own virtual machine? So that’s a pretty big research area, and one we will be exploring at Argonne. SN: How do you think cloud computing would impact education? PB: Oh I think cloud computing is just amazingly fun and fantastic for education, largely because of its low barrier to entry. If you look at the Beowulf, the cluster mailing lists, there are people who have set up their own clusters at various places. These are school-aged kids who have enormous amounts of energy, and they get a couple of old or new computers and wire them together. Occasionally, you’ll see stories about folks who have a 16-node cluster bought from the cheapest parts possible on the planet. These machines have no cases! They’re just sitting on a table with their motherboards, and they work just fine. There certainly is value in that sort of hacking, but a lot of colleges do not have that sort of expertise. Yet, they want to teach parallel processing, MPI programming and scientific calculations, MATLAB. Cloud computing offers promise here. I can imagine in the future a student just being handed 100 credithours on the cloud. The professor would say, “We are going to do a homework assignment. We are going to write a code that calculates the surface area to volume ratio of this system with each particle moving around.” Now, each student has his or her own credit hours that he or she uses in the cloud to do the computation. That’s the sort of thing where we are likely to be headed, and it’s fantastic for universities! More and more students can get access to
www.acm.org/crossroads
Crossroads
The Cloud at Work
resources. You can do a simple MATLAB thing or parallel processing, or write a new ray-tracer and do some visualization. However, it will still be mid-range computing. It won’t have the impact of a 5,000-core supercomputer that has Infiniband, but it’s a great way to get students started and tinkering. I can imagine five years from now universities routinely handing out cloud credit hours to students. SN: What are the opportunities for students to consider, both at ANL and outside? PB: Argonne has a lot of student interns. We have a fantastic student program. Usually, our problem is that we cannot find enough students! It’s not that we don’t have enough slots for undergraduates or graduates who are in computational science or computer science—we don’t have enough students! Argonne has a catalog (www.dep.anl.gov) that lists all its projects. Students can essentially apply to a project for a summer position. But with respect to the Magellan project, we have a web site that we are still working on: www.megallen.alcf.anl.gov. There, we will have a place to apply for cycles or time on the cloud machine. And if a student has a fantastic idea for exploring cloud computing in some way that benefits the lab and is in line with the mission of this lab in understanding cloud computing, then they can get time on the machine. SN: What is your vision for the future, with respect to cloud computing? Where do you see cloud computing in ten years? Fifty years? PB: We are slowly moving to faster and faster networks, even with respect to our homes. We can imagine that as we improve that last mile, more things will be stored in the cloud, everything from your home network and 802.11x, all the way to businesses relying on cloud services that will be spread out around multiple data servers. You’re never going to be without email, your pictures, your data because it is replicated somewhere in the cloud. We are rapidly moving towards that. Now, providing for that mid-range computing is where the science will go: We will be hosting climate data, earth data, and so forth, and we’ll allow scientists to slice and dice and explore the data in the cloud here at Argonne and elsewhere. There will always be the need for a supercomputer that goes beyond commodity inexpensive computing to provide a capability that you can do in a high-end environment. You probably have read stories about Google’s data centers and how cheap they are. Google carefully calculates every penny spent, in terms of motherboards and CPUs and hard disks. For high-performance computing, we do that too, but on a different level, optimizing different variables. We’re not optimizing the cost to run queries per second that can force an embarrassingly parallel search, but instead, we’re trying to figure out how fast we can simulate 100 years of climate change. We need specialized architecture for that, and we will always need high-end architectures like Blue Gene that are low-powered but still massively parallel. In the future, we will probably see a move toward cloud computing for mid-range capabilities like email, calendaring, and other services that businesses now sometimes host on site. And, we will also have space where we will solve the world’s most challenging problems in climate, materials, biology or genomics on very high-performance machines like exa-scale machines. Crossroads
Bradley Horowitz, vice president of Product Management, Google, interview by Chris Heiden Bradley Horowitz oversees product management for Google Apps, including Gmail, Calendar, Google Talk, Google Voice, Google Docs, Blogger, and Picasa. Before joining Google, he led Yahoo!’s advanced development division, which developed new products such as Yahoo! Pipes, and drove the acquisition of products such as Flickr and MyBlogLog. Previously, he was co-founder and CTO of Virage, where he oversaw the technical direction of the company from its founding through its IPO and eventual acquisition by Autonomy. Horowitz holds a bachelor’s degree in computer science from the University of Michigan, and a master’s degree from the MIT Media Lab and was pursuing his PhD there when he co-founded Virage. Here, he discusses the issues affecting cloud computing and how we must address them going forward. The ubiquitous nature of computing today is a perfect fit for computing in the cloud. —Chris Heiden Chris Heiden: Describe your background and how it brought you into cloud computing. Bradley Horowitz: Much of my research has involved looking at what computers do well and what people do well, and figuring out how to marry the two most effectively. Over time, that line has shifted, and computers now do many things that used to require a lot of manual effort. Much of it requires very large networked resources, and very large data sets. A good example is face recognition. These kinds of tasks are much more easily accomplished in the cloud, both in terms of computing power and data sets. And Google runs one of the largest— if not the largest—“cloud computers” on the planet. So it’s great to be able to build applications that run on this massively scaled architecture.
www.acm.org/crossroads
Spring 2010/ Vol. 16, No. 3
21
More broadly, I’ve always been interested in the way people socialize and collaborate online. The most interesting part of cloud computing is in the network and the interaction between computers—not in individual computers themselves. CH: What do you see as the single most important issue affecting cloud computing today? BH: The network is getting exponentially faster. It’s Moore’s Law meeting Nielsen’s Law. Not only can you pack tremendous computing power into a mobile phone to access the cloud anywhere, but bandwidth is growing exponentially as well, roughly 50 percent per year. And it’s starting to be available everywhere. At this point you expect the network to be fast and always-on. Airplanes used to be off-limits to cloud computing, but not anymore. WiFi is becoming standard on planes. People now buy phones expecting 3G data connections—talking has become the “oh yeah” feature at the bottom of a list of a dozen web and data features.
pens behind closed doors. The nice thing about cloud computing is that all these service providers are so publicly accountable for even the slightest glitch. There’s no tolerance for downtime, which is great for users. Google’s cloud is made up of a highly resilient network of thousands and thousands of disposable computers, with apps and data spread out on them across geographies. Gmail, for example, is replicated live across multiple datacenters, so if a meteor hits one, the app and data keeps on running smoothly on another. And efforts like the Data Liberation Front and Google Dashboard make it clear that users maintain control of their data, even when they’re using the cloud. They can take their stuff to go anytime they like, and they can always see how the data is being used. The value to you if you’re a cloud provider is not in locking in users’ data; rather, it’s in the service, the flow of data, transactions, choices. Lock-in and closed formats are a losing strategy. They make you lazy as a provider. And on the web, laziness is deadly, since competitors are a click away.
CH: Describe how much of an impact cloud computing will have in CH: What are some of the aspects of cloud computing that you are the next evolution in computing. How will it affect the everyday comworking on that will revolutionize the field? puter user? BH: Everything “cloud” is really about collaboration. It’s not just a BH: This shift to cloud computing has already started invisibly for larger computer—cloud computing gives birth to a new way of work- most users. Most people use webmail without realizing it’s essentially ing, where the stuff you produce lives in the center of a social circle a “cloud” app. That may be how they start using other cloud apps. They and can be collaborated on by everyone in that circle. We’re trying to get invited to collaborate on a document online, and they use that doc make that collaboration process every day, and then another, and Lock-in and closed formats are a losing more seamless, so sharing is easy one day they wake up and realize it’s and you can work on stuff togethstrategy. They make you lazy as a provider. been months since they’ve opened er without worrying about the up their desktop software. And on the web, laziness is deadly, since mechanics of version control and But it’s going to start getting competitors are a click away. invites and the like. more obvious as people switch to I’m not sure people have quite —Bradley Horowitz netbooks and smartphones, and grasped how much of computing as they stop having to worry about can and will shift to the cloud. It’s not that there isn’t ever-growing all the mechanics of backing of their discs, worrying about where they computing power on the desktop or in beefy all-purpose servers. But stored something, or hassling with document versions. It’s already the part of computing people really care about—that developers are making daily life more mobile and more fluid. Even the word developer developing for, and that yields the most interesting real-world appli- is becoming more and more synonymous with web developer. The web cations—is the networked part. As an example, you don’t use Google is now the primary platform for building apps, not an afterthought you Docs to write solo reports and print them out to stick on a shelf some- hook into an app after it’s built. where. You use it to get 10 people to hack out a plan everyone’s happy Cloud computing is already changing the way businesses, governwith in one hour. It’s the networked process that’s revolutionary, not ments, and universities run. The City of Los Angeles just switched to the app used in isolation. the cloud using Google Apps. Philadelphia International Airport cuts down on delays and costs by coordinating their operations using CH: What about concerns people have voiced about trusting cloud Google Docs, and they’re looking at technologies like Google Wave too. computing? Do you see these concerns as slowing adoption? And it’s individuals, too. We hear about soldiers in the desert in BH: I’d flip that on its head. The providers actually can’t keep up with Iraq keeping in touch with their families using video chat in Gmail, user and customer demand. They can’t build this stuff fast enough for and checking their voicemail on Google Voice. what people want to do in the cloud. At universities, for example, it’s The movement away from the desktop and on-premise solutions students demanding their administrators switch to Gmail. Universities has been sluggish for software makers with entrenched interests on the like Arizona State and Notre Dame have switched to cloud-based email, desktop. They’re moving so slowly compared to what users want. and we’re seeing big businesses like Motorola making the switch, too. These fast, lightweight, social, mobile apps are what people are actually It helps that the stats are finally starting to get out comparing apples using to communicate with each other and get work done. They to apples. Desktop and on-premises applications break down far more sprouted in the cloud, they’re built to run in the cloud, and they’re now often than web apps, even if you don’t hear about it as often since it hap- “growing down” into big enterprises, driven by user demand.
❝
❞
22
Spring 2010/ Vol. 16, No. 3
www.acm.org/crossroads
Crossroads
State of Security Readiness By Ramaswamy Chandramouli and Peter Mell
C
loud computing is a model for enabling convenient, on-demand network access to a shared pool of configurable computing resources that can be rapidly provisioned and released with minimal management effort or service provider interaction. With this pay-as-you-go model of computing, cloud solutions are seen as having the potential to both dramatically reduce costs and increase the rapidity of development of applications. However, the security readiness of cloud computing is commonly cited among IT executives as the primary barrier preventing organizations from immediately leveraging this new technology. These problems are real and arise from the nature of cloud computing: broad network access, resource pooling, and on-demand service. In this article, we survey some of these challenges and the set of security requirements that may be demanded in the context of various cloud service offerings (noted in the article as No. 1, No. 2, and so on). The security challenges and requirements we survey not only involve core security operations, such as encryption of data at rest and in transit, but also contingency-related operations, such as failover measures. The survey touches upon the various artifacts or entities involved in IT services, such as the users, data, applications, computing platforms and hardware. We call the enterprise or government agency subscribing to the cloud services as the “cloud user” and the entity hosting the cloud services as the “cloud provider.” To further refine the definition of cloud computing presented above, we classify cloud computing service offerings into three service models.
Service Models Software as a service (SaaS). The capability provided to the consumer is the use of a provider’s applications running on a cloud infrastructure. The applications are accessible from various client devices through a thin client interface such as a web browser. The consumer does not manage or control the underlying cloud infrastructure, including network, servers, operating systems, storage, or even individual application capabilities, with the possible exception of limited user-specific application configuration settings. Examples of this include the case of a cloud provider offering a software application used for a specific business function, such as customer relationship management or human resources management, on a subscription or usage basis rather than the familiar purchase or licensing basis. Platform as a service (PaaS). The capability provided to the consumer is the deployment of consumer-created or acquired applications onto the cloud infrastructure. These applications are created using programming languages and tools supported by the provider. The consumer does not manage or control the underlying cloud infrastructure including network, servers, operating systems, or storage, but has control over the deployed applications and possibly application hosting environment configurations. Examples of this include the case of a cloud provider providing a set of tools for developing and deploying applications using various languages (for example, C, C++, Java) under a whole application framework (JEE, .NET, and so forth). Crossroads
Infrastructure as a service (IaaS). The capability provided to the consumer is provision processing, storage, networks, and other fundamental computing resources where the consumer is able to deploy and run arbitrary software, which can include operating systems and applications. The consumer does not manage or control the underlying cloud infrastructure but has control over operating systems, storage, deployed applications, and possibly limited control of select networking components (for example, host firewalls). Examples of this include the case of a cloud provider providing physical and virtual hardware (servers, storage volumes) for hosting and linking all enterprise applications and storing all enterprise data—in other words, the infrastructure backbone for an enterprise’s data center.
Survey of Security Challenges In reviewing the security challenges and requirements of cloud computing, we will look first at the necessary interactions between the cloud users, the users’ software clients, and the cloud infrastructure or services.
The Users When an enterprise subscribes to a cloud service, it may have a diverse user base consisting of not only its own employees but also its partners, suppliers, and contractors. In this scenario, the enterprise may need an effective identity and access management function and therefore require the following security requirements:
•
support for a federation protocol for authentication of users (No. 1) and
•
support for a standardized interface to enable the cloud user (or the cloud user’s system administrator) to provision and de-provision members of their user base (No. 2).
Many commercial cloud services are now beginning to provide support for the security assertion markup language (SAML) federation protocol (which contains authentication credentials in the form of SAML assertions) in addition to their own proprietary authentication protocol, and hence we do not see a big obstacle in meeting the first of the above requirements. As far as the user provisioning and de-provisioning requirement is concerned, many of the cloud providers still use their own proprietary interfaces for user management. There exist common, machine-neutral formats or XML vocabularies for expressing user entitlements or access policies, such as the extensible access control markup language (XACML), and for user provisioning and de-provisioning with capabilities such as the service provision markup language (SPML). Until
www.acm.org/crossroads
Spring 2010/ Vol. 16, No. 3
23
Ramaswamy Chandramouli and Peter Mell
the user management interface of the cloud provider provides supports for these kinds of protocols, the cloud user’s control of this important security function cannot be realized.
Access to Data Data is an enterprise’s core asset. What are the security challenges and requirements surrounding access to data stored in the cloud infrastructure? Driven by citizen safety and privacy measures, government agencies and enterprises (for example, healthcare organizations) may demand of a SaaS, PaaS, or IaaS cloud provider that the data pertaining to their applications be:
•
hosted in hardware located within the nation’s territory or a specific region, for example, for disaster recovery concerns (No. 3), and
•
protected against malicious or misused processes running in the cloud (No. 4).
For many cloud providers, hosting hardware within a specific region can be done easily. However, protecting the data itself from malicious processes in the cloud is often more difficult. For many cloud providers, the competitiveness of the service offering may depend upon the degree of multi-tenancy. This represents a threat exposure as the many customers of a cloud could potentially gain control of processes that have access to other customers’ data. Given the challenges in protecting access to cloud data, encryption may provide additional levels of security. Some enterprises, due to sensitive or proprietary nature of data and due to other protection requirements such as intellectual property rights, may need to protect the confidentiality of data and hence may require that both data in transport and data at rest (during storage) be encrypted (Nos. 5 and 6). While encryption of data in transit can be provided through various security protocols such as transport layer security and web services-security based on robust cryptographic algorithms, encryption of data at rest requires the additional tasks of key management (for example, key ownership, key rollovers, and key escrow). The cloud environment has a unique ownership structure in the sense that the owner of the data is the cloud user while physical resources hosting the data are owned by the cloud provider. In this environment, best practices for key management have yet to evolve, and this is one of the areas the standard bodies or industry consortiums have to address in order to meet the encryption requirements of data at rest. Data protection, depending upon the criticality of data, may call for either periodical backups or real time duplication or replication. This is true in any enterprise IT environment. Hence the cloud user has to look for these capabilities in an IaaS provider offering storage service. We will call this subclass of IaaS cloud provider a cloud storage provider. Further, if the cloud storage provider has experienced a data breach or if the cloud user is not satisfied with the data recovery features or data availability (which is also a security parameter) provided by that organization, the latter should have the means to rapidly migrate the data from one cloud storage provider to another.
In some cases, the data protection may also call for capabilities for segmenting data among various cloud storage providers. As a result, secure and rapid data backup and recovery capabilities should be provided for all mission-critical data (No. 7), and common APIs should be required to migrate data from one cloud storage provider to another (No. 8).
Vulnerabilities for PaaS When developing applications in a PaaS cloud environment, especially for PaaS solutions, what might leave the application security vulnerable? Vulnerabilities represent a major security concern whether applications are hosted internally at an enterprise or offered as a service in the cloud. In the cloud environment, the custom applications developed by the cloud user are hosted using the deployment tools and run time libraries or executables provided by the PaaS cloud provider. While it is the responsibility of cloud users to ensure that vulnerabilities such as buffer overflows and lack of input validation are not present in their custom applications, they might expect similar and additional properties, such as lack of parsing errors and immunity to SQL injection attacks, to be present in the application framework services provided by a PaaS cloud provider. Additionally, they have the right to expect that persistent programs such as web servers will be configured to run not as a privileged user (such as root). Further, the modern application frameworks based on service oriented architectures provide facilities for dynamically linking applications based on the dynamic discovery capabilities provided by a persistent program called the Directory Server. Hence this directory server program also needs to be securely configured. Based on the above discussion, two security requirements may arise from cloud users. First, the modules in the application framework provided are free of vulnerabilities (No. 9). Second, persistent programs such as web servers and directory servers are configured properly (No. 10). The biggest business factors driving the use of IaaS cloud providers is the high capital costs involved in purchase and operation of high performance servers and the network gears involved in linking up the servers to form a cluster to support compute-intensive applications. The economy of service offered by an IaaS cloud provider comes from the maximum utilization of physical servers and hence it is difficult to think of an IaaS cloud offering without a virtual machine. While it’s critical in PaaS to offer services to ensure the security of developed applications, in IaaS it’s critical for the cloud provider to rent to the users secure operating systems. IaaS cloud providers usually offer a platform for subscribers (cloud users) to define their own virtual machines to host their various applications and associated data by running a user-controlled operating system within a virtual machine monitor or hypervisor on the cloud provider’s physical servers. In this context, a primary concern of a subscriber to an IaaS cloud service is that their virtual machines are able to run safely without becoming targets of an attack, such as a side channel attack, from rogue virtual machines collocated on the same physical server.
readiness is commonly cited ❝Security among IT executives as the primary barrier
preventing organizations from immediately leveraging cloud computing.
24
Spring 2010/ Vol. 16, No. 3
❞
www.acm.org/crossroads
Crossroads
State of Security Readiness
If cloud users are not satisfied with the services provided by the current cloud provider due to security or performance reasons, they should have the capability to de-provision the virtual machines from the unsatisfactory cloud provider and provision them on a new cloud provider of their choice. Users may need to migrate from one virtual machine to another in real time, so as to provide a seamless computing experience for the end users. These needs translate to the following security requirements:
•
the capability to monitor the status of virtual machines and generate instant alerts (No. 11),
•
the capability for the user to migrate virtual machines (in non-real time) from one cloud provider to another (No. 12), and
•
the capability to perform live migration of VMs from one cloud provider to another or from one cloud region to another (No. 13).
Tools to continuously monitor the vulnerabilities or attack on virtual machines running on a server have already been developed or are under development by many vendors, and hence the first of the above requirements can be easily met. Large scale adoption of virtual machine import format standards such as open virtualization format will enable the user to rapidly provision virtual machines into one cloud provider environment and de-provision at another cloud provider environment which is no longer needed by the cloud user and thus meet the second requirement above. Further, a virtual machine migrated using a common import format should not require extensive time to reconfigure under the new environment. Hence common run time formats are also required to enable the newly migrated virtual machine to start execution in the new environment. Live migration of virtual machines (in situations of peak loads) is now possible only if the source and target virtual machines run on physical servers with the same instruction set architecture. The industry is already taking steps to address this limitation. However, since the majority of virtualized environments run the x86 ISA, this is not a major limitation.
issues. While some issues may have ready answers (such as existing security standards), others may be more problematic (such as threat exposure due to multi-tenancy). The ultimate answer is almost certainly multifaceted. Technical solutions will be discovered and implemented. Security standards will enable new capabilities. Finally, differing models and types of clouds will be used for data of varying sensitivity levels to take into account the residual risk.
Biographies Dr. Ramaswamy Chandramouli is a supervisory computer scientist in the Computer Security Division, Information Technology Laboratory at NIST. He is the author of two text books and more than 30 peer-reviewed publications in the areas of role-based access control models, model-based test development, security policy specification and validation, conformance testing of smart card interfaces and identity management. He holds a PhD in information technology security from George Mason University. Peter Mell is a senior computer scientist in the Computer Security Division at the NIST, where he is the cloud computing and security project lead, as well as vice chair of the interagency Cloud Computing Advisory Council. He is also the creator of the United States National Vulnerability Database and lead author of the Common Vulnerability Scoring System (CVSS) version 2 vulnerability metric used to secure credit card systems worldwide.
Standards With respect to standards and cloud security readiness, we have made four major observations. First, some requirements are already met today using existing standards (such as Federation protocols for authentication) and technologies (automatic real-time duplication of data for disaster recovery). Second, some requirements can be met if there is more market support for existing standards (XACML and SPML for user provisioning, open virtualization format for virtual machines migration). Third, some requirements such as data location and non multi-tenancy can be met by restructuring cost models for associated cloud service offerings. And fourth, some requirements can only be met by developing new standards (common run time formats for virtual machines, common APIs for migration of data from one cloud storage provider to another). While cloud computing presents these challenges, it has the potential to revolutionize how we use information technology and how we manage datacenters. The impact may be enormous with respect to IT cost reduction and increased rapidity and agility of application deployment. Thus, it is critical that we investigate and address these security Crossroads
www.acm.org/crossroads
Spring 2010/ Vol. 16, No. 3
25
The Business of Clouds By Guy Rosen
A
t the turn of the 20th century, companies stopped generating their own power and plugged into the electricity grid. In his now famous book The Big Switch, Nick Carr analogizes those events of a hundred years ago to the tectonic shift taking place in the technology industry today.
Just as with electricity, businesses are now turning to on-demand, mass-produced computing power as a viable alternative to maintaining their IT infrastructure in-house. In this article, we’ll try to hunt down some hard data in order to shed some light on the magnitude of this shift. We’ll also take a look at why it is all so significant, examining what the cloud means for businesses and how it is fueling a new generation of tech startups.
What is the Cloud? While the exact definition of cloud computing is subject to heated debate, we can use one of the more accepted definitions from NIST, which lays out five essential characteristics: on-demand self-service, broad network access, resource pooling, rapid elasticity and measured service. Of particular interest to us are the three service models NIST describes: Infrastructure as a service (IaaS) displaces in-house servers, storage and networks by providing those resources on-demand. Instead of purchasing a server, you can now provision one within minutes and discard it when you’re finished, often paying by the hour only for what you actually used. (See also “Elasticity in the Cloud,” page 3, for more.) Platform as a service (PaaS) adds a layer to the infrastructure, providing a platform upon which applications can be written and deployed. These platforms aim to focus the programmers on the business logic, freeing them from the worries of the physical (or virtual) infrastructure. Software as a service (SaaS) refers to applications running on cloud infrastructures, typically delivered to the end user via a web browser. The end-user need not understand a thing about the underlying infrastructure or platform! This model has uprooted traditional software, which was delivered on CDs and required installation, possibly even requiring purchase of a server to run on.
The Hype Research outfit Gartner describes cloud computing as the most hyped subject in IT today. IDC, another leading firm, estimated that cloud IT spending was at $16 billion in 2008 and would reach $42 billion by 2012. Using Google Trends, we can find more evidence of the growing interest in cloud computing by analyzing search volume for the term cloud computing. It’s extraordinary that a term that was virtually unheard of as recently as 2006 is now one of the hottest areas of the tech industry.
In an attempt to shed some light on de facto adoption of cloud infrastructure, I conducted some research during 2009 that tries to answer these questions. The first study, a monthly report titled “State of the Cloud” (see www.jackofallclouds.com/category/state-of-the-cloud/), aims to estimate the adoption of cloud infrastructure among public web sites. It’s relatively straightforward to determine whether a given site is running on cloud infrastructure, and if so, from which provider, by examining the site’s DNS records as well as the ownership of its IP. Now all we need is a data set that will provide a large number of sites to run this test on. For this, we can use a site listing such as that published by marketing analytics vendor Quantcast. Quantcast makes available a ranked list of the Internet’s top million sites (see www.quantcast.com/top-sites-1). To complete this survey, we’ll test each and every one of the sites listed and tally the total number of sites in the cloud and the total number of sites hosted on each provider. In practice the top 500,000 of these million were used. The caveat to this technique is that it analyzes a particular cross section of cloud usage and cannot pretend to take in its full breadth. Not included are back end use cases such as servers used for development, for research or for other internal office systems. This adoption of the cloud among enterprises and backend IT systems has been likened to the dark matter of the universe—many times larger but nearly impossible to measure directly. For now, let’s focus on the achievable and examine the results for the high-visibility category of public web sites. See Figures 1 and 2. From this data, we can draw two main conclusions: First, on the one hand, cloud infrastructure is in its infancy with a small slice of the overall web hosting market. On the other hand, the cloud is growing rapidly. So rapidly in fact, that Amazon EC2 alone grew 58 percent in the four months analyzed, equivalent to 294 percent annual growth.
The Reality The big question is whether cloud computing is just a lot of hot air. To add to the mystery, hard data is exceedingly hard to come by. Amazon, the largest player in the IaaS space, is deliberately vague. In its financial reports, the revenues from its IaaS service are rolled into the “other” category.
26
Spring 2010/ Vol. 16, No. 3
Figure 1: Amazon EC2 has a clear hold on the cloud market.
www.acm.org/crossroads
Crossroads
The Business of Clouds
Figure 2: The top 500,000 sites by cloud provider are shown. Second, Amazon EC2 leads the cloud infrastructure industry by a wide margin. Amazon is reaping the rewards of being the innovating pioneer. Its first cloud service was launched as early as 2005 and the richness of its offering is at present unmatched. The second study we’ll discuss examined the overall usage of Amazon EC2, based on publicly-available data that had been overlooked. Every time you provision a resource from Amazon EC2 (for example, request to start a new server instance) that resource is assigned an ID, such as i-31a74258. The ID is an opaque number that is used to refer to the resource in subsequent commands. In a simplistic scenario, that ID would be a serial number that increments each time a resource is provisioned. If that were the case, we could perform a very simple yet powerful measurement: we could provision a resource and record its ID at a certain point in time. Twenty-four hours later, we could perform the same action, again recording the ID. The difference between the two IDs would represent the number of resources provisioned within the 24-hour period. Unfortunately, at first glance Amazon EC2’s IDs appear to have no meaning at all and are certainly not trivial serial numbers. A mixture of luck and investigations began to reveal patterns in the IDs. One by one, these patterns were isolated and dissected until it was discovered that underlying the seemingly opaque ID there is, after all, an incrementing serial number. For example, the resource ID 31a74258 can be translated to reveal the serial number 00000258. (This process was published in detail and can be found in the blog post Anatomy of an Amazon EC2 Resource ID: www.jackofallclouds.com/2009/09/anatomy-of-an-amazon-ec2resource-id.) With these serial numbers now visible, we can perform our measurement as described above. Indeed, during a 24-hour period in September 2009 the IDs for several types of resources were recorded, translated and the resource usage calculated from the differences. See Figure 3. Over the 24-hour period observed, the quantities of resources provisioned were:
• • • •
Instances (servers): 50,242 Reservations (atomic commands to launch instances): 41,121 EBS volumes (network storage drives): 12,840 EBS snapshots (snapshots of EBS volumes): 30,925.
These numbers are incredible to say the least. They show the use of Amazon EC2 to be extensive as well as dynamic. We should recall that Crossroads
Figure 3: The chart shows resource usage of Amazon EC2 in the eastern United States in September 2009 over a 24-hour period. these numbers represent the number of resources created and do not provide clues to how many of them exist at any given point in time, because we do not know which resources were later destroyed and when. The above view is of a single 24-hour period. RightScale, a company that provides management services on top of IaaS, collected IDs from the logs it has stored since its inception and broadened the analysis to a much larger timeframe—almost three years (see http://blog. rightscale. com/2009/10/05/amazon-usage-estimates). With this perspective, we can clearly witness the substantial growth Amazon EC2 has seen since its launch, from as little as a few hundred instances per day in the early days to today’s volumes of 40,000-50,000 daily instances and more.
Is the Cloud Good for Business? What’s driving adoption is the business side of the equation. We can fold the benefits of the cloud into two primary categories: economics and focus. The first and foremost of the cloud’s benefits is cost. Informal polls among customers of IaaS suggest that economics trumps all other factors. Instead of large up-front payments, you pay as you go for what you really use. From an accounting point of view, there are no assets on the company’s balance sheet: CAPEX (capital expenditure) becomes OPEX (operating expenditure), an accountant’s dream! When it comes to IT, economies of scale matter. Maintaining 10,000 servers is cheaper per server than maintaining one server alone. Simple geographic factors also come into play: whereas an on-premise server must be powered by the electricity from your local grid, cloud datacenters are often constructed near sources of low-cost electricity such as hydroelectric facilities (so the cloud is good for the environment as well). These cost savings can then be passed on to customers. The second reason you might use the cloud is in order to focus on your core competencies and outsource everything else. What is the benefit of holding on to on-premise servers, air conditioned server rooms and enterprise software—not to mention the IT staff necessary to maintain them—when you can outsource the lot? In the new model, your company, be it a legal firm, a motor company, or a multinational bank, focuses on its core business goals. The cloud companies, in turn, focus on their core competency, providing better, more reliable and cheaper IT. Everyone wins.
Start-Up Companies Love the Cloud One sector particularly boosted by cloud computing is the tech startup space. Just a few years ago, building a web application meant you
www.acm.org/crossroads
Spring 2010/ Vol. 16, No. 3
27
Guy Rosen
had to estimate (or guesstimate) the computing power and bandwidth needed and purchase the necessary equipment up front. In practice this would lead to two common scenarios: 1) Underutilization: Before that big break comes and millions come swarming to your web site, you’re only using a small fraction of the resources you purchased. The rest of the computing power is sitting idle—wasted dollars. 2) Overutilization: Finally, the big break comes! Unfortunately, it’s bigger than expected and the servers come crashing down under the load. To make up for this, teams scramble to set up more servers and the CEO, under pressure, authorizes the purchase of even more costly equipment. To make things worse, a few days later the surge subsides and the company is left with even more idle servers. If there’s something start-up companies don’t have much of, it’s money, particularly up front. Investors prefer to see results before channeling additional funds to a company. Additionally, experience shows that new companies go through a few iterations of their idea before hitting the jackpot. Under this assumption, what matters is not to succeed cheaply but to fail cheaply so that you have enough cash left for the next round. Along comes cloud computing. Out goes up-front investment and in comes pay-per-use and elasticity. This elasticity—the ability to scale up as well as down—leaves the two scenarios described above as moot points. Before the big break, you provision the minimal number of required servers in the cloud and pay just for them. When the floods arrive, the cloud enables you to provision as many resources as needed to handle the load, so you pay for what you need but not a penny more. After the surge, you can scale your resources back down. One of the best-known examples of this is a start-up company called Animoto. Animoto is a web-based service that generates animated videos based on photos and music the user provides. Video generation is a computation-intensive operation, so computing power is of the utmost importance. At first, Animoto maintained approximately 50 active servers running on Amazon EC2, which was enough to handle the mediocre success they were seeing at the time. Then, one day, its marketing efforts on Facebook bore fruit, the application went viral, and the traffic went through the roof. Over the course of just three days, Animoto scaled up its usage to 3,500 servers. How would this have been feasible, practically or economically, before the age of cloud computing? Following the initial surge, traffic continued to spike up and down for a while. Animoto took advantage of the cloud’s elasticity by scaling up and down as necessary, paying only for what they really had to. The Animoto story illustrates the tidal change for start-ups. It’s not surprising to see, therefore, that the number of such companies is consistently on the rise. If you like, cloud computing has lowered the price of buying a lottery ticket for the big game that is the startup economy.
28
Spring 2010/ Vol. 16, No. 3
It’s become so cheap to take a shot that more and more entrepreneurs are choosing the bootstrap route, starting out on their own dime. When they do seek external investment, they find that investors are forking over less and less in initial funding, out of realization that it now takes less to get a start-up off the ground.
A Bounty of Opportunity Cloud computing isn’t just an enabler for start-ups—the number of start-ups providing cloud-related services is growing rapidly, too. The colossal change in IT consumption has created a ripe opportunity for small, newly formed companies to outsmart the large, well-established, but slow-to-move incumbents. The classic opportunity is in SaaS applications at the top of the cloud stack. The existing players are struggling to rework their traditional software offerings into the cloud paradigm. In the meantime, start-ups are infiltrating the market with low-cost, on-demand alternatives. These start-ups are enjoying both sides of the cloud equation: on the one hand the rising need for SaaS and awareness of its validity from consumers; on the other hand the availability of PaaS and IaaS which lower costs and reduce time-to-market. Examples of such organizations include Unfuddle (SaaS-based source control running on the Amazon EC2 IaaS) and FlightCaster (flight delay forcaster running on the Heroku PaaS). The second major opportunity is down the stack. Although providing IaaS services remains the realm of established businesses, a category of enabling technologies is emerging. Users of IaaS tend to need more than what the provider offers, ranging from management and integration to security and inter-provider mechanisms. The belief among start-ups and venture capitalists alike is that there is a large market for facilitating the migration of big business into the cloud. Examples of such companies include RightScale, Elastra, and my own start-up, Vircado. The third and final category of start-ups aims to profit from the increased competition between IaaS providers. These providers are in a constant race to widen their portfolio and lower their costs. Startups can innovate and be the ones to deliver that sought-after edge, in areas ranging from datacenter automation to virtualization technologies and support management. Examples in this category include Virtensys and ParaScale. I for one am convinced that beyond the hype and excitement the world of IT is undergoing a very real period of evolution. Cloud computing is not a flash flood: it will be years before its full effect is realized.
Biography Guy Rosen is co-founder and CEO of Vircado, a startup company in the cloud computing space. He also blogs about cloud computing at JackOfAllClouds.com, where he publishes original research and analysis of the cloud market.
www.acm.org/crossroads
Crossroads
acm
STUDENT MEMBERSHIP APPLICATION AND ORDER FORM Join ACM online: www.acm.org/joinacm CODE: CRSRDS
Name
INSTRUCTIONS
Please print clearly
Carefully complete this application and return with payment by mail or fax to ACM. You must be a full-time student to qualify for student rates.
Address
City
State/Province
Country
E-mail address
Area code & Daytime phone
Postal code/Zip
CONTACT ACM
Fax
Member number, if applicable
MEMBERSHIP BENEFITS AND OPTIONS • Electronic subscription to Communications of the ACM magazine
• ACM e-news digest TechNews (thrice weekly)
• Electronic subscription to Crossroads magazine
• ACM online newsletter MemberNet (twice monthly) • Student Quick Takes, ACM student e-newsletter (quarterly) • ACM's Online Guide to Computing Literature • Free "acm.org" email forwarding address plus filtering through Postini
• Free software and courseware through the ACM Student Academic Initiative • 2,500 online courses in multiple languages, 1,000 virtual labs and 500 online books • ACM's CareerNews (twice monthly)
• Free e-mentoring service from MentorNet
For immediate processing, FAX this application to +1-212-944-1318.
PLEASE CHOOSE ONE: ❏ Student Membership: $19 (USD) ❏ Student Membership PLUS Digital Library: $42 (USD) ❏ Student Membership PLUS Print CACM Magazine: $42 (USD) ❏ Student Membership w/Digital Library PLUS Print CACM Magazine: $62 (USD) P R I N T P U B L I C AT I O N S Check the appropriate box and calculate amount due. • acmqueue (online only) • Computers in Entertainment (online only) Computing Reviews • Computing Surveys • Crossroads • interactions (included in SIGCHI membership) Int’l Journal of Network Management (Wiley) Int’l Journal on Very Large Databases • Journal of Educational Resources in Computing (see TOCE) • Journal of Experimental Algorithmics (online only) • Journal of Personal and Ubiquitous Computing • Journal of the ACM • Journal on Computing and Cultural Heritage • Journal on Data and Information Quality • Journal on Emerging Technologies in Computing Systems • Linux Journal (SSC) • Mobile Networks & Applications • netWorker • Wireless Networks Transactions on: • Accessible Computing • Algorithms • Applied Perception • Architecture & Code Optimization • Asian Language Information Processing • Autonomous and Adaptive Systems • Computational Biology and Bio Informatics • Computer-Human Interaction • Computational Logic • Computation Theory (NEW) • Computer Systems • Computing Education (formerly JERIC) • Database Systems • Design Automation of Electronic Systems • Embedded Computing Systems • Graphics • Information & System Security • Information Systems • Internet Technology • Knowledge Discovery From Data • Mathematical Software • Modeling and Computer Simulation • Multimedia Computing, Communications, and Applications • Networking • Programming Languages & Systems • Reconfigurable Technology & Systems • Sensor Networks • Software Engineering and Methodology • Speech and Language Processing (online only) • Storage • Web marked • are available in the ACM Digital Library *Check here to have publications delivered via Expedited Air Service. For residents outside North America only.
PAYMENT INFORMATION Payment must accompany application
Please check one Issues per year 6 4 12 4 4 6 6 4 4 12 6 6 4 4 4 12 4 4 4
Code 143 247 104 103 XRoads 123 136 148 239 129 144 102 173 171 154 137 130 133 125
4 4 4 4 4 4 4 4 4
174 151 145 146 138 158 149 119 135
4
114 277 109 128 142 112 134 113 140 170 108 116 156 118 110 172 155 115 153 157 159
4 4 4 4 4 4 4 4 4 4 4 6 6 4 4 4 4 4 4
PUBLICATION SUBTOTAL:
Member Member Rate Rate PLUS Air* Visit www.acmqueue.org for more info. N/A $44 ❐ N/A $60 ❐ $62 ❐ $37 ❐ $35 ❐ $53 ❐ $55 ❐ $84 ❐ $85 ❐ $110 ❐ $85 ❐ $110 ❐ N/A N/A N/A N/A $119 ❐ $86 ❐ $56 ❐ $107 ❐ $50 ❐ $68 ❐ $50 ❐ $68 ❐ $43 ❐ $61 ❐ $27 ❐ $60 ❐ $64 ❐ $89 ❐ $56 ❐ $81 ❐ $64 ❐ $89 ❐ $50 ❐ $53 ❐ $44 ❐ $44 ❐ $40 ❐ $42 ❐ $36 ❐ $46 ❐ $45 ❐ $50 ❐ $48 ❐ N/A $47 ❐ $44 ❐ $45 ❐ $52 ❐ $45 ❐ $48 ❐ $43 ❐ $42 ❐ $48 ❐ $52 ❐ $43 ❐ $55 ❐ $60 ❐ $50 ❐ $43 ❐ $44 ❐ N/A $43 ❐ $42 ❐
phone: 800-342-6626 (US & Canada) +1-212-626-0500 (Global) hours: 8:30am–4:30pm US Eastern Time fax: +1-212-944-1318 email:
[email protected] mail: Association for Computing Machinery, Inc. General Post Office P.O. Box 30777 New York, NY 10087-0777
$68 ❐ $71 ❐ $62 ❐ $62 ❐ $58 ❐ $60 ❐ $77 ❐ $64 ❐ $63 ❐ $68 ❐ $66 ❐ N/A $65 ❐ $62 ❐ $63 ❐ $70 ❐ $63 ❐ $66 ❐ $61 ❐ $60 ❐ $66 ❐ $70 ❐ $61 ❐ $108 ❐ $85 ❐ $68 ❐ $61 ❐ $62 ❐ N/A $61 ❐ $60 ❐
$ Member dues ($19, $42, or $62) To have Communications of the ACM sent to you via Expedited Air Service, add $50 here (for residents outside of $ North America only).
Publications
$
Total amount due
$
Check or money order (make payable to ACM, Inc. in U.S. dollars or equivalent in foreign currency)
❏ Visa/Mastercard
❏ American Express
Card number
Exp. date
Signature Member dues, subscriptions, and optional contributions are tax deductible under certain circumstances. Please consult with your tax advisor.
EDUCATION
Name of School Please check one: ❐ High School (Pre-college, Secondary School) College: ❐ Freshman/1st yr. ❐ Sophomore/2nd yr. ❐ Junior/3rd yr. ❐ Senior/4th yr. Graduate Student: ❐ Masters Program ❐ Doctorate Program ❐ Postdoctoral Program ❐ Non-Traditional Student
Major
Expected mo./yr. of grad.
Age Range: ❐ 17 & under ❐ 18-21 ❐ 22-25 ❐ 26-30 ❐ 31-35 ❐ 36-40 ❐ 41-45 ❐ 46-50 ❐ 51-55 ❐ 56-59 ❐ 60+ Do you belong to an ACM Student Chapter? ❐ Yes ❐ No I attest that the information given is correct and that I will abide by the ACM Code of Ethics. I understand that my membership is non transferable.
Signature
