Large-scale Distributed Systems and Energy Efficiency
Description
More details
Other editions
Additional editions
Person
Jean-Marc Pierson is a Professor in Computer Science at the University of Toulouse?(France). Jean-Marc Pierson received his PhD from the ENS-Lyon, France in1996. He was an Associate Professor at the University Littoral?Cote-d'Opale (1997-2001) in Calais, then at INSA-Lyon (2001-2006). He is a member of the IRIT Laboratory and Chair of the SEPIA Team on distributed systems. His research focuses on energy- aware distributed systems, in particular?monitoring, job placement and scheduling, green networking, autonomic computing, and mathematical modelling.
Content
Preface xv
Acknowledgment xvii
1 INTRODUCTION TO ENERGY EFFICIENCY IN LARGE-SCALE DISTRIBUTED SYSTEMS 1
Jean-Marc Pierson and Helmut Hlavacs
1.1 Energy Consumption Status 1
1.2 Target of the Book 3
1.3 The Cost Action IC0804 4
1.3.1 Birth of the Action 4
1.3.2 Development of the Action 5
1.3.3 End and Future of the Action 10
1.4 Chapters Preview 11
Acknowledgement 12
References 12
2 HARDWARE LEVERAGES FOR ENERGY REDUCTION IN LARGE-SCALE DISTRIBUTED SYSTEMS 17
Davide Careglio, Georges Da Costa, and Sergio Ricciardi
2.1 Introduction 17
2.1.1 Motivation for Energy-Aware Distributed Computing 17
2.2 Processor 19
2.2.1 Context 19
2.2.2 Advanced Configuration and Power Interface (ACPI) 20
2.2.3 Vendors 21
2.2.4 General-Purpose Graphics Processing Unit (GPGPU) 23
2.2.5 ARM Architecture 24
2.3 Memory (DRAM) 25
2.3.1 Context 25
2.3.2 Power Consumption 25
2.3.3 Energy Efficiency Techniques 26
2.3.4 Vendors 26
2.4 Disk/Flash 27
2.4.1 Spindle Speed 28
2.4.2 Seek Speed 28
2.4.3 Power Modes 29
2.4.4 Power Consumption 29
2.4.5 Solid-State Drive (SDD) 29
2.5 Fan 30
2.6 Power Supply Unit 30
2.7 Network Infrastructure 31
2.7.1 Current Scenario 31
2.7.2 New Energy-Oriented Model 32
2.7.3 Current Advances in Networking 33
2.7.4 Adaptive Link Rate (ALR) 34
2.7.5 Low Power Idle (LPI) 34
2.7.6 Energy-Aware Dynamic RWA Framework 34
2.7.7 Energy-Aware Network Attacks 35
References 36
3 GREEN WIRED NETWORKS 41
Alfonso Gazo Cervero, Michele Chincoli, Lars Dittmann, Andreas Fischer, Alberto E. Garcia, Jaime Galán-Jiménez, Laurent Lefevre, Hermann de Meer, Thierry Monteil, Paolo Monti, Anne-Cecile Orgerie, Louis-Francois Pau, Chris Phillips, Sergio Ricciardi, Remi Sharrock, Patricia Stolf, Tuan Trinh, and Luca Valcarenghi
3.1 Economic Incentives and Green Tariffing 44
3.1.1 Regulatory, Economic, and Microeconomic Measures 44
3.1.2 Pricing Theory in Relation to Green Policies 46
3.1.3 COST Action Results 50
3.2 Network Components 51
3.2.1 Router 51
3.2.2 Network Interface Card 55
3.2.3 Reconfigurable Optical Add-Drop Multiplexer 56
3.2.4 Digital Subscriber Line Access Multiplexer 56
3.3 Architectures 57
3.3.1 Access Networks 57
3.3.2 Carrier Networks 58
3.3.3 Grid Overlay Networks 58
3.4 Traffic Considerations 59
3.5 Energy-Saving Mechanisms 60
3.5.1 Static Mechanisms 60
3.5.2 Dynamic Mechanisms 61
3.6 Challenges 72
3.7 Summary 72
References 73
4 GREEN WIRELESS-ENERGY EFFICIENCY IN WIRELESS NETWORKS 81
Vitor Bernardo, Torsten Braun, Marilia Curado, Markus Fiedler, David Hock, Theus Hossmann, Karin Anna Hummel, Philipp Hurni, Selim Ickin, Almerima Jamakovic-Kapic, Simin Nadjm-Tehrani, Tuan Ahn Trinh, Ekhiotz Jon Vergara, Florian Wamser, and Thomas Zinner
4.1 Introduction 81
4.2 Metrics and Trade-Offs in Wireless Networks 83
4.2.1 Metrics 83
4.2.2 Energy Optimization Trade-Offs 84
4.2.3 Summary 85
4.3 Measurement Methodology 85
4.3.1 Energy Measurement Testbeds 86
4.3.2 Energy Estimation Techniques 90
4.3.3 Energy Measurements versus Estimation 97
4.3.4 Summary 99
4.4 Energy Efficiency and QoE in Wireless Access Networks 100
4.4.1 Energy Issues in Cellular Networks 100
4.4.2 Energy Efficiency and QoE in Wireless Mesh Networks 101
4.4.3 Reducing Energy Consumption of the End User Device 105
4.4.4 Energy Measurements Revealing Video QoE Issues 108
4.4.5 Energy Issues in Environmental WMNs 110
4.4.6 Summary 112
4.5 Energy-Efficient Medium Access in Wireless Sensor Networks 113
4.5.1 MaxMAC - An Energy-Efficient MAC Protocol 113
4.5.2 Real-World Testbed Experiments with MaxMAC 116
4.5.3 Summary 119
4.6 Energy-Efficient Connectivity in Ad-Hoc and Opportunistic Networks 119
4.6.1 Ad-Hoc Networking 120
4.6.2 Opportunistic and Delay-Tolerant Networking 121
4.6.3 Summary 123
4.7 Summary and Conclusions 124
References 125
5 POWER MODELING 131
Jason Mair, Zhiyi Huang, David Eyers, Leandro Cupertino, Georges Da Costa, Jean-Marc Pierson, and Helmut Hlavacs
5.1 Introduction 131
5.2 Measuring Power 133
5.2.1 External Power Meters 133
5.2.2 Internal Power Meters 134
5.3 Performance Indicators 135
5.3.1 Source Instrumentation 135
5.3.2 Binary Instrumentation 136
5.3.3 Performance Monitoring Counters 136
5.3.4 Operating System Events 137
5.3.5 Virtual Machine Performance 138
5.4 Interaction between Power and Performance 138
5.4.1 Central Processing Unit (CPU) 138
5.4.2 Memory 140
5.4.3 Input/Output (I/O) 141
5.4.4 Network 141
5.4.5 Idle States 142
5.5 Power Modeling Procedure 143
5.5.1 Variable Selection 143
5.5.2 Training Data Collection 144
5.5.3 Learning from Data 145
5.5.4 Event Correlation 145
5.5.5 Model Evaluation Concepts 146
5.5.6 Power Estimation Errors 148
5.5.7 Related Work 149
5.6 Use-Cases 151
5.6.1 Applications 151
5.6.2 Single-Core Systems 152
5.6.3 Multi-core and Multiprocessor 152
5.6.4 Distributed Systems 153
5.7 Available Software 154
5.8 Conclusion 155
References 156
6 GREEN DATA CENTERS 159
Robert Basmadjian, Pascal Bouvry, Georges Da Costa, László Gyarmati, Dzmitry Kliazovich, Sébastien Lafond, Laurent Lefèvre, Hermann De Meer, Jean-Marc Pierson, Rastin Pries, Jordi Torres, Tuan Anh Trinh, and Samee Ullah Khan
6.1 Introduction 160
6.2 Overview of Energy Consumption of Hardware Infrastructure in Data Center 161
6.2.1 Energy Consumption Rankings and Metrics 161
6.2.2 Processing: CPU, GPU, and memory 162
6.2.3 Storage 168
6.2.4 Communicating Elements 168
6.3 Middleware Solutions that Regulate and Optimize the Energy Consumption in Data Centers 169
6.3.1 An Overview of the Middleware 169
6.3.2 System Modeling 171
6.3.3 Control Mechanisms 172
6.3.4 A Use Case of Leveraging Energy Efficiency in Data Centers 174
6.4 Data Center Network Architectures 177
6.4.1 Architectures 177
6.4.2 Power Consumption of Data Center Architectures 181
6.4.3 Additional Proposals for Energy-Efficient Data Centers 182
6.5 Solutions for Cooling and Heat Control in Data Center 184
6.5.1 Mechanical-Based Approaches 185
6.5.2 Software-Based Approaches 187
Acknowledgments 187
References 188
7 ENERGY EFFICIENCY AND HIGH-PERFORMANCE COMPUTING 197
Pascal Bouvry, Ghislain Landry Tsafack Chetsa, Georges Da Costa, Emmanuel Jeannot, Laurent Lefèvre, Jean-Marc Pierson, Frédéric Pinel, Patricia Stolf, and Sébastien Varrette
7.1 Introduction 197
7.2 Overview of HPC Components and Latest Trends Toward Energy Efficiency 198
7.2.1 Architecture of the Current HPC Facilities 198
7.2.2 Overview of the Main HPC Components 201
7.2.3 HPC Performance and Energy Efficiency Evaluation 203
7.3 Building the Path to Exascale Computing 206
7.3.1 The Exascale Challenge: Hardware and Architecture Issues 206
7.3.2 Energy Efficiency and Resource and Job Management System (RJMS) 207
7.3.3 Energy-Aware Software 210
7.3.4 A Methodology for Energy Reduction in HPC 210
7.4 Energy Efficiency of Virtualization and Cloud Frameworks over HPC Workloads 216
7.5 Conclusion: Open Challenges 221
Acknowledgments 222
References 222
8 SCHEDULING AND RESOURCE ALLOCATION 225
Pragati Agrawal, Damien Borgetto, Carmela Comito, Georges Da Costa, Jean-Marc Pierson, Payal Prakash, Shrisha Rao, Domenico Talia, Cheikhou Thiam, and Paolo Trunfio
8.1 Introduction: Energy-Aware Scheduling 225
8.2 Use of Linear Programming in Energy-Aware Scheduling 226
8.2.1 Finding the Optimal Solution Using a Linear Program 226
8.2.2 Benefits and Limitations of LP 227
8.3 Heuristics in Large Instances 228
8.3.1 Energy-Aware Greedy Algorithms 229
8.3.2 Vector Packing 229
8.3.3 Improving Fast Algorithms 229
8.4 Comparing Allocation Heuristics for Energy-Aware Scheduling 230
8.4.1 Problem Formulation 230
8.4.2 Allocation Heuristics 232
8.4.3 Results 234
8.5 Energy-Aware Task Allocation in Mobile Environments 236
8.5.1 Reference Architecture 237
8.5.2 Task Allocation Strategy 238
8.5.3 Task Allocation Algorithm 239
8.5.4 Performance Results 241
8.6 An Energy-Aware Scheduling Strategy for Allocating Computational Tasks in a Fully Decentralized Way 243
8.6.1 Decentralized Resources in Cloud: Overview 243
8.6.2 Cooperative Scheduling Anti-Load Balancing Algorithm for Cloud (CSAAC) 244
8.6.3 Simulation Results 245
8.6.4 Evaluation 248
8.7 Cost-Aware Scheduling with Smart Grids 248
8.7.1 Cost-Aware Scheduling 248
8.7.2 Cost-Aware Scheduling Using DE 252
8.7.3 Comparison of DE with Other Approaches 254
8.8 Heterogeneity, Cooling, DVFS, and Migration 257
8.8.1 Lever Interactions 257
8.8.2 Infrastructures 257
8.8.3 Resource Allocation as a Whole 258
8.9 Conclusions 259
References 260
9 ENERGY EFFICIENCY IN P2P SYSTEMS AND APPLICATIONS 263
Simone Brienza, Sena Efsun Cebeci, Seyed-Saeid Masoumzadeh, Helmut Hlavacs, Öznur Özkasap, Giuseppe Anastasi
9.1 Introduction 264
9.2 General Approaches to Energy Efficiency 264
9.2.1 Sleep/Wakeup Approaches 264
9.2.2 Hierarchical Approaches 266
9.2.3 Resource Allocation 268
9.3 Energy Efficiency in File-Sharing Applications 269
9.3.1 Client-Server versus P2P File Sharing 269
9.3.2 Energy Efficiency in P2P File Sharing 270
9.3.3 Energy Efficiency in BitTorrent 270
9.3.4 Energy Efficiency in Other File-Sharing Protocols 279
9.4 Energy Efficiency in P2P Epidemic Protocols 280
9.5 Conclusions 282
References 283
10 TOWARD SUSTAINABILITY FOR LARGE-SCALE COMPUTING SYSTEMS: ENVIRONMENTAL, ECONOMIC, AND STANDARDIZATION ASPECTS 287
Christina Herzog, Jean-Marc Pierson, and Laurent Lefèvre
10.1 Introduction 287
10.2 Green IT for Innovation and Innovation for Green IT 288
10.2.1 Defining Green IT and Its Link with Sustainability 288
10.2.2 Differences between Academia and Companies 291
10.2.3 Describing the Loop between Academia and Industry 294
10.3 Standardization Landscape in Green IT 295
10.3.1 Different Standardization Levels 296
10.3.2 Standardization Bodies 297
10.3.3 Regulations 299
10.3.4 Industry Groups and Professional Bodies 299
10.3.5 Analysis of the Standardization Actors 301
10.4 Modeling Actors of Innovation in Green IT and their Links 301
10.4.1 Researcher 301
10.4.2 Universities 302
10.4.3 Technology Transfer Office (TTO) 302
10.4.4 Industry 302
10.4.5 Funding Organization 303
10.4.6 Standardization Body 303
10.4.7 Links between Actors 303
10.4.8 Rating the Relationships between Actors 304
10.5 Using the Modeling for Deciding 306
10.5.1 Methodology to be Developed 306
10.6 Conclusion 307
Acknowledgment 307
References 307
Author Index 309
Subject Index 311
Chapter 1
Introduction to Energy Efficiency in Large-Scale Distributed Systems
Jean-Marc Pierson1 and Helmut Hlavacs2
1IRIT, University of Toulouse, France
2Faculty of Computer Science, University of Vienna, Austria
1.1 Energy Consumption Status
The demand for research in energy efficiency in large-scale systems is supported by several incentives [1Â-3], including financial incentives by government or institutions to energy efficient industries/companies [4-5]. Indeed, studies such as [6] reported already in 2006 that the information technology (IT) consumption accounts for 5% to 10% of the growing global electricity demand and for a mere 2% of the energy while data centers alone account for 14% of the information and communication technology (ICT) footprint. It was projected that by 2020, the energy demand of data centers will represent 18% of the ICT footprint, the carbon footprint rising at an annual 7% pace, doubling between 2007 and 2020 [7]. The study of Koomey [8] in 2011 highlights that the rise of energy consumption is not as bad as expected in 2007: between 2005 and 2010, the electricity demand for data centers increased by (only) about 56% worldwide instead of the projected doubling and even as low as 36% in the United States. Altogether the electricity used worldwide for operating data centers in 2010 accounted for about 1.3% of total electricity use.
The past 5 years have witnessed the increase of research focusing especially in energy reduction. While being a major concern in embedded systems since decades, the problem is quite new in the large-scale infrastructures where performances have been for long the sole parameters to optimize. The motivation comes from two complementary concerns: first, the electrical cost of running such infrastructure is equivalent nowadays to the purchase costs of the equipment during a 4-year usage [9]. Second, electricity providers are not always able to deliver the needed power to run the machines, capping the amount of electricity delivered to one particular client.
Modern usage of ITs relies on the existence of large data centers, high-performance infrastructures, and performance networks, core and mobile networks.
Cloud computing is one of the major evolutions in IT in the past decade. It mainly relies on data centers, some hosting thousands of servers. In 2010, Google was hosting already 900,000 servers (almost 1 million must be the case today, and is estimated to be even more than 1.5 million). In 2013, Microsoft's CEO Steve Ballmer claimed hosting more than 1 million servers. Amazon is guessed to have about the same number of servers. For 1 million servers, at about 200 W per server, plus something like 50 W for cooling and electricity distribution losses, it represents a total power consumption of 250 MW, which is likely 2 TWh/year. However, in [8], it is shown that less than 1% of electricity used by data centers worldwide was attributable to Google's data center operations: The big players are often cited as examples, but they represent only a few percentage of the problem. When they exhibit better energy efficiency, it must be remembered that most other companies have less advances and the average is far from these big players.
While, traditionally, supercomputers have been mainly compared by their raw performance measured now in PFlops (petaflops), they are now also assessed based on their energy efficiency. The ranking of supercomputers by their energy efficiency places great emphasis on their energy consumption through the number of GFlops (gigaflops) they can achieve per watt. For instance, the Tianhe-2 machine, the leading one of the top performance list (Top500 1), delivers a computing power of over 33 PFlops and shows an energy efficiency of 1.9 GFlops/W; while the CINECA machine, which tops the green list (Green500 list 2) with an energy efficiency of 3.9 GFlops/W, delivers a low computing power of less than 2 PFlops. Nevertheless, it can be noted that supercomputers are getting greener, or more exactly, their energy efficiency is continuously increasing, while their energy consumption itself is nevertheless growing. Despite this trend, it will be difficult to achieve exascale computing for 20 MW by 2020, the limit given by the US Department of Energy (DoE).
Network operators are among the most power-consuming players. Telecom Italia [10] estimated that its consumption represented 1% of the Italian total power consumption in 2011 (compared to 0.7% in 2008). Similarly, British Telecom estimates 0.7% to be its share of electricity usage in the United Kingdom (2.3 TWh), same as that of NTT in Japan. These numbers account not only for networks but also for associated infrastructures to operate them. For instance, for Telecom Italia, 65% of electricity is consumed in the networks (wired and mobile) and 10% by their data centers. However, these numbers do not account for the equipment at final clients. In France, a study from IDATE [11] shows that the total electricity consumption of the telecom is 8.5 TWh in 2012 (for 6.7 TWh in 2008). The share is 40% for wired and mobile networks, 6% for data centers, 24% for the ADSL boxes at client places, and, finally, 18% for the fix and mobile phones themselves. We can notice that the total energy consumption of the internet boxes at clients' home is estimated to be 3.3 TWh in 2012 (40 millions of boxes).
The share of power consumption in servers is evolving continuously, because of the improvement in electronics for individual components. Processors (central processing unit, CPU) and memory account together for about 54% of the total consumption, with a rough 37% share for CPU and 17% for memory while the other components are consuming less: Peripheral Component Interconnect (PCI) slots (23%), motherboard (12%), disk (6%), and fans (5%) [12] (see Chapter 2 for details). When graphics processing unit (GPU) are present, they can represent up to a tremendous 50% share of the total consumption. It is, therefore, not surprising that most of the efforts have been put on reducing the power consumption of processors and memory. However, despite the urge for proportional computing already demonstrated in 2007 [13], the current servers are not consuming proportionally to their usage. This makes a lot of work trying to switch off components (consolidation in clouds) or using them at lower speed and capacity (dynamic voltage frequency scaling (DVFS) for CPU, Low Power Idle (LPI) for network cards, disk spin down for hard disk) very valuable. It should be noted that the situation is improving: 5 years ago, a server was consuming as much as 50% of its peak power when idle. Now this drops to 20%, and the peak power itself is decreasing. One can wonder if the works based on the nonproportionality of power consumption will still be interesting in the future. We believe that the delay is still long enough to see achievements in this dimension and also that the aforementioned researches may be used in conjunction and transferred at lower levels (at the components architecture), to allow for actual proportional computing.
One must not forget also the impact of cooling in the global consumption, especially in data centers or large-scale networking equipment rooms. The power usage effectiveness (PUE), promoted since 2007 as a criteria for assessing the power efficiencies of infrastructures, is the ratio between the global power usage to the power usage for IT. While it was common to have a PUE of 2 or more (meaning that as much electricity was used for the infrastructure-mainly cooling and distribution losses) the state-of-the-art values are now at about 1.5 or 1.6. Still 50-60% of power is used for cooling IT equipment. However, as outlined earlier, many data centers do not operate with state-of-the-art solutions, and their PUE are more likely to be at about 1.8 or 1.9 [8].
Energy concerns have been integrated in many works at the different levels of the IT stack: hardware, network, middleware, and software levels in large-scale distributed systems, being high-performance computing (HPC), clouds, or networks.
In the following, we exhibit some actions undertaken at these levels, in particular in the scope of an European-funded initiative.
1.2 Target of the Book
The focus and context of the book is on large-scale distributed systems. We will not study embedded systems in this book. Also we are not investigating hardware-specific optimization for energy saving. Instead, we focus in this work on energy-efficient computation and communication in large-scale distributed systems. These systems consist of thousands of heterogeneous elements that communicate via heterogeneous networks and provide different memory, storage, and processing capabilities. Examples for very large-scale distributed systems are computational and data grids, data centers, clouds, core and sensor networks, and so on.
The target audiences of the book are manifold: from IT and environmental researchers to operators of large-scale systems, up to small and medium sized enterprises (SMEs) and startups willing to understand the global picture and the state of the art in the field. It helps in building strategies and understanding upcoming developments in the rapid field of energy efficiency to speedup transfer of technologies to industries [14].
1.3 The Cost Action IC0804
This section introduces the European Cooperation in Science and Technology (COST) Action IC0804. The COST Action instrument is a 4-year funding scheme in European research framework aimed at helping the development of...
