Maintaining Mission Critical Systems in a 24/7 Environment
2nd Edition
Published on 12. August 2011
Book
Hardback
512 pages
978-0-470-65042-4 (ISBN)
Description
This book is meant to offer Architects, Property Mangers, Facility Managers, Building Engineers, Information Technology Professionals, Data Center Personnel, Electrical & Mechanical Technicians and students in undergraduate, graduate, or continuing education programs relevant insight into the Mission Critical Environment with an emphasis on business resiliency, data center efficiency, and green power technology. Industry improvements, standards, and techniques have been incorporated into the text and address the latest issues prevalent in the Mission Critical Industry. An emphasis on green technologies and certifications is presented throughout the book. In addition, a description of the United States energy infrastructure's dependency on oil, in relation to energy security in the mission critical industry, is discussed. In conjunction with this, either a new chapter will be created on updated policies and regulations specifically related to the mission critical industry or updates to policies and regulations will be woven into most chapters. The topics addressed throughout this book include safety, fire protection, energy security and data center cooling, along with other common challenges and issues facing industry engineers today.
Reviews / Votes
This book addresses a very timely issue: Keeping critical facilities and businesses operating in an age of physical and cyber terrorism, global warming, technical evolution and revolution, and the emergence of a rapidly maturing digital society with global reach. The book provides insights into all aspects of building critical systems, with thought-out and concise recommendations which will get even the most novice building engineers started on the road to zero building downtime. George ElloMore details
Series
Edition
2nd edition
Language
English
Place of publication
United States
Publishing group
John Wiley & Sons Inc
Target group
Professional and scholarly
Illustrations
Photos: 50 B&W, 0 Color; Drawings: 70 B&W, 0 Color; Screen captures: 54 B&W, 0 Color
Dimensions
Height: 239 mm
Width: 165 mm
Thickness: 31 mm
Weight
816 gr
ISBN-13
978-0-470-65042-4 (9780470650424)
Copyright in bibliographic data and cover images is held by Nielsen Book Services Limited or by the publishers or by their respective licensors: all rights reserved.
Schweitzer Classification
Other editions
New editions
Peter M. Curtis
Maintaining Mission Critical Systems in a 24/7 Environment
Book
01/2021
Wiley-Blackwell
€130.50
Shipment within 15-20 days
Previous edition
Peter M. Curtis
Maintaining Mission Critical Systems in a 24/7 Environment
Book
04/2007
Wiley-Blackwell
€89.08
Article exhausted; check for reprint
Person
PETER M. CURTIS is the founder of Power Management Concepts, LLC. He has twenty-eight years of experience designing, operating, managing, and educating in the mission critical facilities engineering industry. A keynote speaker for numerous semiannual conferences, Curtis is a member of the 7x24 Exchange, Association for Facilities Engineering, Institute of Electrical and Electronics Engineers, International Electrical Testing Association, International Facility Management Association, and the Association of Energy Engineers.
Content
Foreword xvii
Preface xix
Acknowlegments xxi
1 An Overview of Reliability and Resiliency in Today's Mission Critical Environment 1
1.1 Introduction 2
1.2 Risk Assessment 4
1.2.1 Levels of Risk 6
1.3 Capital Cost Versus Operation Cost 6
1.4 Critical Environment Workflow and Change Management 8
1.4.1 Change Management 9
1.4.2 Escalation Procedures 10
1.5 Testing and Commissioning 10
1.6 Documentation and the Human Factor 14
1.7 Education and Training 18
1.8 Operation and Maintenance 19
1.9 Employee Certification 20
1.10 Standards and Benchmarking 21
1.11 Conclusion 22
1.12 Risk Analysis and Improvement 22
2 Energy Security and Its Effect on Business Resiliency 25
2.1 Introduction 25
2.2 Risks Related to Information Security 29
2.3 How Risks are Addressed 34
2.4 Use of Distributed Generation 37
2.5 Documentation and Its Relation to Information Security 40
2.6 Smart Grid 42
2.7 Conclusion 44
2.8 Risk Analysis and Improvement 45
3 Mission Critical Engineering with an Overview of Green Technologies 47
3.1 Introduction 47
3.2 Companies' Expectations: Risk Tolerance and Reliability 48
3.3 Identifying the Appropriate Redundancy in a Mission Critical Facility 50
3.3.1 Load Classifications 51
3.4 Improving Reliability, Maintainability, and Proactive Preventative Maintenance 52
3.5 The Mission Critical Facilities Manager and the Importance of the Boardroom 53
3.6 Quantifying Reliability and Availability 54
3.6.1 Re vie w of Reliability Terminology 55
3.7 Design Considerations for the Mission Critical Data Center 56
3.7.1 Data Center Certification 57
3.8 The Evolution of Mission Critical Facility Design 58
3.9 Human Factors and the Commissioning Process 59
3.10 Short-Circuit and Coordination Studies 60
3.10.1 Short-Circuit Study 60
3.10.2 Coordination Study 63
3.11 Introduction to Direct Current in the Data Center 65
3.11.1 Advantages of DC Distribution 65
3.11.2 DC Lighting 67
3.11.3 DC Storage Options 67
3.11.4 Renewable Energy Integration 68
3.11.5 DC and Combined Cooling, Heat, and Power 68
3.11.6 Current State of the Art 68
3.11.7 Safety Issues 70
3.11.8 Maintenance 71
3.11.9 Education and Training 71
3.11.10 Future Vision 71
3.12 Containerized Systems Overview 72
3.13 Conclusion 73
4 Mission Critical Electrical System Maintenance and Safety 81
4.1 Introduction 81
4.2 The History of the Maintenance Supervisor and the Evolution of the Mission Critical Facilities Engineer 83
4.3 Internal Building Deficiencies and Analysis 85
4.4 Evaluating Your System 86
4.5 Choosing a Maintenance Approach 87
4.5.1 Annual Preventive Maintenance 88
4.6 Safe Electrical Maintenance 89
4.6.1 Standards and Regulations 89
4.6.2 Electrical Safety: Arc Flash 90
4.6.3 Personal Protective Equipment (PPE) 92
4,6.4 Lockout/Tagout 98
4.7 Maintenance of Typical Electrical Distribution Equipment 99
4.7.1 Thermal Scanning and Thermal Monitoring 100
4.7.2 15 kV Class Equipment 102
4.7.3 480 Volt Switchgear 102
4.7.4 Motor Control Centers and Panel Boards 103
4.7.5 Automatic Transfer Switches 103
4.7.6 Automatic Static Transfer Switches (ASTS) 104
4.7.7 Power Distribution Units 105
4.7.8 277/480 Volt Transformers 105
4.7.9 Uninterruptible Power Systems 105
4.8 Being Proactive in Evaluating Test Reports 107
4.9 Conclusion 107
5 Standby Generators: Operations and Maintenance 109
5.1 Introduction 109
5.2 The Necessity for Standby Power 110
5.3 Emergency, Legally Required, and Optional Systems 111
5.4 Standby Systems That Are Legally Required 112
5.5 Optional Standby Systems 113
5.6 Understanding Your Power Requirements 113
5.7 Management Commitment and Training 113
5.7.1 Lockout/Tagout 114
5.7.2 Training 115
5.8 Standby Generator Systems Maintenance Procedures 115
5.8.1 Maintenance Record Keeping and Data Trending 116
5.8.2 Engine 116
5.8.3 Coolant System 117
5.8.4 Control System 117
5.8.5 Generator Mechanics 117
5.8.6 Automatic and Manual Switchgear 117
5.8.7 Load Bank Testing 118
5.9 Documentation Plan 118
5.9.1 Proper Documentation and Forms 118
5.9.2 Record Keeping 119
5.10 Emergency Procedures 119
5.11 Cold Start and Load Acceptance 120
5.12 Nonlinear Load Problems 121
5.12.1 Line Notches and Harmonic Current 121
5.12.2 Step Loading 121
5.12.3 Voltage Rise 122
5.12.4 Frequency Fluctuation 122
5.12.5 Synchronizing to Bypass 122
5.12.6 Automatic Transfer Switch 123
5.13 Conclusion 123?
6 Fuel Systems Design and Maintenance 125
6.1 Introduction 125
6.2 Brief Discussion on Diesel Engines 126
6.3 Bulk Storage Tank Selection 126
6.3.1 Aboveground Tanks 127
6.3.2 Modern Underground Tanks and Piping Systems 128
6.4 Codes and Standards 128
6.5 Recommended Practices for All Tanks 129
6.6 Fuel Distribution System Configuration 133
6.7 Day Tank Control System 135
6.8 Diesel Fuel and A Fuel Quality Assurance Program 139
6.8.1 Fuel Needs and Procurement Guidelines 141
6.8.2 New Fuel Shipment Prereceipt Inspection 141
6.8.3 Analysis of New Fuel Prior to Transfer to On-Site Storage 144
6.8.4 Monthly Fuel System Maintenance 145
6.8.5 Quarterly or Semiannual Monitoring of On-Site Bulk Fuel 146
6.8.6 Remediation 146
6.9 Conclusion 148
7 Power Transfer Switch Technology, Applications, and Maintenance 149
7.1 Introduction 149
7.2 Transfer Switch Technology and Applications 151
7.3 Types of Power Transfer Switches 152
7.3.1 Manual Transfer Switches 152
7.3.2 Automatic Transfer Switches 153
7.4 Control Devices 163
7.4.1 Time Delays 163
7.4.2 ln-Phase Monitor 164
7.4.3 Test Switches 165
7.4.4 Exercise Clock 165
7.4.5 Voltage and Frequency Sensing Controls 166
7.5 Design Features 166
7.5.1 Close Against High In-Rush Currents 166
7.5.2 Withstand and Closing Rating (WCR) 167
7.5.3 Carry Full Rated Current Continuously 167
7.5.4 Interrupt Current 167
7.6 Additional Characteristics and Ratings of ATS 167
7.6.1 NEMA Classification 167
7.6.2 System Voltage Ratings 168
7.6.3 ATS Sizing 168
7.6.4 Seismic Requirement 168
7.7 Installation and Commissioning, Maintenance, and Safety 168
7.7.1 Installation and Commissioning 168
7.7.2 Maintenance and Safety 170
7.7.3 Maintenance Tasks 173
7.7.4 Drawing and Manuals 173
7.7.5 Testing and Training 173
7.8 General Recommendations 176
7.9 Conclusion 176
8 Static Transfer Switch 179
8.1 Introduction 179
8.2 Overview 180
8.2.1 Major Components 180
8.3 Typical Static Switch, One-Line Diagram 181
8.3.1 Normal Operation 181
8.3.2 Bypass Operation 182
8.3.3 STS and STS/Transformers Configurations 183
8.4 STS Technology and Application 183
8.4,1 General Parameters 184
8.4.2 STS Location and Type 184
8.4.3 Advantages and Disadvantages of the Primary and Secondary STS/Transformer Systems 184
8.4.4 Monitoring, Data Logging, and Data Management 184
8.4.5 Downstream Device Monitoring 185
8.4.6 STS Remote Communication 185
8.4.7 Security 186
8.4,8 Human Engineering and Eliminating Human Errors 187
8.4.9 Reliability and Availability 187
8.4.10 Repairability and Maintainability 189
8.4.11 Fault Tolerance and Abnormal Operation 189
8.5 Testing 190
8.6 Conclusion 190
9 Fundamentals of Power Quality 193
9.1 Introduction 193
9.2 Electricity Basics 195
9.2.1 Basic Circuit 196
9.2.2 Power Factor 196
9.3 Transmission of Power 197
9.3.1 Life Cycle of Electricity 198
9.3.2 Single-Phase and Three-Phase Power Basics 199
9.3.3 Unreliable Power versus Reliable Power 201
9.4 Understanding Power Problems 202
9.4.1 Power Quality Transients 202
9.4.2 RMS Variations 204
9.4.3 Causes of Power Line Disturbances 207
9.4.4 Power Line Disturbance Levels 212
9.5 Tolerances of Computer Equipment 212
9.5.1 CBEMA Curve 214
9.5.2 ITIC Curve 215
9.5.3 Purpose of Curves 215
9.6 Power Monitoring 215
9.7 The Deregulation Wildcard 217
9.8 Conclusion 221
10 UPS Systems: Applications and Maintenance with an Overview of Green Technologies 223
10.1 Introduction 223
10.1.1 Green Technologies and Reliability Overview 223
10.2 Purpose of UPS Systems 225
10.3 General Description of UPS Systems 228
10.3.1 What is a UPS System? 228
10.3.2 How Does a UPS System Work? 228
10.3.3 Static UPS Systems 229
10.3.4 Online 230
10.3.5 Double Conversion 230
10.3.6 Double Conversion UPS Power Path 231
10.4 Components of a Static UPS System 232
10.4.1 Power Control Devices 232
10.5 Online Line Interactive UPS Systems 238
10.6 Offline (Standby) 239
10.7 The Evolution of Static UPS Technology 240
10.7.1 Emergence of the IGBT 240
10.7.2 Two- and Three-Level Rectifier/Inverter Topology 241
10.8 Rotary UPS Systems 242
10.8.1 UPSs Using Diesel 243
10.8.2 Hybrid UPS Systems 244
10.9 Redundancy, Configurations, and Topology 245
10.9.1 N Configuration 245
10.9.2 N+1 Configuration 246
10.9.3 Isolated Redundant Configuration 246
10.9.4 N+2 Configurati on 246
10.9.5 2N Configuration 248
10.9.6 2N+ 1 Configuration 248
10.9.7 Distributed Redundant/Catcher UPS 249
10.9.8 Eco-Mode for Static UPS 249
10.9.9 Availability Calculations 250
10.10 Energy Storage Devices 251
10.10.1 Battery 251
10.10.2 Flywheel Energy 255
10.11 UPS Maintenance and Testing 256
10.11.1 Physical Preventive Maintenance (PM) 257
10,11,2 Protection Settings, Calibration, and Guidelines 258
10.11.3 Functional Load Testing 258
10.11.4 Steady-State Load Test 259
10.11.5 Steady-State Load Test at 0%, 50%, and 100% of Load 259
10.11.6 Harmonic Analysis and Testing 259
10.11.7 Filter Integrity and Testing 260
10.11.8 Transient Response Load Test 261
10.11.9 Module Fault Test 261
10.11.10 Battery Rundown Test 261
10.12 Static UPS and Maintenance 262
10.13 UPS Management 262
10.14 Conclusion 263
11 Data Center Cooling Systems 265
11.1 Introduction 265
11.2 Background Information 266
11.3 Cooling within Datacom Rooms 266
11.4 Cooling Systems 267
11.4,1 Air Side 267
11.4.2 Cooling-Medium Side 267
11.5 Components Outside the Datacom Room 269
11.5.1 Refrigeration Equipment-Chillers 269
11.5.2 Heat-Rejection Equipment 273
11.5.3 Energy-Recovery Equipment 282
11.5.4 Heat Exchangers 287
11.6 Components Inside the Datacom Rooms 290
11.6,1 CRAC Units 290
11.7 Conclusion 295
12 Data Center Cooling Efficiency: Concepts and Advanced Technologies 297
12.1 Introduction 297
12.1.1 Data Center Efficiency Measurement
12.2 Heat Transfer Inside Data Centers 300
12.2.1 Heat Generation 301
12.2.2 Heat Return 302
12.2.3 Cooling Air 302
12.3 Cooling and Other Airflow Topics 303
12.3.1 Leakage 303
12.3.2 Mixing and Its Relationship to Efficiency 303
12.3.3 Recirculation 303
12.3.4 Venturi Effect 304
12.3.5 Vortex Effect 304
12.3.6 CRAC/CRAH Types 304
12.3.7 Potential CRAC Operation Issues 305
12.3.8 Sensible Versus Latent Cooling 305
12.3.9 Humidity Control 307
12.3.10 CRAC Fighting-Too Many CRACs 308
12.4 Design Approaches for Data Center Cooling 308
12.4.1 Hot Aisle/Cold Aisle 308
12.4.2 Cold-Aisle Containment 309
12.4.3 In-Row Cooling with Hot-Aisle Containment 309
12.4.4 Overhead Supplemental Cooling 309
12.4.5 Chimney or Ducted Returns 310
12.4.6 Advanced Active Airflow Management for Server Cabinets 310
12.5 Additional Considerations 310
12.5.1 Active Air Movement 310
12.5.2 Adaptive Capacity 311
12.5.3 Liquid Cooling 311
12.5.4 Cold Storage 312
12.6 Hardware and Associated Efficiencies 312
12.6.1 Server Efficiency 312
12.6.2 Server Virtualization 313
12.6.3 Multicore Processors 313
12.6.4 Blade Servers 313
12.6.5 Energy-Efficient Servers 313
12.6.6 Power Managed Servers 313
12.6.7 Effects of Dynamic Server Loads on Cooling 313
12.7 Best Practices 314
12.8 Efficiency Problem Solving 314
12.9 Conclusion 316
12.10 Conversions, Formulas, Guidelines 316
13 Raised Access Floors 317
13.1 Introduction 317
13.1.1 What is an Access Floor? 317
13.1.2 What are Typical Applications for Access Floors? 318
13.1.3 Why Use an Access Floor? 319
13.2 Design Considerations 319
13.2.1 Determine the Structural Performance Required 320
13.2.2 Determine the Required Finished Floor Height 322
13.2.3 Determine the Understructure Support Design Type Required 323
13.2.4 Determine the Appropriate Floor Finish 325
13.2.5 Airflow Requirements 326
13.3 Safety Concerns 328
13.3.1 Removal and Reinstallation of Panels 328
13.3.2 Removing Panels 328
13.3.3 Stringer Systems 330
13.3.4 Protection of the Floor from Heavy Loads 331
13.3.5 Grounding the Access Floor 336
13.3.6 Fire Protection 337
13.3.7 Zinc Whiskers 337
13.4 Panel Cutting 328
13.4.1 Safety Requirements for Cutting Panels 328
13.4.2 Guidelines for Cutting Panels 328
13.4.3 Cutout Locations in Panel-Supplemental Support for Cut Panels 338
13.4.4 Saws and Blades for Panel Cutting 339
13.4.5 Interior Cutout Procedure 339
13.4.6 Round Cutout Procedure 339
13.4.7 Installing Protective Trim Around Cut Edges 340
13.4.8 Cutting and Installing the Trim 340
13.5 Access Floor Maintenance 340
13.5.1 Best Practices for Standard High-Pressure Laminate Floor Tile (HPL) and for Vinyl Conductive and Static Dissipative Tile 341
13.5.2 Damp Mopping Procedure for HPL and Conductive and Static Dissipative Vinyl Tile 342
13.5.3 Cleaning the Floor Cavity 342
13.6 Troubleshooting 343
13.6.1 Making Pedestal Height Adjustments 343
13.6.2 Rocking Panel Condition 343
13.6.3 Panel Lipping Condition (Panel Sitting High) 343
13.6.4 Out-of-Square Stringer Grid (Twisted Grid) 344
13.6.5 Tipping at Perimeter Panels 345
13.6.6 Tight Floor or Loose Floor-Floor Systems Laminated with HPL Tile 345
13.7 Additional Design Considerations 346
13.7.1 LEED Certification 346
13.7.2 Energy Efficiency-Hot and Cold Air Containment 346
13.7.3 Airflow Distribution and CFD Analysis 347
13.8 Conclusion 354
14 Fire Protection in Mission Critical Infrastructures 357
14.1 Introduction 357
14.2 Philosophy 358
14.3 Alarm and Notification 359
14.4 Early Detection 361
14.5 Fire Suppression 362
14.6 System Designs 364
14.6.1 Stages of a Fire 364
14.6.2 Fire and Building Codes 365
14.7 Fire Detection 366
14.8 Fire Suppression Systems 374
14.8.1 Water Mist Systems 379
14.8.2 Carbon Dioxide Systems 382
14.8.3 Clean Agent Systems 384
14.8.4 Inert Gas Agents 384
14.8.5 IG-541 385
14.8.6 IG-55 385
14.8.7 Chemical Clean Agents 386
14.8.8 Portable Fire Extinguishers 390
14.8.9 Clean Agents and the Environment 390
14.8.10 Conclusion 391
Appendix A Policies and Regulations 393
A.1 Introduction 393
A.2 Industry Policies and Regulations 395
A.2.1 USA PATRIOT Act 396
A.2.2 Sarbanes-Oxley Act (SOX) 397
A.2.3 Comprehensive Environmental Response, Compensation, and Liability Act of 1980 399
A.2.4 Executive Order 13423-Strengthening Federal Environmental, Energy, and Transportation Management 399
A.2.5 IS027000 Information Security Management Systems (ISMS) 400
A.2.6 The National Strategy for the Physical Protection of Critical Infrastructure and Key Assets 403
A.2.7 2009 National Infrastructure Protection Plan 404
A.2.8 North American Electric Reliability Corporation (NERC) Critical Infrastructure Protection Program 405
A.2.9 U.S. Security and Exchange Commission (SEC) 405
A.2.10 Sound Practices to Strengthen the Resilience of the U.S, Financial System 405
A.2.11 C4I-Command, Control, Communications, Computers, and Intelligence 407
A.2.12 Basel II Accord 408
A.2.13 National Institute of Standards and Technology (NIST) 408
A.2.14 Business Continuity Management Agencies and Regulating Organizations410
A.2.15 FFIEC-Federal Financial Institutions Examination Council 412
A.2.16 National Fire Prevention Association 1600 Standard on Disaster/Emergency Management and Business Continuity Programs 412
A.2.17 Private Sector Preparedness Act 414
A.3 Data Protection 414
A.4 Encryption 416
A.4.1 Protecting Critical Data through Security and Vaulting 417
A.5 Business Continuity Plan (BCP) 417
A. 6 Conclusion 419
Appendix B Consolidated List of Key Questions 421
Appendix C Airflow Management: A Systems Approach 441
C.l Introduction 441
C.2 Control is the Key 442
C.2.1 Benefits of Control 444
C.3 Obtaining Control 445
C.3.1 Lower Return Air ?T Versus Higher ?T 445
C.4 Air Management Technologies 451
C.4.1 In-Row Cooling 452
C.4.2 Overhead Cooling 452
C.4.3 Containment Strategies 452
C.4.4 Active-Air Management 453
C.4.5 A Benchmark Study for Comparison 454
C.5 Conclusion 456
Glossary 459
Bibliography 473
Index 479
Preface xix
Acknowlegments xxi
1 An Overview of Reliability and Resiliency in Today's Mission Critical Environment 1
1.1 Introduction 2
1.2 Risk Assessment 4
1.2.1 Levels of Risk 6
1.3 Capital Cost Versus Operation Cost 6
1.4 Critical Environment Workflow and Change Management 8
1.4.1 Change Management 9
1.4.2 Escalation Procedures 10
1.5 Testing and Commissioning 10
1.6 Documentation and the Human Factor 14
1.7 Education and Training 18
1.8 Operation and Maintenance 19
1.9 Employee Certification 20
1.10 Standards and Benchmarking 21
1.11 Conclusion 22
1.12 Risk Analysis and Improvement 22
2 Energy Security and Its Effect on Business Resiliency 25
2.1 Introduction 25
2.2 Risks Related to Information Security 29
2.3 How Risks are Addressed 34
2.4 Use of Distributed Generation 37
2.5 Documentation and Its Relation to Information Security 40
2.6 Smart Grid 42
2.7 Conclusion 44
2.8 Risk Analysis and Improvement 45
3 Mission Critical Engineering with an Overview of Green Technologies 47
3.1 Introduction 47
3.2 Companies' Expectations: Risk Tolerance and Reliability 48
3.3 Identifying the Appropriate Redundancy in a Mission Critical Facility 50
3.3.1 Load Classifications 51
3.4 Improving Reliability, Maintainability, and Proactive Preventative Maintenance 52
3.5 The Mission Critical Facilities Manager and the Importance of the Boardroom 53
3.6 Quantifying Reliability and Availability 54
3.6.1 Re vie w of Reliability Terminology 55
3.7 Design Considerations for the Mission Critical Data Center 56
3.7.1 Data Center Certification 57
3.8 The Evolution of Mission Critical Facility Design 58
3.9 Human Factors and the Commissioning Process 59
3.10 Short-Circuit and Coordination Studies 60
3.10.1 Short-Circuit Study 60
3.10.2 Coordination Study 63
3.11 Introduction to Direct Current in the Data Center 65
3.11.1 Advantages of DC Distribution 65
3.11.2 DC Lighting 67
3.11.3 DC Storage Options 67
3.11.4 Renewable Energy Integration 68
3.11.5 DC and Combined Cooling, Heat, and Power 68
3.11.6 Current State of the Art 68
3.11.7 Safety Issues 70
3.11.8 Maintenance 71
3.11.9 Education and Training 71
3.11.10 Future Vision 71
3.12 Containerized Systems Overview 72
3.13 Conclusion 73
4 Mission Critical Electrical System Maintenance and Safety 81
4.1 Introduction 81
4.2 The History of the Maintenance Supervisor and the Evolution of the Mission Critical Facilities Engineer 83
4.3 Internal Building Deficiencies and Analysis 85
4.4 Evaluating Your System 86
4.5 Choosing a Maintenance Approach 87
4.5.1 Annual Preventive Maintenance 88
4.6 Safe Electrical Maintenance 89
4.6.1 Standards and Regulations 89
4.6.2 Electrical Safety: Arc Flash 90
4.6.3 Personal Protective Equipment (PPE) 92
4,6.4 Lockout/Tagout 98
4.7 Maintenance of Typical Electrical Distribution Equipment 99
4.7.1 Thermal Scanning and Thermal Monitoring 100
4.7.2 15 kV Class Equipment 102
4.7.3 480 Volt Switchgear 102
4.7.4 Motor Control Centers and Panel Boards 103
4.7.5 Automatic Transfer Switches 103
4.7.6 Automatic Static Transfer Switches (ASTS) 104
4.7.7 Power Distribution Units 105
4.7.8 277/480 Volt Transformers 105
4.7.9 Uninterruptible Power Systems 105
4.8 Being Proactive in Evaluating Test Reports 107
4.9 Conclusion 107
5 Standby Generators: Operations and Maintenance 109
5.1 Introduction 109
5.2 The Necessity for Standby Power 110
5.3 Emergency, Legally Required, and Optional Systems 111
5.4 Standby Systems That Are Legally Required 112
5.5 Optional Standby Systems 113
5.6 Understanding Your Power Requirements 113
5.7 Management Commitment and Training 113
5.7.1 Lockout/Tagout 114
5.7.2 Training 115
5.8 Standby Generator Systems Maintenance Procedures 115
5.8.1 Maintenance Record Keeping and Data Trending 116
5.8.2 Engine 116
5.8.3 Coolant System 117
5.8.4 Control System 117
5.8.5 Generator Mechanics 117
5.8.6 Automatic and Manual Switchgear 117
5.8.7 Load Bank Testing 118
5.9 Documentation Plan 118
5.9.1 Proper Documentation and Forms 118
5.9.2 Record Keeping 119
5.10 Emergency Procedures 119
5.11 Cold Start and Load Acceptance 120
5.12 Nonlinear Load Problems 121
5.12.1 Line Notches and Harmonic Current 121
5.12.2 Step Loading 121
5.12.3 Voltage Rise 122
5.12.4 Frequency Fluctuation 122
5.12.5 Synchronizing to Bypass 122
5.12.6 Automatic Transfer Switch 123
5.13 Conclusion 123?
6 Fuel Systems Design and Maintenance 125
6.1 Introduction 125
6.2 Brief Discussion on Diesel Engines 126
6.3 Bulk Storage Tank Selection 126
6.3.1 Aboveground Tanks 127
6.3.2 Modern Underground Tanks and Piping Systems 128
6.4 Codes and Standards 128
6.5 Recommended Practices for All Tanks 129
6.6 Fuel Distribution System Configuration 133
6.7 Day Tank Control System 135
6.8 Diesel Fuel and A Fuel Quality Assurance Program 139
6.8.1 Fuel Needs and Procurement Guidelines 141
6.8.2 New Fuel Shipment Prereceipt Inspection 141
6.8.3 Analysis of New Fuel Prior to Transfer to On-Site Storage 144
6.8.4 Monthly Fuel System Maintenance 145
6.8.5 Quarterly or Semiannual Monitoring of On-Site Bulk Fuel 146
6.8.6 Remediation 146
6.9 Conclusion 148
7 Power Transfer Switch Technology, Applications, and Maintenance 149
7.1 Introduction 149
7.2 Transfer Switch Technology and Applications 151
7.3 Types of Power Transfer Switches 152
7.3.1 Manual Transfer Switches 152
7.3.2 Automatic Transfer Switches 153
7.4 Control Devices 163
7.4.1 Time Delays 163
7.4.2 ln-Phase Monitor 164
7.4.3 Test Switches 165
7.4.4 Exercise Clock 165
7.4.5 Voltage and Frequency Sensing Controls 166
7.5 Design Features 166
7.5.1 Close Against High In-Rush Currents 166
7.5.2 Withstand and Closing Rating (WCR) 167
7.5.3 Carry Full Rated Current Continuously 167
7.5.4 Interrupt Current 167
7.6 Additional Characteristics and Ratings of ATS 167
7.6.1 NEMA Classification 167
7.6.2 System Voltage Ratings 168
7.6.3 ATS Sizing 168
7.6.4 Seismic Requirement 168
7.7 Installation and Commissioning, Maintenance, and Safety 168
7.7.1 Installation and Commissioning 168
7.7.2 Maintenance and Safety 170
7.7.3 Maintenance Tasks 173
7.7.4 Drawing and Manuals 173
7.7.5 Testing and Training 173
7.8 General Recommendations 176
7.9 Conclusion 176
8 Static Transfer Switch 179
8.1 Introduction 179
8.2 Overview 180
8.2.1 Major Components 180
8.3 Typical Static Switch, One-Line Diagram 181
8.3.1 Normal Operation 181
8.3.2 Bypass Operation 182
8.3.3 STS and STS/Transformers Configurations 183
8.4 STS Technology and Application 183
8.4,1 General Parameters 184
8.4.2 STS Location and Type 184
8.4.3 Advantages and Disadvantages of the Primary and Secondary STS/Transformer Systems 184
8.4.4 Monitoring, Data Logging, and Data Management 184
8.4.5 Downstream Device Monitoring 185
8.4.6 STS Remote Communication 185
8.4.7 Security 186
8.4,8 Human Engineering and Eliminating Human Errors 187
8.4.9 Reliability and Availability 187
8.4.10 Repairability and Maintainability 189
8.4.11 Fault Tolerance and Abnormal Operation 189
8.5 Testing 190
8.6 Conclusion 190
9 Fundamentals of Power Quality 193
9.1 Introduction 193
9.2 Electricity Basics 195
9.2.1 Basic Circuit 196
9.2.2 Power Factor 196
9.3 Transmission of Power 197
9.3.1 Life Cycle of Electricity 198
9.3.2 Single-Phase and Three-Phase Power Basics 199
9.3.3 Unreliable Power versus Reliable Power 201
9.4 Understanding Power Problems 202
9.4.1 Power Quality Transients 202
9.4.2 RMS Variations 204
9.4.3 Causes of Power Line Disturbances 207
9.4.4 Power Line Disturbance Levels 212
9.5 Tolerances of Computer Equipment 212
9.5.1 CBEMA Curve 214
9.5.2 ITIC Curve 215
9.5.3 Purpose of Curves 215
9.6 Power Monitoring 215
9.7 The Deregulation Wildcard 217
9.8 Conclusion 221
10 UPS Systems: Applications and Maintenance with an Overview of Green Technologies 223
10.1 Introduction 223
10.1.1 Green Technologies and Reliability Overview 223
10.2 Purpose of UPS Systems 225
10.3 General Description of UPS Systems 228
10.3.1 What is a UPS System? 228
10.3.2 How Does a UPS System Work? 228
10.3.3 Static UPS Systems 229
10.3.4 Online 230
10.3.5 Double Conversion 230
10.3.6 Double Conversion UPS Power Path 231
10.4 Components of a Static UPS System 232
10.4.1 Power Control Devices 232
10.5 Online Line Interactive UPS Systems 238
10.6 Offline (Standby) 239
10.7 The Evolution of Static UPS Technology 240
10.7.1 Emergence of the IGBT 240
10.7.2 Two- and Three-Level Rectifier/Inverter Topology 241
10.8 Rotary UPS Systems 242
10.8.1 UPSs Using Diesel 243
10.8.2 Hybrid UPS Systems 244
10.9 Redundancy, Configurations, and Topology 245
10.9.1 N Configuration 245
10.9.2 N+1 Configuration 246
10.9.3 Isolated Redundant Configuration 246
10.9.4 N+2 Configurati on 246
10.9.5 2N Configuration 248
10.9.6 2N+ 1 Configuration 248
10.9.7 Distributed Redundant/Catcher UPS 249
10.9.8 Eco-Mode for Static UPS 249
10.9.9 Availability Calculations 250
10.10 Energy Storage Devices 251
10.10.1 Battery 251
10.10.2 Flywheel Energy 255
10.11 UPS Maintenance and Testing 256
10.11.1 Physical Preventive Maintenance (PM) 257
10,11,2 Protection Settings, Calibration, and Guidelines 258
10.11.3 Functional Load Testing 258
10.11.4 Steady-State Load Test 259
10.11.5 Steady-State Load Test at 0%, 50%, and 100% of Load 259
10.11.6 Harmonic Analysis and Testing 259
10.11.7 Filter Integrity and Testing 260
10.11.8 Transient Response Load Test 261
10.11.9 Module Fault Test 261
10.11.10 Battery Rundown Test 261
10.12 Static UPS and Maintenance 262
10.13 UPS Management 262
10.14 Conclusion 263
11 Data Center Cooling Systems 265
11.1 Introduction 265
11.2 Background Information 266
11.3 Cooling within Datacom Rooms 266
11.4 Cooling Systems 267
11.4,1 Air Side 267
11.4.2 Cooling-Medium Side 267
11.5 Components Outside the Datacom Room 269
11.5.1 Refrigeration Equipment-Chillers 269
11.5.2 Heat-Rejection Equipment 273
11.5.3 Energy-Recovery Equipment 282
11.5.4 Heat Exchangers 287
11.6 Components Inside the Datacom Rooms 290
11.6,1 CRAC Units 290
11.7 Conclusion 295
12 Data Center Cooling Efficiency: Concepts and Advanced Technologies 297
12.1 Introduction 297
12.1.1 Data Center Efficiency Measurement
12.2 Heat Transfer Inside Data Centers 300
12.2.1 Heat Generation 301
12.2.2 Heat Return 302
12.2.3 Cooling Air 302
12.3 Cooling and Other Airflow Topics 303
12.3.1 Leakage 303
12.3.2 Mixing and Its Relationship to Efficiency 303
12.3.3 Recirculation 303
12.3.4 Venturi Effect 304
12.3.5 Vortex Effect 304
12.3.6 CRAC/CRAH Types 304
12.3.7 Potential CRAC Operation Issues 305
12.3.8 Sensible Versus Latent Cooling 305
12.3.9 Humidity Control 307
12.3.10 CRAC Fighting-Too Many CRACs 308
12.4 Design Approaches for Data Center Cooling 308
12.4.1 Hot Aisle/Cold Aisle 308
12.4.2 Cold-Aisle Containment 309
12.4.3 In-Row Cooling with Hot-Aisle Containment 309
12.4.4 Overhead Supplemental Cooling 309
12.4.5 Chimney or Ducted Returns 310
12.4.6 Advanced Active Airflow Management for Server Cabinets 310
12.5 Additional Considerations 310
12.5.1 Active Air Movement 310
12.5.2 Adaptive Capacity 311
12.5.3 Liquid Cooling 311
12.5.4 Cold Storage 312
12.6 Hardware and Associated Efficiencies 312
12.6.1 Server Efficiency 312
12.6.2 Server Virtualization 313
12.6.3 Multicore Processors 313
12.6.4 Blade Servers 313
12.6.5 Energy-Efficient Servers 313
12.6.6 Power Managed Servers 313
12.6.7 Effects of Dynamic Server Loads on Cooling 313
12.7 Best Practices 314
12.8 Efficiency Problem Solving 314
12.9 Conclusion 316
12.10 Conversions, Formulas, Guidelines 316
13 Raised Access Floors 317
13.1 Introduction 317
13.1.1 What is an Access Floor? 317
13.1.2 What are Typical Applications for Access Floors? 318
13.1.3 Why Use an Access Floor? 319
13.2 Design Considerations 319
13.2.1 Determine the Structural Performance Required 320
13.2.2 Determine the Required Finished Floor Height 322
13.2.3 Determine the Understructure Support Design Type Required 323
13.2.4 Determine the Appropriate Floor Finish 325
13.2.5 Airflow Requirements 326
13.3 Safety Concerns 328
13.3.1 Removal and Reinstallation of Panels 328
13.3.2 Removing Panels 328
13.3.3 Stringer Systems 330
13.3.4 Protection of the Floor from Heavy Loads 331
13.3.5 Grounding the Access Floor 336
13.3.6 Fire Protection 337
13.3.7 Zinc Whiskers 337
13.4 Panel Cutting 328
13.4.1 Safety Requirements for Cutting Panels 328
13.4.2 Guidelines for Cutting Panels 328
13.4.3 Cutout Locations in Panel-Supplemental Support for Cut Panels 338
13.4.4 Saws and Blades for Panel Cutting 339
13.4.5 Interior Cutout Procedure 339
13.4.6 Round Cutout Procedure 339
13.4.7 Installing Protective Trim Around Cut Edges 340
13.4.8 Cutting and Installing the Trim 340
13.5 Access Floor Maintenance 340
13.5.1 Best Practices for Standard High-Pressure Laminate Floor Tile (HPL) and for Vinyl Conductive and Static Dissipative Tile 341
13.5.2 Damp Mopping Procedure for HPL and Conductive and Static Dissipative Vinyl Tile 342
13.5.3 Cleaning the Floor Cavity 342
13.6 Troubleshooting 343
13.6.1 Making Pedestal Height Adjustments 343
13.6.2 Rocking Panel Condition 343
13.6.3 Panel Lipping Condition (Panel Sitting High) 343
13.6.4 Out-of-Square Stringer Grid (Twisted Grid) 344
13.6.5 Tipping at Perimeter Panels 345
13.6.6 Tight Floor or Loose Floor-Floor Systems Laminated with HPL Tile 345
13.7 Additional Design Considerations 346
13.7.1 LEED Certification 346
13.7.2 Energy Efficiency-Hot and Cold Air Containment 346
13.7.3 Airflow Distribution and CFD Analysis 347
13.8 Conclusion 354
14 Fire Protection in Mission Critical Infrastructures 357
14.1 Introduction 357
14.2 Philosophy 358
14.3 Alarm and Notification 359
14.4 Early Detection 361
14.5 Fire Suppression 362
14.6 System Designs 364
14.6.1 Stages of a Fire 364
14.6.2 Fire and Building Codes 365
14.7 Fire Detection 366
14.8 Fire Suppression Systems 374
14.8.1 Water Mist Systems 379
14.8.2 Carbon Dioxide Systems 382
14.8.3 Clean Agent Systems 384
14.8.4 Inert Gas Agents 384
14.8.5 IG-541 385
14.8.6 IG-55 385
14.8.7 Chemical Clean Agents 386
14.8.8 Portable Fire Extinguishers 390
14.8.9 Clean Agents and the Environment 390
14.8.10 Conclusion 391
Appendix A Policies and Regulations 393
A.1 Introduction 393
A.2 Industry Policies and Regulations 395
A.2.1 USA PATRIOT Act 396
A.2.2 Sarbanes-Oxley Act (SOX) 397
A.2.3 Comprehensive Environmental Response, Compensation, and Liability Act of 1980 399
A.2.4 Executive Order 13423-Strengthening Federal Environmental, Energy, and Transportation Management 399
A.2.5 IS027000 Information Security Management Systems (ISMS) 400
A.2.6 The National Strategy for the Physical Protection of Critical Infrastructure and Key Assets 403
A.2.7 2009 National Infrastructure Protection Plan 404
A.2.8 North American Electric Reliability Corporation (NERC) Critical Infrastructure Protection Program 405
A.2.9 U.S. Security and Exchange Commission (SEC) 405
A.2.10 Sound Practices to Strengthen the Resilience of the U.S, Financial System 405
A.2.11 C4I-Command, Control, Communications, Computers, and Intelligence 407
A.2.12 Basel II Accord 408
A.2.13 National Institute of Standards and Technology (NIST) 408
A.2.14 Business Continuity Management Agencies and Regulating Organizations410
A.2.15 FFIEC-Federal Financial Institutions Examination Council 412
A.2.16 National Fire Prevention Association 1600 Standard on Disaster/Emergency Management and Business Continuity Programs 412
A.2.17 Private Sector Preparedness Act 414
A.3 Data Protection 414
A.4 Encryption 416
A.4.1 Protecting Critical Data through Security and Vaulting 417
A.5 Business Continuity Plan (BCP) 417
A. 6 Conclusion 419
Appendix B Consolidated List of Key Questions 421
Appendix C Airflow Management: A Systems Approach 441
C.l Introduction 441
C.2 Control is the Key 442
C.2.1 Benefits of Control 444
C.3 Obtaining Control 445
C.3.1 Lower Return Air ?T Versus Higher ?T 445
C.4 Air Management Technologies 451
C.4.1 In-Row Cooling 452
C.4.2 Overhead Cooling 452
C.4.3 Containment Strategies 452
C.4.4 Active-Air Management 453
C.4.5 A Benchmark Study for Comparison 454
C.5 Conclusion 456
Glossary 459
Bibliography 473
Index 479