Distributed and Parallel Computing
Description
Master the growing field of distributed and parallel computing with this essential guide, offering expert insights into the fundamentals and real-world applications for intelligent and collaborative systems.
Distributed computing, or running programs across multiple computers over a network, is becoming a popular solution for addressing the demands for increased performance across industries, including scientific computing, oil exploration, biotechnology, and medicine. Distributed computing enables seamless communication and collaboration by allowing users from different locations to access and interact with their digital twin simultaneously. Distributed computing enhances the capabilities of digital twins by providing scalability, parallel processing, real-time data integration, collaboration support, resource optimization, fault tolerance, and security features. Distributed and Parallel Computing explores the fundamentals and innovations in intelligent and distributed computing systems and applications, including adaptivity and learning, agents and multi-agent systems, argumentation, case-based reasoning, and collaborative systems. Through expert insights, readers will discover promising real-world applications for this emerging technology.
More details
Other editions
Additional editions
Persons
Sandhya Avasthi, PhD is an assistant professor in the Computer Science and Engineering Department at ABES Engineering College at Dr. Abdul Kalam Technical University with over 18 years of teaching experience. She has published numerous research articles in refereed international journals, conference proceedings, and book chapters. Her research interests include natural language processing, information extraction, information retrieval, data science, and business intelligence.
Suman Lata Tripathi, PhD is a professor at Lovely Professional University with more than 22 years of experience in academics and research. She has published more than 19 books, 125 research papers in refereed science journals, conference proceedings, 13 Indian patents, and four copyrights. Her area of expertise includes microelectronics device modeling and characterization, low-power VLSI circuit design, VLSI design testing, and advanced FET design for IoT.
Content
Preface xvii
1 Introduction to Distributed Systems 1
K. Karthikeyan, S. Hemalatha and S. Vignesh
1.1 Introduction 2
2 Topology in Network Technologies 39
Mohit Malik, Varun Dogra, Ramandeep Sandhu, Ratesh Kumar and Deepika Ghai
2.1 Introduction 40
2.2 Related Work 41
2.3 Network Topology Design 44
2.4 Advantages of Network Topologies 47
2.5 Case Studies of Network Topologies 48
2.6 Distributed and Parallel Computing in Network Topology 49
2.7 Conclusion 53
3 Distributed Processing Technology and Advancements 57
Saranya. N., Shermy. R.P., Sindhu. V., Karthika Renuka. D. and Renugadevi. G.
3.1 Introduction 58
3.2 Distributed Processing in Modern Computing 60
3.3 Evolution of Distributed Processing 61
3.4 Key Concepts and Technologies Driving the Evolution of Distributed Processing 63
3.5 Recent Advancements in Distributed Processing 64
3.6 Changing Landscape of Computing 65
3.7 Security and Privacy Considerations 68
3.8 Opportunities and Challenges in Distributed Computing 71
3.9 Conclusion 73
4 Distributed System Architecture and Computing Models 77
Sandhya Avasthi and Suman Lata Tripathi
4.1 Introduction 77
4.2 Distributed System Architecture 78
4.3 Middleware in Distributed Systems 85
4.4 Distributed Cloud Architecture 88
4.5 Distributed Machine Learning 90
4.6 Conclusion 91
5 Parallel Computing Models and Architecture 95
Aastha Sharma, Swati Sharma and Divya Mishra
5.1 Introduction 96
5.2 Evolution of Parallel Computing Models 98
5.3 Parallel Computing Models 100
6 Network Issues and High-Level Communication Tools in Distributed Computing 115
Sandhya Avasthi and Sachin Kumar Vishwakarma
6.1 Introduction 116
6.2 Latency in Distributed and Parallel Computing 118
6.3 Straggler Effect 122
6.4 Packet Loss in the Distributed Network 125
6.5 Network Congestion 128
6.6 Communication Load 129
6.7 Conclusion 132
7 Infinite Horizons: Empowering Business Education Through Metaverse 135
Shikha Mishra and Sujit S. Nair
7.1 Introduction 136
7.2 Challenges and Concerns 137
7.3 Rules of Law-Based Governance 139
7.4 Possible Implementations in Educational Framework of Metaverse Technology 140
7.5 Understanding Metaverse 141
7.6 Explore the Transformational Potential of Metaverse for Business Education 142
7.7 Innovation and Design Ideas Forum 145
7.8 Continuing Education and Career Development 147
7.9 Takeaways 147
7.10 Ethical and Legal Issues 148
7.11 Future Prospects and Innovations 149
7.12 Conclusion 149
8 Paradigm Shifts and Future Directions in Distributed Data Management for Decentralized Networks 153
Bishnu Kant Shukla, Bhupender Parashar, Vikas Kumar Singla and Shivam Verma
8.1 Introduction 154
8.2 Paradigm Shifts in Distributed Data Management 155
8.3 Emergent Architectures and Frameworks 156
8.4 Integration of IoT 160
8.5 Edge Computing and Big Data Analytics 163
8.6 Data Aggregation and Summarization Techniques 164
8.7 Advanced Applications and Case Studies 168
8.8 Future Directions in Distributed Data Management 172
8.9 Conclusion 174
9 Autonomy and Adaptive Architectures in Distributed Systems 183
Bishnu Kant Shukla, Bhupender Parashar, Vikas Kumar Singla and Shivam Verma
9.1 Introduction 184
9.2 Conceptual Framework 185
9.3 Architectural Models 187
9.4 Adaptive Strategies in Distributed Systems 188
9.5 Case Studies 191
9.6 Application of MASs 195
9.7 Autonomous Navigation Systems 199
9.8 Collision Avoidance Algorithms 202
9.9 Network Properties in Distributed Systems 206
9.10 Challenges and Future Directions 207
9.11 Conclusion 210
10 Distributed Consensus Frequency Control in Networked Microgrid 219
Jeevitha Kandasamy, J. Jenitha, M.R. Mohanraj, Jenifer Amla. L., G. Mahalakshmi and Ponrekha M.
10.1 Introduction 220
10.2 System Model 224
10.3 Distributed Control Technique 225
10.4 Results and Discussion 228
10.5 Conclusion 231
11 Navigating Trust in Distributed Systems 235
Pooja Dehankar and Susanta Das
11.1 Introduction 236
11.2 Transparency Basics 240
11.3 Heterogeneous and Homogeneous DSs 240
11.4 Trust Management 242
11.5 Components' Roles 243
11.6 User Authentication and Access Control 244
11.7 Trust Framework Pillars 244
11.8 Trust Metrics 244
11.9 Trust Models and Mechanisms 245
11.10 Trust in Sensor Networks 247
11.11 Apps of TMSs 248
11.12 DSs Advantages and Challenges 249
11.13 Artificial Intelligence for Trust Management 250
11.14 Conclusion 251
12 Trust in Distributed Systems, Distributed Ledger, Blockchain, and Related Technologies 257
Shikha Verma, Priya Mishra and Sandhya Avasthi
12.1 Introduction 257
12.2 Trust in Distributed Ledger 258
12.3 Trust Management in Edge-Based IoT Systems 260
12.4 A Taxonomy of Trust Computation Frameworks for Edge-Based IoT Systems 261
12.5 Trust in Distributed Databases 262
12.6 Trust in Blockchain 263
12.7 Serverless Computing 268
12.8 Trust in Cloud Computing 273
12.9 Conclusion 275
13 Strategy to Improve Psychological Elements and Customer Service in Intelligent Health Systems 281
Mani Dublish, Anita Pati Mishra and Shikha Mittal
13.1 Introduction 282
13.2 Design with User in Mind 284
13.3 Designing Networked Medication Equipment with a Human Center 285
13.4 Testing Usability with End Users 285
13.5 Framework Required in Healthcare Fiction for Human-Based Design 287
13.6 A Necessary Step to Enable Quick Learning Processes 287
13.7 Requirement for a Planned Approach Setup 288
13.8 Limitations 291
13.9 Conclusion 292
14 New Age Techs Backbone: Role of Distributed Systems in Paving Path for a Durable Future 299
Kadambri Agarwal and Saijal Dahiya
14.1 Introduction 300
14.2 Distributed Systems in Smart Grids 303
14.3 Distributed Systems in IoT 305
14.4 Distributed Systems in Blockchain 308
14.5 Distributed Systems in Machine Learning 310
14.6 Distributed Systems in AR 313
14.7 Distributed Systems in Distributed Databases 315
14.8 Distributed Systems in Metaverse 317
14.9 Conclusion 320
15 Power Optimization in 4Bit Comparator Architecture Through Precomputation Technique 323
Adarsh Sharma, Suman Lata Tripathi, Deepika Ghai and Khilda Afifah
15.1 Introduction 324
15.2 Comparator 327
15.3 Precomputation Technique 328
15.4 Reported Precomputation Circuit 329
15.5 Comparator Design with Cadence Virtuoso 330
15.6 Future Scopes 331
15.7 Conclusion 335
Acknowledgement 335
References 335
Index 339
1
Introduction to Distributed Systems
K. Karthikeyan1*, S. Hemalatha2 and S. Vignesh1
1Department of Computer Science and Business Systems, K.S. Rangasamy College of Technology, Tiruchengode, India
2Department of Computer Science and Design, Kongu Engineering College, Perundurai, India
Abstract
A distributed system is an assembly of independent computers linked by a network that cooperates to accomplish a shared objective. Distributed systems, as opposed to conventional centralized systems, which assign all work to one machine, provide for greater performance, fault tolerance, and scalability. The goal of communication-efficient distributed machine learning is to train machine learning models across several nodes while minimizing the communication overhead in distributed systems. In distributed machine learning, data are distributed across various nodes, and computation is performed in parallel. However, exchanging data and model parameters between nodes can lead to significant communication costs, especially in scenarios with large datasets or complex models. Training complex machine learning models on large datasets is very computationally intensive. Distributed ML systems partition the data and workload across multiple machines to parallelize and speed up training. However, the communication cost of synchronizing model parameters can dominate system performance. Blockchain technology is inherently based on distributed systems principles. It decentralizes the storage and management of data, ensuring transparency, security, and reliability. The legitimate network state and history must be agreed upon by all nodes running the blockchain software. To establish the canonical chain, consensus techniques including proof-of-work, proof-of-stake, and Raft are employed. By agreement, fault tolerance is provided. Each node in the blockchain network keeps a local copy of the whole ledger or state. Transparency is enabled by many mechanisms that broadcast and replicate changes among nodes. Trends such as serverless computing, edge network growth, distributed ledgers, and autonomic systems point to more diffusion of distribution across the computing stack. For sectors like healthcare, distributed technologies promise to transform care delivery by connecting data and providers across geographic and organizational silos. Distributed systems have clearly emerged as a transformational paradigm in modern computing. But truly realizing their potential while managing their complexities continues to be an active domain for innovations in algorithms, mathematical models, systemic architectures, and practical tools.
Keywords: Distributed, blockchain, Internet of Things (IoT), raft, machine learning, nodes, healthcare
1.1 Introduction
A group of separate computers that appear to users to be a single, cohesive system is known as a distributed system. These systems are designed to share resources and coordinate their actions to achieve a common goal. The computers in a distributed system are connected through a communication network and communicate with each other by passing messages.
1.1.1 Background and Context
The primary motivation behind distributed systems is to achieve better performance, scalability, fault tolerance, and resource sharing. By distributing the workload across multiple computers, distributed systems can handle larger computational tasks and support more users than a single computer system. Additionally, distributed systems can provide redundancy, ensuring that if one component fails, the system as a whole can continue to operate.
Some key characteristics and challenges of distributed systems include:
Concurrency: Multiple components in a distributed system may execute concurrently, which can lead to synchronization issues and data inconsistencies.
Lack of a global clock: It is difficult to maintain a consistent global clock across distributed components, which can make it challenging to order events consistently.
Scalability: Distributed systems should be able to scale up or down by adding or removing components without significantly affecting the overall performance or functionality.
Transparency: Ideally, the distribution of resources and components should be transparent to users, who should perceive the system as a single, cohesive unit.
Heterogeneity: Numerous hardware and software components are frequently used in distributed systems. Heterogeneity, coming from several vendors, may provide compatibility and integration issues [1].
Figure 1.1 shows the evidence of distributed systems are widely used in various domains, including web applications, cloud computing, peerto-peer (P2P) networks, big data processing, and distributed databases. Examples of popular distributed systems include Apache Hadoop and Kubernetes. To design and implement distributed systems, developers often rely on architectural patterns and principles, such as the client-server model, message queues, load balancing, replication, and consensus algorithms (e.g., Paxos, Raft). Additionally, distributed systems may leverage technologies such as remote procedure calls (RPCs), distributed file systems, and distributed caching to facilitate communication and data sharing among components.
A common diagram used to illustrate the concept of a distributed system would be a network diagram, depicting multiple computers or nodes connected through a communication network.
- Numerous hardware and software components are frequently used in distributed systems. Heterogeneity, coming from several vendors, may provide compatibility and integration issues.
- The communication network that allows these nodes to communicate with one another is represented by the lines joining them, which might be the Internet, a local area network (LAN), or any other appropriate network.
- The nodes are independent and autonomous but can share resources and coordinate their actions by sending and receiving messages over the network.
- The diagram shows how distributed systems are decentralized, meaning that several nodes cooperate to accomplish a particular objective rather than depending on a single hub for failure or control.
Figure 1.1 Distributed systems.
This is a simplified representation, and real-world distributed systems can be much more complex, involving different types of nodes (e.g., servers, clients, load balancers, databases), different network topologies, and various communication protocols and technologies [2].
1.1.2 Objectives of the Study
The intrinsic difficulties and demands of distributed systems motivate a number of important goals that the study of these systems seeks to accomplish. The goal of studying distributed systems is to develop scalable, secure, dependable, and efficient systems that can meet the ever-increasing needs of contemporary computer settings. These goals can be met by researchers and developers to build reliable and efficient distributed systems that support a variety of services and applications.
1.1.3 Scope and Limitations
Distributed systems have a broad scope, enabling large-scale computing tasks, high availability, fault tolerance, efficient resource sharing, load balancing, scalability, geographic distribution, and interoperability across heterogeneous environments. They are essential in areas such as the Internet of Things (IoT) and enable parallel computing. However, distributed systems also face several limitations. Their complexity arises from synchronization requirements, fault tolerance mechanisms, and coordination among multiple nodes. Network latency, bandwidth constraints, and maintaining data consistency and integrity across nodes pose challenges. Security concerns related to data privacy, integrity, and access control are heightened due to communication and data sharing among nodes. Integrating diverse hardware and software components can lead to compatibility issues. Debugging, troubleshooting, and handling partial failures become more difficult compared with centralized systems. Efficient resource allocation and load balancing across nodes require sophisticated algorithms. While offering numerous benefits, the inherent complexity and potential limitations of distributed systems must be carefully considered during design, implementation, and operation to mitigate risks and maximize their advantages as mentioned in Table 1.1.
Table 1.1 Scope and limitations of distributed systems.
SN. Scope Limitations 1 Large-scale computing Complexity 2 High availability and fault tolerance Network latency and bandwidth 3 Resource sharing and load balancing Consistency and data integrity 4 Scalability Security and trust 5 Geographic distribution Heterogeneity challenges 6 Heterogeneous environments Debugging and troubleshooting 7 Parallel processing Partial failure handling 8 Internet of Things (IoT) Resource allocation and load balancing