Graph Convolutional Neural Networks for Computer Vision
Description
Revolutionize your machine learning practice with this essential book that provides expert insights into leveraging Graph Convolutional Networks (GCNNs) to overcome the limitations of traditional CNNs.
In the last decade, computer vision has become a major focus for addressing the world's growing processing needs. Many existing deep learning architectures for computer vision challenges are based on convolutional neural networks (CNNs). Despite their great achievements, CNNs struggle to encode the intrinsic graph patterns in specific learning tasks. In contrast, graph convolutional networks have been used to address several computer vision issues with equivalent or superior results. The use of GCNNs has shown significant achievement in image classifications, video understanding, point clouds, meshes, and other applications in natural language processing. This book focuses on the applications of graph convolutional networks in computer vision. Through expert insights, it explores how researchers are finding ways to perform convolution algorithms on graphs to improve the way we use machine learning.
Malini Alagarsamy, PhD is an assistant professor at the Thiagarajar College of Engineering. She has published more than 30 research papers in journals and national and international conferences. Her research interests include software engineering, mobile application development, green computing, Internet of Things, blockchain, and machine learning.
Rajesh Kumar Dhanaraj, PhD is a Professor in the School of Computing Science and Engineering at Galgotias University. He has authored and edited more than 25 books and 53 articles in international journals and conferences and holds 21 patents. His research interests include machine learning, cyber-physical systems, and wireless sensor networks.
J. Felicia Lilian is an Assistant Professor at the Thiagarajar College of Engineering. She has published more than 10 articles in international journals and conferences. Her research interests include natural language processing, machine learning, and deep learning.
Vandana Sharma, PhD is an Associate Professor at the Amity Institute of Information Technology at the Amity University Noida Campus with more than 14 years of teaching experience. She has published 25 research papers in international journals and conferences. Her primary areas of interest include artificial intelligence, machine learning, blockchain technology, and the Internet of Things (IoT).
Gheorghita Ghinea, PhD is a Professor in the Department of Computer Science at Brunel University London. He has more than 600 publications to his credit, including book chapters and research articles in international journals of repute. His research centers on extending the notion of multimedia with that of mulsemedia, a term to denote multiple sensorial media.
More details
Other editions
Additional editions
Content
Preface xv
1 Role of Graph Convolutional Neural Networks (GCNN) in Computer Vision Applications 1
A. Malini, Vandana Sharma, J. Felicia Lilian, Rajesh Kumar Dhanaraj, Sharangapriyan S. and Shrinivas S.
1.1 Introduction 2
1.2 Understanding Convolutional Neural Network in Computer Vision 2
1.3 Core Components of CNN 3
1.4 Extending CNNs to Handle Graph-Structured Data 3
1.5 Application of GCNN in Computer Vision 6
1.6 Enhancing Performance and Interpretability with GCNN 8
1.7 Future Directions and Emerging Trends 10
1.8 Challenges and Open Research Questions 13
1.9 Case Studies: Real-World Applications 16
1.10 Conclusion 18
2 Scene Graph Generation from Static Images: Overview, Methods, and Applications 21
K. Krishnakishore, R. Vijayarangan, V. Jagan Naveen and V. Kannan
2.1 Introduction 22
2.2 Definition 24
2.3 Challenge 25
2.4 Scene Graph Generation 25
2.5 Static Image 25
2.6 Degradation of a Static Image 26
2.7 Method 1: Wavelet Feature Extraction 29
2.8 Psychological Perspective 32
2.9 Linguistic Perspective 33
2.10 Concepts and Conceptual Structures in Artificial Intelligence Perspective 35
2.11 Applications of CGS 37
2.12 Linguistic and Psychological Perspective 39
2.13 Image Synthesis from Layouts 41
2.14 Method Comparison 42
2.15 Conclusion 43
3 Transformation from CNN to Graph-Structured Data: Node Classification and Edge Prediction 47
R. Vijayarangan, R. Satish Kumar, K. Umadevi and K. Ashok Kumar
3.1 Why Graphs 48
3.2 SVM (Support Vector Machine) 57
3.3 XGBOOST 58
3.4 Artificial Neural Network (ANN) 59
3.5 Auto Encoder (AE) 62
3.6 Demographic and Related Data: Health Condition, Type of Gender, Age, Family Condition 63
3.7 Naïve Bayes (NB) 64
3.8 Random Forest (RF) 66
3.9 Conclusions 68
4 Research Trends and Challenges of GCNN Over CNN and Digital Image Processing Techniques 73
Rithish Kanna S., Suganthi P. and Kavitha P.
4.1 Introduction 74
4.2 Introduction to Convolutional Neural Network 75
4.3 Neural Style Transfer-Artistic View 78
4.4 Various Existing Works of NST 79
4.5 Hybrid Neural Style Transfer 81
4.6 Implementation of HNST 85
4.7 Results and Inference 86
4.8 Further Ideas of HNST 91
4.9 Conclusion 92
5 Classification of Graph Filtering Operations and Inductive Learning by Exploiting Multiple Graphs in GCNN 95
S. Kayalvizhi, Harish Sekar and Prasanna Guptha M.P.
5.1 Introduction 96
5.2 Graph Basics 96
5.3 Graph Convolutional Filters 98
5.4 Graph Filter Banks 107
5.5 Graph Neural Networks 110
5.6 Conclusion 112
6 GCNN with Adaptive Filters for Hyperspectral Image Classification 117
U. Moulali, R. Vijayarangan, S. Khaleel Ahamed and Kamakshaiah Kolli
6.1 Introduction 118
6.2 Related Works 120
6.3 Classification of Graph Filtering Operations 123
6.4 Experimental Analysis and Discussion 134
6.5 Conclusion 136
7 Graph Convolution Neural Network on Human Motion Prediction 141
B. Subbulakshmi, M. Nirmala Devi and Srimadhi J.
7.1 Introduction 141
7.2 Graph Convolution Neural Network (GCN) 146
7.3 Forms of GCN on Human Motion Prediction 148
7.4 Types of Graphs Employed on GCN 156
7.5 Conclusion 157
8 GraphChXNet: A Graph Convolutional Neural Network-Based Model for Detecting Chest Diseases Using X-Ray Images 161
D. Kiruthika, N. Vinothini, G. Jegan and G. Ananthi
8.1 Introduction 162
8.2 Proposed Methodology 164
8.3 Results and Discussion 171
8.4 Conclusion 178
9 Aspect-Based Sentiment Analysis Using GCN 181
Sachin K., Santhosh K.M.R., Sugindar A.D. and J. Felicia Lilian
9.1 Introduction 181
9.2 GCN and ABSA 185
9.3 Advancements of GCN and ABSA over the Years 189
9.4 Advancement of Technology with GCN and Algorithm Used 196
9.5 Case Study on GCN Application: Recommendation Systems 199
9.6 Summary 202
10 Analysis and Classification Using Graph Convolutional Neural Networks in Medical Imaging 205
M. Suguna and Priya Thiagarajan
10.1 Introduction 206
10.2 Literature Review-GCNN in Healthcare 210
10.3 Methodology 213
10.4 Results and Discussion 218
10.5 Conclusion 220
11 Case Studies and Real-World Applications of Graph Convolutional Networks in Computer Vision 225
Yogeesh N.
11.1 Introduction 226
11.2 Graph Convolutional Networks: A Brief Review 228
11.3 Case Study 1: Graph Convolutional Networks for Image Classification 231
11.4 Case Study 2: Object Detection and Localization Using Graph Convolutional Networks 236
11.5 Case Study 3: Semantic Segmentation with Graph Convolutional Networks 238
11.6 Case Study 4: 3D Vision and Point Cloud Processing of Graph Convolutional Networks 240
11.7 Case Study 5: Graph Convolutional Networks for Video Understanding and Action Recognition 243
11.8 Other Notable Case Studies and Applications 244
11.9 Discussion and Future Directions 249
11.10 Conclusion 250
12 Case Study and Use Cases of Dynamic Graphs in GCNN for Computer Vision 255
S. Anubha Pearline and S. Geetha
12.1 Introduction 255
12.2 Graph Convolutional Neural Networks (GCNNs) 259
12.3 GCNN Case Studies 265
12.4 Challenges and Issues in GCNN for CV 270
12.5 Conclusion 270
References 271
About the Editors 275
Index 279
1
Role of Graph Convolutional Neural Networks (GCNN) in Computer Vision Applications
A. Malini1*, Vandana Sharma2, J. Felicia Lilian3, Rajesh Kumar Dhanaraj4, Sharangapriyan S.3 and Shrinivas S.3
1School of Computer Science and Engineering, Vellore Institute of Technology, Chennai, Tamil Nadu, India
2Christ University, Bengaluru, India
3Department of Computer Science and Business Systems, Thiagarajar College of Engineering, Tamil Nadu, India
4Symbiosis Institute of Computer Studies and Research (SICSR), Symbiosis International (Deemed University), Pune, India
Abstract
Graph Convolution Neural Networks (GCNNs) are an important concept in advancing computer vision by transforming the understanding and modeling of graph-structured data. They have a unique capability to capture intricate relations along with the visual content that goes beyond the traditional and usual convolutional neural networks, it also empowers computers to observe and interpret the complex interconnection between the elements in images, which enhances the depth and nuance of visual dentata analysis. As a revolutionary study in computer vision, GCNNs are poised to transform various industries by unleashing new frontiers in the visual information domain's analysis and interpretation. Their multifaceted applications promise to reshape the landscape of computer vision.
Keywords: Graph convolutional neural networks (GCNNs), computer vision, graph-structured data, visual data analysis, visual information analysis
1.1 Introduction
In our rapidly changing universe, computer vision's sphere is undergoing an amazingly unusual growth transforming several areas and enhancing human life's standards significantly. This paradigm change is supported by the huge expansion of Profound Learning and Nervous Networks in the computer vision realm.
Unlike humans, who reflexively gather footage and images, computers are carefully designed and trained to understand data enabling sophisticated analysis and rapid decision making. This innovative path has extensive implications for several fields, including the progress of autonomous vehicles, advanced medical testing, and social media recommendation system enhancement.
In today's world, GCNN has many applications. In social media analysis, GCNNs can analyze using nodes and edges as social networks by modeling users and their interactions, improving suggestion systems, class detection, and studies on influence propagation and predicting atomic behavior, molecular properties, and interactions by molecule models as graphical structures are crucial for genomics research in bioinformatics. For autonomous vehicles, GCNNs can model the environment as a graph, where nodes represent objects, and edges represent spatial relationships allowing for better scene understanding and decision making.
Additionally, GCNNs are employed in 3D computer vision tasks, such as shape analysis and 3D reconstruction, where 3D objects are represented as graphs of connected vertices. In conclusion, Graph Convolutional Neural Networks are revolutionizing the way complex data structures are analyzed and processed expanding the horizons of what is achievable in computer vision and beyond. Their ability to leverage the power of graphs opens up new possibilities for innovation and application across various domains further enhancing the impact of computer vision technologies on our everyday lives and enhancing human life's standards significantly.
1.2 Understanding Convolutional Neural Network in Computer Vision
Space Invariant Artificial Neural Networks have been the spearhead of reform in computer vision, which is the foundation of many refinements in visual analysis and interpretations. It has a profound effect on object recognition and scene understanding. CNN is often referred to as ConvNets representing a confined type of artificial neural network originated to operate mesh-like data such as images and videos. They have a very long history heading back to the 1980s, with seminal efforts by researchers like Yann LeCun, Geoffrey Hinton, and Yoshua Bengio. However, CNN came to action in the 2010s largely due to advances in computational power, the availability of large labeled datasets, and improved training algorithms.
1.3 Core Components of CNN
- Convolutional Layers: These layers apply filters (also termed as kernels) to the feed-in data to pull features. These filters are accustomed to extract the computing dot in every position, and it captures the local patterns available in the given input data.
- Layers for Pooling: Pooling layers lessen the geometrical extents of the characteristic map produced by convolutional layers. General pooling functions consist of average pooling and max pooling, which retain the most important data while reducing the computational complexity.
- Connected Layers (Fully Connected): The layers that are fully connected are considered standard neural networks that connect all axons from one layer to another typically leading to the final output. They serve as classifiers or regressors in the environs of computer vision assignments.
1.3.1 Hierarchy Feature Learning
One of CNNs' key advantages is its capacity for hierarchical learning from unfiltered pixel data. Higher layers combine these properties into more complicated representations as the data traverses through the network allowing the network to eventually recognize objects, patterns, and abstract concepts. Lower layers collect basic elements like edges and corners.
1.4 Extending CNNs to Handle Graph-Structured Data
For their remarkable performance in grid-like data, CNN has been utilized for a long time, especially in computer vision applications involving photos and videos. Graph-structured data, which has its own special set of difficulties and potential, is included in the broad landscape of data kinds, which goes far beyond grids.
1.4.1 Graphs-A Universal Data Structure
For data with complicated linkages, graphs provide a flexible representation. Each node can carry important information, and they are made up of vertices and the edges that connect them. Graphs are used in many different sectors, such as social media networks, biology, transportation, recommendation engines, and more. To derive useful insights from graphs, it is essential to interpret and analyze the patterns inside graph-structured data.
1.4.2 Challenges in Processing Graphs with CNN
- Irregular Data Structures: Graphs do not have homogeneous grids like ordinary grids do, which makes it challenging to extract the crucial data from the input.
- Variable Neighborhoods: Traditional CNNs struggle to handle the variable-sized neighborhoods that nodes in a graph can have.
1.4.3 Graph Convolutional Neural Networks (GCNNs): Bridging the Gap
To effectively extract features and patterns from structured data, GCNNs expand the concepts of CNNs to the world of graphs. By using graph convolutions that operate on nodes and their neighboring nodes, they provide an advantage by capturing the contextual information required for various applications.
1.4.4 Architectural Components of Graph Convolutional Layers
At the heart of GCNN is the representation of the input graph. This typically involves encoding both the graph structure and node attributes. Common representations include the following as given in Figure 1.1.
- Adjacency Matrix: The adjacency matrix, often referred to as A, is a key data structure that encodes the connectivity of nodes in a graph. In a plot with N nodes, the adjacency matrix is an N × N matrix, where each element (i,j) indicates whether there is an interrelation between node i, j. Typically, it is a binary matrix where the value of 1 signifies the presence of an edge, while a value of 0 indicates no connection.
- Node Feature Matrix: This is a matrix in which each row indicates a node, and each column indicates an attribute. The node feature matrix is a structured data representation where each row conforms to a node in a graph, and each node represents the specific feature or attribute associated with that node. In essence, it acts as a table where nodes are entities, and the attributes are the properties of characteristics of those nodes.
- Edge Feature Matrix: If the edge features are available, they can be encoded in a separate matrix. Edge feature matrices are responsible for enhanced connectivity between the nodes, which will give important information that is present in between the nodes. These added facts can reproduce various particulars of the relationships between nodes.
Figure 1.1 Components of GCNN.
1.4.5 Graph Convolutional Layers: Adaptation of Convolutional Layers
The convolutional neural networks (CNNs) are famous for their effectiveness on structured data like time signals and images. They rely on the core operation of convolution with learned filters to extract meaningful features from input data. However, intrinsic limitations come with CNNs specifically as suitable only for regular structured data sources. This drawback restricts their use in cases where data have irregular or non-uniform patterns. In response to this limitation, Graph Signal Processing has emerged as a powerful replacement paradigm. GSP provides an entirely new way of dealing with signals that are inherently represented by graphs or networks....
