Data-Variant Kernel Analysis

List of Figures xiii

List of Tables xix

Preface xxiii

Acknowledgments xxv

Chapter 1 Survey 1

1.1 Introduction of Kernel Analysis 1

1.2 Kernel Offline Learning 2

1.2.1 Choose the Appropriate Kernels 3

1.2.2 Adopt KA into the Traditionally Developed Machine Learning Techniques 6

1.2.3 Structured Database with Kernel 9

1.3 Distributed Database with Kernel 12

1.3.1 Multiple Database Representation 12

1.3.2 Kernel Selections Among Heterogeneous Multiple Databases 13

1.3.3 Multiple Database Representation KA Applications to Distributed Databases 14

1.4 Kernel Online Learning 16

1.4.1 Kernel-Based Online Learning Algorithms 16

1.4.2 Adopt "Online" KA Framework into the Traditionally Developed Machine Learning Techniques 17

1.4.3 Relationship Between Online Learning and Prediction Techniques 21

1.5 Prediction with Kernels 22

1.5.1 Linear Prediction 22

1.5.2 Kalman Filter 23

1.5.3 Finite-State Model 23

1.5.4 Autoregressive Moving Average Model 24

1.5.5 Comparison of Four Models 25

1.6 Future Direction and Conclusion 26

References 26

Chapter 2 Offline Kernel Analysis 41

2.1 Introduction 41

2.2 Kernel Feature Analysis 43

2.2.1 Kernel Basics 43

2.2.2 Kernel Principal Component Analysis (KPCA) 45

2.2.3 Accelerated Kernel Feature Analysis (AKFA) 46

2.2.4 Comparison of the Relevant Kernel Methods 48

2.3 Principal Composite Kernel Feature Analysis (PC-KFA) 49

2.3.1 Kernel Selections 49

2.3.2 Kernel Combinatory Optimization 52

2.4 Experimental Analysis 54

2.4.1 Cancer Image Datasets 54

2.4.2 Kernel Selection 56

2.4.3 Kernel Combination and Reconstruction 58

2.4.4 Kernel Combination and Classification 59

2.4.5 Comparisons of Other Composite Kernel Learning Studies 60

2.4.6 Computation Time 61

2.5 Conclusion 61

References 62

Chapter 3 Group Kernel Feature Analysis 69

3.1 Introduction 69

3.2 Kernel Principal Component Analysis (KPCA) 71

3.3 Kernel Feature Analysis (KFA) for Distributed Databases 73

3.3.1 Extract Data-Dependent Kernels Using KFA 73

3.3.2 Decomposition of Database Through Data Association via Recursively Updating Kernel Matrices 75

3.4 Group Kernel Feature Analysis (GKFA) 78

3.4.1 Composite Kernel: Kernel Combinatory Optimization 79

3.4.2 Multiple Databases Using Composite Kernel 81

3.5 Experimental Results 83

3.5.1 Cancer Databases 83

3.5.2 Optimal Selection of Data-Dependent Kernels 84

3.5.3 Kernel Combinatory Optimization 84

3.5.4 Composite Kernel for Multiple Databases 86

3.5.5 K-NN Classification Evaluation with ROC 87

3.5.6 Comparison of Results with Other Studies on Colonography 89

3.5.7 Computational Speed and Scalability Evaluation of GKFA 90

3.6 Conclusions 91

References 92

Chapter 4 Online Kernel Analysis 97

4.1 Introduction 97

4.2 Kernel Basics: A Brief Review 99

4.2.1 Kernel Principal Component Analysis 99

4.2.2 Kernel Selection 100

4.3 Kernel Adaptation Analysis of PC-KFA 101

4.4 Heterogeneous vs. Homogeneous Data for Online PC-KFA 102

4.4.1 Updating the Gram Matrix of the Online Data 103

4.4.2 Composite Kernel for Online Data 104

4.5 Long-Term Sequential Trajectories with Self-Monitoring 104

4.5.1 Reevaluation of Large Online Data 105

4.5.2 Validation of Decomposing Online Data into Small Chunks 106

4.6 Experimental Results 107

4.6.1 Cancer Datasets 107

4.6.2 Selection of Optimum Kernel and Composite Kernel for Offline Data 108

4.6.3 Selection of Optimum Kernel and Composite Kernel for the New Online Sequences 110

4.6.4 Classification of Heterogeneous Versus Homogeneous Data 111

4.6.5 Online Learning Evaluation of Long-term Sequence 112

4.6.6 Evaluation of Computational Time 116

4.7 Conclusions 117

References 117

Chapter 5 Cloud Kernel Analysis 121

5.1 Introduction 121

5.2 Cloud Environments 123

5.2.1 Server Specifications of Cloud Platforms 123

5.2.2 Cloud Framework of KPCA for AMD 124

5.3 AMD for Cloud Colonography 125

5.3.1 AMD Concept 125

5.3.2 Data Configuration of AMD 126

5.3.3 Implementation of AMD for Two Cloud Cases 129

5.3.4 Parallelization of AMD 132

5.4 Classification Evaluation of Cloud Colonography 135

5.4.1 Databases with Classification Criteria 135

5.4.2 Classification Results 137

5.5 Cloud Computing Performance 140

5.5.1 Cloud Computing Setting with Cancer Databases 140

5.5.2 Computation Time 142

5.5.3 Memory Usage 144

5.5.4 Running Cost 145

5.5.5 Parallelization 145

5.6 Conclusions 146

References 147

Chapter 6 Predictive Kernel Analysis 153

6.1 Introduction 153

6.2 Kernel Basics 154

6.2.1 KPCA and AKFA 155

6.3 Stationary Data Training 157

6.3.1 Kernel Selection 157

6.3.2 Composite Kernel: Kernel Combinatory Optimization 159

6.4 Longitudinal Nonstationary Data with Anomaly/Normal Detection 160

6.4.1 Updating the Gram Matrix Based on Nonstationary Longitudinal Data 160

6.4.2 Composite Kernel for Nonstationary Data 162

6.5 Longitudinal Sequential Trajectories for Anomaly Detection and Prediction 163

6.5.1 Anomaly Detection of Nonstationary Small Chunks Datasets 164

6.5.2 Anomaly Prediction of Long-Time Sequential Trajectories 167

6.6 Classification Results 169

6.6.1 Cancer Datasets 169

6.6.2 Selection of Optimum Kernel and Composite Kernel for Stationary Data 170

6.6.3 Comparisons with Other Kernel Learning Methods 172

6.6.4 Anomaly Detection for the Nonstationary Data 174

6.7 Longitudinal Prediction Results 175

6.7.1 Large Nonstationary Sequential dataset for Anomaly Detection 175

6.7.2 Time Horizontal Prediction for Risk Factor Analysis of Anomaly Long-Time Sequential Trajectories 178

6.7.3 Computational Time for Complexity Evaluation 179

6.8 Conclusions 180

References 181

Chapter 7 Conclusion 185

Appendix A 189

Appendix B Representative Matlab codes 195

B.1 Accelerated Kernel Feature Analysis 196

B.2 Experimental Evaluations 198

B.3 Group Kernel Analysis 201

B.4 Online Composite Kernel Analysis 206

B.5 Online Data Sequences Contol 208

B.6 Alignment Factor 209

B.7 Cloud Kernel Analysis 210

B.8 Plot Computation Time 211

B.9 Parallelization 212

Index 215