Data Analysis and Related Applications, Volume 5
Description
This book is a collective work by several leading scientists, analysts, engineers, mathematicians and statisticians, who have been working at the forefront of data analysis and related applications, arising from data science, operations research, engineering, machine learning or statistics.
Data Analysis and Related Applications 5 represents a cross-section of current concerns and research interests in the above scientific areas. The collected material has been divided into appropriate sections to provide the reader with both theoretical and applied information on data analysis methods, models and techniques, along with appropriate applications.
More details
Other editions
Additional editions
Persons
Yiannis Dimotikalis works in the Department of Management Science and Technology at the Hellenic Mediterranean University, Greece. His research areas include teaching operations research (analytics), focusing on simulation and optimization.
Christos H. Skiadas is the Founder and Director of the Data Analysis and Forecasting Laboratory in the Technical University of Crete, Greece. He is also the former Vice-Rector of the Technical University of Crete and Chairman of the Department of Production Engineering and Management.
Content
Chapter 1 Modeling/Forecasting Patient Recruitment in Multicenter Clinical Trials Using Time-dependent Models 1
Volodymyr ANISIMOV and Lucas OLIVER
1.1 Introduction 1
1.2 Poisson-gamma model with time-dependent rates 5
1.2.1 The case of homogeneous rates 5
1.3 Non-homogeneous PG model 7
1.3.1 Estimation at the interim stage 9
1.3.2 Simulation of non-homogeneous PG model 10
1.4 Testing the recruitment rates for homogeneity 11
1.4.1 Poisson-type test 12
1.4.2 Criterion for testing hypothesis H 0 13
1.4.3 Poisson-gamma test 15
1.5 Implementations 19
1.6 Acknowledgment 20
1.7 References 20
Chapter 2 Forecasting the Next Megacycle of the Economy 23
George S. ATSALAKIS and Ioanna ATSALAKI
2.1 Introduction 23
2.2 2024: the end of an economic megacycle 24
2.3 The role of technology in shaping the future 25
2.4 The economic consequences of the new cycle 25
2.4.1 Presenting past megacycles 26
2.4.2 The structure of economic megacycles 26
2.4.3 Technology as a catalyst for megacycles 27
2.4.4 Historical patterns of energy and economic growth 27
2.5 The future of economic megacycles 28
2.5.1 The 2024-2080 megacycle: a new era of exponential change and growth 28
2.5.2 Stagnation phase: 2024-2052 29
2.5.3 Growth phase: 2052-2080 29
2.5.4 Geopolitical and societal implications 30
2.5.5 Role of labor and automation 31
2.6 Conclusions 31
2.7 References 32
Chapter 3 Modeling Functioning as a Determinant of Wellbeing: A Mediation Analysis 35
Anastasia CHARALAMPI and Catherine MICHALOPOULOU
3.1 Introduction 35
3.2 Methods 37
3.2.1 Procedure and participants 37
3.2.2 Measures and item selection 38
3.2.3 Statistical analyses 40
3.3 Results 41
3.3.1 Univariate analysis 41
3.3.2 Bivariate analysis: correlation analysis 42
3.3.3 Multivariate analysis: mediation analysis 42
3.4 Conclusions 47
3.5 References 48
3.6 Appendix 51
Chapter 4 Cross-Cultural Issues in Psychological Assessment: A Multistrategy Approach 53
Franca CRIPPA, Giulia GOTTI, Raffaella CALATI, Mariangela ZENGA, Kainaat DANYAL and Naved IQBAL
4.1 Introduction 53
4.2 Suicide risk among university students: India and Italy in direct comparison 54
4.3 Merging techniques: A better way in cross-cultural studies? 56
4.4 Do different people respond in the same way to common items? 58
4.5 Conclusions 61
4.6 References 61
Chapter 5 A Control Chart for Zero-Inflated Semi-Continuous Data 65
Fernanda Otília FIGUEIREDO, Adelaide FIGUEIREDO and M. Ivette GOMES
5.1 Introduction 65
5.1.1 Zero-inflated and hurdle models for count data 66
5.1.2 Inflated distributions for semi-continuous data 66
5.2 Zero-inflated Lomax distribution 67
5.2.1 The Lomax and the zero-inflated Lomax distributions 67
5.2.2 Maximum likelihood estimates 68
5.3 Shewhart control chart for monitoring zero-inflated Lomax data 69
5.4 Performance of the proposed control chart 70
5.5 Conclusions 74
5.6 Acknowledgments 76
5.7 References 76
Chapter 6 Further Results on Location Invariant Estimation of the Weibull Tail Coefficient 79
M. Ivette GOMES, Frederico CAEIRO and Lígia HENRIQUES-RODRIGUES
6.1 Introduction 79
6.2 Hill and GMs EVI and WTC-estimators 81
6.2.1 Power-mean-of-exponent- p (PM p) and Holder's mean-of-order-p (MO p = H p) EVI estimation 81
6.2.2 WTC estimation 83
6.3 Classes of PORT-GMs (PGMs) WTC-estimators 84
6.4 Monte Carlo simulation of the PORT-GPM p (PGPM p) WTC-estimators 85
6.5 Overall comments and open research topics 89
6.6 Acknowledgments 91
6.7 References 91
Chapter 7 What Can we Learn from Malta? An Exploration of Gender Disparities in Education, Work and Money in Europe 95
Erika GRAMMATICA, Francesca GRESELIN and Mariangela ZENGA
7.1 Introduction 95
7.2 Gender gap: education, work and money 96
7.3 Gender Equality Index 98
7.4 Three-way data approach based on principal component analysis 100
7.5 Evidence from principal component analysis 102
7.6 Results of trajectory analysis 103
7.7 Conclusions 106
7.8 Acknowledgments 106
7.9 References 107
Chapter 8 Financial Analysis of a Public Hospital: The Case of the Corfu General Hospital 109
Margarita IOANNIDOU and George MATALLIOTAKIS
8.1 Introduction 109
8.2 Materials and methods 110
8.3 Results 110
8.3.1 Liquidity ratios 110
8.3.2 Financial structure and viability ratios 111
8.3.3 Activity ratios 112
8.3.4 Profitability ratios 113
8.4 Discussion 114
8.5 References 115
Chapter 9 EWMA Control Charts for Skewed Distributions 117
Derya KARAGÖZ and Moustapha Aminou TUKUR
9.1 Introduction 117
9.2 Exponentially weighted moving average control charts 119
9.3 EMMA control charts for the non-normal process 120
9.3.1 The WV EWMA control chart 121
9.3.2 The WSD EWMA control chart 121
9.3.3 Newly proposed SC EWMA control chart 122
9.4 Real data 122
9.5 Simulation study 125
9.6 Simulation algorithm 126
9.7 Results and discussion 127
9.8 Conclusion 131
9.9 References 133
Chapter 10 Assessing the Impact of Renewable Energy Sources on Energy Economics: A Non-Linear Regression Analysis of Hellenic Energy Exchange Market Clearing Prices 135
Emmanuel KARAPIDAKIS, Yiannis KATSIGIANNIS, Konstantinos BLAZAKIS, Marios NIKOLOGIANNIS, George MATALLIOTAKIS, Georgios STAVRAKAKIS, Nikos VENIANAKIS and Paolo BONFINI
10.1 Introduction 135
10.2 Methodology 137
10.2.1 Spearman's rank 137
10.2.2 Sparse autoencoder 139
10.3 Results 140
10.4 Discussion 142
10.5 Conclusions 143
10.6 Acknowledgments 143
10.7 References 144
Chapter 11 Enhancing Energy Market Stability: Comparative Analysis of Forecasting Techniques for Market Clearing Prices in the Day-Ahead Market 147
Emmanuel KARAPIDAKIS, Yiannis KATSIGIANNIS, Konstantinos BLAZAKIS, Marios NIKOLOGIANNIS, George MATALLIOTAKIS, Georgios STAVRAKAKIS, Nikos VENIANAKIS and Nikolaos SCHETAKIS
11.1 Introduction 147
11.2 Methodology 150
11.3 Results 152
11.4 Discussion 155
11.5 Conclusions 155
11.6 Acknowledgments 155
11.7 References 156
Chapter 12 Using the Coxian Continuous-Time Hidden Markov Model to Analyze Lombardy Region Wards for Older Individuals 159
Hannah MITCHELL, Adele H. MARSHALL and Mariangela ZENGA
12.1 Introduction 160
12.2 Methodology 161
12.3 Data and results 165
12.3.1 Data 165
12.3.2 Results 165
12.4 Conclusions 172
12.5 Practice implications 172
12.6 Conflict of interest 173
12.7 References 173
Chapter 13 Estimators for Extreme Value Index: Advancements in Tail Inference 177
Ayana MATEUS and Frederico CAEIRO
13.1 Introduction 177
13.2 Estimators for the tail parameters 179
13.2.1 The new class of estimators for the EVI 179
13.2.2 Asymptotic properties of the GPWM estimators 181
13.2.3 Estimating an extreme quantile 182
13.3 Monte Carlo simulation study of the GPWM estimators 182
13.3.1 Methodology 183
13.3.2 Results 183
13.4 Conclusion 185
13.5 Acknowledgments 185
13.6 References 185
Chapter 14 Determinants of Students' Attitude Toward History: An Empirical Approach 187
Aristea MAVROGIANNI, Eleni VASILAKI and Maria GRYDAKI
14.1 Introduction 188
14.2 Previous research 190
14.2.1 Attitude toward history 190
14.2.2 Educational factors 191
14.2.3 Socioeconomic factors 191
14.3 Data and methods 194
14.3.1 Data 194
14.3.2 Empirical methodology 197
14.4 Results 198
14.5 Summary and conclusions 203
14.6 Appendices 204
14.6.1 Appendix A: the initial full questionnaire for the attitude survey toward history (EDIS) 204
14.6.2 Appendix B: the final questionnaire for the attitude survey toward history (EDIS) 206
14.6.3 Appendix C 207
14.7 References 208
Chapter 15 Methodological Procedures for Assessing the Quality of Death Certificates Due to Unknown Causes 217
Neir Antunes PAES
15.1 Introduction 217
15.2 Methods 219
15.2.1 First step: correction of underregistration of deaths (f) 220
15.2.2 Second step: redistribution of deaths due to ill-defined causes 223
15.2.3 Third step: redistribution of deaths due to non-specific causes (garbage codes) 226
15.3 Illustrative example 227
15.4 Conclusions 230
15.5 References 230
Chapter 16 Health Status, Cancer and Pneumonia Death Rates in Europe: 2019-2022 233
Elena RÍHOVÁ and Kornélia SVACINOVÁ
16.1 Introduction 233
16.2 Background 234
16.3 Methods 235
16.4 Results and discussion 237
16.4 Conclusions 244
16.5 References 244
Chapter 17 A Bayesian Asymmetric Approach to Modeling Volatility on Portfolios with Many Assets 247
David SUDA, Monique Borg INGUANEZ and Matthew CAMILLERI
17.1 Introduction 247
17.2 Dynamic principal component analysis 248
17.3 Bayesian Student-t GJR(1,1) model 250
17.4 Asymmetric modeling of a portfolio with many assets 251
17.5 Forecasting, predictive ability and risk estimation 253
17.6 Conclusion 256
17.7 References 256
Chapter 18 Pandemic-Driven Innovations: Utilizing Online Learning and Big Data Analysis for Decision-Making in Educational Environments 259
Leonidas THEODORAKOPOULOS, Ioanna KALLIAMPAKOU, Alexandra THEODOROPOULOU and Gerasimos KALOGERATOS
18.1 Introduction 260
18.2 Literature review 260
18.2.1 Difficulties during the COVID-19 period 260
18.2.2 Effects of COVID-19 on education 261
18.2.3 Big data analysis in educational research 262
18.2.4 Related work 263
18.3 Methodology 264
18.4 Research questions 264
18.4.1 Dataset presentation 265
18.5 Conclusion 273
18.6 Suggestions for further research 274
18.7 References 274
Chapter 19 Credit Card Fraud Detection with Machine Learning and Big Data Analytics: A PySpark Framework Implementation 281
Leonidas THEODORAKOPOULOS, Ioanna KALLIAMPAKOU, Alexandra THEODOROPOULOU and Fotini ZAKKA
19.1 Introduction 281
19.2 Literature review 283
19.2.1 Introduction to credit card fraud detection 283
19.2.2 The importance of detecting credit card fraud 284
19.2.3 Role of machine learning in improving decision-making processes in fraud detection 284
19.2.4 Automated pattern recognition 285
19.2.5 Predictive modeling 285
19.2.6 Dynamic risk scoring 285
19.2.7 Anomaly detection 286
19.2.8 Natural language processing (NLP) 286
19.2.9 Integration with existing systems 286
19.2.10 Credit card fraud detection: machine learning applications 286
19.2.11 Credit card fraud detection using Apache Spark 287
19.2.12 How can machine learning algorithms enhance decision quality in detecting fraud? 288
19.2.13 Improved detection accuracy 288
19.2.14 Real-time processing and analysis 288
19.2.15 Handling big data and complex variables 289
19.2.16 Adaptive learning for evolving threats 289
19.2.17 Cost efficiency through automation 289
19.2.18 Enhanced scalability 289
19.3 Materials and methods 290
19.3.1 Performance evaluation 292
19.4 Results 307
19.4.1 Comparative analysis 314
19.5 Conclusions 315
19.6 Future work 317
19.7 Limitations 317
19.8 References 318
Chapter 20 Quantitative Modeling of the Demographic Aging Process 323
Grazyna TRZPIOT
20.1 Introduction 323
20.2 Literature review 324
20.3 Methodology 325
20.3.1 Dependency ratios - double aging index 327
20.3.2 Multivariate regression model for the double aging index 330
20.4 Research results and discussion 330
20.5 Conclusions 331
20.6 References 332
Chapter 21 Hotel Sales During COVID-19: Evidence from the United States 333
Dimitrios VORTELINOS, Christos FLOROS, Alexandros APOSTOLAKIS and Ioannis PASSAS
21.1 Introduction 333
21.2 Literature review 334
21.3 Travel and tourism economic impact in 2022 335
21.3.1 Travel and tourism GDP 335
21.3.2 Travel and tourism employment 335
21.3.3 Travel and tourism forecasts 336
21.4 Key developments in the hospitality sector in 2022 336
21.4.1 Overall highlights 336
21.4.2 Hotel room demand 337
21.4.3 Occupancy 337
21.4.4 Room revenue 338
21.4.5 Workforce 338
21.4.6 State and local tax revenue 338
21.4.7 US hotel markets 339
21.5 Data and methodology 340
21.5.1 Data description 340
21.5.2 Methodology 340
21.5.3 LARC score 340
21.5.4 Graphical analysis of variables 341
21.6 Results 343
21.7 Concluding remarks and future research 346
21.8 Acknowledgments 346
21.9 References 346
Chapter 22 Investigating the Mediating Role of Religious Services Attendance in the Relationship Between Religion Variables and Social Class Perceptions 349
Aggeliki YFANTI and Catherine MICHALOPOULOU
22.1 Introduction 349
22.2 Method 351
22.2.1 Procedure and participants 351
22.2.2 Measures 352
22.2.3 Statistical analyses 353
22.3 Results 355
22.3.1 Univariate analyses 355
22.3.2 Bivariate analyses: correlation analyses 359
22.3.3 Multivariate analyses: mediation analyses 360
22.4 Conclusions 362
22.5 References 363
Chapter 23 New Methods of Constructing Confidence Intervals of a Sensitive Proportion in Survey Statistics 365
Marta ZALEWSKA and Wojciech NIEMIRO
23.1 Introduction 365
23.2 Method of moments (MM) for ICT data 367
23.3 EM estimation for ICT data and parametric bootstrap 367
23.3.1 Maximum likelihood via the EM algorithm 368
23.3.2 Percentile parametric bootstrap confidence intervals 369
23.4 "Almost exact" confidence intervals for ICT data 369
23.5 Simulation results 371
23.6 Conclusions 375
23.7 References 375
Chapter 24 Spreading Diseases Models Under Vaccination 377
S. ZIMERAS and D. VASILEIOU
24.1 Introduction 377
24.2 Modeling epidemic disease 378
24.3 Vaccination models analysis 380
24.4 SEIVR model 380
24.5 SIRV model 382
24.6 Conclusions 384
24.7 References 385
List of Authors 387
Index 393
1
Modeling/Forecasting Patient Recruitment in Multicenter Clinical Trials Using Time-dependent Models
Clinical trials in the modern era are characterized by their complexity and high costs and usually involve hundreds/thousands of patients to be recruited across multiple clinical centers in many countries, as typically it is required to achieve a rather large sample size to prove the efficiency of a particular drug.
As the imperative to enroll vast numbers of patients across multiple clinical centers has become a major challenge, an accurate forecasting of patient recruitment is one of the key factors for the operational success of clinical trials. A classic Poisson-gamma (PG) recruitment model assumes time-homogeneous recruitment rates. However, there can be potential time-trends in the recruitment driven by various factors, for example, seasonal changes and exhaustion of patients on particular treatments in some centers. Recently, a few authors have considered some extensions of the PG model to time-dependent rates under some particular assumptions. In this chapter, a natural generalization of the original PG model to a PG model with non-homogeneous time-dependent rates is introduced. Some tests on homogeneity of the rates (non-parametric using a Poisson model and parametric using a PG model) are considered. The techniques for modeling and simulation of the recruitment using time-dependent model are discussed; for re-projection of the remaining recruitment, it is proposed to use a moving window and re-estimating parameters at every interim time. The results are supported by simulation of some artificial data sets.
1.1. Introduction
Contemporary late-phase clinical trials involve hundreds or even thousands of patients that need to be recruited across multiple clinical centers in many countries, as typically it is required to achieve a rather large sample size to prove the efficiency of a particular drug. Citing Ken Getz, director, Tufts Center for Study of Drug Development, USA, patient recruitment and retention are among the greatest challenges that the clinical research enterprise faces today, and they are a major cause of drug development delays.
To address these challenges, novel approaches and predictive analytic techniques are required for efficient data analysis, monitoring and decision-making.
Let us consider typical multicenter clinical trials used in the pharmaceutical industry where the patients are recruited by many clinical centers in different countries and then randomized to different treatments.
Using Poisson models for modeling patient recruitment in clinical trials is a well-accepted approach. Several papers use Poisson processes in different clinical centers with fixed recruitment rates (Carter et al. (2005); Senn (1997, 1998)).
However, in real trials, different centers typically have different capacity and productivity, and the recruitment rates vary. To model this variation, Anisimov and Fedorov (2007) introduced a Poisson-gamma (PG) model, where the patient arrival is modeled using Poisson processes with some rates and the variation in rates in different centers is modeled using a gamma distribution.
This model can be described in the framework of the empirical Bayesian approach where the prior distribution of the rates is a gamma distribution with parameters that are evaluated either using historical data or data provided by study investigators. In Anisimov and Fedorov (2007), a maximum likelihood technique was proposed for estimating the parameters of the rates and the Bayesian technique for adjusting the posterior distribution of the rates at any interim time using recruitment data in the individual centers. Various applications to real trials are considered in Anisimov and Fedorov (2007), Anisimov et al. (2007) and Anisimov (2011a).
This technique was developed further to account for random delays and the closure of clinical centers (Anisimov (2011a, 2020)) and for modeling events in event-driven trials (oncology) (Anisimov (2011b, 2020); Anisimov et al. (2022)).
Note that independently, Gajewski et al. (2008) later considered a similar approach to modeling recruitment as a Poisson process with gamma distributed rate, however their model is applicable to only one clinical center.
The PG model was also used in Mijoule et al. (2012) to consider some extensions related to other distributions of the recruitment rates and sensitivity analysis, in Minois et al. (2017) to evaluate the duration of recruitment process when historical trials are available, and in Bakhshi et al. (2013) for the evaluation of parameters of a PG model using meta-analytic techniques for historic trials. A survey on using mixed Poisson models is provided in Anisimov (2016).
It is also worth mentioning the recent investigations devoted to centralized statistical monitoring and forecasting recruitment performance and also forecasting patient recruitment under various restrictions and creating an optimal cost-efficient recruitment design (Anisimov and Austin (2020, 2022, 2023); Anisimov (2023)).
There are also other approaches to recruitment modeling described in the literature; however, they mainly deal with the analysis of the global recruitment and therefore have some limitations. These approaches typically require rather large number of centes and patients and cannot be applied to predicting recruitment on center/country level (see survey papers by Barnard et al. (2010), Heitjan et al. (2015) and Gkioni et al. (2019)).
Currently, the originally developed PG model (Anisimov and Fedorov (2007); Anisimov (2011a)) has gained world-wide recognition and in some recent papers is now called "one of the most popular techniques" and an "industry-standard" model.
This model assumes that patient recruitment rates do not change over time. However, in real trials there can be some time-trends, seasonable changes, and recruitment can change in some countries over time due to different reasons, for example, a slowdown in recruitment near the end of a trial.
Therefore, to capture these situations, in some recent papers (Lan et al. (2019); Best et al. (2022); Perperoglou et al. (2023); Turchetta et al. (2023); Urbas et al. (2022)), the standard PG model was extended to the cases where the recruitment rates can be time dependent.
Lan et al. (2019) proposed a non-homogeneous Poisson process model that allows for staggered center activation and heterogeneity within centers modeled by a gamma distribution and assuming that after a period of steady recruitment, the center mean recruitment rate gradually declines as a negative exponential with some coefficient. They calculated the posterior distribution and considered some applications to real trials. Urbas et al. (2022) proposed a more general time-dependent PG model that allows for a wider range of recruitment rate functions. The model is fitted using a maximum likelihood approach and is used to select the best model among a set of candidate models. Perperoglou et al. (2023) compared the performance of the standard PG model to two time-dependent models: the models proposed in Lan et al. (2019) and Urbas et al. (2022). The study found that the time-dependent models outperformed the PG model in terms of prediction accuracy, especially in trials with time-varying recruitment rates. Best et al. (2022) considered a standard PG model as the starting point and developed a more flexible version that introduces variation in rates over time as a function of COVID-19-dependent covariates, which is implemented in a fully Bayesian probabilistic framework. Turchetta et al. (2023) proposed a time-dependent PG model allowing recruitment rates to vary over time through B-splines that is fitted using a Bayesian approach. The model was evaluated in a simulation study and found to perform well in a variety of scenarios.
All five papers above contribute to the advancement of recruitment modeling in multicenter clinical trials by extending the standard time-homogeneous PG model to accommodate time-dependent recruitment rates.
The flexibility of the original PG model allows a rather natural extension to model time-dependent rates on the center level. However, the main question here is: What type of time-dependence to use, especially, how to detect the change time-point of the behavior of the rates, and how to test this dependence in real trials? Thus, the choice of a suitable time-dependent model is trial specific.
In this chapter, the authors consider a natural generalization of a PG model to a time-dependent model using a general framework of a standard PG model, which is similar to the way considered in Urbas et al. (2022). The basic methodology of using a PG model with time-dependent rates is developed by the first author in Anisimov (2024). This chapter extends some results of Anisimov (2024); in particular, a new PG criterion is investigated for testing the recruitment rates for homogeneity. In Anisimov (2024), a novel analytic methodology is also proposed for modeling/forecasting patient recruitment using the recent results in Anisimov and Austin (2020) about a PG approximation of the sums of PG processes in different centers that can be applied to modeling recruitment even in not so large countries/regions, which is also described in this chapter for completeness of presentation.
Note that this...
