Numerical Methods for Simulation and Optimization of Piecewise Deterministic Markov Processes
Application to Reliability
1st Edition
Published on 15. December 2015
Book
Hardback
304 pages
978-1-84821-839-0 (ISBN)
Description
Mark H.A. Davis introduced the Piecewise-Deterministic Markov Process (PDMP) class of stochastic hybrid models in an article in 1984. Today it is used to model a variety of complex systems in the fields of engineering, economics, management sciences, biology, Internet traffic, networks and many more. Yet, despite this, there is very little in the way of literature devoted to the development of numerical methods for PDMDs to solve problems of practical importance, or the computational control of PDMPs.
This book therefore presents a collection of mathematical tools that have been recently developed to tackle such problems. It begins by doing so through examples in several application domains such as reliability. The second part is devoted to the study and simulation of expectations of functionals of PDMPs. Finally, the third part introduces the development of numerical techniques for optimal control problems such as stopping and impulse control problems.
This book therefore presents a collection of mathematical tools that have been recently developed to tackle such problems. It begins by doing so through examples in several application domains such as reliability. The second part is devoted to the study and simulation of expectations of functionals of PDMPs. Finally, the third part introduces the development of numerical techniques for optimal control problems such as stopping and impulse control problems.
More details
Language
English
Place of publication
London
United Kingdom
Target group
Professional and scholarly
Product notice
sewn/stitched
Cloth over boards
Dimensions
Height: 240 mm
Width: 161 mm
Thickness: 21 mm
Weight
616 gr
ISBN-13
978-1-84821-839-0 (9781848218390)
Copyright in bibliographic data and cover images is held by Nielsen Book Services Limited or by the publishers or by their respective licensors: all rights reserved.
Schweitzer Classification
Other editions
Additional editions
Benoite De Saporta | François Dufour | Huilong Zhang
Numerical Methods for Simulation and Optimization of Piecewise Deterministic Markov Processes
Application to Reliability
E-Book
12/2015
Wiley-ISTE
€139.99
Available for download
Benoite De Saporta | François Dufour | Huilong Zhang
Numerical Methods for Simulation and Optimization of Piecewise Deterministic Markov Processes
Application to Reliability
E-Book
12/2015
Wiley-ISTE
€139.99
Available for download
Persons
Benoite de Saporta is Professor in Applied Probabilities at the University of Montpellier 2 in France.
Francois Dufour is Professor at the University of Bordeaux in France.
Huilong Zhang is a lecturer at INRIA in Bordeaux, France.
Francois Dufour is Professor at the University of Bordeaux in France.
Huilong Zhang is a lecturer at INRIA in Bordeaux, France.
Content
Preface ix
Introduction xi
Part 1. Piecewise Deterministic Markov Processes and Quantization 1
Chapter 1. Piecewise Deterministic Markov Processes 3
1.1. Introduction 3
1.2. Notation 4
1.3. Definition of a PDMP 5
1.4. Regularityassumptions 8
1.4.1. Lipschitz continuity along the flow 8
1.4.2. Regularity assumptions on the local characteristics 9
1.5. Time-augmentedprocess 11
1.6. EmbeddedMarkovchain 15
1.7. Stopping times 16
1.8. ExamplesofPDMPs 20
1.8.1. Poisson processwith trend 20
1.8.2. TCP 21
1.8.3. Air conditioningunit 22
1.8.4. Crack propagationmodel 23
1.8.5. Repairworkshopmodel 24
Chapter 2. Examples in Reliability 27
2.1. Introduction 27
2.2. Structure subject to corrosion 28
2.2.1. PDMPmodel 29
2.2.2. Deterministic time to reach the boundary 32
2.3. The heatedhold-uptank 33
2.3.1. Tank dynamics 34
2.3.2. PDMPmodel 36
Chapter 3. Quantization Technique 39
3.1. Introduction 39
3.2. Optimal quantization 40
3.2.1. Optimal quantization of a random variable 40
3.2.2. Optimal quantization of a Markovchain 42
3.3. SimulationofPDMPs 44
3.3.1. Simulation of time-dependent intensity 45
3.3.2. Simulation of trajectories 45
3.4. QuantizationofPDMPs 47
3.4.1. Scale of coordinates of the state variable 48
3.4.2. Cardinality of the mode variable 50
Part 2. Simulation of Functionals 53
Chapter 4. Expectation of Functionals 55
4.1. Introduction 55
4.2. Recursive formulation 57
4.2.1. Lipschitz continuity 58
4.2.2. Iterated operator 60
4.2.3. Approximationscheme 61
4.3. Lipschitz regularity 62
4.4. Rate of convergence 69
4.5. Time-dependent functionals 71
4.6. Deterministic time horizon 74
4.6.1. Direct estimation of the running cost term 74
4.6.2. Bounds of the boundary jump cost term 77
4.6.3. Bounds in the general case 79
4.7. Example 81
4.8. Conclusion 84
Chapter 5. Exit Time 87
5.1. Introduction 87
5.2. Problem setting 88
5.2.1. Distribution 90
5.2.2. Moments 91
5.2.3. Computationhorizon 92
5.3. Approximationschemes 92
5.4. Convergence 95
5.4.1. Distribution 95
5.4.2. Moments 100
5.5. Example 101
5.6. Conclusion 108
Chapter 6. Example in Reliability: Service Time 109
6.1. Mean thickness loss 109
6.2. Service time 112
6.2.1. Mean service time 114
6.2.2. Distribution of the service time 118
6.3. Conclusion 121
Part 3. Optimization 123
Chapter 7. Optimal Stopping 125
7.1. Introduction 125
7.2. Dynamic programming equation 128
7.3. Approximation of the value function 130
7.4. Lipschitz continuity properties 132
7.4.1. Lipschitz properties of J and K 132
7.4.2. Lipschitz properties of the value functions 135
7.5. Error estimation for the value function 138
7.5.1. Second term 140
7.5.2. Third term 141
7.5.3. Fourth term 147
7.5.4. Proof of theorem 7.1 148
7.6. Numerical construction of an [1]-optimal stopping time 149
7.7. Example 161
Chapter 8. Partially Observed Optimal Stopping Problem 165
8.1. Introduction 165
8.2. Problem formulation and assumptions 167
8.3. Optimal filtering 170
8.4. Dynamicprogramming 175
8.4.1. Preliminaryresults 176
8.4.2. Optimal stopping problem under complete observation 180
8.4.3. Dynamic programming equation 181
8.5. Numerical approximation by quantization 188
8.5.1. Lipschitz properties 189
8.5.2. Discretization scheme 195
8.5.3. Numerical construction of an [1]-optimal stopping time 205
8.6. Numerical example 211
Chapter 9. Example in Reliability: Maintenance Optimization 215
9.1. Introduction 215
9.2. Corrosionprocess 216
9.3. Air conditioningunit 219
9.4. The heatedhold-uptank 221
9.4.1. Problem setting and simulation 222
9.4.2. Numerical results and validation 224
9.5. Conclusion 228
Chapter 10. Optimal Impulse Control 231
10.1. Introduction 231
10.2. Impulse controlproblem233
10.3. Lipschitz-continuity properties 236
10.3.1. Lipschitz properties of the operators 236
10.3.2. Lipschitz properties of the operator L 239
10.4. Approximation of the value function 242
10.4.1. Time discretization 245
10.4.2. Approximation of the value functions on the control grid U 246
10.4.3. Approximation of the value function 255
10.4.4. Step-by-step description of the algorithm 259
10.4.5. Practical implementation 259
10.5. Example 262
10.6. Conclusion 264
Bibliography 269
Index 277
Introduction xi
Part 1. Piecewise Deterministic Markov Processes and Quantization 1
Chapter 1. Piecewise Deterministic Markov Processes 3
1.1. Introduction 3
1.2. Notation 4
1.3. Definition of a PDMP 5
1.4. Regularityassumptions 8
1.4.1. Lipschitz continuity along the flow 8
1.4.2. Regularity assumptions on the local characteristics 9
1.5. Time-augmentedprocess 11
1.6. EmbeddedMarkovchain 15
1.7. Stopping times 16
1.8. ExamplesofPDMPs 20
1.8.1. Poisson processwith trend 20
1.8.2. TCP 21
1.8.3. Air conditioningunit 22
1.8.4. Crack propagationmodel 23
1.8.5. Repairworkshopmodel 24
Chapter 2. Examples in Reliability 27
2.1. Introduction 27
2.2. Structure subject to corrosion 28
2.2.1. PDMPmodel 29
2.2.2. Deterministic time to reach the boundary 32
2.3. The heatedhold-uptank 33
2.3.1. Tank dynamics 34
2.3.2. PDMPmodel 36
Chapter 3. Quantization Technique 39
3.1. Introduction 39
3.2. Optimal quantization 40
3.2.1. Optimal quantization of a random variable 40
3.2.2. Optimal quantization of a Markovchain 42
3.3. SimulationofPDMPs 44
3.3.1. Simulation of time-dependent intensity 45
3.3.2. Simulation of trajectories 45
3.4. QuantizationofPDMPs 47
3.4.1. Scale of coordinates of the state variable 48
3.4.2. Cardinality of the mode variable 50
Part 2. Simulation of Functionals 53
Chapter 4. Expectation of Functionals 55
4.1. Introduction 55
4.2. Recursive formulation 57
4.2.1. Lipschitz continuity 58
4.2.2. Iterated operator 60
4.2.3. Approximationscheme 61
4.3. Lipschitz regularity 62
4.4. Rate of convergence 69
4.5. Time-dependent functionals 71
4.6. Deterministic time horizon 74
4.6.1. Direct estimation of the running cost term 74
4.6.2. Bounds of the boundary jump cost term 77
4.6.3. Bounds in the general case 79
4.7. Example 81
4.8. Conclusion 84
Chapter 5. Exit Time 87
5.1. Introduction 87
5.2. Problem setting 88
5.2.1. Distribution 90
5.2.2. Moments 91
5.2.3. Computationhorizon 92
5.3. Approximationschemes 92
5.4. Convergence 95
5.4.1. Distribution 95
5.4.2. Moments 100
5.5. Example 101
5.6. Conclusion 108
Chapter 6. Example in Reliability: Service Time 109
6.1. Mean thickness loss 109
6.2. Service time 112
6.2.1. Mean service time 114
6.2.2. Distribution of the service time 118
6.3. Conclusion 121
Part 3. Optimization 123
Chapter 7. Optimal Stopping 125
7.1. Introduction 125
7.2. Dynamic programming equation 128
7.3. Approximation of the value function 130
7.4. Lipschitz continuity properties 132
7.4.1. Lipschitz properties of J and K 132
7.4.2. Lipschitz properties of the value functions 135
7.5. Error estimation for the value function 138
7.5.1. Second term 140
7.5.2. Third term 141
7.5.3. Fourth term 147
7.5.4. Proof of theorem 7.1 148
7.6. Numerical construction of an [1]-optimal stopping time 149
7.7. Example 161
Chapter 8. Partially Observed Optimal Stopping Problem 165
8.1. Introduction 165
8.2. Problem formulation and assumptions 167
8.3. Optimal filtering 170
8.4. Dynamicprogramming 175
8.4.1. Preliminaryresults 176
8.4.2. Optimal stopping problem under complete observation 180
8.4.3. Dynamic programming equation 181
8.5. Numerical approximation by quantization 188
8.5.1. Lipschitz properties 189
8.5.2. Discretization scheme 195
8.5.3. Numerical construction of an [1]-optimal stopping time 205
8.6. Numerical example 211
Chapter 9. Example in Reliability: Maintenance Optimization 215
9.1. Introduction 215
9.2. Corrosionprocess 216
9.3. Air conditioningunit 219
9.4. The heatedhold-uptank 221
9.4.1. Problem setting and simulation 222
9.4.2. Numerical results and validation 224
9.5. Conclusion 228
Chapter 10. Optimal Impulse Control 231
10.1. Introduction 231
10.2. Impulse controlproblem233
10.3. Lipschitz-continuity properties 236
10.3.1. Lipschitz properties of the operators 236
10.3.2. Lipschitz properties of the operator L 239
10.4. Approximation of the value function 242
10.4.1. Time discretization 245
10.4.2. Approximation of the value functions on the control grid U 246
10.4.3. Approximation of the value function 255
10.4.4. Step-by-step description of the algorithm 259
10.4.5. Practical implementation 259
10.5. Example 262
10.6. Conclusion 264
Bibliography 269
Index 277