Numerical Methods for Simulation and Optimization of Piecewise Deterministic Markov Processes
Description
More details
Other editions
Additional editions
Persons
Benoîte de Saporta is Professor in Applied Probabilities at the University of Montpellier 2 in France.
François Dufour is Professor at the University of Bordeaux in France.
Huilong Zhang is a lecturer at INRIA in Bordeaux, France.
Content
Preface ix
Introduction xi
Part 1. Piecewise Deterministic Markov Processes and Quantization 1
Chapter 1. Piecewise Deterministic Markov Processes 3
1.1. Introduction 3
1.2. Notation 4
1.3. Definition of a PDMP 5
1.4. Regularityassumptions 8
1.4.1. Lipschitz continuity along the flow 8
1.4.2. Regularity assumptions on the local characteristics 9
1.5. Time-augmentedprocess 11
1.6. EmbeddedMarkovchain 15
1.7. Stopping times 16
1.8. ExamplesofPDMPs 20
1.8.1. Poisson processwith trend 20
1.8.2. TCP 21
1.8.3. Air conditioningunit 22
1.8.4. Crack propagationmodel 23
1.8.5. Repairworkshopmodel 24
Chapter 2. Examples in Reliability 27
2.1. Introduction 27
2.2. Structure subject to corrosion 28
2.2.1. PDMPmodel 29
2.2.2. Deterministic time to reach the boundary 32
2.3. The heatedhold-uptank 33
2.3.1. Tank dynamics 34
2.3.2. PDMPmodel 36
Chapter 3. Quantization Technique 39
3.1. Introduction 39
3.2. Optimal quantization 40
3.2.1. Optimal quantization of a random variable 40
3.2.2. Optimal quantization of a Markovchain 42
3.3. SimulationofPDMPs 44
3.3.1. Simulation of time-dependent intensity 45
3.3.2. Simulation of trajectories 45
3.4. QuantizationofPDMPs 47
3.4.1. Scale of coordinates of the state variable 48
3.4.2. Cardinality of the mode variable 50
Part 2. Simulation of Functionals 53
Chapter 4. Expectation of Functionals 55
4.1. Introduction 55
4.2. Recursive formulation 57
4.2.1. Lipschitz continuity 58
4.2.2. Iterated operator 60
4.2.3. Approximationscheme 61
4.3. Lipschitz regularity 62
4.4. Rate of convergence 69
4.5. Time-dependent functionals 71
4.6. Deterministic time horizon 74
4.6.1. Direct estimation of the running cost term 74
4.6.2. Bounds of the boundary jump cost term 77
4.6.3. Bounds in the general case 79
4.7. Example 81
4.8. Conclusion 84
Chapter 5. Exit Time 87
5.1. Introduction 87
5.2. Problem setting 88
5.2.1. Distribution 90
5.2.2. Moments 91
5.2.3. Computationhorizon 92
5.3. Approximationschemes 92
5.4. Convergence 95
5.4.1. Distribution 95
5.4.2. Moments 100
5.5. Example 101
5.6. Conclusion 108
Chapter 6. Example in Reliability: Service Time 109
6.1. Mean thickness loss 109
6.2. Service time 112
6.2.1. Mean service time 114
6.2.2. Distribution of the service time 118
6.3. Conclusion 121
Part 3. Optimization 123
Chapter 7. Optimal Stopping 125
7.1. Introduction 125
7.2. Dynamic programming equation 128
7.3. Approximation of the value function 130
7.4. Lipschitz continuity properties 132
7.4.1. Lipschitz properties of J and K 132
7.4.2. Lipschitz properties of the value functions 135
7.5. Error estimation for the value function 138
7.5.1. Second term 140
7.5.2. Third term 141
7.5.3. Fourth term 147
7.5.4. Proof of theorem 7.1 148
7.6. Numerical construction of an [1]-optimal stopping time 149
7.7. Example 161
Chapter 8. Partially Observed Optimal Stopping Problem 165
8.1. Introduction 165
8.2. Problem formulation and assumptions 167
8.3. Optimal filtering 170
8.4. Dynamicprogramming 175
8.4.1. Preliminaryresults 176
8.4.2. Optimal stopping problem under complete observation 180
8.4.3. Dynamic programming equation 181
8.5. Numerical approximation by quantization 188
8.5.1. Lipschitz properties 189
8.5.2. Discretization scheme 195
8.5.3. Numerical construction of an [1]-optimal stopping time 205
8.6. Numerical example 211
Chapter 9. Example in Reliability: Maintenance Optimization 215
9.1. Introduction 215
9.2. Corrosionprocess 216
9.3. Air conditioningunit 219
9.4. The heatedhold-uptank 221
9.4.1. Problem setting and simulation 222
9.4.2. Numerical results and validation 224
9.5. Conclusion 228
Chapter 10. Optimal Impulse Control 231
10.1. Introduction 231
10.2. Impulse controlproblem233
10.3. Lipschitz-continuity properties 236
10.3.1. Lipschitz properties of the operators 236
10.3.2. Lipschitz properties of the operator L 239
10.4. Approximation of the value function 242
10.4.1. Time discretization 245
10.4.2. Approximation of the value functions on the control grid U 246
10.4.3. Approximation of the value function 255
10.4.4. Step-by-step description of the algorithm 259
10.4.5. Practical implementation 259
10.5. Example 262
10.6. Conclusion 264
Bibliography 269
Index 277
