Markov Chains and Stochastic Stability

I Communication and Regeneration.- 1 Heuristics.- 1.1 A Range of Markovian Environments.- 1.2 Basic Models in Practice.- 1.3 Stochastic Stability For Markov Models.- 1.4 Commentary.- 2 Markov Models.- 2.1 Markov Models in Time Series.- 2.2 Nonlinear State Space Models.- 2.3 Models in Control And Systems Theory.- 2.4 Markov Models with Regeneration Times.- 2.5 Commentary.- 3 Transition Probabilities.- 3.1 Defining a Markovian Process.- 3.2 Foundations on a Countable Space.- 3.3 Specific Transition Matrices.- 3.4 Foundations for General State Space Chains.- 3.5 Building Transition Kernels for Specific Models.- 3.6 Commentary.- 4 Irreducibility.- 4.1 Communication and Irreducibility: Countable Spaces.- 4.2 ?-Irreducibility.- 4.3 ?-Irreducibility for Random Walk Models.- 4.4 ?-Irreducible Linear Models.- 4.5 Commentary.- 5 Pseudo-atoms.- 5.1 Splitting ?-Irreducible Chains.- 5.2 Small Sets.- 5.3 Small Sets for Specific Models.- 5.4 Cyclic Behavior.- 5.5 Petite Sets and Sampled Chains.- 5.6 Commentary.- 6 Topology and Continuity.- 6.1 Feller Properties and Forms of Stability.- 6.2 T-chains.- 6.3 Continuous Components for Specific Models.- 6.4 e-Chains.- 6.5 Commentary.- 7 The Nonlinear State Space Model.- 7.1 Forward Accessibility and Continuous Components.- 7.2 Minimal Sets and Irreducibility.- 7.3 Periodicity for nonlinear state space models.- 7.4 Forward Accessible Examples.- 7.5 Equicontinuity and the nonlinear state space model.- 7.6 Commentary.- II Stability Structures.- 8 Transience and Recurrence.- 8.1 Classifying chains on countable spaces.- 8.2 Classifying ?-irreducible chains.- 8.3 Recurrence and transience relationships.- 8.4 Classification using drift criteria.- 8.5 Classifying random walk on IR+.- 8.6 Commentary.- 9 Harris and Topological Recurrence.- 9.1 Harris recurrence.- 9.2 Non-evanescent and recurrent chains.- 9.3 Topologically recurrent and transient states.- 9.4 Criteria for stability on a topological space.- 9.5 Stochastic comparison and increment analysis.- 9.6 Commentary.- 10 The Existence of ?.- 10.1 Stationarity and Invariance.- 10.2 The existence of ?: chains with atoms.- 10.3 Invariant measures: countable space models.- 10.4 The existence of ?: ?-irreducible chains.- 10.5 Invariant Measures: General Models.- 10.6 Commentary.- 11 Drift and Regularity.- 11.1 Regular chains.- 11.2 Drift, hitting times and deterministic models.- 11.3 Drift criteria for regularity.- 11.4 Using the regularity criteria.- 11.5 Evaluating non-positivity.- 11.6 Commentary.- 12 Invariance and Tightness.- 12.1 Chains bounded in probability.- 12.2 Generalized sampling and invariant measures.- 12.3 The existence of A ?-finite invariant measure.- 12.4 Invariant Measures for e-Chains..- 12.5 Establishing boundedness in probability.- 12.6 Commentary.- III Convergence.- 13 Ergodicity.- 13.1 Ergodic chains on countable spaces.- 13.2 Renewal and regeneration.- 13.3 Ergodicity of positive Harris chains.- 13.4 Sums of transition probabilities.- 13.5 Commentary.- 14 ƒ-Ergodicity and ƒ-Regularity.- 14.1 ƒ-Properties: chains with atoms.- 14.2 ƒ-Regularity and drift.- 14.3 ƒ-Ergodicity for general chains.- 14.4 ƒ-Ergodicity of specific models.- 14.5 A Key Renewal Theorem.- 14.6 Commentary.- 15 Geometric Ergodicity.- 15.1 Geometric properties: chains with atoms.- 15.2 Kendall sets and drift criteria.- 15.3 ƒ-Geometric regularity of ?and ?n.- 15.4 ƒ-Geometric ergodicity for general chains.- 15.5 Simple random walk and linear models.- 15.6 Commentary.- 16 V-Uniform Ergodicity.- 16.1 Operator norm convergence.- 16.2 Uniform ergodicity.- 16.3 Geometric ergodicity and increment analysis.- 16.4 Models from queueing theory.- 16.5 Autoregressive and state space models.- 16.6 Commentary.- 17 Sample Paths and Limit Theorems.- 17.1 Invariant ?-Fields and the LLN.- 17.2 Ergodic Theorems for Chains Possessing an Atom.- 17.3 General Harris Chains.- 17.4 The Functional CLT.- 17.5 Criteria for the CLT and the LIL.- 17.6 Applications.- 17.7 Commentary.- 18 Positivity.- 18.1 Null recurrent chains.- 18.2 Characterizing positivity using Pn.- 18.3 Positivity and T-chains.- 18.4 Positivity and e-Chains.- 18.5 The LLN for e-Chains.- 18.6 Commentary.- 19 Generalized Classification Criteria.- 19.1 State-dependent drifts.- 19.2 History-dependent drift criteria.- 19.3 Mixed drift conditions.- 19.4 Commentary.- IV Appendices.- A Mud Maps.- A.l Recurrence versus transience.- A.2 Positivity versus nullity.- A.3 Convergence Properties.- B Testing for Stability.- B.l A Glossary of Drift Conditions.- B.2 The scalar SETAR Model: a complete classification.- C A Glossary of Model Assumptions..- C.l Regenerative Models.- C.2 State Space Models.- D Some Mathematical Background.- D.l Some Measure Theory.- D.2 Some Probability Theory.- D.3 Some Topology.- D.4 Some Real Analysis.- D.5 Some Convergence Concepts for Measures.- D.6 Some Martingale Theory.- D.7 Some Results on Sequences and Numbers.- References.- Symbols Index.