Markov Decision Processes
Discrete Stochastic Dynamic Programming
Published on 4. November 2009
672 pages
978-0-470-31772-3 (ISBN)
Description
The Wiley-Interscience Paperback Series consists of selected booksthat have been made more accessible to consumers in an effort toincrease global appeal and general circulation. With these newunabridged softcover volumes, Wiley hopes to extend the lives ofthese works by making them available to future generations ofstatisticians, mathematicians, and scientists.
"This text is unique in bringing together so many resultshitherto found only in part in other texts and papers. . . . Thetext is fairly self-contained, inclusive of some basic mathematicalresults needed, and provides a rich diet of examples, applications,and exercises. The bibliographical material at the end of eachchapter is excellent, not only from a historical perspective, butbecause it is valuable for researchers in acquiring a goodperspective of the MDP research potential."
--Zentralblatt fur Mathematik
". . . it is of great value to advanced-level students,researchers, and professional practitioners of this field to havenow a complete volume (with more than 600 pages) devoted to thistopic. . . . Markov Decision Processes: Discrete Stochastic DynamicProgramming represents an up-to-date, unified, and rigoroustreatment of theoretical and computational aspects of discrete-timeMarkov decision processes."
--Journal of the American Statistical Association
More details
Other editions
Additional editions
Martin L. Puterman
Markov Decision Processes - Discrete Stochastic Dynamic Programming
Software
05/2008
Wiley
€214.69
No shipping information available
Book
02/2005
Wiley
€160.50
Shipment within 10-20 days
Book
04/1994
Wiley
€107.09
Article exhausted; check different version
Person
Martin L. Puterman, PhD, is Advisory Board Professor of Operations and Director of the Centre for Operations Excellence at The University of British Columbia in Vancouver, Canada.
Content
Preface.
1. Introduction.
2. Model Formulation.
3. Examples.
4. Finite-Horizon Markov Decision Processes.
5. Infinite-Horizon Models: Foundations.
6. Discounted Markov Decision Problems.
7. The Expected Total-Reward. Criterion.
8. Average Reward and Related Criteria.
9. The Average Reward Criterion-Multichain and CommunicatingModels.
10. Sensitive Discount Optimality.
11. Continuous-Time Models.
Afterword.
Notation.
Appendix A. Markov Chains.
Appendix B. Semicontinuous Functions.
Appendix C. Normed Linear Spaces.
Appendix D. Linear Programming.
Bibliography.
Index.
