
Approximate Dynamic Programming
Solving the Curses of Dimensionality
Warren B. Powell(Author)
Wiley (Publisher)
1st Edition
Published on 16. October 2007
Book
Hardback
488 pages
978-0-470-17155-4 (ISBN)
Article exhausted; check for reprint
Description
A complete and accessible introduction to the real-world applications of approximate dynamic programming
With the growing levels of sophistication in modern-day operations, it is vital for practitioners to understand how to approach, model, and solve complex industrial problems. Approximate Dynamic Programming is a result of the author's decades of experience working in large industrial settings to develop practical and high-quality solutions to problems that involve making decisions in the presence of uncertainty. This groundbreaking book uniquely integrates four distinct disciplines-Markov design processes, mathematical programming, simulation, and statistics-to demonstrate how to successfully model and solve a wide range of real-life problems using the techniques of approximate dynamic programming (ADP). The reader is introduced to the three curses of dimensionality that impact complex problems and is also shown how the post-decision state variable allows for the use of classical algorithmic strategies from operations research to treat complex stochastic optimization problems.
Designed as an introduction and assuming no prior training in dynamic programming of any form, Approximate Dynamic Programming contains dozens of algorithms that are intended to serve as a starting point in the design of practical solutions for real problems. The book provides detailed coverage of implementation challenges including: modeling complex sequential decision processes under uncertainty, identifying robust policies, designing and estimating value function approximations, choosing effective stepsize rules, and resolving convergence issues.
With a focus on modeling and algorithms in conjunction with the language of mainstream operations research, artificial intelligence, and control theory, Approximate Dynamic Programming:
*
Models complex, high-dimensional problems in a natural and practical way, which draws on years of industrial projects
*
Introduces and emphasizes the power of estimating a value function around the post-decision state, allowing solution algorithms to be broken down into three fundamental steps: classical simulation, classical optimization, and classical statistics
*
Presents a thorough discussion of recursive estimation, including fundamental theory and a number of issues that arise in the development of practical algorithms
*
Offers a variety of methods for approximating dynamic programs that have appeared in previous literature, but that have never been presented in the coherent format of a book
Motivated by examples from modern-day operations research, Approximate Dynamic Programming is an accessible introduction to dynamic modeling and is also a valuable guide for the development of high-quality solutions to problems that exist in operations research and engineering. The clear and precise presentation of the material makes this an appropriate text for advanced undergraduate and beginning graduate courses, while also serving as a reference for researchers and practitioners. A companion Web site is available for readers, which includes additional exercises, solutions to exercises, and data sets to reinforce the book's main concepts.
With the growing levels of sophistication in modern-day operations, it is vital for practitioners to understand how to approach, model, and solve complex industrial problems. Approximate Dynamic Programming is a result of the author's decades of experience working in large industrial settings to develop practical and high-quality solutions to problems that involve making decisions in the presence of uncertainty. This groundbreaking book uniquely integrates four distinct disciplines-Markov design processes, mathematical programming, simulation, and statistics-to demonstrate how to successfully model and solve a wide range of real-life problems using the techniques of approximate dynamic programming (ADP). The reader is introduced to the three curses of dimensionality that impact complex problems and is also shown how the post-decision state variable allows for the use of classical algorithmic strategies from operations research to treat complex stochastic optimization problems.
Designed as an introduction and assuming no prior training in dynamic programming of any form, Approximate Dynamic Programming contains dozens of algorithms that are intended to serve as a starting point in the design of practical solutions for real problems. The book provides detailed coverage of implementation challenges including: modeling complex sequential decision processes under uncertainty, identifying robust policies, designing and estimating value function approximations, choosing effective stepsize rules, and resolving convergence issues.
With a focus on modeling and algorithms in conjunction with the language of mainstream operations research, artificial intelligence, and control theory, Approximate Dynamic Programming:
*
Models complex, high-dimensional problems in a natural and practical way, which draws on years of industrial projects
*
Introduces and emphasizes the power of estimating a value function around the post-decision state, allowing solution algorithms to be broken down into three fundamental steps: classical simulation, classical optimization, and classical statistics
*
Presents a thorough discussion of recursive estimation, including fundamental theory and a number of issues that arise in the development of practical algorithms
*
Offers a variety of methods for approximating dynamic programs that have appeared in previous literature, but that have never been presented in the coherent format of a book
Motivated by examples from modern-day operations research, Approximate Dynamic Programming is an accessible introduction to dynamic modeling and is also a valuable guide for the development of high-quality solutions to problems that exist in operations research and engineering. The clear and precise presentation of the material makes this an appropriate text for advanced undergraduate and beginning graduate courses, while also serving as a reference for researchers and practitioners. A companion Web site is available for readers, which includes additional exercises, solutions to exercises, and data sets to reinforce the book's main concepts.
Reviews / Votes
"Perhaps the most appealing aspect of Professor Powell's book is the fact that it spans both theory and practice...Problems, deemed intractable a few years ago, are now easily solved by using the exhibited techniques in this book. I would strongly recommend the book to any practitioner facing complex, dynamic models involving constantly changing information streams." (IIE Transactions-Operations Engineering, 2008) "Focus[es] on the core ... of dynamic programming with a simple and clear exposition of the material ... while ... elevating the standard of the theory."(Computing Reviews, 2008) "Motivated by examples from modern-day operations research, Approximate Dynamic Programming is an accessible introduction to dynamic modeling and is also a valuable guide for the development of high-quality solutions to problems that exist in operations research and engineering. (Mathematical Reviews, 2008)More details
Series
Edition
1., Auflage
Language
English
Place of publication
Hoboken
United Kingdom
Publishing group
John Wiley and Sons Ltd
Target group
Professional and scholarly
Illustrations
Illustrations
Dimensions
Height: 25.6 cm
Width: 18.2 cm
Thickness: 29 mm
Weight
964 gr
ISBN-13
978-0-470-17155-4 (9780470171554)
Schweitzer Classification
Other editions
New editions

Book
11/2011
2nd Edition
Wiley
€144.50
Shipment within 10-20 days
Person
Warren B. Powell, PhD, is Professor of Operations Research and Financial Engineering at Princeton University, where he is founder and Director of CASTLE Laboratory, a research unit that works with industrial partners to test new ideas found in operations research. The recipient of the 2004 INFORMS Fellow Award, Dr. Powell has authored over 100 refereed publications on stochastic optimization, approximate dynamic programming, and dynamic resource management.
Content
Preface.
Acknowledgments.
1. The challenges of dynamic programming.
2. Some illustrative models.
Problems.
3. Introduction to Markov decision processes.
Problems
4. Introduction to approximate dynamic programming.
Problems.
5. Modeling dynamic programs.
Problems.
6. Stochastic approximation methods.
Problems.
7. Approximating value functions.
Problems.
8. ADP for finite horizon problems.
Problems.
9. Infinite horizon problems.
Problems.
10. Exploration vs. exploitation.
Problems.
11. Value function approximations for special functions.
Problems.
12. Dynamic resource allocation.
Problems.
13. Implementation challenges.
04701717663ENIntroduction.
Round One. Understand the cost of hiring recklessly.
Round Two. Become a proactive recruiter.
Round Three. Create a potent interview structure.
Round Four. Ask tough interview questions.
Round Five. Effectively check references.
Round Six. How to use predictive testing hiring profiles.
Round Seven. How to get new hires off to a great start.
Round Eight. How to retain eagles once you find them.
Round Nine. Differentiate your employees for development and retention.
Round Ten. Knockout summary & follow through.
Acknowledgments.
About the Author.
Acknowledgments.
1. The challenges of dynamic programming.
2. Some illustrative models.
Problems.
3. Introduction to Markov decision processes.
Problems
4. Introduction to approximate dynamic programming.
Problems.
5. Modeling dynamic programs.
Problems.
6. Stochastic approximation methods.
Problems.
7. Approximating value functions.
Problems.
8. ADP for finite horizon problems.
Problems.
9. Infinite horizon problems.
Problems.
10. Exploration vs. exploitation.
Problems.
11. Value function approximations for special functions.
Problems.
12. Dynamic resource allocation.
Problems.
13. Implementation challenges.
04701717663ENIntroduction.
Round One. Understand the cost of hiring recklessly.
Round Two. Become a proactive recruiter.
Round Three. Create a potent interview structure.
Round Four. Ask tough interview questions.
Round Five. Effectively check references.
Round Six. How to use predictive testing hiring profiles.
Round Seven. How to get new hires off to a great start.
Round Eight. How to retain eagles once you find them.
Round Nine. Differentiate your employees for development and retention.
Round Ten. Knockout summary & follow through.
Acknowledgments.
About the Author.