Preface to the Planning Chapters

This preface chapter establishes the philosophical and modeling foundations for the planning section of the book. It motivates the MDP framework through concrete examples including board games, robot control and inventory management. The chapter articulates the key modeling assumptions underlying MDPs: discrete regular time, finite action and state spaces, single-currency rewards, known decision horizons, and Markov state evolution. The discussion addresses different types of uncertainty in decision making – aleatoric (inherent randomness), epistemic (parameter uncertainty) and partial observability. The chapter justifies the MDP model’s simplicity and generality.