Deterministic Decision Processes

This chapter introduces optimal planning in deterministic sequential decision problems. The deterministic dynamic system framework is presented with state transitions, encompassing both finite discrete models and continuous systems. Two fundamental problem settings are covered: finite-horizon decision problems solved via backward dynamic programming, and average cost criteria. Classic graph-theoretic algorithms are developed including Bellman–Ford, Dijkstra’s algorithm and A* search. The chapter extends to continuous optimal control, presenting the linear quadratic regulator (LQR) and iterative LQR.