The Bellman equation characterizes the optimal policy, and solution methods (value iteration, policy iteration, linear programming) provide constructive algorithms. Partially Observable MDPs -- The ...