Utilizing a real-world instance to clarify MDPs, the Bellman equation and worth iteration
Markov Determination Issues (MDP) are a central matter in robotics and AI as they’re the gateway to extra advanced matters corresponding to Reinforcement Studying and Partially Observable MDPs. Within the giant majority of on-line supplies, MDP are defined utilizing a “grid world” instance. In case you discover it troublesome to see the true world functions of this instance and in search of a extra classical MDP illustration, this text is for you! After motivating MDPs utilizing a robotic resolution making drawback, we’ll formally mannequin the MDP, introduce the Bellman equation and worth iteration, and supply a easy Python implementation.
Think about this video displaying a PR2 robotic getting a sandwich: