Feedback

Why is your figure a POMDP? Isn't it just an MDP? Perhaps add the P(o|s) somehow. More examples would make this page much more readable.

"T is a set of conditional transition probabilities between states" Is not accurate. T specifies a conditional probability of the next state given the preceding state and action. It is less confusing to be more accurate sometimes. (Can you add a space around your math -- the lack of spaces in the text is distracting?)

In the belief MDP, the most important part -- the definition of "belief state" is missing. A belief state is a function from S -> [0,1] that sums to 1.

Mention that belief update is exactly what happens in an HMM.

It might be useful to mention that a belief state is not part of the defintion of a POMDP. [ Indeed it is possible (and commom) to solve a POMDP without using a belief state.]

I am not sure that the Games is appropriate. Multiple agents with partial observability is much more complex. Is the Hansen reference relavant as it is about games.

Your bibliography doesn't include any papers about POMDPs in general. Where did you get this formulation from?

Also "markov" should be "Markov" as it is someone's name.