Course talk:CPSC522/Partially Observable Markov Decision Processes

I found this paper concise and well structured; you made good use of figures throughout the page. I think it would be helpful to discuss the limitations of POMDP in the applications you listed at the end of the page. (5) The topic is relevant for the course.

(5) The writing is clear and the English is good.

(5) The page is written at an appropriate level for CPSC 522 students (where the students have diverse backgrounds).

(4) The formalism (definitions, mathematics) was well chosen to make the page easier to understand.

(5) The abstract is a concise and clear summary.

(5) There were appropriate (original) examples that helped make the topic clear.

(5) There was appropriate use of (pseudo-) code.

(5) It had a good coverage of representations, semantics, inference and learning (as appropriate for the topic).

(5) It is correct.

(5) It was neither too short nor too long for the topic

(4) It was an appropriate unit for a page (it shouldn't be split into different topics or merged with another page).

(5) It links to appropriate other pages in the wiki.

(4) The references and links to external pages are well chosen.

(5) I would recommend this page to someone who wanted to find out about the topic.

(4) This page should be highlighted as an exemplary page for others to emulate.

If I was grading it out of 20, I would give it: 19/20

Good structure and initial part. Good conclusion section. I like the examples of applications in the end. I find the explanation with the table helpful to get the background, although it can be not clear to a non-expert reader since it talks about topics that are given for granted. The part from “Formal Definition” to “Belief MDP” could benefit from some rewriting to make the English more easy to follow. I would make the notation in the section “Belief MDP” more clear, as there are overlappings with the notation in the previous section. In particular, “b” is used in both section, but I think it refers to different things; I think it would be more clear if “b” was openly defined in the second section. There is a typo “oroginal POMDP”.

On a scale of 1 to 5, where 1 means "strongly disagree" and 5 means "strongly agree" please rate and comment on the following: • The topic is relevant for the course: 5 • The writing is clear and the English is good: 3 • The page is written at an appropriate level for CPSC 522 students (where the students have diverse backgrounds): 5 • The formalism (definitions, mathematics) was well chosen to make the page easier to understand: 3 • The abstract is a concise and clear summary: 5 • There were appropriate (original) examples that helped make the topic clear: 5 • There was appropriate use of (pseudo-) code: 5 • It had a good coverage of representations, semantics, inference and learning (as appropriate for the topic): 5 • It is correct: 5 • It was neither too short nor too long for the topic: 5 • It was an appropriate unit for a page (it shouldn't be split into different topics or merged with another page): 5 • It links to appropriate other pages in the wiki: 5 • The references and links to external pages are well chosen: 5 • I would recommend this page to someone who wanted to find out about the topic: 4 • This page should be highlighted as an exemplary page for others to emulate: 4 If I was grading it out of 20, I would give it: 17.5

Why is your figure a POMDP? Isn't it just an MDP? Perhaps add the P(o|s) somehow. More examples would make this page much more readable.

"T is a set of conditional transition probabilities between states" Is not accurate. T specifies a conditional probability of the next state given the preceding state and action. It is less confusing to be more accurate sometimes. (Can you add a space around your math -- the lack of spaces in the text is distracting?)

In the belief MDP, the most important part -- the definition of "belief state" is missing. A belief state is a function from S -> [0,1] that sums to 1.

Mention that belief update is exactly what happens in an HMM.

It might be useful to mention that a belief state is not part of the defintion of a POMDP. [ Indeed it is possible (and commom) to solve a POMDP without using a belief state.]

I am not sure that the Games is appropriate. Multiple agents with partial observability is much more complex. Is the Hansen reference relavant as it is about games.

Your bibliography doesn't include any papers about POMDPs in general. Where did you get this formulation from?

Also "markov" should be "Markov" as it is someone's name.