Course talk:CPSC522/Reinforcement Learning with Function approximation

Excellent page Mehrdad Ghomi,Jiahong Chen and Jordon Johnson. Neither full of a lot of convoluted details nor too simplistic, a perfect balance in my opinion. Use of external links was great. I would simply like to mention one point

• Is it possible to include conditions on the bounds on how to get a solution that is close to the optimal value function upto some finite error $\epsilon$

Hi Abed, Thanks for your feedback and positive comments, I believe you should be able to get your answer from the page after the modifications that were done to the page recently.

Hi Mehrdad Ghomi,

I am glad to read your page. I appreciate your work on this topic. I have a suggesion for you. Considering there is another page introducing Reinforcement Learning, you should place more emphasis on the part of function approximation and avoid too much overlapping content with Reinforcement Learning. Further, you should extend section Generalization with Multi-Layer Perceptrons (Neural Network) and compress other parts of your page. For example, you can explain how neural networks can be applied in reinforcement learning, and also talk about the strength and weeknesses of Reinforcement Learning with Fucntion approximation compared to Reinforcement Learning with a Look-up-table. Thanks

Best regards,

Ke Dai

Thanks Ke for your comments and feedback, I factored out a lot of the RL content.

Hi Mehrdad,

Nice work! Your page is very well written and informative. I have a few suggestions:

1. You page could focus more on the part of function approximation and avoid overlapping content since there is a separate page on RL.
2. You could talk a bit more about the relative merits of RL with Function approximation compared to RL with a Look-up-table.
3. The presentation of the page can be improved by using latex commands instead of using images for mathematical constructs.

Scores:

• [5] The topic is relevant for the course.
• [5] The writing is clear and the English is good.
• [5] The page is written at an appropriate level for CPSC 522 students (where the students have diverse backgrounds).
• [5] The formalism (definitions, mathematics) was well chosen to make the page easier to understand.
• [5] The abstract is a concise and clear summary.
• [4] There were appropriate (original) examples that helped make the topic clear.
• [3] There was appropriate use of (pseudo-) code.
• [5] It had a good coverage of representations, semantics, inference and learning (as appropriate for the topic).
• [5] It is correct.
• [4] It was neither too short nor too long for the topic.
• [5] It was an appropriate unit for a page (it shouldn't be split into different topics or merged with another page).
• [5] It links to appropriate other pages in the wiki.
• [5] The references and links to external pages are well chosen.
• [4] I would recommend this page to someone who wanted to find out about the topic.
• [4] This page should be highlighted as an exemplary page for others to emulate.

Thanks for your feedback Adnan, will I did add some Pseudo code after your comments.

Hi Mehrdad,

Such a well written and solid draft! Here are my scores and more details below that. Please let me know if you need any further clarification.
Scale of 1 to 5, where 1 = strongly disagree and 5 = strongly agree:

---

[5] The topic is relevant for the course.
[5] The writing is clear and the English is good.
[5] The page is written at an appropriate level for CPSC 522 students (where the students have diverse backgrounds).
[4] The formalism (definitions, mathematics) was well chosen to make the page easier to understand.
[5] The abstract is a concise and clear summary.
[3.5] There were appropriate (original) examples that helped make the topic clear.
[3] There was appropriate use of (pseudo-) code.
[4] It had a good coverage of representations, semantics, inference and learning (as appropriate for the topic).
[5] It is correct.
[4] It was neither too short nor too long for the topic.
[4] It was an appropriate unit for a page (it shouldn't be split into different topics or merged with another page).
[5] It links to appropriate other pages in the wiki.
[4] The references and links to external pages are well chosen.
[5] I would recommend this page to someone who wanted to find out about the topic.
[4] This page should be highlighted as an exemplary page for others to emulate.

Comments:

I enjoyed reading this page, however, I believe this page can use the following suggestions:

An Original Motivational Example

An original motivational example can be presented to summarize the learning materials, especially for this page that has required grounds for an example to be built upon.

Focus

As Adnan already suggested, there is a dedicated page for RL. Therefore, you might want to expand the function approximation part and pay less attention to RL itself.

Images quality

The quality of figures that are used is a bit low. You may want to find replacements with higher qualities or redraw them.

Captions

You may want to use captions for figures to briefly explain what they illustrate.

Mathematical Equation

I suggest you to replace the images of equations and expressions with Math tags, to let them following the policy of the stylesheet of the wiki.

Aside from above suggestions, you did a great job.




Sincerely,
Yaashaar Hadadian

Thanks for your fantastic feedback Yaashaar, will try to improve my page as we go.