Feedback

This feedback will be brief as it still isn't finished.

You need to spend more of the space describing the actual contribution of the second paper over the first (which was the point of the assignment). The rest of the paper is giving the context to understand this. I think a lot of the other details can be omitted (which will save you time). For example; does the paper use stochastic or deterministic policies? Describe the one used; you are not expected to give a survey of all alternatives.

Don't expect the reader to understand terms like "generic context space". Perhaps just have one running example, and refer to that example when defining terms.