Course talk:CPSC522/Character Level Language Models using LSTM

Critique from Gudbrand:

A really good article. Easy to follow, nice structure, clear content, etc. Well done. No big objections to anything in particular, here are some small things I noticed:

- Perplexity is first used and then defined. Might be OK since it's just a small mention. - LSTM cell equations are misaligned. Also there is some mismatch between super- and subscripts here. - You write the "gradient vanishing problem". I thought it more common to say the "vanishing gradient problem" - Since the words "eventful, eventfully", etc are meant to be meta, perhaps use italics or quotation marks? Not sure what is correct. - Somewhere in the text you write "matrix character embeddings", I think you're missing an "of"? - When you write j \in i, f, o g, it would be nice to see the set of indices wrapped up in curly brackets. - The PTB corpus isn't really defined. Perhaps a sentence about the dataset? - I would like to see just a few more blue links for easy reference of undefined concepts, e.g. edit distance.

One last thing; it isn't really clear that you're presenting two particular papers here. Perhaps be more explicit about this. Sometimes you write "our ..." as if you're actually presenting your own material. Perhaps change wording there.. Anyway, I enjoyed reading your article, I certainly have a better overview of ML for NLP after reading it.

The topic is relevant for the course. 5 The writing is clear and the English is good. 5 The page is written at an appropriate level for CPSC 522 students (where the students have diverse backgrounds). 5 The formalism (definitions, mathematics) was well chosen to make the page easier to understand. 5 The abstract is a concise and clear summary. 5 There were appropriate (original) examples that helped make the topic clear. 5 There was appropriate use of (pseudo-) code. - It had a good coverage of representations, semantics, inference and learning (as appropriate for the topic). 5 It is correct. 5 It was neither too short nor too long for the topic. 5 It was an appropriate unit for a page (it shouldn't be split into different topics or merged with another page). 5 It links to appropriate other pages in the wiki. 5 The references and links to external pages are well chosen. 5 I would recommend this page to someone who wanted to find out about the topic. 5 This page should be highlighted as an exemplary page for others to emulate. 5 If I was grading it out of 20, I would give it: 19

Will address the above-mentioned issues. Thanks :)

Good work on the article! It was well articulated and i like the use of images. Some things concerning from the article were:

• In "Long-Short-Term-Memory" section, while you have written the equations, but there is not sufficient explanation accompanying the equations and meaning of $h_{t}-1$ is missing.
• May be there is some confusion due to structure of the page, but i didn't get that how LSTM, Recurrent Neural Network Language model, Character-level Convolutional Neural Network and Highway Network are related to each other.
• Also, from reading the article its difficult to make out the boundary between the two papers who are presenting. I think you need to change your use of words to reflect that you are explaining some else's paper.

I a scale of 1 to 5, where 1 means "strongly disagree" and 5 means "strongly agree" please rate and comment on the following:

• The topic is relevant for the course. -5
• The writing is clear and the English is good. -5
• The page is written at an appropriate level for CPSC 522 students (where the students have diverse backgrounds). -5
• The formalism (definitions, mathematics) was well chosen to make the page easier to understand. -4
• The abstract is a concise and clear summary. -5
• There were appropriate (original) examples that helped make the topic clear. -4
• There was appropriate use of (pseudo-) code. -4
• It had a good coverage of representations, semantics, inference and learning (as appropriate for the topic). -4
• It is correct. -5
• It was neither too short nor too long for the topic. -5
• It was an appropriate unit for a page (it shouldn't be split into different topics or merged with another page). -5
• It links to appropriate other pages in the wiki. -5
• The references and links to external pages are well chosen. -5
• I would recommend this page to someone who wanted to find out about the topic. -4
• This page should be highlighted as an exemplary page for others to emulate. -4

If I was grading it out of 20, I would give it: 16

Will add a paragraph connecting the dots. Thanks for the feedback.

• The topic is relevant for the course. 5
• The writing is clear and the English is good. 5
• The page is written at an appropriate level for CPSC 522 students (where the students have diverse backgrounds). 5
• The formalism (definitions, mathematics) was well chosen to make the page easier to understand.5
• The abstract is a concise and clear summary. 3
• There were appropriate (original) examples that helped make the topic clear. 4
• There was appropriate use of (pseudo-) code. 5
• It had a good coverage of representations, semantics, inference and learning (as appropriate for the topic). 5
• It is correct. 5
• It was neither too short nor too long for the topic. 5
• It was an appropriate unit for a page (it shouldn't be split into different topics or merged with another page). 5
• It links to appropriate other pages in the wiki. 5
• The references and links to external pages are well chosen. 5
• I would recommend this page to someone who wanted to find out about the topic.4
• This page should be highlighted as an exemplary page for others to emulate.4
• If I was grading it out of 20, I would give it: 19

Well written, the figures and equations are meaningful and well used. A couple comments: The one sentence summery should be expanded Each figure caption should reference which paper it came from I think you should end add a paragraph explicitly comparing the papers and talking about the contributions of the second paper, I'm not sure if the relation is clear (expect that the both address the same problem)

Thank you for your suggestions. Will address the issues you mentioned.