Course talk:CPSC522/NormalizingFlows

Thank you Mehar and Harshinee for your feedback on my post. To the best of my knowledge I've addressed both of your concerns and added a section on applications which talks about how normalizing flows are used and what the current state of research in the area is.

The article reads well and is easy to grasp.

Some clarifications that would be helpful

- It wasn't fully clear as to why a 1-to-1 mapping is required in a normalizing flow

- How does the composition of more straightforward transformations improve complexity? Do they use non-linearity to improve complexity?

- Is s_\theta a neural network in RealNVP?

Suggestions:

- Can you add some images from Flow-based models to give some motivation?

- Can you give some cons of this method and why it's not preferred over say something like Diffusion-based models?

Some minor language errors,

- "generally known, model it by transforming samples from a source distribution"

- "Such a transformation must b.."

Overall, an excellent article - probably one of the best ways someone could have explained Normalizing Flows. Every equation is well-introduced beforehand and the gradual steps are easy to understand. Great job! :)

Critique
1) "As a result, the authors of this method train their flow using the reverse KL." It would be great if you can explain how reverse KL is different from forward KL (and how that helps with transformations that aren't invertible).
2) Although very well written, overall, the article feels a little short. Would it be possible to include some further details (i.e. pros and cons of normalizing flows, why choose normalizing flows over other generative models, some applications of normalizing flows etc.)?

Minor edits
1) Since the analytic form of $p(\mathbf {x} )$ is not generally known, *it *is *modelled by transforming samples from a source distribution $p(\mathbf {z} )$ through a transformation $T$, *which *is generally parameterized with some parameters $\theta$.

2) Such a transformation must be complex enough to model the data distribution. [This sentence has been accidentally split across two lines].
3) To *achieve this complexity in the finite normalizing flow paradigm, the overall transformation...
4) Then, we define the transformation of each partition *separately...
5) *Notably, unlike the finite normalizing flows, computing...
6) Training continuous normalizing flows is also challenging, as it requires *backpropagating through the ODE solver.