Course talk:CPSC522/Bayesian Coresets

Clearly written and well organized. The scope is somewhat limited for the "Construction" subsections but I think it's still appropriate.

This page is very well written and organized. The way you have summarized the many points in coresets and Bayesian coresets is very great.

As David has mentioned, I think that any example of how finding a coreset is done and how good the resulting approximation is and how hard it is to achieve such approximation (both computationally and methodologically) would help the reader. I do not mean that you should add details about exact algorithms and bounds, but mentioning some examples in only the first few sections would definitely help the reader to follow those sections more easily and better understand the final sections. Right now, I understand what the goal of coresets are, but I do not know how they actually do that.

This is good and well written, but it needs more insight (for the intended audience). I suggest you add:

• Some hints at the theorems. For example, in the definition of coresets, you talk about "for any C of appropriate size" (why isn't it P?), and then it would be good to say what size is needed. (Can we just select some at random?) You give us the definitions of Bayesian coresets, but then don't tell us what is known about the approximation (are there known bounds?) It would be good to have some details about what is known (for at least one case), instead of teasing us with "several convergence results exist" - perhaps give us the simplest one. It would be nice to have both theoretical and empirical results (or say why some are missing).
• Some case(s) in more detail. It is usually easiest for a reader to understand a theoretical topic if they have one example in detail. People are good at generalizing from examples (even single examples). I suggest that you choose one of the constriction examples, and do it in more detail (tell us how the examples are chosen, and what bounds are known in theory and in practice). Then say how the other methods relate.

I'd like to see some hints about the relevance for continual learning. Note that "retaining past experiences" is ambiguous; indeed the standard solution used in reinforcement learning is to keep retraining on past experiences that have been stored. But I don't think you mean that. But that seems to be more related to Bayesian coresets than what you gave (I think). Are these proposals theoretical (with proofs) or empirical (with experimental results) or just proposals. It would be nice to know. I'm thinking about a student who knows that you talked about catastrophic forgetting, and wants some idea of what is known.

Minor comments:

• I kept reading it as "co-resets" not "core-sets". I'm not sure you can do anything about that
• You need a space after many of your equations
• I was trying to work out what \overline means in "streaming and parallel computation", but then realized it was a name constructor.

I don't think I'm suggesting a big change, essentially just one case in more detail.