Feedback

I think you need to be much clearer about what the problem is. You know the problem is that we want to learn about multiple tasks, where we want to both not forgot how to solve earlier tasks and want to use what is learned in previous tasks for subsequent tasks. (Is this correct). But you don't tell us; we have to guess.

You need to set up the definitions clearer. Eg., "However, most ANN’s fail to learn a new task without becoming inadequate in the original task". You should add something like "when learning multiple tasks sequentially"

In your defintion of "Continual learning" are you assuming that the learning knows when the task is changing? Do you really mean "at each time step a system (eg. a neural network) receives a new sample from a different task"? This seems to imply that the task changes for each sample. Is that correct? surely then X_t and y_t should be multiple pairs, not one pair. Why do you use {} for pairs, or do you mean sets? I think the setup needs to be explained more clearly in straightforward language.

I found the math impenetrable and unhelpful. Notation should only ever be introduced when it makes the description simpler.

Do progressive neural networks - the topic of this page - degrade performance of previous tasks? (I'm guessing not, but that assumes that we know what the task is, so you should be more explicit about this). If not, it seems like the introduction of the slack variables is an irrelevant detail. You could just say that other methods degrade the performance.

It might be useful in the progressive NN section to say what happend with more than 2 tasks. The formulation looks weird to me. Why would we sum the outputs of the previous layers -- which I think you are doing -- rather that letting the number of inputs nodes grow linearly for the n-th task. Maybe you need to define U notation better.

"Please note that the notations follow the notations from [3]." Note that this shouldn't mean that can get away without defining the notation. (I can't tell whether you intend that). It would be simpler to say "The notation follows [3]."