Continual Learning Challenges

Updating model parameters with new data often leads to catastrophic forgetting that requires architectural tweaks and better optimization rules to retain proficiency in old tasks.