[Continual Learning] Replay-based Methods (Memory Buffer)

Herding: Distance from each class sample to the mean sample of class

 

Mean of feature, Histogram: Distance from each class sample to the mean of class

 

 

 

 

Offline Replay-based Continual Learning Methods

 

[1] iCaRL: Incremental Classifier and Representation Learning (iCaRL)

• CVPR 2017
• Mean of feature-based sample selection

 

[2] Gradient Episodic Memory for Continual Learning (GEM)

• NeurIPS 2017
• Last m samples of each task

 

[3] End-to-End Incremental Learning

• ECCV 2018
• Herding-based sample selection

 

[4] Learning to learn without forgetting by maximizing transfer and minimizing interference (MER)

• ICLR 2019
• Reservoir sampling-based selection

 

[5] IL2M: Class Incremental Learning With Dual Memory (IL2M)

• ICCV 2019
•  

 

[6] Large Scale Incremental Learning (BiC)

• CVPR 2019
• Mean of feature-based sample selection

 

[7] Efficient Lifelong Learning with A-GEM (A-GEM)

• ICLR 2019
• Uniform random sampling

 

[8] Experience Replay for Continual Learning (ER)

• NeurIPS 2019
• Reservoir sampling-based selection

 

[9] Dark Experience for General Continual Learning: a Strong, Simple Baseline (DER)

• NeurIPS 2020
• Reservoir sampling-based selection

 

[10] Gdumb: A simple approach that questions our progress in continual learning (Gdumb)

• ECCV 2020
• Uniform random sampling

 

[11] Maintaining Discrimination and Fairness in Class Incremental Learning (WA)

• CVPR 2020
• Herding-based sample selection

 

[12] Using Hindsight to Anchor Past Knowledge in Continual Learning (HAL)

• AAAI 2021
•  

 

 

 

 

 

 

Online Replay-based Continual Learning Methods