Task-Agnostic Continual Learning Using Base-Child Classifiers

Continual learning (CL) aims to learn new tasks by forward transfer of information learnt from previous tasks and without forgetting them. In task incremental CL, task information is vital during both strategy development and inference. Providing such partial knowledge about the test sample demands additional complexity and may become intractable, especially when the sample source is ambiguous. In this work, we design a task-agnostic approach that uses {\em base-child} hybrid setup to incrementally learn tasks while mitigating forgetting. Multiple base classifiers guided by reference points learn new tasks and this information is {\em distilled} via feature space induced sampling strategy. A central child classifier consolidates information across tasks and infers the task identifier automatically. Experimental results on standard datasets show that the proposed approach outperforms the various state-of-the-art regularization and replay CL algorithms in terms of accuracy, by 50\% and 7\% with homogeneous and heterogeneous tasks, respectively, in task-agnostic scenarios.