Optimizing AV1 Encoder For Real-Time Communication

Computer vision datasets usually present long-tailed training distributions where the classes are not represented with the same number of training samples. This so-called class imbalance problem hinders the proper learning of inference models, biasing them towards over-represented classes and decreasing their generalization. Adopted solutions to tackle the effect of class imbalance are based on weighting the training loss according to the number of class samples, leading to regimes where low-represented classes guide the learning just accounting for their cardinal number. To also incorporate class complexity in the process, we propose a novel training scheme called CCL: Class-wise Curriculum Learning. Classes are first sorted based on a difficulty criterion which not only accounts for the number of training samples but also for their training outcomes. The curriculum is then used to guide the training: easy classes are fed first and?incrementally, the more difficult ones are added. The proposed approach is validated for image classification using long-tailed datasets. Results show that when the proposed Class-wise Curriculum Learning scheme is used, trained models outperform specific state-of-the-art methods devoted to handle the class imbalance problem. The code, data and reported models described along this paper are publicly available at \url{https://github.com/vpulab/CCL}.