Skip to main content

LABEL-EFFICIENT AND ROBUST LEARNING FROM MULTIPLE EXPERTS

Bojan Kolosnjaji (Technical University of Munich); Apostolis Zarras (Delft University of Technology)

  • SPS
    Members: Free
    IEEE Members: $11.00
    Non-members: $15.00
07 Jun 2023

Learning from multiple sources of labeled data presents unique challenges, including the cost of using many annotators in the training, test, and production time and their limited reliability. In this paper, we analyze the problem of label-efficient learning and propose a method for training a classification system from data labeled by multiple annotators, where only a small subset of them is chosen adaptively to reduce later communication effort and the effect of malicious annotators, especially in the validation and test phase. We define an iterative optimization procedure where the annotation weight vector’s sparsity is enforced while reducing the overall classification error. Using real-world data, we show that our approach can pertain to the classification performance on the complete set of annotators while reducing the need for annotator labels by over 50% on several different tasks. Furthermore, we demonstrate that label sparsity reduces label flip attacks’ effect on performance.

More Like This

  • SPS
    Members: Free
    IEEE Members: $11.00
    Non-members: $15.00
  • SPS
    Members: Free
    IEEE Members: $11.00
    Non-members: $15.00
  • SPS
    Members: Free
    IEEE Members: $11.00
    Non-members: $15.00