DEEP SUBCLASS LINEAR DISCRIMINANT ANALYSIS FOR MULTIMODAL FEATURE SPACE LEARNING

In this work, we target a known problem in representation learning that is: beyond coarse classification, how can we better model fine- grained categorization? To address this problem, we introduce Deep Subclass Linear Discriminant Analysis (DeepSDA), which utilizes intra-class variation and inter-class similarity during training. We could achieve multimodal classification by maximizing the ratio of between-subclass scatter matrix and within-subclass scatter matrix. We maximize the eigenvalues along the discriminative eignevector directions. Hence the deep neural network is able to learn more dis- criminative representation space and thus has higher class separation in the linearly separable latent space. We show that DeepSDA leads to significant improvements on diverse fine-grained categorization and attribute learning benchmarks.