Multi-View Abnormality Detection In Clinical Knee MRI Studies Using Transformers

We propose a novel neural network architecture based on transformers that is capable of combining information from multiple three-dimensional MRI scans taken in different views, i.e. sagittal, coronal and axial. We demonstrate its performance on a public dataset and show that it is on par with other state-of-the-art methods in terms of area-under-curve (0.93 ± 0.01), sensitivity (0.86 ± 0.02), specificity (0.89 ± 0.05) and accuracy (0.86 ± 0.01) in detecting abnormalities. We further show that robustness to missing data can be increased when applying multi-view dropout and outline the potential of such an architecture in the field of medical imaging.