Prediction Of Cognitive Scores By Movie-Watching FMRI Connectivity And Eye Movement Via Spectral Graph Convolutions

Brain functional connectivity has been demonstrated to serve as a “fingerprint” to predict individual behavior and phenotypes. Hence, a precise mapping of the functional connectivity patterns to behavior and phenotype could provide insightful clues to brain architecture and generation of cognition. In this context, the naturalistic paradigm provides more engaging conditions and richer fMRI information and preserve or even enhance individual features and increase sensitivity to phenotypic measures, compared with other functional MRI modalities including resting-state and task paradigms. However, to the best of our knowledge, only linear methods were developed for predicting phenotypic measures from brain activity under naturalistic stimulus, while the brain activity is highly dynamic and nonlinear. Hence, in this work, we adopted nonlinear graph convolutional network (GCN) to predict cognition-related phenotypic score from brain functional connectivity under naturalistic stimulus with the behavior patterns of eye movement integrated, where subjects are the nodes, eye movement trajectory similarity across subjects is used to defined graph edges, functional connectivity is used as node feature. A few nodes are labeled by their phenotypic score, and the model is trained to predict the scores of those unlabeled nodes. The prediction accuracy of this method outperforms those from the linear classification method, resting-state based functional node feature and random edge tests.