Individualized representation learning of resting-state fMRI

Modest success has been made in using resting state fMRI (rs-fMRI) to predict individualized behaviors. Here, we describe a scalable and modular system for individualized representation learning of rs-fMRI. The system includes a “deep” base with “shallow” adds-on. The “deep” base is trained through multiple stages of self-supervised learning with unlabeled rs-fMRI. After the base is trained and fixed, the “shallow” adds-on are trained with supervised learning for behavior prediction. The base includes a variational auto-encoder (VAE) for unsupervised learning of spatial representations, a Bi-directional Encoder Representations from Transformer (BERT) for self-supervised learning of temporal dynamics, and a subspace learning of individualized representations. In each module, an encoder is paired with a decoder to ensure reversible transformation from the brain to the feature space and vice versa. The add-on uses a multilayer perceptron (MLP) to predict behavioral scores with individualized representation extracted from the base model. We demonstrate that this system can predict individualized scores on cognition, significantly outperforming the state-of-the-art connectome-based predictive modeling method. The system also uncovers the brain networks underlying individual variations of cognitive phenotypes.