Cross-subject Decoding of Continuous Language from Non-invasive fMRI Recordings

It also contains preprocessed data of two subjects.Decoding continuous language from non-invasive brain recordings across individuals remains a formidable challenge. Existing non-invasive decoders are constrained to isolated linguistic units or rely on subject-specific calibration, which limits their generalizability across individuals.

Here we present an approach for cross-subject decoding of continuous language from single-trial, non-invasive fMRI data. Our approach employs a three-dimensional convolutional neural network to extract invariant feature representations shared across individuals, augmented with an information bottleneck that accurately identifies voxels most responsive to linguistic stimuli. The resulting decoder can produce intelligible textual sequences capturing the general meaning of both perceived and imagined language, significantly outperforming existing decoders in within- and cross-subject evaluations.