‘Soundless answer. Firm safety.’ ERPs showing perceptual simulation since actual 160 ms (Bernabeu, Willems, & Louwerse, in prep.)

However, the critical manipulation is the transition of conceptual modality across trials—e.g., haptic to visual—, which is enabled by modality-normed context and target trials. Switching across trials in different modalities, compared to maintaining a modality, incurs a switching cost. Previous experiments measured this from ERPs time-locked to the second word of target trials, and then from response times.

In the current follow-up, we tackled more precisely the time frame of lexical and semantic access by time-locking ERPs to the first word of target trials, which also avoided semantic confounds on the target word. Next, the experiment featured different types of switch—from auditory to visual, and from haptic to visual—, which were compared to the non-switch—visual to visual. Further, we had a quick response group ( n = 21), and a self-paced group ( n = 21), alongside a few participants with no speed instructions ( n = 5).

The results, analyzed with mixed effects models, reveal ERP effects of modality-switching in four typical time windows between 160 and 750 ms after word onset. The effects are generally characterized by a more negative amplitude for modality-switching than not switching, and they arise with both types of switch, in both groups, and in anterior as well as posterior brain regions. In sum, the early start and broad scope of this effect suggest that perceptual simulation contributes fundamentally to word comprehension.