Examinations of Audiovisual Speech Processes, the McGurk Effect and the Heteromodal Superior Temporal Sulcus in the Human Brain Across Numerous Approaches

Abstract

Speech can be perceived through the speech sounds we hear and the facial/lip movements we see. In this dissertation, I explore the fundamental question of how the brain processes audiovisual (AV) speech using different approaches. In Chapter 2, we present a general framework for the comparison of AV speech neuroimaging experiments in terms of whether the AV signals were dissimilar (conflicting AV speech) or equivalent (validating AV speech). Using this framework, we conduct an Activation Likelihood Estimation meta-analysis to determine which brain regions are consistently activated for experiments that stressed conflicting vs. validating AV speech. Our main finding reveals that bilateral posterior temporal regions, left inferior parietal lobule and other dorsal-stream regions are recruited for conflicting AV speech. In Chapter 3, we focus on the superior temporal sulcus (STS), which has been associated with AV speech (Calvert et al., 2000; Chapter 2). Across cognitive functions, we assess the consistent functional coactivation patterns associated with anatomical bilateral STS subregions using meta-analytic connectivity modeling. These results demonstrate that the posterior STS consistently coactivates with auditory/language dorsal-stream regions, whereas anterior STS consistently coactivates with ventral-stream regions (although less strongly). In Chapter 4, we examine the causal relationship between brain structure and AV speech perception in 33 left-hemisphere stroke survivors and 39 matched controls, using the McGurk paradigm. We investigate behavior, and brain structure and AV speech relationships using a multivariate lesion symptom mapping technique in the stroke cohort and complementary voxel-based morphometry in the controls. We report that damage to the left angular gyrus correlates with decreased visually-influenced speech percepts. This finding is supported by brain structural correlations identified in controls. Using complementary approaches, this dissertation presents evidence that the auditory/language dorsal stream is involved in processing conflicting AV speech signals, consistently coactivates with the posterior STS across cognitive functions, and is necessary for various AV speech integrative processes based on corroborative findings in people with left-hemisphere lesions and controls.