An integrative approach to studying real-world speech and music processing

Communication is inherently social and requires an efficient exchange of complex cues between speakers and listeners. However, language processing is typically studied using individual listeners and simplistic stimuli. What are the interpersonal mechanisms that allow us to understand and connect with others across the lifespan? My lab studies everyday interactions using behavioral, computational, and dual-brain neuroimaging techniques in real-life environments. To understand the real-time dynamics of communication at the biological level, I have used brain-to-brain coupling (child-caregiver, adult-adult) as a measure of interpersonal alignment to predict communicative success and learning outcomes. In one fNIRS study, we found that activation in the infant prefrontal cortex preceded and drove similar activation in the adult brain, a result that advances our understanding of children’s influence over the accommodative behaviors of caregivers. In ongoing work using dual-brain EEG during adult dialogue, we are exploring the causal relationship between representations of fine-grained linguistic features, speaker-listener coupling, and overall communication quality. Across several studies, we have developed new methods for quantifying the acoustic and semantic structure of naturalistic speech in different communicative modes (e.g., infant-directed speech, podcasting to diverse audiences) and measuring how this structure relates to overall impressions of a speaker (e.g., creativity, personality). Finally, another branch of the lab focuses on music cognition, including the fine-grained processing of hierarchical features of naturalistic music, how these features impact memory, and how they are represented in the brain. Ongoing work will investigate how processes of perception and production coordinate during everyday music-making. This collection of findings provides a new understanding of how our brains and behaviors both shape and reflect different audiences during everyday communication.

Dr. Elise Piazza is an Assistant Professor at the University of Rochester.

Dr. William Hodos NACS Seminars are free and open to the public.