Open research questions in Multisensory perception and integration

What the literature leaves open

• The study does not explicitly state the technical challenges, but it mentions the need to prevent higher-order cognitive processes often triggered by arithmetic or letter tasks. The experiment required the participants to maintain fixation on a central fixation cross throughout the experiment. The study had to account for multiple comparisons in the univariate analyses.

  Cross-modal processing of auditory and visual symbol representations in the temporo-parietal cortex · 2026 · DOI

• further research is needed to investigate the role of the TPOJ in cross-modal processing, - research could explore the implications of the findings for developmental learning difficulties such as dyslexia, - studies could investigate the ability to form efficient associations between letters and speech sounds in different populations

  Cross-modal processing of auditory and visual symbol representations in the temporo-parietal cortex · 2026 · DOI

• There is a lack of understanding of how spiciness intensity influences emotional responses to food. The effects of nationality and sensation-seeking tendencies on emotional responses to spicy food are not well studied. The study aims to address this gap by exploring emotional responses to Xianglajiang among Korean and Chinese consumers.

  Effects of Spiciness Intensity on Emotional Responses to Xianglajiang: A Comparison Between Korean and Chinese Consumers · 2026 · DOI

• The complexity of audiovisual speech perception in individuals with hearing loss. The variability in response to different linguistic stimuli. The need for personalized rehabilitation strategies and supportive systems to meet individual needs.

  The impact of visual information in speech perception for individuals with hearing loss: a mini review · 2024 · DOI

• The variability in response to different linguistic stimuli highlights the complexity of audiovisual speech perception. Investigating the efficacy of tailored interventions across diverse linguistic contexts remains an essential avenue for future research. There is a need for further comparative analysis across linguistic levels.

  The impact of visual information in speech perception for individuals with hearing loss: a mini review · 2024 · DOI

• While active inference provides a principled framework for uncertainty-guided inference, the role of dynamically inferred sensory precision in generative multimodal arbitration under sensory conflict remains comparatively underexplored.

  Sensory Precision Inference for Multimodal Arbitration under Uncertainty · 2026

• How these multisensory cues form stable percepts of viscosity that drive behaviour remains unknown.

  Both optical and rheological properties contribute to viscosity judgements when comparing real liquids using vision and touch · 2025 · DOI

• Poems describing urban and soundscape heritage have been poorly researched due to the obscurity, fragmentation, and inaccessibility of historical materials.

  Historical sensing: the spatial pattern of soundscape occurrences recorded in poems between the Tang and the Qing Dynasties amid urbanization · 2024 · DOI

• Since the underlying mechanisms of these aberrations are not yet fully understood, the present study aims to investigate multisensory processing in schizophrenia in more detail.

  Neural correlates of audiovisual integration in schizophrenia – an ERP study · 2024 · DOI

• Future studies can build upon the findings by examining the neural mechanisms underlying emotion-dependent brain-eye interactions. The development of more advanced affective computing systems and emotion recognition technologies can be informed by the study's results.

  Quantifying emotion-dependent brain–eye interactions during audiovisual emotional stimulation · 2026 · DOI

• Traditional affective models have primarily focused on identifying the neural substrates of emotion, neglecting the embodied nature of emotional experience. The study addresses the gap in understanding emotion-dependent brain-eye interactions during audiovisual emotional stimulation.

  Quantifying emotion-dependent brain–eye interactions during audiovisual emotional stimulation · 2026 · DOI

• The study had a limited sample size, with 38 EL and 59 TL infants. Four participants were excluded from the analyses due to issues with fussiness or withdrawal from the study.

  Developmental changes in audio-visual speech integration during the first year of life in infants at elevated and typical likelihood of autism · 2026 · DOI

• There is limited longitudinal research on how audio-visual integration develops across the first year of life, particularly in infants at elevated likelihood of autism. Previous studies have not investigated the developmental changes in audio-visual speech integration during the first year of life in infants at elevated and typical likelihood of autism.

  Developmental changes in audio-visual speech integration during the first year of life in infants at elevated and typical likelihood of autism · 2026 · DOI

• Future studies could examine the effects of visual distractors on sound localization in more complex and realistic environments. The development of more sophisticated models of audiovisual integration could provide further insights into the dissociation in audiovisual integration.

  SOUND LOCALIZATION IS BIASED BY SIMULTANEOUS AND DELAYED BY PRECEDING VISUAL DISTRACTORS · 2026 · DOI

• The neural basis of iconicity and its role in sound-to-meaning mappings in natural language remain unclear. Prior studies have not investigated iconic shape associations of auditory real words and pseudowords.

  DECODING HOW THE SOUNDS OF WORDS AND PSEUDOWORDS SIGNIFY SHAPE: AN fMRI STUDY · 2026 · DOI

• Extending to arbitrary style transfer, - Improving the efficiency and flexibility of image style transfer, - Exploring other applications of synesthesia in AI

  Deep Hybrid Synesthesia Model for Audio-Image Transfer · 2026 · DOI

• Existing neural style transfer models are typically unimodal, operating either on 2D images or on 1D/2D audio representations. There is a lack of deep models that can jointly handle 2D spatial image features and time-series audio features in a unified framework. The lack of deep models that explicitly learn to transfer style between images and audio.

  Deep Hybrid Synesthesia Model for Audio-Image Transfer · 2026 · DOI

• Attention mechanisms do not inherently guarantee robustness and may fail under several conditions. Limited interpretability and high computational cost are also limitations. Sensitivity to noisy or missing modalities is another challenge.

  A Structured Survey of Attention Mechanisms in Audio-Visual Fusion: Architectures, Challenges, and Evaluation Frameworks · 2026 · DOI

• Improving the robustness and interpretability of attention mechanisms. Developing more efficient and effective evaluation practices. Exploring new applications of audio-visual fusion.

  A Structured Survey of Attention Mechanisms in Audio-Visual Fusion: Architectures, Challenges, and Evaluation Frameworks · 2026 · DOI

• The sample size was limited to 18 participants per experiment. The study used a specific stimulus list composed of 432 unique words from the French language. The assumptions for the two-way ANOVA were not fully met.

  Consciously detecting and recognizing a past visual word after its sensory trace is gone · 2026 · DOI

• Future research could investigate the neural mechanisms underlying the effect of retro-cueing on conscious perception. Additional studies could explore the application of the findings to the development of novel therapies for individuals with impaired conscious perception. Further research could also examine the effectiveness of retro-cueing in different contexts and populations.

  Consciously detecting and recognizing a past visual word after its sensory trace is gone · 2026 · DOI

• Less is known about how nouns, verbs, and non-verbal sounds differentially activate action-related conceptual information. The study addresses the gap in understanding how action congruence influences the activation of action-related information.

  The power of words depends on context: action congruence differentially shapes activation by words and sounds · 2026 · DOI

• The study faced challenges in terms of the small sample size and the limited range of noise vibration levels investigated. The study faced challenges in terms of the potential for bias and variability in the results.

  Verification of stochastic resonance effects in tactile sensitivity of the tongue · 2026 · DOI

• The study suggests that future research should investigate the effects of stochastic resonance on tactile sensitivity in larger samples. The study suggests that future research should explore the potential applications of stochastic resonance in improving tactile sensitivity in individuals with swallowing disorders. The study suggests that future research should investigate the optimal parameters for applying noise vibration to the jaw to improve tactile sensitivity.

  Verification of stochastic resonance effects in tactile sensitivity of the tongue · 2026 · DOI

• Further research could explore the specific auditory cues that are most important for assessing conversation success. Investigating the role of other sensory modalities, such as tactile or olfactory cues, in assessing conversation success could be a future research direction. Longitudinal studies could examine how conversation success is assessed over time.

  Sound is essential for observers to accurately assess unsuccessful conversations · 2026 · DOI

Most-cited papers in Multisensory perception and integration

• A feature-integration theory of attention · Cognitive Psychology · 1980 · 8,697 citations
• Hearing lips and seeing voices · Nature · 1976 · 4,091 citations
• Half a century of research on the Stroop effect: An integrative review. · Psychological Bulletin · 1991 · 3,962 citations
• Analysis of Individual Differences in Multidimensional Scaling Via an N-way Generalization of “Eckart-Young” Decomposition · Psychometrika · 1970 · 3,676 citations
• Evoked-Potential Correlates of Stimulus Uncertainty · Science · 1965 · 2,292 citations
• Some informational aspects of visual perception. · Psychological Review · 1954 · 2,140 citations
• An Experimental Study of Apparent Behavior · The American Journal of Psychology · 1944 · 2,098 citations
• Categorizing sounds and learning to read—a causal connection · Nature · 1983 · 2,003 citations
• The motor theory of speech perception revised · Cognition · 1985 · 1,768 citations
• Multidimensional Scaling: I. Theory and Method · Psychometrika · 1952 · 1,714 citations

Most recent work

• Hearing sounds when the eyes move: A case study implicating the tensor tympani in eye movement-related peripheral auditory activity · bioRxiv · 2026
• Domain-specificity in phonological and acoustic characteristics of pseudoword iconicity across multiple domains · bioRxiv · 2026
• Sounds Sweet: Sound Reduplication in Brand Names Enhances Sweet Taste Expectations · Psychology and Marketing · 2026
• Multisensory coding of audiovisual movies in the human hippocampus · bioRxiv · 2026
• Higher-order crossmodal representations in the auditory cortex of deaf and hearing individuals · bioRxiv · 2026
• The addressee’s face as a cue for gesture production · Language Cognition and Neuroscience · 2026
• Decoding the dynamic formation of bolus: A multimodal fusion and simulation framework for texture perception-regulated oral processing · Food Research International · 2026
• Progressive neural organization across the IFG and pMTG tracks gesture-speech information-theoretic structure · bioRxiv · 2026
• Next-generation virtual consumer sensory and behaviour science: The evolving role of virtual reality eye-tracking—A graphical perspective · Acta Psychologica · 2026
• The temporal effects of auditory and visual immersion on speech level in virtual environments · The Journal of the Acoustical Society of America · 2026

Related topics in Psychology