Although large language models often produce impressive outputs, it remains unclear how they perform in real-world scenarios requiring strong reasoning skills and expert domain knowledge.

The gap

Consensus across the literature

Supporting evidence — 5 representative gaps

• Mitigating Perceptual Judgment Bias in Multimodal LLM-as-a-Judge via Perceptual Perturbation and Reward Modeling (2026)

  Recent multimodal large language models have demonstrated strong reasoning ability, yet their reliability as automated evaluators remains limited by a critical weakness: when visual evidence conflicts with textual cues, MLLM judges tend to reward plausible narratives over perceptually correct answers.

  Keywords: recent multimodal large language models strong reasoning ability reliability automated evaluators remains limited critical weakness
• Attention-guided Fine-tuning of Multimodal Large Language Models Improves Chain-of-Thought Reasoning (2026)

  The effectiveness of Chain-of-Thought (CoT) prompting in Multimodal Large Language Models (MLLMs) remains uncertain: across several visual reasoning benchmarks, CoT prompting often degrades performance compared to direct prompting.

  Keywords: prompting effectiveness chain thought multimodal large language models mllms remains uncertain across several visual reasoning
• Moment-Video: Diagnosing Temporal Fidelity of Video MLLMs on Momentary Visual Events (2026)

  Video multimodal large language models (MLLMs) have made rapid progress on general and long-form video understanding, yet their ability to preserve brief answer-critical visual evidence remains underexplored.

  Keywords: video multimodal large language models mllms made rapid progress general long form understanding ability preserve
• Can large language models reason about medical questions? (2024) · doi

  Although large language models often produce impressive outputs, it remains unclear how they perform in real-world scenarios requiring strong reasoning skills and expert domain knowledge.

  Keywords: large language models often produce impressive outputs remains unclear perform real world scenarios requiring strong
• Can MLLMs Reason Beyond Language? VisReason: A Comprehensive Benchmark for Vision-Centric Reasoning (2026)

  Recent multimodal large language models (MLLMs) achieve strong performance on visual reasoning benchmarks, yet it remains unclear to what extent such performance reflects reasoning directly grounded in visual evidence.

  Keywords: performance visual reasoning recent multimodal large language models mllms achieve strong benchmarks remains unclear extent