Open research questions in Topic Modeling

What the literature leaves open

• Existing RAG frameworks face challenges such as semantic drift and knowledge conflicts. Traditional retrieval strategies disrupt the logical structure of power documents. There is a need for a framework that can effectively filter retrieval noise and resolve knowledge conflicts.

  Multi-Route Search and Adaptive Fusion for Power QA with Small Language Model Guidance · 2026 · DOI

• The practical reliability and pedagogical value of LLM-based graders in real classrooms remain underexplored. Most prior works rely on controlled or synthetic settings, leaving open the question of how LLM-based graders perform on authentic student responses.

  LLM-as-a-Grader: Practical Insights from Large Language Models for Short-Answer and Report Evaluation · 2026 · DOI

• We presented CRAFT, a claim-centric pipeline for grounded multi-video question answering that combines keyframe selection, ASR-based evidence extraction, critic-guided verification, and citation- backed report generation. CRAFT improves over the baseline through atomic-claim formatting, ASR, and the critic loop. However, recall and cita- tion recall remain challenging, suggesting that future work should improve evidence coverage, cross-video retrieval, multilingual ASR, and pre- cise claim-to-video attribution.

  CRAFT: Critic-Refined Adaptive Key-Frame Targeting for Multimodal Video Question Answering · 2026

• Future work should investigate whether these patterns reflect fundamental differences in how models allocate cognitive resources across task types, and whether architectural interventions (e. Future work should investigate whether these patterns reflect fundamental differences in how models process spatial vs linguistic representations during active navigation. Causal and Memory Worlds: Mixed Results Causal worlds (memory canonical) showed 5/8 wins for memory, with grid competitive on dynamic state-change worlds (C2_fire_crossing, C3_flood_room).

  HalluWorld: A Controlled Benchmark for Hallucination via Reference World Models · 2026

• We find that the benefits of CES are not limited to the original 7B math setting, but generalize to a smaller backbone and to out-of-domain tasks in both general reasoning and code generation.

  Taming the Thinker: Conditional Entropy Shaping for Adaptive LLM Reasoning · 2026

• Additionally, we compare two conditioning formats, templated quality tokens and natural language critiques, but we do not ablate more fine-grained training-time conditioning choices such as the granularity or verbosity of the critique, different formats of the natural language critique, or mixing conditioning signals—deeming it beyond the scope of this work.

  Introspective X Training: Feedback Conditioning Improves Scaling Across all LLM Training Stages · 2026

• The major factors are: benchmark contamination, synthetic reasoning bias, and 42 the type of disloyal stream of thought that appears when understanding is lacking, both numerically and symbolically.

  Mathematical Reasoning in Large Language Models: Benchmarks, Architectures, Evaluation, and Open Challenges · 2026

• We evaluated four capabilities of LLM agents in engineering design workflows: tool execution, retrieval-assisted parameter selection, remote training-job orchestration, and training-code authoring. ENGIAI provides the multi-agent, tool-connected implementation used in these experiments. 19 EngiAI: Capability-Based Evaluation of Tool-Connected LLM Agents for Engineering Design PREPRINT The experiments yield four main findings. First, GPT-5-mini and Gemini-3-Flash complete 96–97% of the structured Beams2D workflows, compared with 55% for Qwen3-4B and 78% for Qwen3.5-4B. Second, indexed documents improve numerical parameter selection relative to an empty retrieval index. Third, Gemini-3-Flash completes every tested HPC pipeline, whereas GPT-5-mini reaches the final evaluation step in 70% of explicit-prompt trials and 50% of natural-prompt trials. Fourth, successful orchestration of a supplied training program does not imply uniform success when the agent must write that program. The authoring benchmark identifies separate failures in checkpoint compatibility, sampling behavior, and model quality. At T6v2, both proprietary models implement conditional VAEs rather than the supplied cGAN; Gemini-3-Flash obtains lower MMD than the seed-matched Beams2D reference in seven of ten trials. The study covers two engineering problems and four LLM backends, but the RAG, HPC, and authoring evaluations use subsets of those backends. The retrieval index contains only two papers, the authoring benchmark uses two proprietary models and one prompt formulation, and model quality is compared with a single reference run per seed. The results should therefore not be generalized beyond the tested models, prompts, and problems. Future work should evaluate additional engineering problems, model backends, prompt formulations, and sampling temperatures. Retrieval should be tested with larger and noisier document collections. The authoring benchmark should include other generative-model families, including diffusion models, and compare authored models against distributions of repeated reference runs. A further extension would raise the authoring target from the training program to the simulator itself, supplying a multi-page problem specification as input and scoring the generated simulator against reference solutions and physical validity checks. A matched single-agent baseline is also needed to determine whether the multi-agent decomposition itself improves performance. Finally, experiments with larger tool sets exposed through tool APIs or MCP would test how workflow reliability changes as the system expands. The principal implication is that engineering agents should not be assessed from successful demonstrations alone. Reliable deployment requires capability-specific tests, controlled information conditions, inspection of intermediate artifacts, and validation of the resulting engineering artifact. ENGIAI provides one implementation of this evaluation approach, while the four evaluation dimensions define a basis for future comparisons among systems that expose compatible evidence.

  EngiAI: A Multi-Agent Framework and Benchmark Suite for LLM-Driven Engineering Design · 2026

• The lack of a comprehensive framework for assessing LLM vulnerabilities. The need for a systematic evaluation of LLM vulnerabilities. The gap in understanding how to improve the faithfulness and reliability of LLMs.

  A red teaming framework for large language models: a case study on faithfulness evaluation · 2026 · DOI

• Further research is needed to explore the applications of Modular RAG in diverse scenarios - Investigating the trade-offs between efficiency and effectiveness in Modular RAG

  Modular RAG: Transforming RAG Systems into LEGO-like Reconfigurable Frameworks · 2026 · DOI

• The lack of a unified paradigm for designing and optimizing RAG systems. The rapid advancements in RAG are outpacing the foundational RAG paradigm. Many methods struggle to be unified under the process of retrieve-then-generate.

  Modular RAG: Transforming RAG Systems into LEGO-like Reconfigurable Frameworks · 2026 · DOI

• The study does not establish bedside clinician behavior or clinically meaningful undertriage, - Clinician-anchored validation is needed before attributing the findings to bedside decision-making, - The neutralization analysis was conducted on MIMIC-IV only, - The study only analyzed emergency triage decisions

  Auditing sex/gender disparities in emergency triage with LLM-based paired comparisons · 2026 · DOI

• Compared to earlier studies using more conventional annotation methods, TGSM facilitates capturing multiple valid answers for individual questions, thus addressing critical open challenges identified in previous QA research, such as low question clarity and variability in annotation quality [24]. Given the unstructured and informal nature of online discussions, further research into automated and semi-automated methods for validating and filtering clinically relevant content from patient-generated data could significantly enhance the quality and usability of forum-based QA datasets.

  Assisting patients with reliable information gathering: A guide to building efficient QA systems for medical use · 2026 · DOI

• The study used a limited number of models and datasets, - The evaluation was text-only, - The study did not account for potential biases in the data, - The LLM-as-judge may not be perfect, - The study did not evaluate the models on image inputs

  The economics of accuracy for medical reasoning with large language models · 2026 · DOI

• Investigating the effects of model scale, medical fine-tuning, test-time scaling, and context grounding on other clinical AI tasks, - Evaluating the models on image inputs, - Exploring the use of other prompting strategies and context grounding methods, - Investigating the potential biases in the data and their effects on the models

  The economics of accuracy for medical reasoning with large language models · 2026 · DOI

• One of the challenges is the high rate of life-threatening medication recommendations produced by the LLMs. Another challenge is the need for LLMs to consistently evaluate patient-specific cases. The study also highlights the challenge of using LLMs in domains marked by highly technical language or rarely occurring scenarios, such as medical and pharmacy domains.

  Large language models management of complex medication regimens: a case-based evaluation · 2025 · DOI

• Most LLMs are trained on a widely available corpus, which creates the potential for problems in domains marked by highly technical language or rarely occurring scenarios, - The study used a small sample of patient cases, - The study used a limited number of LLMs, - The study did not evaluate the long-term effects of using LLMs for medication management

  Large language models management of complex medication regimens: a case-based evaluation · 2025 · DOI

• During the data generation pipeline of MOTAB, the student generates reasoning steps delimited by the stop sequence ".

  Backtracking When It Strays: Mitigating Dual Exposure Biases in LLM Reasoning Distillation · 2026

• The underlying formalizations are language- agnostic, but generalization of the natural-to-formal evaluation to other languages and writing conventions remains to be empirically validated.

  LLMEval-Logic: A Solver-Verified Chinese Benchmark for Logical Reasoning of LLMs with Adversarial Hardening · 2026

• Cross-linguistic analysis, - Evaluation of other LLMs, - Investigation of different prompt strategies

  A red teaming framework for large language models: a case study on faithfulness evaluation · 2026 · DOI

• Difficulty in obtaining large-scale and unbiased medical datasets. Significant privacy concerns. Inefficiencies due to manual dataset annotation.

  Assisting patients with reliable information gathering: A guide to building efficient QA systems for medical use · 2026 · DOI

• This suggests that our findings are not limited to a specific model scale or agent framework.

  DocOS: Towards Proactive Document-Guided Actions in GUI Agents · 2026

• This work takes a step toward formalizing what should count as a fact in T2I generation, an aspect that has remained largely underexplored.

  FAGER: Factually Grounded Evaluation and Refinement of Text-to-Image Models · 2026

• We note one limitation: BERTScore relies on bert-base-multilingual-cased, which it- self exhibits English-centric biases (Conneau et al.

  Lost in Interpretation: The Plausibility-Faithfulness Trade-off in Cross-Lingual Explanations · 2026

• However, open-source LLM performance in domain-specific ESG tasks remains insufficiently understood.

  Empirical Evaluation of Open-Source Large Language Models for Retrieval-Augmented Generation in ESG Domain · 2026

Most-cited papers in Topic Modeling

• Toward a model of text comprehension and production. · Psychological Review · 1978 · 3,139 citations
• Large language models encode clinical knowledge · Nature · 2023 · 3,027 citations
• A Survey on Evaluation of Large Language Models · ACM Transactions on Intelligent Systems and Technology · 2024 · 2,472 citations
• A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions · ACM Transactions on Information Systems · 2024 · 1,585 citations
• A Brief Overview of ChatGPT: The History, Status Quo and Potential Future Development · IEEE/CAA Journal of Automatica Sinica · 2023 · 1,366 citations
• Toward expert-level medical question answering with large language models · Nature Medicine · 2025 · 1,009 citations
• Unifying Large Language Models and Knowledge Graphs: A Roadmap · IEEE Transactions on Knowledge and Data Engineering · 2024 · 952 citations
• Lost in the Middle: How Language Models Use Long Contexts · Transactions of the Association for Computational Linguistics · 2024 · 944 citations
• G-Eval: NLG Evaluation using Gpt-4 with Better Human Alignment · 2023 · 714 citations
• Detecting hallucinations in large language models using semantic entropy · Nature · 2024 · 665 citations

Most recent work

• A Survey on Retrieval-Augmented Text Generation for Large Language Models · ACM Computing Surveys · 2026
• Exploring Large Language Models for Scientific Question Answering via Natural Language to SPARQL Translation · ACM Transactions on Intelligent Systems and Technology · 2026
• A Survey of Mamba · ACM Transactions on Intelligent Systems and Technology · 2026
• Towards Trustworthy Retrieval Augmented Generation for Large Language Models: A Survey · ACM Computing Surveys · 2026
• Benchmarking contextual understanding for in-car conversational systems · Journal of Systems and Software · 2026
• KIS: COLIEE 2025 Task 4 Solver Using Japanese LLM · The Review of Socionetwork Strategies · 2026
• Accuracy and fairness of generative AI in automated essay scoring: Comparing GPT-4o, feature-based models, and human raters · Assessing Writing · 2026
• Metadata Augmentation Using NLP, Machine Learning and AI-Chatbots: A Comparison · Journal of Library Metadata · 2026
• Not all bonds are created equal: Dyadic latent class models for relational event data · Social Networks · 2026
• Bayesian RAG: uncertainty-aware retrieval for reliable financial question answering · Frontiers in Artificial Intelligence · 2026

Related topics in Computer Science