Open research questions in Topic Modeling

What the literature leaves open

• Integration of real-time, dynamic information retrieval with persistent personal memory in RAG-based assistants remains unaddressed; current systems either use static knowledge bases or live web retrieval but do not unify time-sensitive external data with evolving personal episodic memory.

  Web-Integrated RAG System for Real-Time AI Responses · 2026 · DOI

• Shrinking precision cuts down memory demands while speeding up output creation; however, what remains unclear is how such squeezing affects the way models rate their own sureness, measured via average prediction strength for each produced word unit.

  The Effects of Low-Bit Quantization on Confidence Calibration in Small Language Models · 2026 · DOI

• Future work is oriented in four directions. First, expanding SRB-ES with more inference subtypes, including causal, temporal, counterfactual, and simple arithmetic reasoning. This would make it possible to evaluate whether the improvement observed in RTE holds across more diverse forms of reasoning. Second, we propose evaluating other model families while keeping the same benchmark and experimental protocol. Comparing Qwen2.5 with Llama, Gemma, or Mistral would help distinguish which patterns are family-specific and which can be considered more general. Third, it would be worthwhile to incorporate log-probability-based metrics when the infrastructure allows it. This would facilitate comparing discrete accuracy with continuous measures of preference for the correct option, connecting more directly with the debate on whether emergent abilities are real or threshold artifacts. Finally, the benchmark can be used to analyze models specifically trained or fine-tuned on Spanish. This would make it possible to study whether the observed biases stem mainly from model size, instruction tuning, the linguistic distribution of the pretraining data, or the binary formulation of the task.

  Does reasoning emerge with scale? Response biases in binary yes/no evaluation of SLMs in Spanish · 2026 · DOI

• This study addresses the challenge of accurate user story effort estimation in agile software development by introducing a hybrid DL model which combines semantic understanding with sequential modeling. Combining the pre-trained BERT encoder with an LSTM network, the model can effectively learn contextual and structural patterns in the user story nar- rations, which can be used to produce more accurate and data-driven predictions. One of the important contributions of this study is that it has created a large scale dataset of 6,956 user stories that have been comprehensively preprocessed and filtered to 4,079 high quality stories through expert validation. The curated dataset offers a solid base to enhance the accuracy of DL based estimation models. The proposed model demonstrates superior performance, achieving a MAE of 0.6481, RMSE of 1.4559, and a Coefficient of Determination R2 of 0.6581. Furthermore, when mapping the continuous regression outputs to the discrete Fibonacci sequence used in Scrum, the model achieves a classification accuracy of 72%, demonstrating its direct applicability to practical sprint planning. The findings support the efficacy of the hybrid model in improving predictive accuracy in effort estimation. Moreover, the statistical validation with the Wilcoxon Signed Rank Test (p < 0.05) confirms the fact that the improvements are statistically significant and not a mere coincidence. Besides its methodological contributions, the mod el is also deployed as a web-based decision support system, which proves its practical applicability in estimating real-time in agile environments. Altogether, the research identifies the possibility of using large scale curated datasets and hybrid DL methods to enhance the objectivity of the subject and the reliability of estimations in Scrum-based project management. Future efforts will concentrate on incorporating XAI methods and testing the model on real world industrial data to promote transparency and generalizability.

  A hybrid deep learning model for user story effort estimation · 2026 · DOI

• In simulation, calibration-optimal is not decision-optimal; in generation, fine-tuning on richer targets buys vividness that collapses into hallucination when the grounding state is sparse -- a failure a four-base replication traces to base-model instruction adherence, not scale, and that sparse-context auditing removes from the production model.

  Pitwall: Faithful Natural-Language Race-Strategy Briefings from a Calibrated Real-Time Monte Carlo Engine · 2026

• Reinforcement learning (RL) can optimize beyond this ceiling using external quality measures, but how the reward function should be designed when quality is multi-dimensional remains unexplored.

  Improving LLM-Generated Process Model Quality Through Reinforcement Learning: The Role of Reward Function Design · 2026

• Several concrete directions naturally extend the present study. Larger and domain-specific datasets. The current 500-query benchmark is sufficient to expose the main qualitative differences between method families but, as discussed above, is too small to support definitive conclusions for narrow specialized domains. A natural next step is to scale the dataset by at least an order of magnitude and to add domain-specific subsets (medical, legal, financial, source-code evaluation), with corre- sponding domain experts providing the gold-standard scores, so that the per-category trends observed here can be tested for robustness. Multiple annotators and inter-annotator agreement. The gold standard in this study is produced by a single human annotator. Replicating the annotation procedure with several independent experts and reporting inter-annotator agreement (e.g. Cohen’s κ, Krippendorff ’s α) would establish an upper bound on automated-metric correlation and would allow disentangling annotator subjectivity from method error, especially in the borderline medium class that we identified as the hardest region for all methods. Meshkov Journal of Electrical Systems and Information Technology (2026) 13:66 Page 26 of 28 Updated method snapshots. The empirical numbers reported here depend on specific model snapshots (gpt−4.1-mini for the LLM-judge, text-embedding-3-small for Ope- nAI Embeddings, all-MiniLM-L6-v2 for SBERT, microsoft/deberta-base-mnli for NLI, word2vec-google-news-300 for Word2Vec). Since these models evolve quickly, periodic re-running of the experimental pipeline with refreshed versions would keep the recom- mendations of the selection algorithm calibrated. The fully scripted nature of the pipe- line (section “Experimental research methodology”) makes such revalidation low-effort. Reference-free and adversarial evaluation. All embedding-based and NLI-based methods used here require a reference answer, which restricts their applicability. Investigating reference-free variants, for instance, LLM-as-a-Judge configurations that grade a response in isolation, or self-consistency between multiple generations of the same query, and benchmarking them on the same dataset is a natural follow-up. Equally important is studying adversarial robustness: how the eight methods behave on responses that are crafted to game superficial features (very long answers, keyword stuff- ing, sycophantic but factually wrong content), since deployed evaluation pipelines are exposed to such inputs in production. Selection algorithm as a service. Finally, the decision-tree algorithm of section “Algo- rithm to choose an evaluation method” can be packaged as an open-source tool that, given a user’s constraints (compute, budget, latency, dataset size, task category, inter- pretability requirement), returns a ranked list of recommended methods together with the expected accuracy, processing time, and cost. This would make the contributions of the paper directly actionable for practitioners building generative AI systems.

  Comparative analysis of evaluation methods for generative artificial intelligence systems and development of selection algorithm · 2026 · DOI

• Overall, the findings imply that scalable RAG-oriented data selection is best treated as an engineering and measurement problem: humans and LLMs can generate and normalize case descriptions, but selection should be validated against the retrieval behaviour of the intended deployment stack. Human evaluation of many items is costly and difficult to scale in industrial settings where domain expertise is scarce, while LLMs can deliver immediate and consistent decisions at minimal marginal cost.

  Evaluating Data Suitability for RAG Systems in Manufacturing: A Comparison Between Human and LLM Judgments · 2026 · DOI

• Future studies should investigate these aspects in greater depth to better understand how specific design configurations contribute to higher conceptual divergence. A higher diversity score may reflect a richer and more granular internal conceptual organization, which is hypothesized to enhance a model’s ability to generalize and retrieve knowledge effectively, although this relationship remains to be empirically validated.

  Probing Maximally Divergent Conceptual Regions in Large Language Models Through Prompt Engineering · 2026 · DOI

• Furthermore, PM accuracy degrades substantially under implicit constraints or triggers overloaded by concurrent user requests, indicating that robust PM remains an open challenge.

  TriggerBench: Investigating Prospective Memory for Large Language Models · 2026

• Despite their prevalence, relatively little academic work has examined RL-based training of these models, likely due to difficult benchmarks, a lack of data, and a lack of simple baseline recipes.

  Tmax: A simple recipe for terminal agents · 2026

• MORPHEUS v2.0 demonstrates that algorithmic parsimony—arriving at correct solutions via the shortest path—can effectively mitigate LLM hallucinations without generative verification overhead. Future work includes multilingual expansion, RAG integration for dynamic knowledge bases, and community-driven benchmark extension.

  MORPHEUS v2.0: A Deterministic Triple-Judge Protocol for Anti-Hallucination in Large Language Models · 2026 · DOI

• The findings address three interconnected challenges: the lack of standardized evaluation methods for language model performance grounded in practice documentation; uncertainty about whether architectural advances translate into measurable performance gains in domain-specific applications; and persistent questions about the minimum computational requirements for adequate accuracy in resource-constrained settings.

  Small Models Achieve Large Language Model Performance: Evaluating Reasoning-Enabled AI for Secure Child Welfare Research · 2026 · DOI

• Several limitations circumscribe the interpretation of these findings. First, the benchmarks are derived from a single state's child welfare system over a specific time period (2016-2018). Documentation practices, terminology, and case characteristics vary across jurisdictions and over time. Models that perform well on Michigan data may show reduced accuracy when applied to records from states with different practice standards, reporting requirements, or demographic compositions. Validation across multiple jurisdictions would strengthen confidence in generalizability. Second, the binary classification framework simplifies the complexity of assessing mentions of potential risk aspects in case notes. Practitioners rarely make simple present/absent determinations; they assess severity, chronicity, impact, and context-dependent factors requiring nuanced judgment. The benchmarks evaluate whether models can match expert determinations in cases where clear classifications exist. Still, they cannot assess performance on genuinely ambiguous cases that require integrating multiple information sources and applying professional expertise. Extensions to ordinal classifications or continuous ratings might better capture the complexity of practice decisions. Third, the study examines the extended reasoning architecture in detail but provides only limited comparisons with alternative innovations. Other recent developments including retrieval- augmented generation, fine-tuning approaches, or multimodal architectures combining text with structured data, may offer advantages for specific applications (Stoll et al., 2025a; Stoll et al., 2025b). Systematic comparison across architectural approaches using common evaluation frameworks would provide more comprehensive guidance for model selection. Fourth, the evaluation used n-shot classification without task-specific fine-tuning or few- shot learning approaches. While this design tests models' general capabilities applicable across diverse tasks, fine-tuning on domain-specific data typically improves performance for narrow applications. Organizations with sufficient resources to curate training datasets and conduct fine- tuning may achieve accuracy gains beyond those documented here. Research comparing no-shot classification against fine-tuned alternatives would clarify the magnitude of potential improvements and inform cost-benefit analyses of fine-tuning investments.

  Small Models Achieve Large Language Model Performance: Evaluating Reasoning-Enabled AI for Secure Child Welfare Research · 2026 · DOI

• Prior AQDE studies report mixed evidence on whether adding distractors as additional text to the question stem and the correct key consistently improves difficulty prediction.

  Structure-Aware Modeling of Multiple-Choice Questions Improves Automatic Difficulty Estimation · 2026

• Extending these methods to long-form generation exposes a fundamental yet underexamined limitation: token-level distillation implicitly treats all output tokens as equally informative, but long-form outputs are dominated by high-frequency template and grammatical tokens, while the tokens that actually determine output quality are sparsely distributed.

  Not All Tokens Matter Equally: Dynamic In-context Vector Distillation with Decisive-Token Supervision for Long-form Medical Report Generation · 2026

• To investigate the underexplored sources of heterogeneity in prompt performance, we conduct a causal inference-inspired observational analysis of optimized prompts across a diverse set of optimization frameworks, LLM backbones, and NLP benchmarks.

  Why Prompt Optimization Works, and Why It Sometimes Doesn't: A Causal-Inspired Edit-Level Analysis · 2026

• While retrieval-augmented generation (RAG) helps address single-hop questions, many retrievers presently focus on semantic similarity in a dense vector space, which is insufficient for handling multi-hop questions specifically.

  Towards Multi-Hop Retrieval Using Bipartite Question-Oriented Graphs · 2026

• Aligning AI systems with diverse human values requires value specifications grounded in concrete examples, but generating such examples without extensive human supervision remains an open challenge.

  Coherence Maximization Improves Pluralistic Alignment · 2026

• Existing prompt-based methods for IDRR often rely heavily on predicting surface connectives as an indicator for the discourse relation, which is inherently limited by the capacity of pre-trained language models.

  Role-Focused Prompt Framework for Implicit Discourse Relation Recognition · 2026 · DOI

• Coreference resolution is the process of identifying and linking expressions in a text that refer to the same real-world entity. It plays an important role in many Natural Language Processing (NLP) applications, such as machine translation, question answering, and text summarization. In this study, we proposed a transformer-based approach using BERT for Amharic coreference resolution. The system includes both training and testing phases, with components for preprocessing, contextual embedding, feature extraction, and mention prediction. By leveraging BERT, the model captures contextual information from both directions, allowing it to better understand relationships between words in Amharic text. The experimental results show that the proposed model performs well across standard evaluation metrics, achieving F-scores of 80%, 85.71%, 90.9%, 88.86%, and 81.7% for MUC, B3, BLANC, CEAF-m, and CEAF-e, respectively. These results indicate that transformer-based approaches can be effectively applied to coreference resolution in low-resource languages such as Amharic. However, the findings should be interpreted with some caution. Coreference resolution remains a challenging task that requires understanding at multiple linguistic levels, including morphology, syntax, and semantics. In addition, transformer-based models typically perform better with large annotated datasets, which are still limited for Amharic. Despite these promising results, coreference resolution remains a highly complex task that requires linguistic knowledge at multiple levels, including morphological, syntactic, semantic, and world knowledge. Incorporating all such knowledge sources is time-consuming and computationally demanding. Consequently, this study focused on exploiting the strengths of BERT-based contextual representations to model coreference in Amharic. However, transformer-based models typically require large amounts of annotated training data to achieve optimal performance. The relatively limited size of available Amharic datasets may therefore constrain the accuracy of the proposed model. To further improve performance, we suggest the following directions for future work: • Incorporating syntactic parse trees to reduce noisy mention spans and enable the identification of long-dis- tance coreference relations, potentially spanning multiple sentences or entire documents. • Developing POS-tagged and chunked Amharic corpora, which are currently scarce, to support richer linguis- tic feature extraction and improve model performance.

  Transformer-based coreference resolution modeling for Amharic text · 2026 · DOI

• Despite the encouraging results, this study has several limitations. First, the dataset used is relatively small, consisting of only 312 documents and 18,763 mentions, which may limit the model’s ability to generalize to more diverse or complex text. The current work also focuses only on intra-document coreference and does not address cross-document relationships, which are important in many real-world applications. In addition, the manual features used in this study, such as mention distance and string matching, are relatively simple and may not fully capture deeper linguistic patterns. There is also a risk of overfitting due to the combination of a small dataset and a high-capacity model like mBERT, even though techniques such as dropout and early stopping were applied. Another challenge is that mBERT, while multilingual, is not specifically designed for Amharic, which may limit its ability to capture certain linguistic characteristics of the language. Finally, transformer-based models require significant computational resources, which may limit their scalability and practical use in resource- constrained environments.

  Transformer-based coreference resolution modeling for Amharic text · 2026 · DOI

• studies, the explicit investigates knowledge address to Vietnamese EE baselines., in • Proposed Approach: We will encode AMR and KG structures as structural embeddings or learned prefixes injected into the model’s internal layers. By providing a graph-based structural prior, this approach enables the generative SLM to better connect distant triggers and arguments, thereby addressing long-range dependencies that are often lost in purely sequential processing. • Validation Strategy: The approach will be evaluated through long-range dependency cases, along with ablation studies to assess the effectiveness of the structural components. experiments targeted on Question 3: To what extent can the proposed framework mitigate such as word segmentation errors and complex entity boundaries, by leveraging linguistically informed features? language-specific ambiguities, • Theoretical Anchor: This question is motivated by analyses in BKEE and VHE, which identify word segmentation ambiguity and complex morphology as key challenges for Event Extraction in Vietnamese. • Proposed Approach: We propose incorporating Vietnamese-specific external knowledge into the SLM framework. This integration enables the model to better structural handle complexities without relying solely on potentially errorprone automated segmenters. ambiguities boundary and • Validation Strategy: The approach will be validated by benchmarking the framework on standard Vietnamese EE datasets. In addition, ablation studies will be conducted the performance gains contributed by the Vietnamese-specific external knowledge compared to a baseline SLM. to quantify VI. FUTURE RESEARCH PLANS Building upon the proposed knowledge-augmented SLM framework, our future work will proceed in three critical directions to establish a robust Event Extraction solution for Vietnamese: 1. We plan to conduct a comprehensive survey of recent LLM-based EE approaches, including those identified in our literature review (e.g.,,,,) and other relevant studies utilizing LLMs. This will systematically analyze their strengths and weaknesses, and provide a solid comparative baseline and justifying the definitive shift toward SLM-based method. 2. To address the critical data scarcity, a key focus will be the construction of new Vietnamese domain-specific EE datasets, with an emphasis on financial sector. We will establish comprehensive annotation guidelines and conduct manual labeling, ensuring dataset reliability through rigorous Inter-Annotator Agreement (IAA) evaluation. 3. Finally, we will experiment with and refine the framework by optimizing knowledge-aware prompting or the injection of structural embeddings into the SLM. This involves testing how to best integrate AMR/KGbased structural priors to mitigate Vietnamese-specific challenges, such as word segmentation ambiguities. Our goal is to empirically determine the most effective configuration that maintains a strong balance between extraction performance and computational efficiency. ACKNOWLEDGEMENT We acknowledge Ho Chi Minh City University of Technology (HCMUT), VNU-HCM for supporting this study. CONFLICT OF INTEREST The authors declare that there is no conflict of interest regarding the publication of this paper. REFERENCES the 2024 Kulkarni, A., & Dogra, V. (2024). Comprehensive survey of event extraction methods in natural language processing.

  Towards Resource-Constrained Event Extraction: A Knowledge-Augmented Framework for Overcoming Challenges in Vietnamese NLP · 2026 · DOI

• While our study demonstrated the effectiveness of lightweight document embedding and ML models in extracting relevant COPD documents, several limitations should be considered. First, the dataset used in this study was specific to COPD, and the generalizability of our findings to other clinical conditions remains to be evaluated. Future research should examine the applicability of this framework across a broader range of diseases and clinical document types. Second, the data were derived from a single institution (VUMC), which may limit external validity due to variations in clinical documentation practices across health care systems.

  Scalable Identification of Clinically Relevant Chronic Obstructive Pulmonary Disease Documents in Large-Scale Electronic Health Record Datasets With a Lightweight Natural Language Processing Model: Retrospective Cohort Study · 2026 · DOI

• Possible extensions include: - Development of a lightweight Python implementation usable as an agent skill. - Creation of curated datasets of triadic mappings across domains. - Empirical evaluation protocols measuring circularity reduction and downstream reasoning improvements. - Integration with emerging standards for agent memory and reasoning transparency. We anticipate that the core protocol will remain stable while its implementations and evaluation frameworks evolve.

  Triadic Semantic Mapper: A Relational Protocol for Grounding Meaning in Artificial Intelligence · 2026 · DOI

Most-cited papers in Topic Modeling

• Large language models encode clinical knowledge · Nature · 2023 · 3,027 citations
• A Survey on Evaluation of Large Language Models · ACM Transactions on Intelligent Systems and Technology · 2024 · 2,472 citations
• A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions · ACM Transactions on Information Systems · 2024 · 1,585 citations
• Unifying Large Language Models and Knowledge Graphs: A Roadmap · IEEE Transactions on Knowledge and Data Engineering · 2024 · 952 citations
• Lost in the Middle: How Language Models Use Long Contexts · Transactions of the Association for Computational Linguistics · 2024 · 944 citations
• Detecting hallucinations in large language models using semantic entropy · Nature · 2024 · 665 citations
• Adapted large language models can outperform medical experts in clinical text summarization · Nature Medicine · 2024 · 651 citations
• AI models collapse when trained on recursively generated data · Nature · 2024 · 567 citations
• Bias and Fairness in Large Language Models: A Survey · Computational Linguistics · 2024 · 518 citations
• Graph of Thoughts: Solving Elaborate Problems with Large Language Models · Proceedings of the AAAI Conference on Artificial Intelligence · 2024 · 406 citations

Most recent work

• A Survey on Retrieval-Augmented Text Generation for Large Language Models · ACM Computing Surveys · 2026
• Benchmarking contextual understanding for in-car conversational systems · Journal of Systems and Software · 2026
• KIS: COLIEE 2025 Task 4 Solver Using Japanese LLM · The Review of Socionetwork Strategies · 2026
• Accuracy and fairness of generative AI in automated essay scoring: Comparing GPT-4o, feature-based models, and human raters · Assessing Writing · 2026
• Metadata Augmentation Using NLP, Machine Learning and AI-Chatbots: A Comparison · Journal of Library Metadata · 2026
• Small Models Achieve Large Language Model Performance: Evaluating Reasoning-Enabled AI for Secure Child Welfare Research · Journal of Evidence-Based Social Work · 2026
• Reference-Based Retrieval-Augmented Unit Test Generation · ACM Transactions on Software Engineering and Methodology · 2026
• RAQAG : A framework for automatically generating Q&A datasets with retrieval-augmented generation · Knowledge-Based Systems · 2026
• Clinical Note Generation From Doctor-Patient Conversations Using Parameter-Efficient Fine-Tuning Large Language Models: Comparative Study · JMIR Medical Informatics · 2026
• CLINES: Clinical LLM-based Information Extractionand Structuring Agent · medRxiv · 2026

Related topics in Computer Science