Open research questions in Explainable Artificial Intelligence (XAI)

What the literature leaves open

• further research is needed to develop explainability mechanisms that provide genuine interactive control, - future studies should investigate the impact of HCAI principles on AIoT systems

  Explainable and Human-Centered AIoT: A Systematic Review of Integration, Interaction, and Impact · 2026 · DOI

• There is a lack of explainability and human accountability in AIoT systems. Most AIoT implementations offer only superficial or transient explainability. There is a need to establish explainability by design as an inherent architectural requirement.

  Explainable and Human-Centered AIoT: A Systematic Review of Integration, Interaction, and Impact · 2026 · DOI

• Traditional RUL prediction methods are primarily categorized into physics-based models and data-driven approaches. There is a need to balance prediction accuracy with model transparency. The paper identifies a gap in the existing literature regarding the application of SHAP-based explainable machine learning in RUL prediction.

  Application of SHAP-based explainable machine learning in remaining useful life prediction for aircraft engine systems · 2026 · DOI

• further research on improving the interpretability and transparency of deep learning models, - exploring the application of AFFA in other natural language processing tasks

  An AFFA-Integrated Hybrid Deep Learning Framework for Explainable Sentiment Analysis · 2026 · DOI

• The increasing architectural complexity of deep learning models often obscures their internal decision-making processes, raising concerns regarding interpretability and transparency. The need for an effective mechanism to improve both robustness and reliability in sentiment analysis tasks. The potential for inconsistencies between labels and comments to introduce noise and negatively affect model performance.

  An AFFA-Integrated Hybrid Deep Learning Framework for Explainable Sentiment Analysis · 2026 · DOI

• Fusing philosophical theory with mechanistic evidence to explore how AI understanding aligns with—and diverges from—our own, - Developing techniques to automatically extract functional circuits for specific prompts from frontier models

  Mechanistic indicators of understanding in large language models · 2026 · DOI

• The gap between the deflationary view of LLMs and the recent findings in mechanistic interpretability. The lack of a theoretical account of understanding that integrates these findings. The need for a framework that distinguishes between different levels of understanding in LLMs.

  Mechanistic indicators of understanding in large language models · 2026 · DOI

• Temporal dynamics and multivariate interactions in time series data, - Lack of ground-truth temporal annotations for real-world datasets, - Scalability of inherently interpretable architectures to higher-order or high-dimensional settings

  When, how long and how much? Interpretable neural networks for time series regression by learning to mask and aggregate · 2026 · DOI

• real-world open-source datasets with ground-truth temporal annotations remain unavailable, - manual labeling requires extensive domain expertise and is prohibitively expensive

  When, how long and how much? Interpretable neural networks for time series regression by learning to mask and aggregate · 2026 · DOI

• Mental models in human–AI interaction have been studied with diverse theoretical underpinnings and methodologies, but none of the provided studies systematically evaluate how mental model accuracy or alignment with actual system capabilities affects user interaction outcomes (trust, reliance appropriateness, task performance) across different AI application domains.

  Users’ mental models within human–AI interaction: a systematic scoping review · 2026 · DOI

• Further research is needed to develop algorithms for minimizing the unified variational objective, - The study of trade-offs between predictive accuracy and structural requirements needs further investigation, - The application of the unified framework to real-world problems needs to be explored

  A Unified Variational Principle for Reliable Machine Learning · 2026 · DOI

• Artificial intelligence models are increasingly scaled to improve predictive accuracy, yet it remains unclear whether scale improves the quality of post-hoc explanations.

  Scaling vision models does not consistently improve localisation-based explanation quality · 2026 · DOI

• However, despite their widespread adoption, the reliability of LLMs in supporting statistically rigorous procedures has not been systematically evaluated, posing risks for unexamined or overly optimistic use.

  Can Large Language Models (LLMs) be Trusted for Power Analysis? An Empirical Evaluation · 2026 · DOI

• The study intentionally excluded treatment variables because of lack of standardization ML Cheilitis.

  Interpreting Machine Learning in Rare Outcomes · 2026 · DOI

• ” The authors should be commended for addressing an important and understudied question: whether readily available clinical and demographic descriptors can be leveraged through machine learning (ML) to support prognostication in actinic cheilitis (AC).

  Interpreting Machine Learning in Rare Outcomes · 2026 · DOI

• Existing artificial intelligence (AI)-based approaches are further limited by their “black-box” nature and insufficient depth in multimodal fusion.

  An intelligent evaluation model of dance artistic expressiveness integrating multimodal information and explainable artificial intelligence · 2026 · DOI

• While XAI is well established in domains like healthcare or finance, its application in sports science remains fragmented and underexplored.

  A scoping review of explainable artificial intelligence in sports science · 2025 · DOI

• In light of these results, it remains unclear for practitioners how to use these methods and choose between them in a principled way.

  Impossibility theorems for feature attribution · 2024 · DOI

• SSAI has not been investigated in a cloud-based setting for generating explainable insights from these new types of Big Data.

  A cloud-based architecture for explainable Big Data analytics using self-structuring Artificial Intelligence · 2024 · DOI

• To explore the use of other explainability techniques, such as saliency maps or feature importance. To apply the results to other jet tagging tasks, such as top tagging or W/Z tagging. To investigate the use of other machine learning architectures, such as transformers or recurrent neural networks.

  The physics behind ML-based quark-gluon taggers · 2026 · DOI

• The physical basis of the network's decisions is unclear. The standard implementation of Shapley values assumes independent inputs and can lead to distorted attributions in the presence of correlations.

  The physics behind ML-based quark-gluon taggers · 2026 · DOI

• The lack of transparency and interpretability in AI-driven marketing automation systems. The need for XAI frameworks that can enhance model transparency without compromising predictive accuracy.

  Designing Explainable AI Based Marketing Automation Architectures for Healthcare and Financial Applications · 2026 · DOI

• Current research remains disease-specific or concentrates on a single interpretability level. Most studies do not easily transfer across different domains.

  Explainable artificial intelligence for cross domain evaluation of predictive models in multi-disease diagnosis · 2026 · DOI

• Scalability and real-time processing remain critical challenges. Maintaining model accuracy in the presence of noisy data. Addressing ethical concerns, such as algorithmic bias and fairness.

  Advancements in Machine Learning Algorithms for Big Data Analytics · 2026 · DOI

• The need for refining hybrid approaches and evaluating their applicability in real-world scenarios. The gap in addressing ethical concerns, such as algorithmic bias and fairness.

  Advancements in Machine Learning Algorithms for Big Data Analytics · 2026 · DOI

Most-cited papers in Explainable Artificial Intelligence (XAI)

• Explanation in artificial intelligence: Insights from the social sciences · Artificial Intelligence · 2018 · 4,019 citations
• Explainable Artificial Intelligence (XAI): What we know and what is left to attain Trustworthy Artificial Intelligence · Information Fusion · 2023 · 1,623 citations
• Machine Learning: An Applied Econometric Approach · The Journal of Economic Perspectives · 2017 · 1,603 citations
• Choosing Prediction Over Explanation in Psychology: Lessons From Machine Learning · Perspectives on Psychological Science · 2017 · 1,535 citations
• Explainable AI: from black box to glass box · Journal of the Academy of Marketing Science · 2019 · 1,161 citations
• Techniques for interpretable machine learning · Communications of the ACM · 2019 · 1,032 citations
• The Mythos of Model Interpretability · Communications of the ACM · 2018 · 1,018 citations
• Artificial Intelligence and Black‐Box Medical Decisions: Accuracy versus Explainability · The Hastings Center Report · 2019 · 760 citations
• Interpretable machine learning: Fundamental principles and 10 grand challenges · Statistics Surveys · 2022 · 736 citations
• Explaining individual predictions when features are dependent: More accurate approximations to Shapley values · Artificial Intelligence · 2021 · 735 citations

Most recent work

• SΔϕ-62 — World Model Kernel: Observed Trace, Inference, UMR, Binding Status, and Revision Path Protocols (v1.1, AI-Readable Kernel Package) · Zenodo (CERN European Organization for Nuclear Research) · 2026
• SΔϕ-62 — World Model Kernel: Trace–UMR–Binding Protocol for AI Inference · Zenodo (CERN European Organization for Nuclear Research) · 2026
• Explainable AI-Driven Quality and Condition Monitoring in Smart Manufacturing · Sensors · 2026
• Reasoning Under Load · 01: Claude Opus 4.8 — An Independent Reasoning-Integrity Evaluation · Zenodo (CERN European Organization for Nuclear Research) · 2026
• Explainable artificial intelligence (XAI): From inherent explainability to large language models · Array · 2026
• Black Box Assessment: rethinking integrity and learning for a time of Generative AI · Assessment & Evaluation in Higher Education · 2026
• The missing link for AI at work: Towards an organization-centered approach to large language models’ explainability · Technology in Society · 2026
• Human-in-the-Loop Large Language Model–Augmented Diagnostic Reasoning in Thoracic Imaging: Impact of Radiologic Expertise · American Journal of Roentgenology · 2026
• Interpretable machine learning models for stroke risk prediction in patients with newly diagnosed atrial fibrillation · npj Digital Medicine · 2026
• Dimensions of AI de-anthropomorphization based on impression integration theory · Technology in Society · 2026

Related topics in Computer Science