Open research questions in Neural Networks and Applications
186 unresolved questions extracted from the limitations and future-work sections of 644 Neural Networks and Applications papers in our library. Each links back to the study that raised it.
What the literature leaves open
The dimensionality issue in optimal transport, where the sample complexity deteriorates poorly with dimension. The robustness issue in optimal transport, where the method needs to handle outliers and heavy-tailed data. The challenge of deriving a dimension-free sample complexity for the optimal transport problem.
E-ROBOT: a dimension-free method for robust statistics and machine learning via Schrödinger bridge · 2026 · DOIparametric inference using the robust Sinkhorn divergence Wε,λ, - applying Wε,λ as loss function to conduct parametric inference in statistics or generative modeling
E-ROBOT: a dimension-free method for robust statistics and machine learning via Schrödinger bridge · 2026 · DOIFurther research on the application of the proposed architecture to other dynamical systems, - investigation of the use of different basis functions for the approximation of the transfer operator
Further research can be done to relax the assumption of M-Lipschitz continuity of f0 on H - The application of the results to other optimization problems can be explored - The extension of the results to other types of vector spaces can be investigated
Lagrange multipliers and duality with applications to constrained support vector machine · 2026 · DOIexploring the application of ASBQ to other types of neural networks, - investigating the effectiveness of ASBQ on different datasets and tasks, - developing new quantization methods that can further improve the performance of BNNs
Adaptive multi-bit progressive quantization for stable training of binary neural networks · 2026 · DOIExisting methods fail to address both stability and expressiveness of BNNs simultaneously. A unified framework that simultaneously addresses training instability, representational degradation, and structural redundancy remains underexplored. Prior work has not effectively reduced the representational gap between full-precision and binary models.
Adaptive multi-bit progressive quantization for stable training of binary neural networks · 2026 · DOIOne limitation of our framework is that it implicitly assumes that the number of training samples is large relative to the dimension of the representation space, so that the sample estimates concentrate around their population counterparts.
Probing for Representation Manifolds in Superposition · 2026Transformers and state-space models (SSMs) are the two dominant families of sequence models, and a central open question is how far the analytical knowledge built for transformers transfers to SSMs.
What Does Layer-Importance Reveal About Transformers and State-Space Models? · 2026However, major branches of CT machine learning have matured in separate research communities, leaving their mathematical relationships and design trade-offs insufficiently characterized.
Continuous-Time Machine Learning: A Unified Mathematical Perspective · 2026Finally, we identify open challenges in approximation theory, training stability, hardware-efficient implementations, benchmarking, foundation models, and scientific machine learning, and discuss an agenda for future research.
Continuous-Time Machine Learning: A Unified Mathematical Perspective · 2026How sensory information is interpreted depends on context, yet the neural mechanisms by which context shapes sensory processing remain poorly understood.
Hierarchy of prediction errors shapes the learning of context-dependent sensory representations · 2026 · DOIThe quantum nature of these results remains to be understood, and existing theoretical approaches fail to capture the observed phenomenology in the far-from-equilibrium regime of driven longitudinal fields.
Magnetic memory and hysteresis from quantum transitions in theory and experiments on quantum annealers · 2026 · DOIThe paper identifies a gap in the understanding of PQCs for satellite image classification. The paper highlights the need for further research on the choice of encoding technique, ansätze, and measurement strategy.
Practical insights on the effect of different encodings, ansätze and measurements in quantum and hybrid convolutional neural networks · 2026 · DOIAdditionally, the evalu- ation does not account for potential hardware-specific constraints or noise effects in quantum computations. We are limited to a specific dataset and predefined PQC configurations, which may not generalize to other tasks or quantum hardware.
Practical insights on the effect of different encodings, ansätze and measurements in quantum and hybrid convolutional neural networks · 2026 · DOIThe paper identifies a gap in traditional prototype-based classification approaches, which do not integrate concepts from multiple domains. The paper also identifies a need for a more interpretable and adaptive classification approach.
Future research could investigate the application of self-modelling to other types of neural networks. Future research could investigate the use of self-modelling in combination with other techniques, such as regularization. Future research could investigate the effects of self-modelling on the interpretability of neural networks.
The benefits of self-models are not well understood. The paper identifies a gap in the understanding of how self-modelling affects network complexity. The paper identifies a gap in the development of techniques for implementing self-modelling in neural networks.
Reducing the training cost, particularly for two-step training - Implementing the AMORE framework for DeepONet with network configurations such as a CNN in a branch network with multiple outputs - Extending the AMORE framework for physics-informed operators network with multiple outputs -
The lack of a reliable operator learning strategy for neural operators (DeepONets) in stiff chemical kinetics. The need for a framework that can effectively reduce the error for multiple output state variables. The challenge of enforcing unity mass-fraction constraint exactly in neural operator training.
The paper identifies a gap in the understanding of neural field equations with random data. The approach addresses the need for a framework to analyze uncertainty quantification schemes. The paper fills a gap in the literature by establishing the well-posedness and regularity of neural field equations with random data.
The system is simulated, not physically implemented. The dataset used is limited to handwritten digits. The system's performance is compared to that of a Restricted Boltzmann Machine implemented on CPU and GPU platforms, but not to other types of neural networks.
Unsupervised and probabilistic learning with Contrastive Local Learning Networks: The Restricted Kirchhoff Machine · 2026 · DOITo physically implement the Restricted Kirchhoff Machine and test its performance on a variety of tasks. To compare the performance of the Restricted Kirchhoff Machine to that of other types of neural networks. To explore the potential applications of the Restricted Kirchhoff Machine in distributed and energy-efficient learning systems.
Unsupervised and probabilistic learning with Contrastive Local Learning Networks: The Restricted Kirchhoff Machine · 2026 · DOIThe inherent opacity of DNNs. The need for transparency and accountability in high-stakes domains. The limitation of existing interpretable approaches to simple additive structures.
Compositional Function Networks: A High-Performance Alternative to Deep Neural Networks with Built-in Interpretability · 2026 · DOIExisting interpretable approaches are limited to simple additive structures. There is a need for a novel framework that can build inherently interpretable models with complex feature interactions.
Compositional Function Networks: A High-Performance Alternative to Deep Neural Networks with Built-in Interpretability · 2026 · DOIGranger Causality marks the final measure covered in this review, and thus the conclusion of the in-depth coverage of the eleven information-theoretic time-series measures. Collectively, we have presented a unified and accessible guide to these eleven key measures for analyzing time series data, with a particular focus on their applications in computational neuroscience. Through a novel schematic overview (with measure-specific decompositions in each section) and a systematic tabular summary, we have sought to clarify conceptual distinctions between measures—such as directionality, temporal dependence, and dimensional scope—and to bridge terminology across disparate literatures. Our aim has been not only to consolidate fundamental definitions but also to offer an integrative perspective that enables researchers with a wide range of background knowledge depths to navigate this methodological landscape with greater fluency. While our examples and emphasis here have focused on neural time-series data, we note that all measures are broadly applicable across diverse scientific domains. Resources like the UEA/UCR Time Series Classification repository [120, 121] provide compelling examples of diverse time-series datasets—from beef spectrograms to yoga pose angles to goose vocalizations—that could benefit from this conceptual framework to yield new insights into how information-theoretic measures relate to each other when computed on diverse empirical data. Each measure in this guide offers a unique lens on dynamics such as uncertainty, dependence, and predictability. Developing an intuitive grasp of how these measures interrelate and differ lays the foundation for richer, more integrative analyses of complex systems like the brain. We hope this guide serves as a starting point, upon which future work may extend this foundation by incorporating additional measures, refining estimation techniques, or exploring emerging applications.
Most-cited papers in Neural Networks and Applications
- Learning representations by back-propagating errors · Nature · 1986 · 24,879 citations
- Reducing the Dimensionality of Data with Neural Networks · Science · 2006 · 17,209 citations
- Deep learning in neural networks: An overview · Neural Networks · 2015 · 14,639 citations
- Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication · Science · 2004 · 3,286 citations
- Artificial neural networks: a tutorial · Computer · 1996 · 2,557 citations
- Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem · The Psychology of learning and motivation/The psychology of learning and motivation · 1989 · 2,451 citations
- Ensembling neural networks: Many could be better than all · Artificial Intelligence · 2002 · 1,616 citations
- Learning and development in neural networks: the importance of starting small · Cognition · 1993 · 1,141 citations
- Long memory relationships and the aggregation of dynamic models · Journal of Econometrics · 1980 · 1,097 citations
- Self-organization in a perceptual network · Computer · 1988 · 1,093 citations
Most recent work
- Hierarchy of prediction errors shapes the learning of context-dependent sensory representations · bioRxiv · 2026
- Function aligns with geometry in locally connected neuronal networks · bioRxiv · 2026
- High-frequency spike inference with particle Gibbs sampling · eLife · 2026
- Precise calcium-to-spike inference using biophysical generative models · bioRxiv · 2026
- Better Neural Network Expressivity: Subdividing the Simplex · 2026
- Next-Generation Neural Mass Models Reproduce Features of Speech Processing · bioRxiv · 2026
- RAP: A reliability-aware pruning framework for deep neural networks · Journal of Systems Architecture · 2026
- Multi-Scale Degradation Trend Perception for Voltage Degradation Prediction of Proton Exchange Membrane Fuel Cells · Batteries · 2026
- Neural networks and deep learning: part II · Machine Learning · 2026
- Multi-resolution enhancement for full-spectrum neural representations · Nature Machine Intelligence · 2026
Find a gap in your own Neural Networks and Applications sub-topic
This page shows what the Neural Networks and Applications literature already flags as unresolved. To narrow it to your specific question, run the guided finder — it searches the gap library on demand and checks candidates against 250M+ OpenAlex works.
Open the Research Gap Finder →