Open research questions in Neural Networks and Applications
266 gap statements mined from Neural Networks and Applications papers in our 7.9M-paper local library, which holds 2,552 papers on the topic — drawn mostly from each paper's own stated research gap, future-work, challenge and limitation notes, and its abstract. The ones listed below are a selection still marked open; each names the study that raised it, with a DOI link where the paper has one.
Choosing where to publish on Neural Networks and Applications? See the ranked Computer Science journals.
What the literature leaves open
Existing methods have limitations in capturing low-dimensional structures within high-dimensional data. Prior work does not effectively process hierarchical compositional data. There is a need for a novel approach that integrates concepts from multiple fields to address the challenges of high-dimensional regression.
The challenge of optimizer convergence is mentioned. The challenge of favorable validation scores not consistently producing lower locked-test error is identified.
The paper identifies the challenge of training NNs, which requires solving a high dimensional, non-convex optimisation problem. The paper identifies the challenge of developing solid theory behind ML, which is imperative for the challenges faced in AI and the future performance of these methods.
The current Artificial Intelligence systems are still mostly statistical and lack a general and biologically inspired approach. There is a need for a more general approach to Artificial Intelligence that can master the final bastions of reasoning and understanding. The current systems are not able to challenge humans in a general sense.
Identifying an appropriate function space for deep neural networks remains a key open question.
However, despite its practical performance, the theoretical analysis of orthonormality in neural networks is still lacking; for example, how orthonormality affects the convergence of the training process.
Additionally, the evaluation does not account for potential hardware-specific constraints or noise effects in quantum computations. We are limited to a specific dataset and predefined PQC configurations, which may not generalize to other tasks or quantum hardware.
Practical insights on the effect of different encodings, ansätze and measurements in quantum and hybrid convolutional neural networks · 2026 · DOIFuture research could investigate the application of self-modelling to other types of neural networks. Future research could investigate the use of self-modelling in combination with other techniques, such as regularization. Future research could investigate the effects of self-modelling on the interpretability of neural networks.
The benefits of self-models are not well understood. The paper identifies a gap in the understanding of how self-modelling affects network complexity. The paper identifies a gap in the development of techniques for implementing self-modelling in neural networks.
The approach addresses the need for a framework to analyze uncertainty quantification schemes. The paper fills a gap in the literature by establishing the well-posedness and regularity of neural field equations with random data.
Further work is needed to formalize the expressive power of different CFN configurations and establish theoretical guarantees on their approximation capabilities.
Compositional Function Networks: A High-Performance Alternative to Deep Neural Networks with Built-in Interpretability · 2026 · DOIEach measure in this guide offers a unique lens on dynamics such as uncertainty, dependence, and predictability. Developing an intuitive grasp of how these measures interrelate and differ lays the foundation for richer, more integrative analyses of complex systems like the brain.
There is a lack of understanding of how hyperparameter settings affect the accuracy and efficiency of neural network approximation to partial differential equations. There is no universal rule for choosing hyperparameters, and the choice is problem-dependent.
Numerical study on hyperparameter settings for neural network approximation to partial differential equations · 2026 · DOIThe paper suggests that future research should focus on understanding the feature-space geometry of neural networks. The study highlights the need for further research on the lottery ticket hypothesis in more complex settings.
Toy Combinatorial Interpretability Models Reveal Lottery Tickets in Early Feature Space · 2026The study highlights the need for a more mechanistic understanding of the hypothesis.
Toy Combinatorial Interpretability Models Reveal Lottery Tickets in Early Feature Space · 2026Prior work has focused on classical scaling and permutation invariances in deep ReLU networks. The gap is that these invariances do not capture all the symmetries present in deep ReLU networks.
The Symmetries of Three-Layer ReLU Networks · 2026Traditional methods of monitoring and analysis are often not effective enough to detect incidents of aggressive behavior on the Internet. The difficulty of identifying cyberbullying is determined by its various forms, hidden nature, and distribution scale.
A Hybrid KAN-BiLSTM Transformer with Multi-Domain Dynamic Attention Model for Cybersecurity · 2025 · DOIThis work has introduced EPs as a novel classification framework that combines prototype-based reasoning, fuzzy representations, and physically inspired deformation mechanisms. The proposed approach departs from conventional classification paradigms by modeling patterns as collections of elastic components, enabling the adaptation of class representations to new instances through controlled deformation.
Further development of the DAD converter for superconducting perceptron-like neural networks. Experimental investigations of the Σ-neuron with improved parameters.
Observations are extremely scarce, - only 17 ordered five-lag input-target rows are used, - the study uses a fixed manual CNN under shuffled and rolling-origin…
TR-HEEP's scalability is limited by the target technology node and the available on-chip SRAM - The area occupied by the CORDIC unit mainly depends on the selected parameters - The converters from floating-to fixed-point and vice versa occupy a significant portion of the total…
TOXOS: Spinning Up Nonlinearity in On-Vehicle Inference with a RISC-V CORDIC Coprocessor · 2025 · DOIExploring the application of TOXOS in other domains - Investigating the potential of TOXOS in improving the performance of other machine learning models - Developing more efficient algorithms for nonlinear function…
TOXOS: Spinning Up Nonlinearity in On-Vehicle Inference with a RISC-V CORDIC Coprocessor · 2025 · DOIAnalyzing the higher-order terms in the expansion. Developing new training methods that take into account the evolution of the function space during training. Applying the spiked covariance expansion framework to other neural network architectures.
How does feature learning reshape the function space? · 2026Investigating continual and adaptive variants of DOODL. Developing tighter integration between operator estimation and geometric representation learning. Applying DOODL to a wide range of domains, including physics, engineering, and biology.
Geometric Dictionary Learning of Dynamical Systems with Optimal Transport · 2026Investigating the topology of fibers, including the number of connected components. Establishing the completeness of the symmetries beyond the generic three-layer bottleneck setting. Analyzing the symmetries in wider architectures.
The Symmetries of Three-Layer ReLU Networks · 2026
Most-cited papers in Neural Networks and Applications
- Learning representations by back-propagating errors · Nature · 1986 · 24,879 citations
- Reducing the Dimensionality of Data with Neural Networks · Science · 2006 · 17,209 citations
- Deep learning in neural networks: An overview · Neural Networks · 2015 · 14,639 citations
- Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) network · Physica D Nonlinear Phenomena · 2020 · 5,170 citations
- Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication · Science · 2004 · 3,286 citations
- Artificial neural networks: a tutorial · Computer · 1996 · 2,557 citations
- Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem · The Psychology of learning and motivation/The psychology of learning and motivation · 1989 · 2,451 citations
- A review on the long short-term memory model · Artificial Intelligence Review · 2020 · 1,906 citations
- Ensembling neural networks: Many could be better than all · Artificial Intelligence · 2002 · 1,616 citations
- Learning and development in neural networks: the importance of starting small · Cognition · 1993 · 1,141 citations
Most recent work
- Novel approach to time series forecasting based on memristive Hopfield Neural Network with hidden heterogeneous and homogeneous extreme multistability · Chaos Solitons & Fractals · 2026
- Multi-head neural operator for modeling interfacial dynamics · International Journal of Mechanical Sciences · 2026
- Discrete Neural Dynamics Combined With Kalman Filter for Cooperative Control of Multirobot Systems With Constraints · IEEE Transactions on Control Systems Technology · 2026
- Hierarchy of prediction errors shapes the learning of context-dependent sensory representations · bioRxiv · 2026
- Cooperative optimization of zeroing neural networks (ZNN) in dynamic systems: Evolution, advances, and applications · Neurocomputing · 2026
- When Deep Learning Meets Polyhedral Theory: A Survey · INFORMS journal on computing · 2026
- Nonparametric Control Koopman Operators · IEEE Transactions on Automatic Control · 2026
- Control of dynamical systems with neural networks · Nonlinear Dynamics · 2026
- From Global to Granular: Revealing IQA Model Performance via Correlation Surface · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2026
- Recurrent Stochastic Configuration Networks With Hybrid Regularization for Nonlinear Dynamics Modeling · IEEE Transactions on Cybernetics · 2026
Find a gap in your own Neural Networks and Applications sub-topic
This page shows what the Neural Networks and Applications literature already flags as unresolved. To narrow it to your specific question, search the Research Gap Finder: the search is free with a free account and lists the papers closest to your topic first. Unlocking that topic (50 credits, charged once) fills the comparison table from our 7.9M-paper local library and writes the gaps from its rows.
Open the Research Gap Finder →