Open research questions in Gaussian Processes and Bayesian Inference

What the literature leaves open

• Future research can focus on extending the methodology to multiple particle observations. Future research can explore the application of the methodology to various real-world problems.

  A Fourier-Based Inference Method for Learning Interaction Kernels in Particle Systems · 2026 · DOI

• The problem of inferring the interaction kernel of stochastic interacting particle systems is not well addressed. The development of efficient inference methodologies for interacting particle systems is of great current interest.

  A Fourier-Based Inference Method for Learning Interaction Kernels in Particle Systems · 2026 · DOI

• The lack of efficient methods for approximating the possibilistic output of inferential models.

  An efficient Monte Carlo method for valid prior-free possibilistic statistical inference · 2026 · DOI

• The simulation model is itself an imperfect approximation of real-world processes. The lack of integration of simulation data and real-world measurement data during surrogate training.

  Bayesian surrogate training on multiple data sources: a hybrid modeling strategy · 2026 · DOI

• Limited number of sensors that can be deployed. Need for accurate spatial mapping of pollutant concentrations.

  Efficient Uncertainty-Guided Sensor Placement for Air Quality Mapping using Gaussian Processes and Spatial Indexing · 2026 · DOI

• Existing automated approaches struggle in high dimensions for two bottlenecks: their kernel search space is limited to additions and multiplications of base kernels, and LLM-based approaches require conditioning on raw observations, which becomes infeasible due to context-length limits and the difficulty of extracting meaningful patterns.

  Automated Kernel Discovery Towards Understanding High-dimensional Bayesian Optimization · 2026

• The lack of robustness of standard SBI algorithms to model mis-specification. The need for a convergence guarantee for SBI algorithms. The need for a principled approach to tuning parameter selection.

  Robust Simulation Based Inference Through Robust Optimal Transport · 2026

• The choice of priors and deployment-time computation methods for various outcome types needs to be investigated. Uncertainty quantification without loss of predictive power for such models, however, might demand alternative approaches beyond the scope of this work. Several methodological extensions warrant further investigation.

  Progression to the mean: A practical Bayesian workflow for the development and deployment of clinical prediction models · 2026

• Existing approaches rely on Bayesian methods or focus primarily on point estimation. Existing approaches have limitations, including computational burden.

  Uncertainty-Aware Ideal Point Estimation via Variational EM · 2026

• The experiments are limited to synthetic and real-world datasets with fleet sizes from 16 to 100 agents.

  Federated Gaussian Process Learning via Pseudo-Representations for Large-Scale Multi-Robot Systems · 2026 · DOI

• Future research can focus on extending pxpGP to asynchronous federated GP formulations. The method can be applied to other domains such as cooperative exploration and model-based control.

  Federated Gaussian Process Learning via Pseudo-Representations for Large-Scale Multi-Robot Systems · 2026 · DOI

• Surrogate models for topology optimization (TO) exhibit highly variable out-of-distribution (OOD) generalization under distribution shifts such as changing loads or boundary conditions, yet the source of this variability remains unclear.

  On the Generalization in Topology Optimization via Sensitivity-Conditioned Bernoulli Flow Matching · 2026

• The need to tune many options and address challenges for each step of a Bayesian model. The vulnerability of Bayesian neural networks to adversarial attacks.

  A Primer on Bayesian Neural Networks: Review and Debates · 2026 · DOI

• The lack of interpretability and uncertainty estimation in conventional neural networks. The vulnerability of conventional neural networks to adversarial attacks. The need for a comprehensive primer on Bayesian neural networks.

  A Primer on Bayesian Neural Networks: Review and Debates · 2026 · DOI

• In this work, we adopted the standard KS algorithm for convergence reconstruction; however, a systematic comparison between shear-based inference and convergence-based results derived from alternative reconstruction techniques is still lacking and should be pursued in future studies.

  The shear-to-cosmology paradigm I. Hybrid field-level and simulation-based framework for weak-lensing surveys · 2026 · DOI

• An extension of copula SSMs is needed, allowing for multivariate latent states. Furthermore, extending copula SSMs by allowing for covariates in the dependence structure would allow for time-varying dependencies between the observations. In Hanebeck (2025) and Hanebeck et al. (2025), we introduce these two extensions and show their observability using the method introduced in the present paper.

  On the Observability of Copula State Space Models using a Bayesian Approach · 2026 · DOI

• The lack of a publicly available code for performing global fits of PDFs with a fixed polynomial parameterisation. The need to assess the impact of aN3LO corrections on NNLO PDF sets.

  Assessing the impact of fitting methodology at aN3LO with FPPDF: an open source tool for extracting Parton distribution functions in the Hessian approach · 2026 · DOI

• Improving the verifier to handle kernels that do not pass the checks for positive semi-definiteness or dimension-agnosticity.

  Automated Kernel Discovery Towards Understanding High-dimensional Bayesian Optimization · 2026

• To investigate the use of other optimization algorithms. To apply the algorithm to real-world problems.

  Robust Simulation Based Inference Through Robust Optimal Transport · 2026

• The study focuses on numerical experiments only. Limited to specific non-Gaussian processes. No real-world dataset or large-scale application is mentioned.

  Tweedie's Formulae and Diffusion Generative Models Beyond Gaussian · 2026

• To further develop the FPPDF code to include additional features and improvements. To apply the FPPDF code to other areas of research, such as precision studies at current and future accelerator facilities.

  Assessing the impact of fitting methodology at aN3LO with FPPDF: an open source tool for extracting Parton distribution functions in the Hessian approach · 2026 · DOI

• Further development of the proposed approaches to improve predictive accuracy and predictive coverage. Application of the approaches to various fields, including hydrology and systems biology.

  Bayesian surrogate training on multiple data sources: a hybrid modeling strategy · 2026 · DOI

• Validation on real urban data. Investigation of batch Bayesian optimization for global designs. Development of adaptive sensing strategies for real-time air quality monitoring.

  Efficient Uncertainty-Guided Sensor Placement for Air Quality Mapping using Gaussian Processes and Spatial Indexing · 2026 · DOI

• Underexplored non-Gaussian diffusion models - Deriving Tweedie's formulae for non-Gaussian processes - Applying denoising score-matching objectives to practical…

  Tweedie's Formulae and Diffusion Generative Models Beyond Gaussian · 2026

• Investigating the use of the proposed method for other types of distributions. Improving the efficiency of the method. Applying the method to real-world problems.

  Transport Quasi-Monte Carlo · 2026 · DOI

Most-cited papers in Gaussian Processes and Bayesian Inference

• Bayesian model averaging: a tutorial (with comments by M. Clyde, David Draper and E. I. George, and a rejoinder by the authors · Statistical Science · 1999 · 2,149 citations
• Benchmark priors for Bayesian model averaging · Journal of Econometrics · 2001 · 702 citations
• Conformal Prediction: A Gentle Introduction · Foundations and Trends® in Machine Learning · 2023 · 618 citations
• Bayesian Computation and Stochastic Systems · Statistical Science · 1995 · 598 citations
• An MCMC approach to classical estimation · Journal of Econometrics · 2003 · 496 citations
• MCMC Methods for Functions: Modifying Old Algorithms to Make Them Faster · Statistical Science · 2013 · 476 citations
• UltraNest - a robust, general purpose Bayesian inference engine · The Journal of Open Source Software · 2021 · 463 citations
• Approximating probabilistic inference in Bayesian belief networks is NP-hard · Artificial Intelligence · 1993 · 397 citations
• Uncertainty quantification in scientific machine learning: Methods, metrics, and comparisons · Journal of Computational Physics · 2023 · 394 citations
• On Particle Methods for Parameter Estimation in State-Space Models · Statistical Science · 2015 · 355 citations

Most recent work

• A review of machine learning with small and limited data · Journal Of Big Data · 2026
• Is K-fold cross validation the best model selection method for machine learning? · Information Fusion · 2026
• Colibri: A new tool for fast-flying PDF fits · The European Physical Journal C · 2026
• L4acados: Learning-Based Models for acados, Applied to Gaussian Process-Based Predictive Control · IEEE Transactions on Control Systems Technology · 2026
• A Primer on Bayesian Neural Networks: Review and Debates · Statistical Science · 2026
• Frequency-domain approach to automated and efficient multivariate kernel density estimation for probabilistic modeling · Reliability Engineering & System Safety · 2026
• Nonlinear Bayesian Filtering With N atural Gr a dient Gaussia n Appr o ximation · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2026
• Sparse Variational Information Bottleneck Gaussian Processes for Uncertainty Estimation · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2026
• An efficient Monte Carlo method for valid prior-free possibilistic statistical inference · Journal of the American Statistical Association · 2026
• HGNJODE: A Hierarchical Gated Neural Jump Ordinary Differential Equation for spatio-temporal event prediction · Advanced Engineering Informatics · 2026

Related topics in Computer Science