Open research questions in Domain Adaptation and Few-Shot Learning

What the literature leaves open

• Moreover, while effective in single-task FT, it remains unclear how best to extend input- dependent adaptation to multi-task or continual learning, or how to merge the learned adapter into the base weights for sustainable deployment.

  Learning When to Adapt · 2026

• In this paper, we propose ODE-driven continual model merg- ing (ODE-M), a controllable framework for merging se- quentially arriving task-adapted models. Instead of treating each merge as a fixed algebraic update between isolated checkpoints, ODE-M formulates the update as a continu- ous trajectory in parameter space and uses lightweight first- order feedback to suppress loss-increasing motion along the path. By constructing barrier-aware trajectories and select- ing operating points through a utility-aware time schedule, ODE-M provides an explicit mechanism for balancing re- tained historical knowledge and incoming task expertise. Extensive experiments on standard continual model merg- ing benchmarks demonstrate consistent improvements over strong baselines across different CLIP ViT backbones and stream lengths. Future work will extend this trajectory- based perspective to larger language and multimodal mod- els, develop more efficient integration strategies to reduce merge-time overhead, and study more practical calibration regimes where only proxy, partial, or dynamically updated calibration data is available.

  Unlocking the Potential of Continual Model Merging: An ODE Perspective · 2026

• The difficulty of generalizing visual perceptual learning to unseen conditions. The challenge of explaining the role of training curriculum in VPL generalization. The need to account for inter-individual variability in visual learning.

  The curriculum effect in visual learning: The role of readout dimensionality · 2026 · DOI

• potential variability induced due to spatial frequency preferences across observers, - limited to a specific visual task, - no explicit comparison to other theories of generalization

  The curriculum effect in visual learning: The role of readout dimensionality · 2026 · DOI

• Conventional predictive models tend to degrade in performance when deployed in novel scenarios due to domain shifts. Single-device models are often difficult to generalize across different devices due to significant variations in structural parameters, operational environments, sensor configurations, and signal patterns.

  Enabling generalizable RUL prediction for equipment: a dual-dimensional continual learning approach based on deep Gaussian process · 2026 · DOI

• Further research on improving the efficiency of TAST, - Exploring the application of TAST to other domains

  TAST: Task-Aware Sparse Topology Learning for Multi-Task Recommendation · 2026 · DOI

• Existing architectures do not explicitly adapt fine-grained parameter connectivity to individual tasks - Fixed sharing architectures cannot provide an appropriate balance between knowledge transfer and task-specific modeling - Lack of flexible sharing structures for effective multi-task recommendation

  TAST: Task-Aware Sparse Topology Learning for Multi-Task Recommendation · 2026 · DOI

• Gains are larger in the one-shot regime, where su- pervision is scarcer, which is consistent with the design in- tent.

  Selective, Regularized, and Calibrated: Harnessing Vision Foundation Models for Cross-Domain Few-Shot Semantic Segmentation · 2026

• Ideal Distributions Definition 3.1 Model & Expert Ideals m / Qj, Q(e) Qm, Q(e) j

  Density-Ratio Losses for Post-Hoc Learning to Defer · 2026

• Fine-tuning large language models with limited data: A survey and practical guide.

  Does Becoming Exceptional at One Domain Reduce Transfer Elsewhere · 2026 · DOI

• Reinforcement learning (RL) is widely utilized in large language model training to improve targeted capabilities, yet how RL reshapes a model remains poorly understood.

  What Does an LLM Learn from Reinforcement Learning? A Mechanistic Interpretability Perspective with Fixed-SAE Track · 2026

• At rho = 100, class-balanced sampling degrades accuracy on every class group, including the tail classes it is designed to help, which we attribute to overfitting caused by extreme oversampling of scarce data; at rho = 50 this failure is confined to head and medium classes.

  Mini-batch Sampling Strategies for Long-Tailed Image Classification: An Empirical Study on CIFAR-100-LT · 2026

• Although originally proposed as an alternative to backpropagation, its behavior has not been systematically characterized.

  A diagnostic evaluation of neural networks trained with the error diffusion learning algorithm · 2026 · DOI

• Existing methods for out-of-distribution detection have limitations. There is a need for a new approach that can effectively detect and reject samples from novel classes.

  Balanced Hyperbolic Embeddings Are Natural Out-of-Distribution Detectors · 2026 · DOI

• At test time, the additional cost is limited to the final-layer hyperbolic mapping and distance computation; in our measurements, the forward-pass time increases by 1–2% on CIFAR-100 and ImageNet-100.

  Balanced Hyperbolic Embeddings Are Natural Out-of-Distribution Detectors · 2026 · DOI

• Investigating the application of transfer learning in new domains. Developing more effective methods for transferring knowledge between domains. Improving the interpretability of models in transfer learning.

  Transfer Learning in Data Science: A Comprehensive Review · 2026 · DOI

• The need for extensive labeled datasets and computational resources in conventional machine learning methods. The lack of effective methods for transferring knowledge between domains.

  Transfer Learning in Data Science: A Comprehensive Review · 2026 · DOI

• The issue of limited feature diversity and poor adaptability in weakly supervised learning. The limitation of few-shot learning in scenarios requiring massive labeled samples.

  Research on a Weakly Supervised Algorithm Based on Few-Shot Learning · 2026 · DOI

• Overall, the Weakly Supervised Few-Shot Feature Expansion algorithm and the Strong-Label Algorithm demonstrate clear advantages when dealing with few-shot learning tasks, class imbalance, and scarce data, particularly in terms of robustness and adaptability to outliers, enhancing both classification accuracy and 55 zeuspress.

  Research on a Weakly Supervised Algorithm Based on Few-Shot Learning · 2026 · DOI

• The lack of a stabilization method for middle-layer insertion under full fine-tuning. The need for a method to mitigate the performance degradation caused by naive middle-layer insertion. The gap in understanding the effects of middle-layer insertion on pre-trained language models.

  Analysis and Mitigation of Performance Degradation from Layer Insertions into the Middle of Pre-Trained Language Models · 2026 · DOI

• To explore the application of the proposed method to other self-supervised learning frameworks and datasets. To investigate the use of different mixing strategies and hyperparameters to further improve the performance of the proposed method.

  Mitigating Feature Overfitting in Barlow Twins via Mixed-Sample Regularization for Stable Long-Horizon Representation Learning · 2026 · DOI

• Feature overfitting is a significant problem in self-supervised learning, particularly in redundancy-reduction frameworks like Barlow Twins. The Barlow Twins framework performs well initially but deteriorates after prolonged training due to limited data diversity and overfitting to feature correlations.

  Mitigating Feature Overfitting in Barlow Twins via Mixed-Sample Regularization for Stable Long-Horizon Representation Learning · 2026 · DOI

• Data scarcity poses a serious threat to modern machine learning and artificial intelligence. The existing methods have various strengths and weaknesses, making it essential to develop new approaches.

  Few-Shot Multi-Task Learning of Linear Invariant Features with Meta Subspace Pursuit · 2026 · DOI

• The closed-set setting is irrational since unknown classes may appear at test time and would be classified into known classes. The challenges of inter-set interference and intra-set inconsistency in federated learning are not addressed in prior works.

  FedPD++: Enhanced Federated Open-Set Recognition with Parameter Disentanglement · 2026 · DOI

• Extending the proposed prototypical augmentation and alignment mechanisms to more complex cross-domain scenarios - Investigating dynamic prototype update strategies to adapt to non-stationary domain shifts in real-world applications - Integrating the proposed framework with transformer-based feature extractors

  Deep transferable label propagation with prototypical augmentation · 2026 · DOI

Most-cited papers in Domain Adaptation and Few-Shot Learning

• Understanding deep learning (still) requires rethinking generalization · Communications of the ACM · 2021 · 2,070 citations
• A survey on deep learning tools dealing with data scarcity: definitions, challenges, solutions, tips, and applications · Journal Of Big Data · 2023 · 867 citations
• A Comprehensive Survey of Continual Learning: Theory, Method and Application · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2024 · 795 citations
• Domain Generalization: A Survey · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2022 · 782 citations
• Generalizing to Unseen Domains: A Survey on Domain Generalization · IEEE Transactions on Knowledge and Data Engineering · 2022 · 734 citations
• Prior Guided Feature Enrichment Network for Few-Shot Segmentation · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2022 · 585 citations
• Deep Long-Tailed Learning: A Survey · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2023 · 579 citations
• SUSTAIN: A Network Model of Category Learning. · Psychological Review · 2004 · 506 citations
• An exemplar-based random walk model of speeded classification. · Psychological Review · 1997 · 434 citations
• A Review of Generalized Zero-Shot Learning Methods · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2022 · 307 citations

Most recent work

• Towards Accurate Model Selection in Deep Unsupervised Domain Adaptation · arXiv (Cornell University) · 2026
• CTJANet: A class-task joint-aware network for enhanced few-shot image classification · Information Processing & Management · 2026
• Preventing Data Leakage in Neural Decoding · bioRxiv · 2026
• Deep transfer learning based on cross-domain subsequence alignment and feature contribution interpretation for remaining useful life prediction · Reliability Engineering & System Safety · 2026
• Target self-guided framework for unsupervised domain adaptation · Pattern Recognition · 2026
• MLSD · The International FLAIRS Conference Proceedings · 2026
• Pseudo-Label Refinement for Multimodal Unsupervised Domain Adaptation · IEEE Transactions on Systems, Man, and Cybernetics: Systems · 2026
• A Visually Explainable Dynamic Similarity Network for Few-Shot Classification · Journal of Artificial Intelligence and Soft Computing Research · 2026
• Automated In-the-Wild Data Collection for Continual AI Generated Image Detection · 2026
• Beyond similarity: Mutual information-guided retrieval for in-context learning in VQA · Pattern Recognition · 2026

Related topics in Computer Science