The development of more comprehensive evaluation methods

The gap

Evidence profile

Stated in the limitations and future work and cells future research sections of the source papers, classified as general, spanning 5 journals.

Supporting evidence — 5 representative gaps

• Systematic Hyperparameter Optimization of Convolutional Neural Networks for Pneumonia Detection from Chest X-rays (2026) · Periodica Polytechnica Electrical Engineering and Computer Science · doi

  One of the main weaknesses of this study is that it uses only one dataset, which prevents the comparison of the model with different patient demographics. The existing data distribution is extremely biased towards adult groups, 10|Nwohiri et al. Period. Polytech. Elec. Eng. Comp. Sci. i.e., the fitted hyperparameters will not perfectly apply to pediatric patients. Moreover, given that the process of data division was done on an image-by-image basis but not on a patient-by- patient basis, two or more radiographs of the same patient can be found on the training and testing sets. External validation should take precedence in future work to tackle the limitations inherent in the use of single datasets. The optimized hyperparameter settings should be tested using independent X-ray data that are acquired from other hospitals to determine whether they are diag- nostic in real clinical setting. Scalability is another consideration because training deep learning models like InceptionV3 is resource- and time-intensive. Future research should focus on the use of hyperparameter tuning and less resource-intensive CNN architectures that train faster.

  generalstated in limitationsevidence 5/5
  Keywords: patient image basis training future hyperparameter resource intensive main weaknesses uses dataset prevents comparison model
• A patient-aware benchmarking of CNN and transformer architectures for breast cancer histopathology classification (2026) · Frontiers in Digital Health · doi

  observed differences were This study establishes a rigorous, leakage-free benchmarking for breast cancer histopathology classification, framework systematically evaluating nine state-of-the-art deep learning architectures. By enforcing a strict patient-aware cross-validation protocol, this work addresses a critical flaw in prior literature: the artificial inflation of model performance due to patient-level data leakage. Our comprehensive quantitative and statistical analyses reveal a fundamental insight: when evaluated under highly controlled, patient-isolated conditions, no statistically across modern significant Convolutional Neural Networks (e.g., ResNet50, DenseNet121) and Vision Transformers (e.g., Swin-Small). While DenseNet121 and ResNet50 demonstrated exceptional stability and high recall, achieving over 92% mean accuracy across 5-fold and 10-fold evaluations, hypothesis incremental architectural complexity did not yield statistically significant differences the magnification-wise analysis highlighted the critical role of spatial context. All models achieved their peak performance at lower magnifications is suffered a measurable decline at higher preserved, and), where limited fields of view obscure magnifications (400 contextual relationships. Ultimately, this study demonstrates that the integrity of the evaluation protocol and data partitioning strategy plays a more critical role in enabling reliable real-world deployment than the choice of baseline architecture. in diagnostic performance. Furthermore, tissue architecture testing confirmed), where global that (40 � � Based on these findings and the recognized limitations of single-scale patch classification, several promising avenues for future research emerge: Instance Learning (MIL) and multi-resolution fusion frameworks (such as Graph Neural Networks or state space models like SlideMamba). These architectures can simultaneously leverage fine-grained cellular atypia at high magnifications and global structural organization at low magnifications, mimicking the holistic approach of a human pathologist. • Cross-Domain Generalization and Multi-Centric Validation: While the models exhibited strong stability on the BreaKHis dataset, histopathological images are highly susceptible to domain shifts caused by variations in tissue preparation, staining protocols (H&E), and digital should prioritize external validation across diverse, multi-centric cohorts (e.g., TCGA or CAMELYON) and integrate domain adaptation or color normalization techniques to ensure robust real-world generalization. scanner calibrations.

  generalstated in future workevidence 5/5
  Keywords: magni cations cation patient validation critical performance across models tissue multi domain differences leakage classi
• The Role of Weight Initialization and Preprocessing Techniques in Analyzing Chest X-ray Images with Deep Neural Networks: A Comparative Study (2026) · DÜMF Mühendislik Dergisi · doi

  The dataset used in this study, although sufficient for initial experimentation, is relatively small and specific to pediatric patients from a single medical center. This limits the generalizability of the findings to broader populations and different age groups. Also, the dataset’s demographic homogeneity may not capture the variability in CXR images that could arise from different ethnicities, geographical locations, or adult populations. The inherent stochastic nature of deep learning training processes posed difficulties in consistently evaluating model performance. Variability in results necessitated the introduction of additional metrics to provide a more robust measure of performance. We ensured reproducibility and stability of the results across different runs by carefully managing random seeds and training conditions. We encountered significant computational demands when training deep learning models, especially with fine-tuning and multiple preprocessing techniques, and limited access to high-performance computing facilities slowed down the experimentation process. While several preprocessing techniques were applied, we did not explore all possible methods. There may be other techniques that could further enhance image quality and model performance. Additionally, some preprocessing methods might not be applicable to CXR images as they could alter the inherent nature of the data. For example, techniques could inadvertently obscure or modify critical diagnostic features, such as white shadows that are indicative of certain pathologies in CXRs. We evaluated the impact of preprocessing incorporating data augmentation, which is commonly used in practice to enhance model robustness. color distributions techniques without adjust that The study focused on EfficientNet B1 and did not extensively compare other state-of-the-art architectures. While EfficientNet is known for its performance, other models might offer different advantages. The decision to limit the comparison was influenced by processing load considerations and computational constraints, which restricted the scope of the study. The ROI selection process relied on an object detection model trained on a small subset of labeled data. This might introduce bias or errors if the object detection model is not generalizable to all types of CXR images. Also, images where the rib cage was not detected were included in the dataset without modifications, potentially affecting the consistency of the dataset.

  generalstated in limitationsevidence 5/5
  Keywords: model performance techniques dataset different images preprocessing training used experimentation small populations variability inherent nature
• Evaluating the Accuracy of Artificial Intelligence Tools in Analyzing X-Ray Images for Detecting Respiratory Diseases: A Comparative Study of Recent Research (2026) · Humanitarian and Natural Sciences Journal · doi

  The development of more comprehensive evaluation methods for artificial intelligence systems. - The investigation of the impact of data diversity on the performance of deep learning models. - The exploration of the application of deep learning models to other medical imaging modalities.

  generalstated in cells future researchevidence 5/5
  Keywords: development comprehensive evaluation methods artificial intelligence systems investigation
• Clinical implementation of an Xception-based deep learning system for multiclass tuberculosis detection on chest X-ray images (2026) · The Egyptian Journal of Radiology and Nuclear Medicine · doi

  Further studies are needed to evaluate the performance of the model in different populations. - Further studies are needed to compare the performance of the model with other deep learning architectures. - Further studies are needed to assess the clinical impact of the model in resource-constrained settings.

  generalstated in cells future researchevidence 4/5
  Keywords: further studies needed evaluate performance model different populations

Questions about this gap