Science AI Journal
Computer Science · Research topic

Open research questions in Web Data Mining and Analysis

77 gap statements mined from Web Data Mining and Analysis papers in our 4.5M-paper local library, which holds 759 papers on the topic — drawn mostly from each paper's own stated research gap, future-work, challenge and limitation notes, and its abstract. The ones listed below are a selection still marked open; each names the study that raised it, with a DOI link where the paper has one.

Choosing where to publish on Web Data Mining and Analysis? See the ranked Computer Science journals.

What the literature leaves open

  • Limited scalability and robustness of existing rule-based and semi-automated approaches. Need for dynamic adaptation to heterogeneous document structures.

    A Self-Adaptive LLM-Based Framework for Automated Extraction and Structuring of Earthquake Information from Heterogeneous Web Sources · 2026 · DOI

  • For cross-modal error detection, we introduce a curated taxonomy of seven error categories injected into 130209 records and show that reliably detecting subtle domain-specific errors such as material anachronisms and temporal shifts remain an open challenge.

    ArtiFact: A Large-Scale Multi-Modal Cultural Heritage Dataset · 2026

  • The use of Large Language Models (LLMs) for web crawling should be further explored, in particular focused crawling as an example application of LLMs to extract relevant links from a web page and to automatically assess the relevance. As future work, concrete operationalisations should be explored, implemented and evaluated, also considering scalability and resource-efficiency.

    Relevance and Diversity – Towards Selection Criteriafor Archiving the German Web · 2026 · DOI

  • Generative Engine Optimization (GEO) is increasingly used to improve content visibility in LLM-based retrieval systems, yet its population-level effects under repeated optimization remain poorly understood.

    CHASE: How Content Ecosystems Are Reshaped When Ranking Is the Only Target · 2026

  • Future research can focus on improving the quality of the retrieval component. Future research can explore the use of the proposed approach in other applications such as text generation and question answering.

    Active Retrieval Augmented Generation · 2023 · DOI

  • The gap is that previous methods do not effectively address hallucination in generative language models.

    Active Retrieval Augmented Generation · 2023 · DOI

  • The lack of a natural language interface for information retrieval systems. The need for a system that can handle uncertainty in natural language semantics. The need for a system that can be applied to most information-management systems.

    Knowledge-assisted document retrieval: I. The natural-language interface · 1987 · DOI

  • There is a need for a distributed system for document retrieval in office environments. The existing systems do not learn document distribution patterns and user interests and preferences. The existing systems do not customize document retrievals for each user.

    An intelligent system for document retrieval in distributed office environments · 1986 · DOI

  • Future research can focus on developing more efficient methods for constructing and maintaining relational thesauri. Future research can explore the application of relational thesauri to other domains, such as question-answering and text generation.

    Relational thesauri in information retrieval · 1985 · DOI

  • The paper notes that traditional statistical association and classification schemes have limitations.

    Relational thesauri in information retrieval · 1985 · DOI

  • Future research can focus on addressing the limitations of the conventional Boolean retrieval model. The development of new techniques for assigning importance factors to terms attached to documents and queries can be explored.

    Advanced feedback methods in information retrieval · 1985 · DOI

  • The conventional Boolean retrieval model has limitations, such as difficulty in controlling the size of the output and lack of ranking of retrieved items. There is a need for alternative approaches to improve retrieval effectiveness.

    Advanced feedback methods in information retrieval · 1985 · DOI

  • Further evaluation of the extended models is needed. The use of other term significance weights should be explored. The application of the extended models to other domains should be considered.

    Document representation in probabilistic models of information retrieval · 1981 · DOI

  • The limitation of probabilistic models of retrieval assuming binary index terms is identified. The need to extend these models to include term significance weights is recognized.

    Document representation in probabilistic models of information retrieval · 1981 · DOI

  • Future research can explore the application of passage retrieval to other domains. The technique can be improved by developing more sophisticated search algorithms. The paper provides a framework for future research in passage retrieval.

    Answer‐passage retrieval by text searching · 1980 · DOI

  • The paper identifies a gap in the knowledge of how employees of institutions undergoing change are linked to the change process. The study indicates that state mental health employees at the institutional level feel genuinely unable to affect the change process.

    Footnotes (items of interest) · 1977 · DOI

  • To further improve the algorithms accuracy, we believe that extending the set of properties to include non-DOM-based properties would be necessary. Analyzing the results revealed that icons were often mismatched, as the algorithm cannot differentiate between them as effectively as it can with text.

    Web element relocalization in evolving web applications: A comparative analysis and extension study · 2026 · DOI

  • Here, EM represents the match rate of knowledge Q&A, while WER and MER can be used to evaluate the overall performance of candidate-based information extraction models on mixed data. 81%, demonstrating that the candidate extractor can effectively extract valid information relevant to the question from mixed datasets.

    Research on the Construction and Appreciation of a Knowledge Graph of Chinese Classical Literature and Culture for International Chinese Language Education · 2026 · DOI

  • The paper notes that complex Boolean queries can be difficult to construct.

    Boolean queries and term dependencies in probabilistic retrieval models · 1986 · DOI

  • The technique has been successfully applied to lawyers, but not to scientists.

    Answer‐passage retrieval by text searching · 1980 · DOI

  • The dataset size is limited (17 documents), - The system relies on predefined patterns, - The study does not evaluate the system in real-world…

    A Self-Adaptive LLM-Based Framework for Automated Extraction and Structuring of Earthquake Information from Heterogeneous Web Sources · 2026 · DOI

  • The paper identifies the challenge of maintaining open research infrastructure. The paper notes the decline of shared truth and the turn away from facts to feeling in policy analysis and decision-making.

    Media Cloud 2.0: An Updated Open Web News Archive · 2026 · DOI

  • The trade-off between accuracy and efficiency in agentic RAG - The need for a token-efficient agentic RAG…

    TeaRAG: A Token-Efficient Agentic Retrieval-Augmented Generation Framework · 2026 · DOI

  • The rapid growth and increasing fragmentation of Earth-system science literature - The difficulty of staying current with the pace of peer-reviewed publications - The need for a holistic Earth-system perspective that integrates work from multiple…

    A domain-adapted retrieval-augmented framework for transparent Earth system assessment · 2026 · DOI

  • To implement the system in a real-world setting. To extend the system to handle other types of document retrievals. To improve the performance of the set of heuristics.

    An intelligent system for document retrieval in distributed office environments · 1986 · DOI

Most-cited papers in Web Data Mining and Analysis

Most recent work

Find a gap in your own Web Data Mining and Analysis sub-topic

This page shows what the Web Data Mining and Analysis literature already flags as unresolved. To narrow it to your specific question, search the Research Gap Finder: the search is free with a free account and lists the papers closest to your topic first. Unlocking that topic (50 credits, charged once) fills the comparison table from our 4.5M-paper local library and writes the gaps from its rows.

Open the Research Gap Finder →

Related topics in Computer Science

Online Learning and Analytics
1,068 open questions
AI in Service Interactions
888 open questions
Virtual Reality Applications and Impacts
796 open questions
Computational Drug Discovery Methods
691 open questions
Quantum Computing Algorithms and Architecture
688 open questions
Teaching and Learning Programming
663 open questions
Digital literacy in education
591 open questions
Topic Modeling
565 open questions
Open questions raised by papers listed on this page include: (1) Generative Engine Optimization (GEO) is increasingly used to improve content visibility in LLM-based retrieval systems, yet its population-level effects under repeated optimization remain poorly understood (“CHASE: How Content Ecosystems Are Reshaped When Ranking Is the Only Target”, 2026); (2) Future research can focus on improving the quality of the retrieval component (“Active Retrieval Augmented Generation”, 2023); (3) The lack of a natural language interface for information retrieval systems (“Knowledge-assisted document retrieval: I. The natural-language interface”, 1987); (4) There is a need for a distributed system for document retrieval in office environments (“An intelligent system for document retrieval in distributed office environments”, 1986); (5) Future research can focus on developing more efficient methods for constructing and maintaining relational thesauri (“Relational thesauri in information retrieval”, 1985). 77 gap statements have been mined from Web Data Mining and Analysis papers in our 4.5M-paper local library, which holds 759 papers on the topic; the gaps come from whichever of those papers state one. They are mostly the research gaps the authors state and the papers' abstracts, plus future-work, limitations and challenges passages. The ones listed below are a selection still marked open; each names the study that raised it, with a DOI link where the paper has one, so you can read the original claim in context.

Tools for your next paper

Gaps & Literature SearchFind open questions in any field — 120,000+ open gaps mined from our own library.Pre-CheckIs your paper ready? Acceptance odds for top, mid and open/emerging journals, in seconds.Journal FinderA ranked shortlist of target journals from a 17,500-venue index.Duplicate Publication CheckerCheck for prior publication across 7 sources and our local library.Citation GeneratorFreeA DOI or title → APA, MLA, Vancouver, Chicago, BibTeX, RIS. No signup.Graphical Abstract MakerFreeTurn your findings into a colorblind-safe graphical abstract.

How Pre-Check worksHow AI Review works

Compare the category — Honest roundups of the AI research tools, ours listed alongside the alternatives.

Command palette

Jump anywhere, run any action.