Open research questions in Natural Language Processing Techniques

What the literature leaves open

• Future work should explore leveraging this longer- context data, potentially by fine-tuning LLMs like Krikri-8B for document-level AG-MG translation, which might better handle discourse phenomena and context-dependent translations, especially for literary texts (Karpinska and Iyyer, 2023; Wang et al.

  Ancient Greek to Modern Greek Machine Translation: A Novel Benchmark and Fine-Tuning Experiments on LLMs and NMT Models · 2026

• Managing linguistic ambiguity and unseen terms in PoS tagging. The limited availability of annotated corpora and linguistic resources for low-resource Sino-Tibetan languages. The challenge of comparing studies directly due to variations in datasets, corpus sizes, tagsets, and evaluation methodologies.

  A structured and comprehensive review of Sino-Tibetan languages PoS taggers · 2026 · DOI

• Limited training data, - Challenges in treating unknown terms, - Unstandardized datasets and tagsets, - Manual rule generation and unseen word processing limited the efficacy of rule-based approaches

  A structured and comprehensive review of Sino-Tibetan languages PoS taggers · 2026 · DOI

• First, the evaluated languages and benchmarks, while substantial, are insufficient to fully represent the global linguistic diversity and task spectrum.

  A Data-Efficient Path to Multilingual LLMs: Language Expansion via Post-training PARAM$Δ$ Integration into Upcycled MoE · 2026

• Transfer to other classical languages, including Latin, Ancient Greek, and Sanskrit, remains to be tested.

  Can We Triage LLM Translation Errors in Classical Texts Without Human References? Source Novelty, GEMBA Scoring, and Budgeted Review through Pali-to-English Translation · 2026

• However, their performance in Chinese grammatical error correction (CGEC) remains unexplored.

  Evaluating LLMs’ grammatical error correction performance in learner Chinese · 2024 · DOI

• Part of our research in the future will be directed at more pattern-matching. We plan to write and test other error patterns as well as refine those described in this paper. Also, we will continue to develop our lexicon and to refine our feature list. And we're interested in developing heuristics which will lower our false positive rates. Because of the limitations of pattern-matching, we are also developing other approaches for error detection. We are currently writing a natural lan- guage parser and a small set of grammar rules and a lexicon to drive it. Rather than searching for particular error patterns, this program will look for well-formed sentences, culling as errors those that violate its grammar rules. Our plan now is to have separate pattern-matching and parsing, but we're beginning to speculate about how the two approaches might be combined for greater accuracy and efficiency. 118 HULL, BALL, FOX, LEVIN, AND McCUTCHEN NOTES REFERENCES 1. We should note that we are developing a separate, more elabo- rate lexicon for a parser. 2. We should note, however, that the original test run was a more general pattern than we specify above and thus probably turned up more false positives than the current pattern would. We esti- mate that the correctly flagged percentage should be about 60 percent. 3. We don't have an actual figure for the percentages because of a windowing limitation in our search program. That is, often it is impossible to tell, from the amount of text that is called up, whether the sentence beginning with the subordinate conjunction is a fragment: Cousins says man does have the capacity to make decisions in life. Whether a man is born with money, no money, a handicap or healthy... We need to see the rest of the sentence to determine its correctness. Bartholomae, D. (1979). Teaching basic writing: An alterna- tive to basic skills. Journal of Basic Writing, 31, 253-269. Cherry, L.L. (1980). PARTS-a system for assigning word classes to English text. (Computing Science Technical Report No. 81.) Bell Laboratories, Murray Hill, NJ. Cohen, M.E., & Lanham, R.A. (1984). HOMER: Teaching style with a microcomputer. In W. Wresch, (Ed.), The computer in composition instruction. Urbana: NCTE. Heidorn, G.E., Jensen, K., Miller, L.A., & Chodorow, M.S. (1982). The EPISTLE text-critiquing system. IBM Systems Journal, 21, 305-326. Hull, Glynda. (1987). Constructing taxonomies for error (or can stray dogs be mermaids?). In T. Enos, (Ed.), A sourcebook for basic writing teachers. New York: Random House. Keifer, K.E., & Smith, C.R. (1983). Textual analysis with com- puters: Tests of Bell Laboratories' computer software. Re- search in the Teaching of

  Computer detection of errors in natural language texts: Some research on pattern-matching · 1987 · DOI

• The approach relies on linguistic proximity between languages. The lack of extensive parallel data for Assamese limits direct alignment with English embeddings.

  Zero-shot English–Assamese neural machine translation via pivot-based cross-lingual embedding alignment and transfer learning · 2026 · DOI

• Exploring the application of the proposed framework to other low-resource languages. Investigating the use of other pivot languages for zero-shot translation.

  Zero-shot English–Assamese neural machine translation via pivot-based cross-lingual embedding alignment and transfer learning · 2026 · DOI

• Orthographic variation and the lack of standardized spelling in dialectal Arabic. Capturing dialectal variations and nuances in NLP applications. Ensuring the quality and consistency of the generated corpus.

  Automated Construction of a Multi-Dialectal Saudi Corpus Using Generative Language Models · 2026 · DOI

• The study only considered three major Saudi dialects, - The study relied on a single LLM model (Gemini 1.5 Pro), - The study used a limited number of native Saudi Arabic reviewers for evaluation, - The study did not consider other factors that may influence dialectal variations

  Automated Construction of a Multi-Dialectal Saudi Corpus Using Generative Language Models · 2026 · DOI

• The choice of translation unit is still debated in neural machine translation. The paper identifies a gap in the optimization of subword vocabulary construction for morphologically rich languages like Korean.

  Korean Subword vocabulary optimization by removing compositional words in neural machine translation · 2026 · DOI

• It can be concluded that the proposed method of subword lexical optimization through compositional word removal is applicable to NMT model of all translation languages with Korean language as the source or target language, and has some degree of effectiveness in improving translation accuracy. This approach may have a greater impact on translation model learning in low-resource language environments, i.e. in the absence of training data. In low-resource language environments, subword lexicon optimization methods are the main future research directions for improving the accuracy of neural machine translation.

  Korean Subword vocabulary optimization by removing compositional words in neural machine translation · 2026 · DOI

• Further research can be conducted to improve the proposed approach. The study can be extended to other languages. The results can be used to improve information retrieval processes for other languages.

  Part-of-Speech Tagging-Based Document Clustering for Kurdish Corpora · 2026 · DOI

• The absence of prior studies that apply document clustering techniques to Kurdish corpora. The scarcity of annotated corpora for the Kurdish language.

  Part-of-Speech Tagging-Based Document Clustering for Kurdish Corpora · 2026 · DOI

• The need for more effective methods to address hallucination and overgeneralization in large language models - The importance of considering prejudice and impartiality in large-scale training corpora

  Large language model based machine translation for universal multilingual understanding and translation quality enhancement · 2026 · DOI

• The comparative analysis shows GPT-4 outperforms other models consistently across language pairs (En-De, En-Cs, En-Zh, En-Ru, De-En, Cs-En, Zh-En, Ru-En), but does not investigate the specific linguistic phenomena or grammatical structures where GPT-4 advantages emerge, particularly for morphologically complex languages like Russian and Czech.

  Large language model based machine translation for universal multilingual understanding and translation quality enhancement · 2026 · DOI

• Exploring the application of the proposed framework to other natural language processing tasks. Investigating the use of other optimization techniques, such as reinforcement learning, for prompt optimization. Evaluating the framework on larger and more diverse datasets.

  Optimization of Adaptive Prompt Engineering for Large Language Models via Bayesian Inference in Low-Resource Settings · 2026 · DOI

• The lack of effective prompt engineering strategies for Large Language Models. The limitations of existing automated prompt optimization techniques, such as requiring significant computational resources or access to model gradients.

  Optimization of Adaptive Prompt Engineering for Large Language Models via Bayesian Inference in Low-Resource Settings · 2026 · DOI

• The semantic gap between human natural language and programming languages. The limitations of the dual-encoder architecture. The need to reduce inference time while maintaining performance.

  SPENCER: Self-Adaptive Model Distillation for Efficient Code Retrieval · 2026 · DOI

• The dual-encoder architecture has limitations in terms of efficiency and effectiveness. There is a need for a framework that combines the strengths of dual-encoders and cross-encoders for code retrieval.

  SPENCER: Self-Adaptive Model Distillation for Efficient Code Retrieval · 2026 · DOI

• The lack of Indian language content on various online platforms. The limited comfort of Indian language users in accessing content on their internet devices.

  A performance analysis of a large language model for Marathi language NLP tasks · 2026 · DOI

• Future research should investigate the application of the proposed framework to other areas of natural language processing. Future research should explore the use of other types of problems and datasets to evaluate linguistic reasoning in large language models.

  Evaluating systematic linguistic reasoning in large language models via linguistics olympiad problems · 2026 · DOI

• Current evaluation benchmarks for large language models are flawed, conflating genuine reasoning ability with memorization and statistical pattern matching. There is a need for a more accurate assessment of a model's ability to induce abstract rules and reason systematically.

  Evaluating systematic linguistic reasoning in large language models via linguistics olympiad problems · 2026 · DOI

• High inference costs of LLMs. Difficulty in capturing query-specific semantics. Need for efficient and accurate systems that can handle semantic queries over unstructured documents.

  ScaleDoc: Scaling LLM-based Predicates over Large Document Collections · 2026 · DOI

Most-cited papers in Natural Language Processing Techniques

• Parameter-efficient fine-tuning of large-scale pre-trained language models · Nature Machine Intelligence · 2023 · 1,014 citations
• The sausage machine: A new two-stage parsing model · Cognition · 1978 · 774 citations
• BabelNet: The automatic construction, evaluation and application of a wide-coverage multilingual semantic network · Artificial Intelligence · 2012 · 707 citations
• A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models · 2024 · 594 citations
• M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation · 2024 · 465 citations
• Revised report on the algorithmic language ALGOL 60 · Communications of the ACM · 1963 · 403 citations
• More than words: Frequency effects for multi-word phrases · Journal of Memory and Language · 2009 · 402 citations
• MTEB: Massive Text Embedding Benchmark · 2023 · 394 citations
• CodeT5+: Open Code Large Language Models for Code Understanding and Generation · 2023 · 382 citations
• FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation · 2023 · 332 citations

Most recent work

• A French Corpus Annotated for Multiword Expressions with Adverbial Function · arXiv (Cornell University) · 2026
• Named Entity Recognition Using Web Document Corpus · 2026
• A Comprehensive Survey on Large Language Model Compression for Artificial Intelligence Applications in Edge Systems · IEEE Internet of Things Journal · 2026
• The Tail-Preserving Alternative: A Design Specification for Variance-Preserving Language Models, and the Political Economy of Why They Are Not Deployed (v1.0) · Zenodo (CERN European Organization for Nuclear Research) · 2026
• Retrieval Settlement Fortification Protocol: Standing SPXI Protocol for Semantic Border Sovereignty (EA-SPXI-RSF-01 v1.0) · Zenodo (CERN European Organization for Nuclear Research) · 2026
• AI_Bleeding, Tail-Pruning, and the Misuse of Semantic Exhaustion: Dossier Executive Summary (EA-AIBLEEDING-DOSSIER-01 v1.0) · Zenodo (CERN European Organization for Nuclear Research) · 2026
• The Threat Model Is Backwards: On Classifying High-Perplexity Text as a Security Threat in an Era of Model Collapse — The AI_Bleeding Mitigation as an Input-Layer Tail-Pruning Instrument (EA-TAILGUARD-01 v1.1) · Zenodo (CERN European Organization for Nuclear Research) · 2026
• stEELlm: An LLM for Generating Semantic Annotations of Tabular Data · ACM Transactions on Intelligent Systems and Technology · 2026
• Lexicons and grammars for language processing: industrial or handcrafted products? · arXiv (Cornell University) · 2026
• Say it better: RL-based prompt tuning for enhancing open-vocabulary recognition · Neurocomputing · 2026

Related topics in Computer Science