Open research questions in Natural Language Processing Techniques

What the literature leaves open

• Future work will investigate the integration of SRE-IDF with semantic embedding and transformer-based models to cover both lexical and semantic meanings of sentences. Therefore, the effectiveness of the SRE-IDF in multilingual and cross-lingual grading scenarios remains an open research question.

  SRE-IDF: A Square Root Exponential Inverse Document Frequency Weighting Scheme for Automated Short Answer Grading · 2026 · DOI

• While the English-Arabic language pair has been extensively investigated in translation studies, it remains underexplored with respect to how different MT and LLM architectures handle VPE reconstruction.

  Assessing English-Arabic translation of verb phrase ellipsis: A comparative study of Google Translate and ChatGPT-4o · 2026 · DOI

• It remains to be seen, however, if the -er in -erwijs behaves similarly or differently in such a phonological context. It remains to be seen if such a constraint is independently motivated. A possible answer to this has significant ramifications for our theory of the role of the lexicon/narrow syntax in the representation of categorial information and is beyond the scope of the current paper.

  Analysing Dutch Present Participle Manner Adverbials · 2025 · DOI

• Unless future empirical evidence suggests otherwise, the present analysis not only advances our understanding of PSE, but also provides valuable insights into ellipsis–prosody interactions, a topic that remains underexplored in the current literature.

  Revisiting Particle-Stranding Ellipsis: A Critical Comparison of Two Analyses · 2025 · DOI

• Notably, RoBERTa outperformed BERT, indicating that when domain-specific training data is limited, more robust pre-trained models are better suited to capture the subtle linguistic cues needed for geolocation tasks.

  A Hybrid Approach for Geo-Referencing Tweets: Transformer Language Model Regression and Gazetteer Disambiguation · 2025 · DOI

• The lack of available data and resources for sign language processing. The limited availability of annotated data for sign languages. The need for advancements in sign language translation technology to enhance communication accessibility.

  Creating a Parallel Corpus for the Kazakh Sign Language and Learning · 2025 · DOI

• Although this idea should be explored further, we believe that this change of reference is the key to understanding why the order in (35) blocks the dual number interpretation for 1PL. Needless to say, further research on this perspective could offer an explanation for the restrictions observed in Section 2. Probably, further research on information structure could give us some answers to this puzzle.

  Agreement and Information Structure in Spanish PRO[PL] with-DP · 2024 · DOI

• We emphasise, however, that reliable large-scale modernisation across genres and authors remains an open challenge and is not yet achieved by the current system.

  A Hybrid NLLB and Large Language Model Pipeline for Diachronic Intralingual Translation of 16th-Century Slovene Literary Heritage · 2026 · DOI

• Active learning emerges as a promising alternative by strategically minimizing labeling effort, yet its application to Arabic sentiment analysis has remained underexplored.

  From human annotation to automation: LLM-in-the-loop active learning for Arabic sentiment analysis · 2026 · DOI

• THEtool’s solid performance in the evaluation shows that its output is reliable and can be used as a foundation for further studies of the writing process.

  Automated Extraction and Analysis of Sentences under Production: A Theoretical Framework and Its Evaluation · 2024 · DOI

• Multilingual Neural Machine Translation (MNMT) enhances low-resource language translation by leveraging related languages, yet its application to Creole languages like Chavacano, a Spanish-based Creole spoken in the Philippines, remains underexplored.

  ChavacanoMNMT: Exploring Language Influences in the Multilingual Neural Machine Translation of a Spanish-Based Creole · 2026 · DOI

• However, many social media posts such as Twitter/X lack coordinate information, especially when search is limited to a specific region or topic.

  A Hybrid Approach for Geo-Referencing Tweets: Transformer Language Model Regression and Gazetteer Disambiguation · 2025 · DOI

• In low-resource language environments, subword lexicon optimization methods are the main future research directions for improving the accuracy of neural machine translation.

  Korean Subword vocabulary optimization by removing compositional words in neural machine translation · 2026 · DOI

• The need to reduce inference time while maintaining performance.

  SPENCER: Self-Adaptive Model Distillation for Efficient Code Retrieval · 2026 · DOI

• The dual-encoder architecture has limitations in terms of efficiency and effectiveness. There is a need for a framework that combines the strengths of dual-encoders and cross-encoders for code retrieval.

  SPENCER: Self-Adaptive Model Distillation for Efficient Code Retrieval · 2026 · DOI

• The lack of Indian language content on various online platforms. The limited comfort of Indian language users in accessing content on their internet devices.

  A performance analysis of a large language model for Marathi language NLP tasks · 2026 · DOI

• Need for efficient and accurate systems that can handle semantic queries over unstructured documents.

  ScaleDoc: Scaling LLM-based Predicates over Large Document Collections · 2026 · DOI

• The parsing efficiency of SQLFlex does not yet match that of rule-based parsers. The approach relies on the effectiveness of the LLM, which may be limited in certain cases. The evaluation is limited to a specific set of use cases and dialects.

  Dialect-Agnostic SQL Parsing via LLM-Based Segmentation · 2026 · DOI

• Limited parallel corpora and training data for low-resource languages, - Vocabulary misalignment, hallucinations, and morphological discrepancies, - Complex morphological structures and extensive linguistic diversity in low-resource languages.

  Dynamic Aggregation and Augmentation for Low-Resource Machine Translation Using Federated Fine-Tuning of Pretrained Transformer Models · 2025 · DOI

• In recent years, machine translation has made significant advancements; however, its effectiveness can vary widely depending on the language pair.

  The Task of Post-Editing Machine Translation for the Low-Resource Language · 2024 · DOI

• The comparative analysis shows GPT-4 outperforms other models consistently across language pairs (En-De, En-Cs, En-Zh, En-Ru, De-En, Cs-En, Zh-En, Ru-En), but does not investigate the specific linguistic phenomena or grammatical structures where GPT-4 advantages emerge, particularly for morphologically complex languages like Russian and Czech.

  Large language model based machine translation for universal multilingual understanding and translation quality enhancement · 2026 · DOI

• First, the benchmark can be extended to include additional languages, scripts, and typological phenomena, particularly from under-represented linguistic families, to further stress-test compositional reasoning and cross-linguistic generalization.

  Evaluating systematic linguistic reasoning in large language models via linguistics olympiad problems · 2026 · DOI

• The authors initially examined Generative Error Correction (GEC) (Yang et al., 2024) to improve the diacritics on the Yor`ub´a language labels using Gemma3-27B. However, this approach also introduced several hallucinated predictions. We postulate that this could be reduced with prompt refinements (Sachdev et al., 2025).

  Sometin Beta Pass Notin (SBPN): Improving Multilingual ASR for Nigerian Languages via Knowledge Distillation · 2026

• Combining cross-lingual learning and federated training to advance NLP for underrepresented languages - Improving MT for low-resource languages requires innovative strategies that better utilize the available resources - Scaling pretrained Transformer models more effectively for low-resource…

  Dynamic Aggregation and Augmentation for Low-Resource Machine Translation Using Federated Fine-Tuning of Pretrained Transformer Models · 2025 · DOI

• Exploring retrieval-augmented generation to improve the effectiveness of SQLFlex. Fine-tuning the LLM to improve its hierarchical reasoning capabilities. Investigating more refined prompting strategies to further improve segmentation effectiveness.

  Dialect-Agnostic SQL Parsing via LLM-Based Segmentation · 2026 · DOI

Most-cited papers in Natural Language Processing Techniques

• Parameter-efficient fine-tuning of large-scale pre-trained language models · Nature Machine Intelligence · 2023 · 1,014 citations
• Multilingual Denoising Pre-training for Neural Machine Translation · Transactions of the Association for Computational Linguistics · 2020 · 807 citations
• The sausage machine: A new two-stage parsing model · Cognition · 1978 · 774 citations
• BabelNet: The automatic construction, evaluation and application of a wide-coverage multilingual semantic network · Artificial Intelligence · 2012 · 707 citations
• A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models · 2024 · 594 citations
• M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation · 2024 · 465 citations
• Revised report on the algorithmic language ALGOL 60 · Communications of the ACM · 1963 · 403 citations
• More than words: Frequency effects for multi-word phrases · Journal of Memory and Language · 2009 · 402 citations
• MTEB: Massive Text Embedding Benchmark · 2023 · 394 citations
• CodeT5+: Open Code Large Language Models for Code Understanding and Generation · 2023 · 382 citations

Most recent work

• A French Corpus Annotated for Multiword Expressions with Adverbial Function · arXiv (Cornell University) · 2026
• Toward Multi-Functional LAWNs With ISAC: Opportunities, Challenges, and the Road Ahead · IEEE Wireless Communications · 2026
• Named Entity Recognition Using Web Document Corpus · 2026
• Large multimodal models for low-resource languages: A survey · Information Fusion · 2026
• A Comprehensive Survey on Large Language Model Compression for Artificial Intelligence Applications in Edge Systems · IEEE Internet of Things Journal · 2026
• AECBench: A hierarchical benchmark for knowledge evaluation of large language models in the AEC field · Advanced Engineering Informatics · 2026
• From the EFT to the UV: the complete SMEFT one-loop dictionary · SciPost Physics · 2026
• The Tail-Preserving Alternative: A Design Specification for Variance-Preserving Language Models, and the Political Economy of Why They Are Not Deployed (v1.0) · Zenodo (CERN European Organization for Nuclear Research) · 2026
• Performance unfairness of large language models in cross-language fact-checking · Information Processing & Management · 2026
• PEXP: A Scalable Parallel Tree-Based Framework for Interpreting Models on Big Data · IEEE Transactions on Big Data · 2026

Related topics in Computer Science