arXiv subjects

Miftahul Jannat Mokarrama

Publications and source records attributed to Miftahul Jannat Mokarrama.

4 recordsLinked to original sources

arXiv2025

BAGELS: Benchmarking the Automated Generation and Extraction of Limitations from Scholarly Text

In scientific research, ``limitations'' refer to the shortcomings, constraints, or weaknesses of a study. A transparent reporting of such limitations can enhance the quality and reproducibility of research and improve public trust in science. However, authors often underreport limitations in their papers and rely on hedging strategies to meet editorial requirements at the expense of readers' clarity and confidence. This tendency, combined with the surge in scientific publications, has created a pressing need for automated approaches to extract and generate limitations from scholarly papers. To address this need, we present a full architecture for computational analysis of research limitations. Specifically, we (1) create a dataset of limitations from ACL, NeurIPS, and PeerJ papers by extracting them from the text and supplementing them with external reviews; (2) we propose methods to automatically generate limitations using a novel Retrieval Augmented Generation (RAG) technique; (3) we design a fine-grained evaluation framework for generated limitations, along with a meta-evaluation of these techniques.

cs.DL↗

arXiv2025

FutureGen: A RAG-based Approach to Generate the Future Work of Scientific Article

The Future Work section of a scientific article outlines potential research directions by identifying gaps and limitations of a current study. This section serves as a valuable resource for early-career researchers seeking unexplored areas and experienced researchers looking for new projects or collaborations. In this study, we generate future work suggestions from a scientific article. To enrich the generation process with broader insights and reduce the chance of missing important research directions, we use context from related papers using RAG. We experimented with various Large Language Models (LLMs) integrated into Retrieval-Augmented Generation (RAG). We incorporate an LLM feedback mechanism to enhance the quality of the generated content and introduce an LLM-as-a-judge framework for robust evaluation, assessing key aspects such as novelty, hallucination, and feasibility. Our results demonstrate that the RAG-based approach using GPT-4o mini, combined with an LLM feedback mechanism, outperforms other methods based on both qualitative and quantitative evaluations. Moreover, we conduct a human evaluation to assess the LLM as an extractor, generator, and feedback provider.

cs.CL↗

arXiv2025

Quantifying the Relevance of Youth Research Cited in the US Policy Documents

In recent years, there has been a growing concern and emphasis on conducting research beyond academic or scientific research communities, benefiting society at large. A well-known approach to measuring the impact of research on society is enumerating its policy citation(s). Despite the importance of research in informing policy, there is no concrete evidence to suggest the research's relevance in cited policy documents. This is concerning because it may increase the possibility of evidence used in policy being manipulated by individual, social, or political biases that may lead to inappropriate, fragmented, or archaic research evidence in policy. Therefore, it is crucial to identify the degree of relevance between research articles and citing policy documents. In this paper, we examined the scale of contextual relevance of youth-focused research in the referenced US policy documents using natural language processing techniques, state-of-the-art pre-trained Large Language Models (LLMs), and statistical analysis. Our experiments and analysis concluded that youth-related research articles that get US policy citations are mostly relevant to the citing policy documents.

cs.CY↗

arXiv2025

Examining the Representation of Youth in the US Policy Documents through the Lens of Research

This study explores the representation of youth in US policy documents by analyzing how research on youth topics is cited within these policies. The research focuses on three key questions: identifying the frequently discussed topics in youth research that receive citations in policy documents, discerning patterns in youth research that contribute to higher citation rates in policy, and comparing the alignment between topics in youth research and those in citing policy documents. Through this analysis, the study aims to shed light on the relationship between academic research and policy formulation, highlighting areas where youth issues are effectively integrated into policy and contributing to the broader goal of enhancing youth engagement in societal decision-making processes.

cs.CY↗