arXiv subjects

Duc-Minh Nguyen

Publications and source records attributed to Duc-Minh Nguyen.

4 recordsLinked to original sources

arXiv2026

Compiling Large Multi-Modal Requirement Documents into Runnable Software Systems: From an Agentic Test-Driven Perspective

Large Language Models (LLMs) have significantly improved programming efficiency by translating natural language into code, yet their performance deteriorates when handling large-scale, multi-modal requirement documents containing hundreds of scenarios, often producing incorrect implementations or missing critical constraints. To address this challenge, we propose ARC (Agentic Requirement Compilation), a framework that compiles DSL-based requirement documents into runnable software systems while automatically generating modular software architecture, comprehensive test suites, and traceability across requirements, design, and code. ARC adopts a bidirectional test-driven agentic workflow, combining a top-down architecture design phase with a bottom-up implementation phase to ensure that generated code satisfies synthesized tests. We evaluate ARC on six runnable web system benchmarks and the AppForge benchmark of 101 mobile app generation tasks. Across three independent trials, ARC consistently outperforms state-of-the-art LLM-based baselines, achieving 50.6% more GUI tests passed on average for web systems, a 100% compilation success rate, and a 68.3% test pass rate on AppForge. A user study with 21 participants further shows that users with limited programming experience can write DSL-based requirement documents containing up to 174 scenarios within an average of 5.6 hours to generate maintainable runnable systems, including a real-world ticket-booking application of approximately 10K lines of code.

cs.SE↗

arXiv2026

WebTestPilot: Agentic End-to-End Web Testing against Natural Language Specification by Inferring Oracles with Symbolized GUI Elements

Visual language model (VLM) agents show great promise in automating end-to-end (E2E) web testing against requirements in natural language. However, the probabilistic nature of language models can have inherent hallucinations. Therefore, given a detected inconsistency between the requirement and the web application, it is hard to distinguish whether it stems from the hallucination or a real application bug. Addressing this issue presents two core technical challenges: the implicit oracle inference challenge, where the agent must act as its own oracle to implicitly decide if the application's behavior is correct without guidance, and the probabilistic inference challenge, where an LLM's inconsistent reasoning undermines its trustworthiness as an oracle. Existing LLM-based approaches fail to capture such implicit oracles, either by treating any page navigation that doesn't crash as a success, or by checking each state in isolation, thus missing bugs dependent on context from prior steps. We introduce WebTestPilot, an LLM-based agent designed to address these challenges. WebTestPilot uses (1) a symbolization layer which detects and symbolizes critical GUI elements on the web application into symbols (i.e., variables) and (2) translates natural language specification into a sequence of steps, each of which is equipped with inferred pre- and post-conditions over the symbols as an oracle. This oracle captures data, temporal, and causal dependencies, enabling the validation of implicit requirements. To advance research in this area, we build a benchmark of bug-injected web apps for evaluating NL-to-E2E testing. The results show that WebTestPilot achieves a task completion rate of 99%, with 96% precision and 96% recall in bug detection, outperforming the best baseline (+70 precision, +27 recall). The agent generalizes across diverse natural language inputs and model scales.

cs.SE↗

arXiv2024

Exploiting LLMs' Reasoning Capability to Infer Implicit Concepts in Legal Information Retrieval

Statutory law retrieval is a typical problem in legal language processing, that has various practical applications in law engineering. Modern deep learning-based retrieval methods have achieved significant results for this problem. However, retrieval systems relying on semantic and lexical correlations often exhibit limitations, particularly when handling queries that involve real-life scenarios, or use the vocabulary that is not specific to the legal domain. In this work, we focus on overcoming this weaknesses by utilizing the logical reasoning capabilities of large language models (LLMs) to identify relevant legal terms and facts related to the situation mentioned in the query. The proposed retrieval system integrates additional information from the term--based expansion and query reformulation to improve the retrieval accuracy. The experiments on COLIEE 2022 and COLIEE 2023 datasets show that extra knowledge from LLMs helps to improve the retrieval result of both lexical and semantic ranking models. The final ensemble retrieval system outperformed the highest results among all participating teams in the COLIEE 2022 and 2023 competitions.

cs.CL↗

arXiv2024

Enhancing Legal Document Retrieval: A Multi-Phase Approach with Large Language Models

Large language models with billions of parameters, such as GPT-3.5, GPT-4, and LLaMA, are increasingly prevalent. Numerous studies have explored effective prompting techniques to harness the power of these LLMs for various research problems. Retrieval, specifically in the legal data domain, poses a challenging task for the direct application of Prompting techniques due to the large number and substantial length of legal articles. This research focuses on maximizing the potential of prompting by placing it as the final phase of the retrieval system, preceded by the support of two phases: BM25 Pre-ranking and BERT-based Re-ranking. Experiments on the COLIEE 2023 dataset demonstrate that integrating prompting techniques on LLMs into the retrieval system significantly improves retrieval accuracy. However, error analysis reveals several existing issues in the retrieval system that still need resolution.

cs.CL↗