arXiv subjects

Yiran Hu

Publications and source records attributed to Yiran Hu.

28 records · Page 2Linked to original sources

arXiv2025

LexRAG: Benchmarking Retrieval-Augmented Generation in Multi-Turn Legal Consultation Conversation

Retrieval-augmented generation (RAG) has proven highly effective in improving large language models (LLMs) across various domains. However, there is no benchmark specifically designed to assess the effectiveness of RAG in the legal domain, which restricts progress in this area. To fill this gap, we propose LexRAG, the first benchmark to evaluate RAG systems for multi-turn legal consultations. LexRAG consists of 1,013 multi-turn dialogue samples and 17,228 candidate legal articles. Each sample is annotated by legal experts and consists of five rounds of progressive questioning. LexRAG includes two key tasks: (1) Conversational knowledge retrieval, requiring accurate retrieval of relevant legal articles based on multi-turn context. (2) Response generation, focusing on producing legally sound answers. To ensure reliable reproducibility, we develop LexiT, a legal RAG toolkit that provides a comprehensive implementation of RAG system components tailored for the legal domain. Additionally, we introduce an LLM-as-a-judge evaluation pipeline to enable detailed and effective assessment. Through experimental analysis of various LLMs and retrieval methods, we reveal the key limitations of existing RAG systems in handling legal consultation conversations. LexRAG establishes a new benchmark for the practical application of RAG systems in the legal domain, with its code and data available at https://github.com/CSHaitao/LexRAG.

cs.CL↗

arXiv2025

CaseGen: A Benchmark for Multi-Stage Legal Case Documents Generation

Legal case documents play a critical role in judicial proceedings. As the number of cases continues to rise, the reliance on manual drafting of legal case documents is facing increasing pressure and challenges. The development of large language models (LLMs) offers a promising solution for automating document generation. However, existing benchmarks fail to fully capture the complexities involved in drafting legal case documents in real-world scenarios. To address this gap, we introduce CaseGen, the benchmark for multi-stage legal case documents generation in the Chinese legal domain. CaseGen is based on 500 real case samples annotated by legal experts and covers seven essential case sections. It supports four key tasks: drafting defense statements, writing trial facts, composing legal reasoning, and generating judgment results. To the best of our knowledge, CaseGen is the first benchmark designed to evaluate LLMs in the context of legal case document generation. To ensure an accurate and comprehensive evaluation, we design the LLM-as-a-judge evaluation framework and validate its effectiveness through human annotations. We evaluate several widely used general-domain LLMs and legal-specific LLMs, highlighting their limitations in case document generation and pinpointing areas for potential improvement. This work marks a step toward a more effective framework for automating legal case documents drafting, paving the way for the reliable application of AI in the legal field. The dataset and code are publicly available at https://github.com/CSHaitao/CaseGen.

cs.CL↗

arXiv2025

SELP: Generating Safe and Efficient Task Plans for Robot Agents with Large Language Models

Despite significant advancements in large language models (LLMs) that enhance robot agents' understanding and execution of natural language (NL) commands, ensuring the agents adhere to user-specified constraints remains challenging, particularly for complex commands and long-horizon tasks. To address this challenge, we present three key insights, equivalence voting, constrained decoding, and domain-specific fine-tuning, which significantly enhance LLM planners' capability in handling complex tasks. Equivalence voting ensures consistency by generating and sampling multiple Linear Temporal Logic (LTL) formulas from NL commands, grouping equivalent LTL formulas, and selecting the majority group of formulas as the final LTL formula. Constrained decoding then uses the generated LTL formula to enforce the autoregressive inference of plans, ensuring the generated plans conform to the LTL. Domain-specific fine-tuning customizes LLMs to produce safe and efficient plans within specific task domains. Our approach, Safe Efficient LLM Planner (SELP), combines these insights to create LLM planners to generate plans adhering to user commands with high confidence. We demonstrate the effectiveness and generalizability of SELP across different robot agents and tasks, including drone navigation and robot manipulation. For drone navigation tasks, SELP outperforms state-of-the-art planners by 10.8% in safety rate (i.e., finishing tasks conforming to NL commands) and by 19.8% in plan efficiency. For robot manipulation tasks, SELP achieves 20.4% improvement in safety rate. Our datasets for evaluating NL-to-LTL and robot task planning will be released in github.com/lt-asset/selp.

cs.RO↗

arXiv2024

LegalAgentBench: Evaluating LLM Agents in Legal Domain

With the increasing intelligence and autonomy of LLM agents, their potential applications in the legal domain are becoming increasingly apparent. However, existing general-domain benchmarks cannot fully capture the complexity and subtle nuances of real-world judicial cognition and decision-making. Therefore, we propose LegalAgentBench, a comprehensive benchmark specifically designed to evaluate LLM Agents in the Chinese legal domain. LegalAgentBench includes 17 corpora from real-world legal scenarios and provides 37 tools for interacting with external knowledge. We designed a scalable task construction framework and carefully annotated 300 tasks. These tasks span various types, including multi-hop reasoning and writing, and range across different difficulty levels, effectively reflecting the complexity of real-world legal scenarios. Moreover, beyond evaluating final success, LegalAgentBench incorporates keyword analysis during intermediate processes to calculate progress rates, enabling more fine-grained evaluation. We evaluated eight popular LLMs, highlighting the strengths, limitations, and potential areas for improvement of existing models and methods. LegalAgentBench sets a new benchmark for the practical application of LLMs in the legal domain, with its code and data available at \url{https://github.com/CSHaitao/LegalAgentBench}.

cs.CL↗

arXiv2024

STARD: A Chinese Statute Retrieval Dataset with Real Queries Issued by Non-professionals

Statute retrieval aims to find relevant statutory articles for specific queries. This process is the basis of a wide range of legal applications such as legal advice, automated judicial decisions, legal document drafting, etc. Existing statute retrieval benchmarks focus on formal and professional queries from sources like bar exams and legal case documents, thereby neglecting non-professional queries from the general public, which often lack precise legal terminology and references. To address this gap, we introduce the STAtute Retrieval Dataset (STARD), a Chinese dataset comprising 1,543 query cases collected from real-world legal consultations and 55,348 candidate statutory articles. Unlike existing statute retrieval datasets, which primarily focus on professional legal queries, STARD captures the complexity and diversity of real queries from the general public. Through a comprehensive evaluation of various retrieval baselines, we reveal that existing retrieval approaches all fall short of these real queries issued by non-professional users. The best method only achieves a Recall@100 of 0.907, suggesting the necessity for further exploration and additional research in this area. All the codes and datasets are available at: https://github.com/oneal2000/STARD/tree/main

cs.IR↗

arXiv2023

MUSER: A Multi-View Similar Case Retrieval Dataset

Similar case retrieval (SCR) is a representative legal AI application that plays a pivotal role in promoting judicial fairness. However, existing SCR datasets only focus on the fact description section when judging the similarity between cases, ignoring other valuable sections (e.g., the court's opinion) that can provide insightful reasoning process behind. Furthermore, the case similarities are typically measured solely by the textual semantics of the fact descriptions, which may fail to capture the full complexity of legal cases from the perspective of legal knowledge. In this work, we present MUSER, a similar case retrieval dataset based on multi-view similarity measurement and comprehensive legal element with sentence-level legal element annotations. Specifically, we select three perspectives (legal fact, dispute focus, and law statutory) and build a comprehensive and structured label schema of legal elements for each of them, to enable accurate and knowledgeable evaluation of case similarities. The constructed dataset originates from Chinese civil cases and contains 100 query cases and 4,024 candidate cases. We implement several text classification algorithms for legal element prediction and various retrieval methods for retrieving similar cases on MUSER. The experimental results indicate that incorporating legal elements can benefit the performance of SCR models, but further efforts are still required to address the remaining challenges posed by MUSER. The source code and dataset are released at https://github.com/THUlawtech/MUSER.

cs.CL↗

arXiv2023

Global in Time Weak Solutions to Singular 3D Quasi-Geostrophic Systems

Geophysicists have studied 3D Quasi-Geostrophic systems extensively. These systems describe stratified flows in the atmosphere on a large time scale and are widely used for forecasting atmospheric circulation. They couple an inviscid transport equation in $\mathbb{R}_{+}\timesΩ$ with an equation on the boundary satisfied by the trace, where $Ω$ is either $2D$ torus or a bounded convex domain in $\mathbb{R}^2$. In this paper, we show the existence of global in time weak solutions to a family of singular 3D quasi-geostrophic systems with Ekman pumping, where the background density profile degenerates at the boundary. The proof is based on the construction of approximated models which combine the Galerkin method at the boundary and regularization processes in the bulk of the domain. The main difficulty is handling the degeneration of the background density profile at the boundary.

math.AP↗

arXiv2022

An epitaxial graphene platform for zero-energy edge state nanoelectronics

Graphene's original promise to succeed silicon faltered due to pervasive edge disorder in lithographically patterned deposited graphene and the lack of a new electronics paradigm. Here we demonstrate that the annealed edges in conventionally patterned graphene epitaxially grown on a silicon carbide substrate (epigraphene) are stabilized by the substrate and support a protected edge state. The edge state has a mean free path that is greater than 50 microns, 5000 times greater that the bulk states and involves a theoretically unexpected Majorana-like zero-energy non-degenerate quasiparticle that does not produce a Hall voltage. In seamless integrated structures, the edge state forms a zero-energy one-dimensional ballistic network with essentially dissipationless nodes at ribbon-ribbon junctions. Seamless device structures offer a variety of switching possibilities including quantum coherent devices at low temperatures. This makes epigraphene a technologically viable graphene nanoelectronics platform that has the potential to succeed silicon nanoelectronics.

cond-mat.mes-hall↗

arXiv2020

Highly Ordered Boron Nitride/Epigraphene Epitaxial Films on Silicon Carbide by Lateral Epitaxial Deposition

Realizing high-performance nanoelectronics requires control of materials at the nanoscale. Methods to produce high quality epitaxial graphene (EG) nanostructures on silicon carbide are known. The next step is to grow Van der Waals semiconductors on top of EG nanostructures. Hexagonal boron nitride (h-BN) is a wide bandgap semiconductor with a honeycomb lattice structure that matches that of graphene, making it ideally suited for graphene-based nanoelectronics. Here, we describe the preparation and characterization of multilayer h-BN grown epitaxially on EG using a migration-enhanced metalorganic vapor phase epitaxy process. As a result of the lateral epitaxial deposition (LED) mechanism, the grown h-BN/EG heterostructures have highly ordered epitaxial interfaces, as desired in order to preserve the transport properties of pristine graphene. Atomic scale structural and energetic details of the observed row-by-row, growth mechanism of the 2D epitaxial h-BN film are analyzed through first-principles simulations, demonstrating one-dimensional nucleation-free-energy-barrierless growth. This industrially relevant LED process can be applied to a wide variety of van der Waals materials.

cond-mat.mtrl-sci↗

arXiv2016

Epitaxial graphene on SiC: 2D sheets, selective growth and nanoribbons

Epitaxial graphene grown on SiC by the confinement controlled sublimation method is reviewed, with an emphasis on multilayer and monolayer epitaxial graphene on the carbon face of 4H-SiC and on directed and selectively grown structures under growth-arresting or growth-enhancing masks. Recent developments in the growth of templated graphene nanostructures are also presented, as exemplified by tens of micron long very well confined and isolated 20-40nm wide graphene ribbons. Scheme for large scale integration of ribbon arrays with Si wafer is also presented.

cond-mat.mes-hall↗