Searcharxiv⌕ Search

arXiv subjects

Jinbiao Huo

Publications and source records attributed to Jinbiao Huo.

3 recordsLinked to original sources

LM-SimBench: A Unified Framework, Dataset, and Benchmark for Large Language Model Training Simulation

Evaluating candidate parallelization strategies through distributed training runs makes strategy selection prohibitively expensive at scale. Performance simulation offers a lower cost alternative, yet controlled comparisons among existing simulators remain difficult because they differ in the training stages they model and the workloads on which they are evaluated. We introduce LM-SimBench, a unified framework, profiling dataset, and benchmark for evaluating LLM training simulators. The framework decomposes simulation into functional stages from model compilation to schedule level aggregation, connecting execution representations and cost estimators to measurable evaluation targets. Following this framework, we profile 23 model settings from 10 dense and Mixture-of-Experts (MoE) LLM families on an Ascend Neural Processing Unit (NPU) cluster. The resulting dataset contains 348 feasible configurations that jointly vary five parallelism dimensions and activation recomputation, with six views spanning configuration, rank, operator, event, and kernel granularity. Evaluations of representative training simulators demonstrate how LM-SimBench supports comparisons of prediction accuracy, assessment of configuration selection, and diagnosis of prediction errors. Matched GPU-NPU experiments validate the cross-platform applicability of the collection and evaluation protocol and demonstrate NPU-guided GPU configuration selection.

cs.AI↗

TRIP-Evaluate: An Open Multimodal Benchmark for Evaluating Large Models in Transportation

Large language models (LLMs) and multimodal large models (MLLMs) are increasingly used for transportation tasks such as regulation question answering, traffic management support, engineering review, and autonomous-driving scene reasoning. Yet transportation workflows are rule-intensive, computation-intensive, safety-critical, and inherently multimodal. Existing general benchmarks provide limited evidence of whether a model can apply regulations correctly, perform verifiable engineering calculations, or interpret traffic scenes reliably, while the small number of public transportation benchmarks remain narrow in scope and rarely support fine-grained diagnosis across text, images, and point-cloud data. To address this gap, we present TRIP-Evaluate, an open multimodal benchmark for large models in transportation. The benchmark organizes 837 items using a role-task-knowledge taxonomy that covers vehicle, traffic-management, traveler, and planning-and-design functions. Each item is annotated with capability, modality, and difficulty labels, enabling diagnosis from overall accuracy down to specific failure modes. The current release includes 596 text items, 198 image items, and 43 point-cloud items. TRIP-Evaluate also standardizes item construction, quality control, prompting, decoding, and scoring to improve cross-model comparability. Results on a diverse panel of models show that text-based performance is improving, but substantial weaknesses remain in multi-step engineering calculation, rule-constrained reasoning, multimodal scene understanding, and point-cloud understanding. Overall, TRIP-Evaluate provides a reproducible, diagnosable, and engineering-aligned evaluation baseline for model selection, regression testing, and safer deployment in transportation applications.

cs.CV↗

The expressway network design problem for multiple urban subregions based on the macroscopic fundamental diagram

As urbanization advances, cities are expanding, leading to a more decentralized urban structure and longer average commuting durations. The construction of an urban expressway system emerges as a critical strategy to tackle this challenge. However, the traditional link-level network design method faces modeling and solution challenges when dealing with the large-scale expressway network design problem (ENDP). To address the challenges, this paper proposes an expressway network design method for multiple urban subregions based on the macroscopic fundamental diagram (MFD). Initially, a mixed road network traffic model that describes traffic dynamics of multiple subregions and candidate expressways is developed by integrating the MFD and the cell transmission model (CTM). Then, treating urban subregions and candidate expressways as route nodes in the mixed road network, a route choice model is established based on stochastic user equilibrium. Finally, a decision model for ENDP is proposed to minimize vehicle travel time under the construction budget constraint. The impact of financial investment and traffic demand on expressway network design schemes in the case study is explored separately. The simulation results indicate that during the initial stages of expressway planning, the construction of new expressways can significantly alleviate traffic congestion. However, as the expressway network expands further, the effectiveness of improving traffic conditions through new expressway construction gradually diminishes if traffic demand does not continue to increase. Additionally, variations in traffic demand between subregions result in different construction schemes, emphasizing the importance of adjusting budget allocations based on specific traffic demands.

eess.SY↗