arXiv subjects

Bo Zeng

Publications and source records attributed to Bo Zeng.

At least 19 recordsLinked to original sources

arXiv2026

Single-Loop Gradient Algorithms for Pessimistic Bilevel Optimization Problems

Bilevel optimization has recently attracted growing attention, particularly in the development of efficient numerical methods. Despite substantial progress on optimistic bilevel optimization, pessimistic bilevel optimization (PBO) remains much less explored, especially the design of fully first-order, single-loop gradient-based methods. To address this gap, we propose a smooth approximation of PBO through reformulation, penalization and regularization, and establish convergence guarantees in terms of both minimizers and stationarity. Building on this framework, we then develop two single-loop algorithms for deterministic and stochastic PBOs, respectively. Both use only first-order gradient information and avoid second-order derivatives and inner-loop subproblem solves. Non-asymptotic convergence rates for the proposed algorithms are established to provide theoretical guarantees. Through a systematic empirical study of both synthetic and practical problem instances, we demonstrate that our algorithms are highly effective and efficient. In particular, our results on spam classification and Smart Predict-then-Optimize further illustrate PBO's advantages over its classical optimistic bilevel counterpart, highlighting its strong potential for practical modeling and the delivery of robust solutions.

math.OC↗

arXiv2026

What Matters for Aggressive Decoding-Time KV Eviction? Temporal Aggregation and Ranking Preservation

Decoding-time KV cache compression research focuses heavily on designing better token scoring functions, while the temporal rule that aggregates scores across decode steps is often treated as an implementation detail. Under aggressive KV compression, we find that exponential-moving-average (EMA) aggregation makes approximately order-preserving scorer modifications largely indistinguishable at the eviction-set level. Value-norm and entropy variants remain highly correlated with attention and produce nearly unchanged retention sets, whereas KeyDiff, key norm, recency, and a learned scorer alter the ranking and degrade substantially. We associate this stability with the evaluated aggregation, which couples layer weighting and temporal retention. Building on this observation, we introduce InertiaKV, an EMA-based decoding-time eviction method, and InertiaKV-Lazy, its periodic-refresh variant, which yields 1.34-1.46x decode throughput relative to full refresh InertiaKV. We also study Score-Free decoding as a separate empirical operating point: it scores the full context once at the first decode step, freezes that ranking, and incurs an average quality change of +0.03 while removing all subsequent scoring. Across six open-weight backbones and the LongBench, LongBench-v2, and RULER benchmarks, the results identify temporal aggregation and ranking preservation as distinct, consequential design factors; they do not imply that scoring quality is irrelevant in general.

cs.AI↗

arXiv2026

CulturalMenuBench: Probing the Knowledge-Application Gap in Multimodal Culinary Reasoning

Multimodal language models achieve near-ceiling scores on food recognition benchmarks, yet it remains unclear whether this success reflects genuine cultural understanding or mere visual matching. To probe this distinction, we introduce CulturalMenuBench, a benchmark of 4,870 items in 10 languages across 18 regions; its 10 tasks pair final-dish and step-by-step cooking images with ingredients, procedural text, and regional labels, spanning basic recognition to process-grounded cultural attribution. Evaluating 12 models exposes a substantial knowledge-application gap: models exceeding 94% on standard multiple-choice tasks drop to at most 56% when attributing dishes to Chinese regional cuisines, despite an identical four-way format. Diagnostic analyses explain why: error patterns are consistent with random guessing, accuracy tracks visual distinctiveness rather than cultural structure, and models classify cuisines more accurately from dish names alone than from images (+7-18 points). The knowledge is thus present but cannot be activated through visual input. An ablation confirms these tasks genuinely require procedural evidence: removing sequential cooking images selectively degrades process-grounded tasks while others remain stable. Overall, CulturalMenuBench shows that near-perfect recognition can conceal an inability to apply cultural knowledge, motivating training that explicitly connects perception, procedure, and cultural context. Code and data are publicly available.

cs.AI↗

arXiv2026

A Primal Perspective on Distributionally Robust Optimization: An Investigation on Modeling and Solution Strategies

As a popular optimization scheme, distributionally robust optimization (DRO) protects decisions against ambiguity in probability distributions. For (single-stage) DRO, prevailing dual reformulations can become difficult when model or ambiguity-set structures are complex. We study DRO from a primal perspective, working directly with distributions in ambiguity sets on closed, potentially unbounded sample spaces. This perspective leads to an algorithmic framework, referred to as BiCS, that constructs and leverages distribution cuts to achieve strong performance. We show that BiCS is applicable to standard DRO, almost-sure DRO, DRO with various chance constraints, and DRO with ambiguity sets strengthened by local information. Numerical experiments with moment and Wasserstein ambiguity sets show that this framework demonstrates superior performance, including solving cases where the examined compact reformulations are unavailable or computationally difficult. The local-information study also makes changes in worst-case distributions directly visible.

math.OC↗

arXiv2026

Designing Hierarchical Hub-and-Spoke Drone-Based Networks for Delivering Time-Sensitive Healthcare Items

Timely distribution of medical items (e.g., whole blood and vaccines) across regional healthcare networks often requires high-volume delivery operations from central facilities (e.g., blood banks) to intermediate regional hospitals, followed by rapid last-mile deliveries to points of injury. Traditional ground-based delivery systems often suffer from limited responsiveness (e.g., traffic congestion) and operational inefficiencies (e.g., blood waste). Leveraging aerial-drone-based delivery systems offers a promising solution for the fast and efficient delivery of time-sensitive medical items across regional healthcare networks. Therefore, we study a hierarchical hub-and-spoke drone-based network for delivering time-sensitive medical items with distinct release and due times to fixed and mobile delivery destinations. We consider a heterogeneous fleet of drones with distinct characteristics (e.g., cost, battery capacity, and speed) and different multi-trip delivery modes. We propose an efficient mixed-integer programming model for location/allocation of mobile delivery destinations, as well as routing and scheduling drones to minimize the total investment and operational costs of the drone delivery network while maintaining the delivery due times. We develop a customized exact solution method integrating problem-specific reformulations and dynamic cutting planes, as well as a fast heuristic algorithm by leveraging a simplified problem variant. Results based on a real-life case study of whole blood delivery data from Pendleton, Oregon, United States, and actual drone flight test data demonstrate that our exact and heuristic solution methods are 31 and 3,375 times faster, respectively, than the Gurobi solver. Results also show that allowing drones to perform multiple trips is 142.8% more cost-efficient than single trips.

math.OC↗

arXiv2026

LISA: Linear-Indexed Sparse Attention for Efficient Long-Context Reasoning

Recent advances in long chain-of-thought reasoning models such as DeepSeek-R1 have led to increasingly longer inference context lengths under the test-time scaling paradigm. However, the O(n^2) computational complexity of standard self-attention causes inference costs to grow sharply with long sequences, limiting the deployment of long-CoT reasoning in production settings. To address this, we propose LISA (Linear-Indexed Sparse Attention), a plug-and-play attention replacement module that requires no pretraining from scratch. LISA integrates two lightweight components in parallel within the original model: (1) a Linear Attention module that provides long-range memory with O(n) time complexity; (2) a Lightning Indexer that selects the top-M important tokens from the full context to feed into a Sparse Self-Attention. The two branches are fused via a gating mechanism, reducing inference complexity from O(n^2) to O(nM) (M << n) for generating n tokens. We design a two-stage training pipeline: Stage 1 initializes the model by integrating the linear attention to capture long-range dependencies, complemented by a sliding-window attention mechanism that is optimized via knowledge distillation to approximate the full self-attention distribution of a frozen teacher model. In Stage 2, we further introduce the Indexer to replace the static sliding-window mechanism, enabling dynamic token selection from broader contexts. The Indexer is trained using a novel per-head KL divergence loss, which aligns its selection behavior with the attention patterns of the teacher model. Experiments on DeepSeek-distilled-Qwen models demonstrate that LISA achieves a 50% inference speedup under 16K-token context, while improving average performance by 5.6% on reasoning benchmarks including AIME and MATH-500.

cs.AI↗

arXiv2026

DuetFair: Coupling Inter- and Intra-Subgroup Robustness for Fair Medical Image Segmentation

Medical image segmentation models can perform unevenly across subgroups. Most existing fairness methods focus on improving average subgroup performance, implicitly treating each subgroup as internally homogeneous. However, this can hide difficult cases within a subgroup, where high-loss samples are obscured by the subgroup mean. We call this problem \textbf{intra-group hidden failure}. To solve this, we propose \textbf{DuetFair} mechanism, a dual-axis fairness framework that jointly considers inter-subgroup adaptation and intra-subgroup robustness. Based on DuetFair, we introduce \textbf{FairDRO}, which combines distribution-aware mixture-of-experts (dMoE) with subgroup-conditioned distributionally robust optimization (DRO) loss aggregation. This design allows the model to adapt across subgroups while also reducing hidden failures within each subgroup. We evaluate FairDRO on three medical image segmentation benchmarks with varying degrees of within-group heterogeneity. FairDRO achieves the best equity-scaled performance on Harvard-FairSeg and improves worst-case subgroup performance on HAM10000 under both age- and race-based grouping schemes. On the 3D radiotherapy target cohort, FairDRO further improves worst-group Dice by 3.5 points ($\uparrow 6.0\%$) under the tumor-stage grouping and by 4.1 points ($\uparrow 7.4\%$) under the institution grouping over the strongest baseline.

cs.CV↗

arXiv2026

Green Manufacturing Capacity Planning by Integrating Distributionally Robust Optimization and Generative AI

Green manufacturing has become a strategic priority for many firms seeking to address sustainability and social responsibility, while improving production efficiency and profitability. However, integrating green technologies and renewable energy unavoidably introduces climate-related randomness that affects both product demand and renewable energy generation, underscoring the need for coordinated planning of production capacity and renewable energy development. To address this challenge, we develop a comprehensive two-stage distributionally robust optimization (DRO) model for green manufacturing capacity planning in a multi-factory, multi-capacity, and multi-product setting, based on an ambiguity set constructed by a data-driven clustering technique that leverages historical data of different availabilities and qualities. To handle the computational challenges of practical instances, an effective generative AI network is integrated into an exact decomposition algorithm, through a novel encoding/decoding scheme designed to provide the AI model with structurally informative training data and to convert AI-generated outputs into algorithm-accessible formats. Experimental results on real-world instances demonstrate that the proposed DRO approach achieves strong economic performance and robust feasibility under demand and renewable generation uncertainty, while also significantly improving computational efficiency and solution consistency relative to the standard approaches. Furthermore, our results highlight the managerial value of integrating green technology adoption with coordinated capacity planning to better utilize renewable energy and align production efficiency with sustainability and corporate social responsibility objectives.

math.OC↗

arXiv2026

Robust Operation of Distribution Networks: Generalized Uncertainty Modelling in Confidence-Level-Based Information Gap Decision

This paper studies the robust optimal operation of distribution networks (DNs) under renewable generation and load demand uncertainties, seeking an improved trade-off between robustness and economic performance. Building upon information gap decision theory (IGDT), a generalized uncertainty modelling is proposed to enhance the expressiveness of the uncertainty characterization. The proposed modelling captures both symmetric and asymmetric uncertainty features, and supports linear or nonlinear expansion of the uncertainty sets driven by confidence level. This advancement leads to the development of a confidence-level-based IGDT (CL-IGDT) framework for DN operation. To solve the resulting model, its equivalence to a family of two-stage robust optimization problems (TSROs) is established, enabling a Fibonacci search over the confidence level. To further improve computational efficiency, a cut-recycling strategy is proposed to exploit invariant information across TSROs. These techniques are integrated into a novel Fibonacci-Parametric Column-and-Constraint Generation algorithm with guaranteed asymptotic convergence. Case studies validate the effectiveness of the proposed framework and demonstrate the performance advantages of the proposed algorithm.

eess.SY↗

arXiv2026

CulturALL: Benchmarking Multilingual and Multicultural Competence of LLMs on Grounded Tasks

Large language models (LLMs) are now deployed worldwide, inspiring a surge of benchmarks that measure their multilingual and multicultural abilities. However, these benchmarks prioritize generic language understanding or superficial cultural trivia, leaving the evaluation of grounded tasks -- where models must reason within real-world, context-rich scenarios -- largely unaddressed. To fill this gap, we present CulturALL, a comprehensive and challenging benchmark to assess LLMs' multilingual and multicultural competence on grounded tasks. CulturALL is built via a human--AI collaborative framework: expert annotators ensure appropriate difficulty and factual accuracy, while LLMs lighten the manual workload. By incorporating diverse sources, CulturALL ensures comprehensive scenario coverage. Each item is carefully designed to present a high level of difficulty, making CulturALL challenging. CulturALL contains 2,610 samples in 14 languages from 51 regions, distributed across 16 topics to capture the full breadth of grounded tasks. Experiments show that the best LLM achieves 44.48% accuracy on CulturALL, underscoring substantial room for improvement.

cs.CL↗

arXiv2026

Two-stage robust bilevel optimization model for facility location considering operational service level under disruption risk

The bilevel facility location problem (BO-FLP) is one of the core optimization problems behind the design of many decentralized industrial systems, e.g., supply chain systems where a supplier constructs some critical facilities and then uses them to serve retailers in a cost-effective fashion, while retailers directly handle customers aiming to minimize the total unmet demand in a rather independent fashion. When uncertainty is considered, scenario-based stochastic approaches are commonly used, but they often become impractical due to insufficient data or an exponential number of scenarios. To address this issue, this paper adopts robust optimization and proposes a novel two-stage robust bilevel facility location model. Several structural properties are derived to improve both theoretical understanding and solution efficiency. Based on this, an enhanced column-and-constraint generation algorithm is developed for robust bilevel optimization with decision-dependent uncertainty, significantly improving exact solution capability over the standard method. Numerical results show that, compared to the centralized two-stage RO model, our model pays more attention to demand fulfillment, typically resulting in higher service efficiency and better utilization of supply capacity. Under a small-scale disruption, this new model delivers better service performance. However, under a large-scale disruption, the centralized model performs more effectively.

math.OC↗

arXiv2026

Difficulty-Estimated Policy Optimization

Recent advancements in Large Reasoning Models (LRMs), exemplified by DeepSeek-R1, have underscored the potential of scaling inference-time compute through Group Relative Policy Optimization (GRPO). However, GRPO frequently suffers from gradient signal attenuation when encountering problems that are either too trivial or overly complex. In these scenarios, the disappearance of inter-group advantages makes the gradient signal susceptible to noise, thereby jeopardizing convergence stability. While variants like DAPO attempt to rectify gradient vanishing, they do not alleviate the substantial computational overhead incurred by exhaustive rollouts on low-utility samples. In this paper, we propose Difficulty-Estimated Policy Optimization (DEPO), a novel framework designed to optimize the efficiency and robustness of reasoning alignment. DEPO integrates an online Difficulty Estimator that dynamically assesses and filters training data before the rollout phase. This mechanism ensures that computational resources are prioritized for samples with high learning potential. Empirical results demonstrate that DEPO achieves up to a 2x reduction in rollout costs without compromising model performance. Our approach significantly lowers the computational barrier for training high-performance reasoning models, offering a more sustainable path for reasoning scaling. Code and data will be released upon acceptance.

cs.AI↗

arXiv2026

New Outer Approximation Algorithms for Nonsmooth Convex MINLP Problems

This paper presents a novel outer approximation algorithm for nonsmooth mixed-integer nonlinear programming (MINLP) problems. The method proceeds by fixing the integer variables and solving the resulting nonlinear convex subproblem. When the subproblem is feasible, valid linear cuts are derived by computing suitable subgradients of the objective and constraint functions at the optimal solution, utilizing KKT optimality conditions. A new parameter, defined through the nonlinear constraint functions, is introduced to facilitate the generation of these cuts. For infeasible subproblems, a feasibility problem is solved, and valid linear cuts are generated via KKT-based subgradients to exclude the infeasible integer assignment. By integrating both types of cuts, a mixed-integer linear programming (MILP) master problem is formulated and proven equivalent to the original MINLP. This equivalence underpins a new outer approximation algorithm, which is guaranteed to terminate after a finite number of iterations. Numerical experiments on smooth convex MINLP problems demonstrate that the proposed algorithm produces tighter MILP relaxations than the classical outer approximation method. Furthermore, the approach offers an alternative mechanism for generating linear cuts, extending beyond reliance solely on first-order Taylor expansions and shows that the efficiency of outer approximation algorithm is strongly dependent on the inherent structure of the MINLP problem.

math.OC↗

arXiv2026

A complete phase-field fracture model for brittle materials subjected to thermal shocks

Brittle materials subjected to thermal shocks experience strong temperature gradients that in turn give rise to mechanical stresses that can be large enough to induce fracture. This work presents a complete model for phase-field fracture for coupled thermo-mechanical problems, wherein the bulk material properties, the material strength, and the fracture toughness are specified independently. The capabilities of the model are assessed across a wide span of scenarios in thermo-mechanical fracture, from the propagation of large pre-existing cracks to crack nucleation under spatially uniform states of stress. In particular, we revisit the controlled quenching of glass plates, and demonstrate how the model captures experimentally observed crack patterns across a range of thermal loads. Ceramic disks subjected to infrared radiation are also examined, with the model reproducing both straight cracks in notched specimens and branching in intact specimens. Finally, ceramic pellets subjected to rapid power pulses are examined, with the model explaining experimental transitions from intact to fractured pellets and the important role of material strength. The results demonstrate that the complete phase-field model unifies the treatment of distinct fracture scenarios under thermal shock, surpassing classical approaches and enabling more reliable prediction of brittle fracture in extreme environments.

physics.comp-ph↗

arXiv2025

$M^2PO$: Multi-Perspective Multi-Pair Preference Optimization for Machine Translation

Aligning Large Language Models (LLMs) with human preferences is pivotal for Machine Translation (MT), yet current approaches are often hindered by misleading reward signals. Our analysis reveals that prevailing Quality Estimation (QE) models exhibit a systematic blind spot toward partial errors, specifically partial hallucinations and omissions, often favoring superficially fluent but unfaithful translations. To address this issue, we propose $M^2PO$ (Multi-Perspective Multi-Pair Preference Optimization), a data-centric framework for preference optimization in machine translation. First, to correct the bias toward fluency, $M^2PO$ uses a dual-perspective mechanism that decouples semantic fidelity from fluency and prioritizes faithfulness through a curriculum strategy. Second, after correcting this bias, partial errors fall between perfect and severely incorrect translations, making them difficult to learn through standard best-versus-worst comparisons. We therefore introduce a multi-pair objective that leverages the full candidate list to capture these fine-grained error signals. Experiments on WMT23, WMT24, and FLORES-200 show that $M^2PO$ enables a 9B model to outperform leading open-source baselines and achieve parity with proprietary models such as GPT-4o and Gemini-2.0-Flash, demonstrating strong potential for efficient and high-fidelity LLM-based translation. Our code and dataset will be released.

cs.CL↗

arXiv2025

Leveraging Electric School Buses for Disaster Recovery: Optimizing Routing and Energy Scheduling via Branch-and-Price

Natural disasters threaten the resilience of power systems, causing widespread power outages that disrupt critical loads (e.g., hospitals) and endanger public safety. Compared to the conventional restoration methods that often have long response times, leveraging government-controlled electric school buses (ESBs) with large battery capacity and deployment readiness offers a promising solution for faster power restoration to critical loads during disasters while traditional maintenance is underway. Therefore, we study the problem of routing and scheduling a heterogeneous fleet of ESBs to satisfy the energy demand of critical isolated loads around disasters addressing the following practical aspects: combined transportation and energy scheduling of ESBs, multiple back-and-forth trips of ESBs between isolated loads and charging stations, and spatial-wise coupling among multiple ESB routes. We propose an efficient mixed-integer programming model for routing and scheduling ESBs, accounting for the practical aspects, to minimize the total restoration cost over a planning horizon. We develop an efficient exact branch-and-price (B&P) algorithm and a customized heuristic B&P algorithm integrating dynamic programming and labeling algorithms. Numerical results based on a real case study of San Antonio disaster shelters and critical facilities demonstrate that our proposed exact B&P and heuristic B&P algorithms are computationally 121 and 335 times faster, respectively, than Gurobi. Using network sparsity to incorporate the limitation in shelter-ESB type compatibility in the model demonstrates that the total restoration cost increases, on average, by 207% as the network becomes fully sparse compared to fully connected. The capacity utilization metric reflects that the proposed practical ESB routing and scheduling enables an ESB to meet the energy demand 4.5 times its effective usable capacity.

math.OC↗

arXiv2025

Beyond Black-Box Interventions: Latent Probing for Faithful Retrieval-Augmented Generation

Retrieval-Augmented Generation (RAG) systems often fail to maintain contextual faithfulness, generating responses that conflict with the provided context or fail to fully leverage the provided evidence. Existing methods attempt to improve faithfulness through external interventions, such as specialized prompting, decoding-based calibration, or preference optimization. However, since these approaches treat the LLM as a black box, they lack a reliable mechanism to assess when and why knowledge conflicts occur. Consequently, they tend to be brittle, data-intensive, and agnostic to the model's internal reasoning process. In this paper, we move beyond black-box interventions to analyze the model's internal reasoning process. We discover that conflicting and aligned knowledge states are linearly separable in the model's latent space, and contextual noise systematically increases the entropy of these representations. Based on these findings, we propose ProbeRAG, a novel framework for faithful RAG that operates in three stages: (i) fine-grained knowledge pruning to filter irrelevant context, (ii) latent conflict probing to identify hard conflicts in the model's latent space, and (iii) conflict-aware attention to modulate attention heads toward faithful context integration. Extensive experiments demonstrate that ProbeRAG substantially improves both accuracy and contextual faithfulness. The related resources are available at https://github.com/LinfengGao/ProbeRAG.

cs.CL↗

arXiv2025

Marco-Bench-MIF: On Multilingual Instruction-Following Capability of Large Language Models

Instruction-following capability has become a major ability to be evaluated for Large Language Models (LLMs). However, existing datasets, such as IFEval, are either predominantly monolingual and centered on English or simply machine translated to other languages, limiting their applicability in multilingual contexts. In this paper, we present an carefully-curated extension of IFEval to a localized multilingual version named Marco-Bench-MIF, covering 30 languages with varying levels of localization. Our benchmark addresses linguistic constraints (e.g., modifying capitalization requirements for Chinese) and cultural references (e.g., substituting region-specific company names in prompts) via a hybrid pipeline combining translation with verification. Through comprehensive evaluation of 20+ LLMs on our Marco-Bench-MIF, we found that: (1) 25-35% accuracy gap between high/low-resource languages, (2) model scales largely impact performance by 45-60% yet persists script-specific challenges, and (3) machine-translated data underestimates accuracy by7-22% versus localized data. Our analysis identifies challenges in multilingual instruction following, including keyword consistency preservation and compositional constraint adherence across languages. Our Marco-Bench-MIF is available at https://github.com/AIDC-AI/Marco-Bench-MIF.

cs.CL↗