Searcharxiv⌕ Search
Home/Archive/arXiv

arXiv · 2610.25710

Evidence Sufficiency and Model Comparisons in MexRegQA: An Item-Level Audit of Mexican-Spanish Regulatory QA

Fred Hu·Yeqi Wang·Xing Zhou
View original source ↗Read source document ↗Export metadata ↓

Abstract

Similarity-gated coverage does not establish that retrieved evidence contains the facts needed to answer a question. We audit Mexican-Spanish financial-regulatory QA and find that 35 of 135 covered items lack some or all load-bearing facts. On the post-hoc full-evidence slice (n=98), eleven prompt-matched models score 95-98/98 and none differs significantly from the anchor; this does not establish equivalence. Removing evidence-defective items changes the nominal significance status of 14 of 55 pairwise comparisons. On 56 zero-evidence items, accuracy spans 33.9%-89.3%. Removing evidence from the audited items under an abstention-permitting prompt reduces accuracy by 63-81 percentage points in all ten tested legs, with no recovered items. Building on sufficient-context research, we contribute MexRegQA v1.0: 192 items with provenance and evidence-status labels, frozen retrieval evidence for 191 evaluated items, predictions, judge outputs and offline checks. The audit was outcome-aware, the later model/panel sweep reused the same questions, and all reported scores and labels rely on machine evaluation without independent human validation. We propose releasing evidence-status labels alongside grounded QA benchmarks so that model comparisons can be examined separately from evidence availability.

Explore related subjects

cs.LG

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Fred Hu, Yeqi Wang, Xing Zhou. 2026-10-19. Evidence Sufficiency and Model Comparisons in MexRegQA: An Item-Level Audit of Mexican-Spanish Regulatory QA. https://arxiv.org/abs/2610.25710

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Simulating, Fast and Slow: Learning Policies for Black-Box Optimization

Simulators are vital in science and engineering, as they faithfully model the influence of design parameters on real-world observations. Optimizing design parameters to minimize a desired objective function is notoriously challenging since simulators are often non-differentiable black boxes. Moreover, each simulation incurs significant compute time, thus doing such optimizations naively may be prohibitively expensive. Furthermore, in experiment design settings, practitioners are typically required to solve the same optimization problem for various input distributions before proceeding to real-world experiments. Starting these related optimizations from scratch each time can be inefficient if the forward simulation model is expensive to evaluate. To address these challenges, this paper introduces a novel method for solving black-box optimization problems on various input distributions by learning a policy that guides the training of a differentiable surrogate. The surrogate's gradients are then used to optimize the simulation parameters with SGD. After training the policy, the cost for downstream optimization of problems involving black-box simulators is amortized, and we require up to $10\times$ fewer expensive simulator calls compared to baselines such as previous local surrogate-based approaches, numerical optimization, and Bayesian optimization.

cs.LG↗
arXiv2026

Physics Inspired Neural Networks Enable Efficient Graph Coloring

Combinatorial optimization problems near algorithmic phase transitions represent a fundamental challenge for both classical algorithms and machine learning approaches. Among them, graph coloring is a prototypical constraint satisfaction problem characterized by phase transitions that fragment the solution space into distinct clusters, making the search for valid solutions challenging. Here we introduce a physics-inspired neural framework that learns to solve large-scale graph coloring instances by combining graph neural networks with statistical-mechanics principles. Our approach integrates a planting-based supervised signal, symmetry-breaking regularization, and iterative noise-annealed neural dynamics to navigate complex solution landscapes. When the number of iterations scales quadratically with graph size, the learned solver successfully tackles increasingly challenging instances, even in regimes where finding valid solutions becomes particularly difficult, demonstrating its ability to extract meaningful information from the graph structure. The model generalizes from small training graphs to instances up to 20 times larger, demonstrating that neural architectures can learn scalable algorithmic strategies that remain effective in hard connectivity regions. These results establish a general framework for learning neural solvers that operate near fundamental phase boundaries in combinatorial optimization and inference.

cs.LG↗
arXiv2026

FiTv2: Scalable and Improved Flexible Vision Transformer for Diffusion Model

\textit{Nature is infinitely resolution-free}. In the context of this reality, existing diffusion models, such as Diffusion Transformers, often face challenges when processing image resolutions outside of their trained domain. To address this limitation, we conceptualize images as sequences of tokens with dynamic sizes, rather than traditional methods that perceive images as fixed-resolution grids. This perspective enables a flexible training strategy that seamlessly accommodates various aspect ratios during both training and inference, thus promoting resolution generalization and eliminating biases introduced by image cropping. On this basis, we present the \textbf{Flexible Vision Transformer} (FiT), a transformer architecture specifically designed for generating images with \textit{unrestricted resolutions and aspect ratios}. We further upgrade the FiT to FiTv2 with several innovative designs, includingthe Query-Key vector normalization, the AdaLN-LoRA module, a rectified flow scheduler, and a Logit-Normal sampler. Enhanced by a meticulously adjusted network structure, FiTv2 exhibits $2\times$ convergence speed of FiT. When incorporating advanced training-free extrapolation techniques, FiTv2 demonstrates remarkable adaptability in both resolution extrapolation and diverse resolution generation. Additionally, our exploration of the scalability of the FiTv2 model reveals that larger models exhibit better computational efficiency. Furthermore, we introduce an efficient post-training strategy to adapt a pre-trained model for the high-resolution generation. Comprehensive experiments demonstrate the exceptional performance of FiTv2 across a broad range of resolutions. We have released all the codes and models at https://github.com/whlzy/FiT to promote the exploration of diffusion transformer models for arbitrary-resolution image generation.

cs.LG↗