Searcharxiv⌕ Search

arXiv subjects

Javad Lavaei

Publications and source records attributed to Javad Lavaei.

At least 19 recordsLinked to original sources
arXiv2026

Strategic Arbitrage in Electricity Markets: A Multi-Agent Policy Optimization Framework

Grid-scale storage and load aggregators are growing quickly in wholesale electricity markets. If highly consolidated, these new participants may distort the competitive market equilibrium through strategic withholding behavior. Traditional market power mitigation techniques are ill-suited to arbitrage-based participants. We provide a multi-agent policy optimization model to study game-theoretic equilibria for real-time markets with consolidated storage and/or aggregators. The model is designed to facilitate natural first-order equilibrium-seeking update rules. We analyze the theoretical properties of different supply functions and their implications on the convergence of the gradient dynamics. Finally, we train equilibrium policies for a modified version of the IEEE RTS-GMLC benchmark across different levels of storage fleet consolidation, to demonstrate the potential of the framework as a tool to characterize and quantify strategic arbitrage.

eess.SY↗
arXiv2026

A Deficiency-Based Approach for Unbalanced Three-Phase Distribution System Expansion Planning

Distribution planners typically characterize grid needs through power flow studies and fit upgrades to those needs by hand. More principled one-shot optimization methods either do not scale or misrepresent unbalanced three-phase networks. This paper proposes an iterative planning process that alternates between two problem classes with mature off-the-shelf solvers. An optimal power flow (OPF) problem quantifies the deficiency of a candidate portfolio by optimizing the dispatch of controllable devices and measuring thermal and voltage violations on a common scale, with an optimal value of zero exactly at adequate portfolios. The OPF multipliers give a first-order approximation of the relief each investment candidate would provide. We bound the error of this approximation and show that structural properties of the deficiency guard it against overestimation. Building on this property, a heuristic cutting-plane search uses a mixed-integer linear program to propose each next portfolio from these approximations. On six unbalanced three-phase feeders from the SMART-DS dataset, the search outperforms a linearized optimization-based benchmark, and the results agree across five variants of the deficiency.

eess.SY↗
arXiv2026

Defense Filters for Robust Learning of Partially Observed Dynamical Systems

Robust estimation of unknown system dynamics from partial observations under attack is a core challenge in learning for control, increasingly critical in multi-agent networks subject to adversarial data poisoning. Existing methods either require highly restrictive assumptions on the adversary or yield weak error bounds. In this paper, we consider identifying partially observed linear dynamical systems subject to adversarial attacks with an attack probability $p < 0.5$ at each time. We propose a defense-filtering framework in which a system designer leverages knowledge of the observation matrix to construct a defense filter satisfying full column rank and subspace separability conditions, which enables the evaluation of a single-attack scenario against observed data. Using this framework, we analyze the constrained $\ell_2$-norm estimator and establish finite-sample error bounds for recovering the Markov parameter matrix $G^*$. This is the first approach to provide provable system identification guarantees via a defense filter for partially observed systems subject to harsh attack environments where the attack probability approaches $0.5$.

math.OC↗
arXiv2026

Groundability, Not Scale Alone: When Weak Reviewers Can Audit Strong Coding Agents

Coding agents can return plausible patches that omit required behavior. These failures are hard to review because long traces and confident summaries often hide what was missed. We ask when a nominally weaker reviewer can reliably decide whether a patch solves its issue. We study 411 execution-labeled traces from three agents and 101 controlled cases. On 154 GPT-5.4 traces, structured but unchecked evidence raises both defect catch and over-rejection. We then provide official execution evidence as an upper-bound diagnostic. After choosing and freezing one of two formats per reviewer, five of six reviewers improve both rates on 122 held-out traces; two classify every trace correctly. Reviewer size is not a consistent predictor of quality. Because official tests are unavailable in deployment, we also evaluate a frozen cascade with patch-caused static errors and generated tests that first fail on the unpatched repository. On 121 scored held-out GPT-5.4 traces and 59 Gemini traces, its coverage is 0.89 and 0.86, risk is 0.33 and 0.26, catch is 0.76 and 0.80, and over-rejection is 0.66 and 0.67. Most false rejections occur when unresolved cases reach the reviewer. Official execution evidence shows the potential of weak review when decisive checks are available. Producing equally reliable checks without official tests remains the main bottleneck.

cs.SE↗
arXiv2026

When Context Changes: Understanding Update Failures in LLMs

As preferences, goals, and facts change, LLM agents must use the current state while earlier versions remain in context. Yet they can answer with an old value of the same variable, a failure that we call stale binding. To study when models use outdated information and why, we introduce Controlled In-Context Memory (CICM), a benchmark for tracking and using updated information in conversations and agent logs. We observe that even frontier reasoning models can fail to recover the current state. We find that in open-source models probes can still recover the updated value when the model answers with an old one, pointing to a failure to select information that remains available. Component tests in Qwen and Pythia identify a mechanism for this selection failure: attention drift, where attention favors old values over the current one when producing an answer. We study a one-layer transformer to mathematically understand how this phenomenon happens: when attention scores are similar, several old values can together receive more attention than the current value. Guided by this explanation, we redirect attention toward the current value without further training. When the current value is requested directly, adjusting this intervention for each input corrects most old-value errors across various model families while preserving nearly all initially correct answers. Reliable context management therefore requires more than remembering updated information: models must use it to guide their answers.

cs.AI↗
arXiv2026

Huber-based Robust System Identification with Near-Optimal Guarantees Across Independent and Adversarial Regimes

Dynamical systems can confront one of two extreme types of disturbances: persistent zero-mean independent noise, and sparse nonzero-mean adversarial attacks, depending on the specific scenario being modeled. While mean-based estimators like least-squares are well-suited for the former, a median-based approach such as the $\ell_1$-norm estimator is required for the latter. In this paper, we propose a Huber-based estimator, characterized by a threshold constant $μ$, to identify the governing matrix of a linearly parameterized nonlinear system from a single trajectory of length $T$. This formulation bridges the gap between mean- and median-based estimation, achieving provably robust error in both extreme disturbance scenarios under mild assumptions. In particular, for persistent zero-mean noise with a positive probability density around zero, the proposed estimator achieves an $\mathcal{O}(1/\sqrt{T})$ error rate if the disturbance is symmetric or the basis functions are linear. For arbitrary nonzero-mean attacks that occur at each time with probability smaller than 0.5, the error is bounded by $\mathcal{O}(μ)$. We validate our theoretical results with experiments illustrating that integrating our approach into frameworks like SINDy yields robust identification of discrete-time systems.

math.OC↗
arXiv2026

System Identification under Noise and Attack Regimes: Agnostic and Composite Robustness

Dynamical systems often confront persistent zero-mean independent noise and/or sparse nonzero-mean adversarial attacks. While mean-based estimators like least-squares handle the former, the median-based $\ell_1$-norm estimator is effective for the latter. In this paper, we develop robust system identification frameworks to identify a linearly-parametrized nonlinear system from a single trajectory of length $T$. We tackle two types of robustness: (1) $\textit{agnostic robustness}$ under either pure noise or pure attacks without knowing which regime is active; and (2) $\textit{composite robustness}$ under concurrent noise and attacks. We first show that the Huber estimator attains agnostic robustness by achieving an $\mathcal{O}(1/\sqrt{T})$ error rate for the noise regime and a bounded error for the attack regime, serving as a one-stage estimator interpolating between mean- and median-based methods. We then prove that no convex one-stage estimator is consistent under both noise and attacks, which motivates a two-stage estimation that sequentially applies median- and mean-based estimators for composite robustness. These dual notions of robustness require a corresponding duality in estimator design, providing a solid foundation for robust control in safety-critical systems.

math.OC↗
arXiv2026

On the Sharp Input-Output Analysis of Nonlinear Systems under Adversarial Attacks

This paper is concerned with learning the input-output mapping of general nonlinear dynamical systems. While the existing literature focuses on Gaussian inputs and benign disturbances, we significantly broaden the scope of admissible control inputs and allow correlated, nonzero-mean, adversarial disturbances. With our reformulation as a linear combination of basis functions, we prove that the $\ell_2$-norm estimator overcomes the challenges posed by an adversary with access to the full information history, provided that the attack times are sparse, i.e., the probability that the system is under adversarial attack at a given time is smaller than a certain threshold. We provide an estimation error bound that decays with the input memory length and prove its optimality by constructing a problem instance that suffers from the same bound under probabilistic adversarial attacks. Our work provides a sharp input-output analysis for a generic nonlinear and partially observed system under significantly generalized assumptions compared to existing works.

math.OC↗
arXiv2026

LLMs Should Express Uncertainty Explicitly

Large language models (LLMs) often produce confident yet incorrect answers, which can lead to risky failures in real-world applications. We study whether post-training can make a model's self-assessment explicit: when the model is uncertain, can it be trained to signal so within its own response? A central design question is where in the response this signal should be exposed -- during reasoning, while the answer is still being formed, or at the end, once the answer has been produced. We study both. For end-of-reasoning self-assessment, we train the model to verbalize a confidence score for its response, with the aim of high confidence on correct answers and low confidence on incorrect ones. For during-reasoning self-assessment, we train the model to emit the marker whenever its current reasoning state appears unreliable. Across factual reasoning tasks, both forms sharply reduce overconfident errors while improving answer quality, and both can be used as triggers for retrieval augmented generation (RAG) to improve the final response. We further analyze their internal mechanisms: end-of-reasoning verbalized confidence sharpens a confidence-related structure already present in the pretrained model, whereas during-reasoning emission teaches the model to mark high-risk reasoning steps, with parameter changes concentrated in the model's late layers.

cs.LG↗
arXiv2026

Structural Correspondence and Universal Approximation in Diagonal plus Low-Rank Neural Networks

The massive computational costs of scaling modern deep learning architectures have driven the widespread use of parameter-efficient low-rank structures, such as LoRA and low-rank factorization. However, theoretical guarantees for their expressive power are less explored, often relying on restrictive priors like a pretrained base matrix, ReLU activations or non-verifiable singularity conditions. We first investigate the limits of neural networks constrained strictly to low-rank manifolds without pretrained dense priors. We demonstrate a theoretical paradox: while purely rank-1 layers can exactly interpolate arbitrary scalar datasets, they collapse for function approximations. To overcome this bottleneck without surrendering parameter efficiency, we introduce a unified \textit{Structural Correspondence} framework. We prove that augmenting low-rank layers with only a minimal sparse diagonal component, say a Diagonal plus Low-Rank (DLoR) structure, is sufficient to reach Universal Approximation. We show that any full-rank transformation can be exactly reconstructed using these DLoR components by trading off network width (additive decomposition) or depth (multiplicative decomposition). By tracking asymptotic Taylor remainders, we prove that DLoR neural networks fully restore the Universal Approximation Theorem for general activation functions. Finally, we establish that multiplicative depth provides superior parameter-to-expressivity scaling compared to additive width. Our results show that dense matrices and specific activation functions are not topological prerequisites for universal expressivity.

cs.LG↗
arXiv2026

StyleBench: Evaluating thinking styles in Large Language Models

Structured reasoning can improve the inference performance of large language models (LLMs), but it also introduces computational cost and control constraints. When additional reasoning structure helps, and when it instead reduces efficiency or robustness, remains poorly understood. We propose StyleBench, where we study reasoning structure as a capacity-constrained design choice rather than a fixed inference recipe. We evaluate five representative reasoning styles: Chain-of-Thought, Tree-of-Thought, Algorithm-of-Thought, Sketch-of-Thought, and Chain-of-Draft across five reasoning tasks and 15 open-source LLMs ranging from 270M to 120B parameters. We find that greater structural complexity improves accuracy only in limited regimes defined by task demands and model capacity. Search-based styles help on open-ended combinatorial problems but fail on smaller models, while concise styles achieve large efficiency gains on structured tasks without sacrificing performance. We also identify systematic failure modes in smaller models, including premature guessing and weak adherence to reasoning-control instructions. To study adaptive reasoning control, we further compare supervised and reinforcement-based strategy selection on Qwen-7B-Instruct. Supervised fine-tuning collapses to shallow style preferences, whereas GRPO learns stronger adaptive control and improves downstream performance. Together, these results clarify when structured reasoning is useful, when it is wasteful, and why learning to choose a reasoning strategy is itself a challenging inference problem, we open source the benchmark in https://github.com/JamesJunyuGuo/Style_Bench.

cs.LG↗
arXiv2026

High Probability Complexity Bounds of Trust-Region Stochastic Sequential Quadratic Programming with Heavy-Tailed Noise

In this paper, we consider nonlinear optimization problems with a stochastic objective and deterministic equality constraints. We propose a Trust-Region Stochastic Sequential Quadratic Programming (TR-SSQP) method and establish its high-probability iteration complexity bounds for identifying first- and second-order $ε$-stationary points. In our algorithm, we assume that exact objective values, gradients, and Hessians are not directly accessible but can be estimated via zeroth-, first-, and second-order probabilistic oracles. Compared to existing complexity studies of SSQP methods that rely on a zeroth-order oracle with sub-exponential tail noise (i.e., light-tailed) and focus mostly on first-order stationarity, our analysis accommodates biased (also referred to as irreducible in the literature) and heavy-tailed noise in the zeroth-order oracle, and significantly extends the analysis to second-order stationarity. We show that under heavy-tailed noise conditions, our SSQP method achieves the same high-probability first-order iteration complexity bounds as in the light-tailed noise setting, while further exhibiting promising second-order iteration complexity bounds. Specifically, the method identifies a first-order $ε$-stationary point in $\mathcal{O}(ε^{-2})$ iterations and a second-order $ε$-stationary point in $\mathcal{O}(ε^{-3})$ iterations with high probability, provided that $ε$ is lower bounded by a constant determined by the bias magnitude (i.e., the irreducible noise) in the estimation. We validate our theoretical findings and evaluate practical performance of our method on CUTEst benchmark test set.

math.OC↗
arXiv2026

A Trust-Region Interior-Point Stochastic Sequential Quadratic Programming Method

In this paper, we propose a trust-region interior-point stochastic sequential quadratic programming (TR-IP-SSQP) method for solving optimization problems with a stochastic objective and deterministic nonlinear equality and inequality constraints. In this setting, exact evaluations of the objective function and its gradient are unavailable, but their stochastic estimates can be constructed. In particular, at each iteration our method builds stochastic oracles, which estimate the objective value and gradient to satisfy proper adaptive accuracy conditions with a fixed probability. To handle inequality constraints, we adopt an interior-point method (IPM), in which the barrier parameter follows a prescribed decaying sequence. Under standard assumptions, we establish global almost-sure convergence of the proposed method to first-order stationary points. We implement the method on a subset of problems from the CUTEst test set, as well as on logistic regression problems, to demonstrate its practical performance.

math.OC↗
arXiv2026

Why is Normalization Preferred? A Worst-Case Complexity Theory for Stochastically Preconditioned SGD under Heavy-Tailed Noise

We develop a worst-case complexity theory for stochastically preconditioned stochastic gradient descent (SPSGD) and its accelerated variants under heavy-tailed noise, a setting that encompasses widely used adaptive methods such as Adam, RMSProp, and Shampoo. We assume the stochastic gradient noise has a finite $p$-th moment for some $p \in (1,2]$, and measure convergence after $T$ iterations. While clipping and normalization are parallel tools for stabilizing training of SGD under heavy-tailed noise, there is a fundamental separation in their worst-case properties in stochastically preconditioned settings. We demonstrate that normalization guarantees convergence to a first-order stationary point at rate $\mathcal{O}(T^{-\frac{p-1}{3p-2}})$ when problem parameters are known, and $\mathcal{O}(T^{-\frac{p-1}{2p}})$ when problem parameters are unknown, matching the optimal rates for normalized SGD, respectively. In contrast, we prove that clipping may fail to converge in the worst case due to the statistical dependence between the stochastic preconditioner and the gradient estimates. To enable the analysis, we develop a novel vector-valued Burkholder-type inequality that may be of independent interest. These results provide a theoretical explanation for the empirical preference for normalization over clipping in large-scale model training.

cs.LG↗
arXiv2026

3DGS$^2$-TR: Scalable Second-Order Trust-Region Method for 3D Gaussian Splatting

We propose 3DGS$^2$-TR,a second-order optimizer for accelerating the scene training problem in 3D Gaussian Splatting (3DGS). Unlike existing second-order approaches that rely on explicit or dense curvature representations, such as 3DGS-LM (Höllein et al., 2025) or 3DGS2 (Lan et al., 2025), our method approximates curvature using only the diagonal of the Hessian matrix, efficiently via Hutchinson's method. Our approach is fully matrix-free and has the same complexity as ADAM (Kingma, 2024), $O(n)$ in both computation and memory costs. To ensure stable optimization in the presence of strong nonlinearity in the 3DGS rasterization process, we introduce a parameter-wise trust-region technique based on the squared Hellinger distance, regularizing updates to Gaussian parameters. Under identical parameter initialization and without densification, 3DGS$^2$-TR is able to achieve better reconstruction quality on standard datasets, using 50% fewer training iterations compared to ADAM, while incurring less than 1GB of peak GPU memory overhead (17% more than ADAM and 85% less than 3DGS-LM), enabling scalability to very large scenes and potentially to distributed training settings.

cs.CV↗
arXiv2026

TRSVR: An Adaptive Stochastic Trust-Region Method with Variance Reduction

We propose a stochastic trust-region method for unconstrained nonconvex optimization that incorporates stochastic variance-reduced gradients (SVRG) to accelerate convergence. Unlike classical trust-region methods, the proposed algorithm relies solely on stochastic gradient information and does not require function value evaluations. The trust-region radius is adaptively adjusted based on a radius-control parameter and the stochastic gradient estimate. Under mild assumptions, we establish that the algorithm converges in expectation to a first-order stationary point. Moreover, the method achieves iteration and sample complexity bounds that match those of SVRG-based first-order methods, while allowing stochastic and potentially gradient-dependent second-order information. Extensive numerical experiments demonstrate that incorporating SVRG accelerates convergence, and that the use of trust-region methods and Hessian information further improves performance. We also highlight the impact of batch size and inner-loop length on efficiency, and show that the proposed method outperforms SGD and Adam on several machine learning tasks.

math.OC↗
arXiv2026

All-Pass Fractional OPF: A Solver-Friendly, Physics-Preserving Approximation of AC OPF

This paper presents a fractional approximation of the AC optimal power flow (AC OPF) problem based on an all-pass approximation of the exponential power flow kernel. The classical AC OPF relies on trigonometric coupling between bus voltage phasors, which yields a nonconvex program with oscillatory derivatives that can slow, or in some cases destabilize, interior-point methods. We replace the trigonometric terms with an all-pass fractional (APF) approximation whose real and imaginary components act as smooth surrogates for the cosine and sine functions, and we introduce a pre-rotation to shift the argument of the approximation toward its most accurate region, ensuring that the reformulated power flow model preserves physical loss behavior, maintains the symmetry of the classical kernels, and improves the conditioning of the Jacobian and Hessian matrices. The proposed APF OPF formulation remains nonconvex, as in the classical model, but it eliminates trigonometric evaluations and empirically produces larger and more stable Newton steps under standard interior-point solvers. Numerical results on more than 25 IEEE and PGLib test systems ranging from 9 to 10{,}000 buses demonstrate that the APF OPF model achieves solutions with accuracy comparable to that of the classical formulation while reducing solver times, indicating a more solver-friendly nonconvex representation of AC OPF. All code, functions, verification scripts, and generated results are publicly available on \href{https://github.com/LSU-RAISE-LAB/APF-OPF}{GitHub}, along with a README describing how to run and reproduce the experiments.

eess.SY↗
arXiv2025

Subgradient Method for System Identification with Non-Smooth Objectives

This paper investigates a subgradient-based algorithm to solve the system identification problem for linear time-invariant systems with non-smooth objectives. This is essential for robust system identification in safety-critical applications. While existing work provides theoretical exact recovery guarantees using optimization solvers, the design of fast learning algorithms with convergence guarantees for practical use remains unexplored. We analyze the subgradient method in this setting, where the optimization problems to be solved evolve over time as new measurements are collected, and we establish linear convergence to the ground-truth system for both the best and Polyak step sizes after a burn-in period. We further characterize sublinear convergence of the iterates under constant and diminishing step sizes, which require only minimal information and thus offer broad applicability. Finally, we compare the time complexity of standard solvers with the subgradient algorithm and support our findings with experimental results. This is the first work to analyze subgradient algorithms for system identification with non-smooth objectives.

math.OC↗