Searcharxiv⌕ Search

arXiv subjects

Kaiqing Zhang

Publications and source records attributed to Kaiqing Zhang.

At least 19 recordsLinked to original sources

Joint Communication-Control Strategy Optimization with Partially Nested Information Structures: The Linear-Quadratic Case

In this paper, we formalize a joint communication-control strategy optimization (JCCO) problem in multi-agent linear systems with quadratic costs, under the common-information-based (CIB) framework from decentralized stochastic control. For computational tractability, we focus on such JCCO problems with partially nested (PN) information structures (ISs). In particular, with a baseline communication protocol that leads to a PN IS, we establish a series of conditions under which the partial nestedness is preserved under the (additional) communication strategies to be optimized, while violating them may cause nonlinearity or non-existence of the optimal control strategies, with open-loop communication strategies. Under these conditions, we then develop a dynamic-programming-based approach to compute the optimal control strategies of JCCO with open-loop communication strategies, which yields a set of closed-form Riccati Equations. As a byproduct of independent interest, such an approach also offers a new way to solve decentralized linear-quadratic control with PN ISs and output feedback, under the CIB framework and the aforementioned conditions. Finally, we extend such an approach to JCCOs with closed-loop communication strategies, yielding a more tractable dynamic program than an infinite-dimensional CIB-belief-based one.

eess.SY↗

Fictitious Play When Agents Are Not Expected-Utility Maximizers: A Geometric Analysis in $2$-Action Games

Fictitious play (FP) is one of the earliest and most fundamental learning procedures in games, where each agent independently chooses a best response to the empirical distribution of opponents' past actions by maximizing the expected payoff with respect to it. However, learning agents may not always be expected-utility maximizers: they may use nonlinear evaluation functionals to account for the uncertainty induced by the opponents, e.g., via risk measures. In this work, we study nonlinear fictitious play (NL-FP), in which each pure action is evaluated by a general, possibly nonlinear functional of the payoff distribution induced by the opponent's empirical belief. To analyze NL-FP, we first identify a single-crossing property (SCP) of evaluation functionals that preserves the classical geometry of FP in $2\times2$ games. Furthermore, we conduct a novel geometric analysis of the convergence behavior of NL-FP in $2\times2$ games, even when the SCP fails. Beyond $2\times2$ games, we show that the convergence of NL-FP in $2\times n$ ($n\ge3$) games may fail even for continuous-time NL-FP, when the two-action player uses conditional value-at-risk (CVaR) in a $2\times3$ game. In addition, we identify a finite-crossing condition that ensures convergence of the selected continuous-time dynamics and the discrete-time iterates in general $2\times n$ games, which also provides new insights into the global landscape of continuous-time classical FP in such games.

cs.GT↗

1.56 THz Full-Field Rydberg-Atom Imaging with 0.3 mm-scale Spatial Response

THz imaging via Rydberg-atom up-conversion offers a powerful full-field modality with high sensitivity and rapid acquisition. However sub-millimeter imaging above 1 THz is still constrained by source power and beam-delivery loss. In this paper, we demonstrate a \(50~\mathrm{mm} \times 50~\mathrm{mm}\) full-field imaging at 1.56~THz by implementing the \(11P_{3/2} \rightarrow 10D_{5/2}\) transition of a cesium Rydberg-atom vapor. The measured spatial response FWHMs are \(0.314~\mathrm{mm}\) in horizontal and \(0.354~\mathrm{mm}\) in vertical for a \(0.5~\mathrm{mm}\) pinhole, alongside a sensitivity of \(31.7~\mathrm{fW}\,\mathrm{s}^{-1/2}/(6.5 \times 6.5)~μ\mathrm{m}^{2}\), and a frame rate up to \(300~\mathrm{fps}\). Under matched illumination, relay, and image-processing conditions, mask imaging at 1.56 THz reproduces finer features and edges with substantially higher edge contrast, deeper intensity-trough separation in line profiles, and recovered detail that remains blurred or unresolved at 0.55 THz. Furthermore, preliminary imaging of a \(200~μ\mathrm{m}\) tissue slice suggests morphological correspondence with H\&E histology histology, offered as a proof-of-concept for label-free tissue characterization. By validating the wavelength-dependent resolution scaling in Rydberg THz imaging, this work establishes frequency scaling as a general design principle for sub-millimeter Rydberg-THz imaging.

physics.optics↗

TeamGym: A Gym for Team Decision-Making by AI Agents with Distributed Information

Many real-world decisions require teams to integrate information distributed across participants. Yet AI agents are still primarily trained and benchmarked in isolation. We introduce TeamGym, an environment for evaluating and training language model-based agents on team decision-making problems constructed from real-world legal, finance, medical, and software engineering corpora. TeamGym introduces a method to identify a minimal set of facts, each element of which is necessary and all of which are collectively sufficient for a correct final decision, which we call pivotal information. This enables TeamGym to both control the distribution of pivotal information and track how well agents identify and integrate pivotal information when making team decisions. Examining 8 language models on 12 source corpora, we find that current AI agents often fail to effectively share information: Distributing pivotal information across participants lowers success rate by 22.9 percentage points on average relative to giving the same information to a single model (a communication gap). We also document that participants do not effectively retrieve decision-relevant information on their own: A team's success rate is lowered by a further 19.9 points when participants must identify and extract pivotal information from source material rather than receiving it directly (a retrieval gap). To help remedy these deficits, we use TeamGym to train policies through reinforcement learning on team decision loss. Across two policy-training experiments, we find that training improves final-decision quality and induces emergent coordination in participation, speaker selection, and communication, even though the reward evaluates only the final decision.

cs.AI↗

Beyond End-to-End Recovery: Towards a Better Understanding of LLM Steganography via Rolewise Decomposition

In large language model (LLM) steganography, an encoder hides a private message in an otherwise ordinary response, as text transmitted to a decoder, who attempts to recover it. The success of this private-message transmission is commonly measured through end-to-end (E2E) recovery, which checks whether the decoder correctly reconstructs the private message. However, E2E recovery cannot distinguish encoding failures from decoding failures or reveal how the private message is represented in the encoder's transmitted text. To address these limitations, we introduce a rolewise decomposition framework that separately examines how private messages are represented in encoder transmitted texts, what decoders recover, and what deterministic verification can establish. Across 7,200 encoder responses, we identify 168 verifiable encodings missed by decoders and 706 responses containing private messages reconstructed through word-length or token-initial patterns not explicitly specified in the encoder's instructions. These findings demonstrate that E2E recovery alone may obscure important distinctions between encoding and decoding, motivating their separate evaluation to better understand and mitigate the safety issues caused by LLM steganography.

cs.CR↗

Coherent collective amplification of terahertz microbunching seeded by laser frequency beating in relativistic electron beams

High-power, continuously tunable terahertz sources based on free-electron lasers require precise control of electron-beam microstructures. Quantitative prediction of the modulation amplitude across a broad frequency range remains challenging because the laser-induced distribution and its collective evolution must be treated together. To describe this coupled evolution, a nonlinear model is developed for microbunching seeded by a frequency-beating laser heater. The non-Gaussian heater-exit distribution is obtained by optical phase averaging and propagated through multistage compression in six-dimensional phase space. Source-induced correlations are retained, with space charge, coherent synchrotron radiation, and radio-frequency wakefields evaluated self-consistently. The wavelength-dependent bunching response is thereby connected to the laser beat frequency, laser power, and compression partition. The short-wavelength double peak is found to be governed mainly by longitudinal space charge, while bunching near a selected wavelength can be enhanced by redistributing compression at fixed total compression. The predicted source modulation and downstream response are benchmarked against Elegant and IMPACT-Z, respectively. Experimental measurements of the wavelength-dependent bunching factor at different laser pulse energies are found to agree well with the trends predicted by theory and simulation. The resulting framework provides a computationally efficient and predictive route to optimizing electron-beam microbunching for tunable, high-power terahertz generation at free-electron-laser facilities.

physics.acc-ph↗

Entropic Risk-Sensitive Evolutionary Learning and Equilibrium Selection in Coordination Games

We study risk-sensitive evolutionary learning dynamics and their long-run equilibrium selection behaviors in coordination games. Agents' risk attitudes enter through the classical entropic risk measure, which evaluates opponent-induced payoff uncertainty and feeds into noisy best responses under two standard revision protocols: best response with mutations and logit choice. We first analyze $2\times 2$ coordination games in both single-population symmetric and two-population asymmetric settings. In the single-population setting, unlike the risk-neutral case where the dynamics are known to favor the risk-dominant equilibrium, we show that risk sensitivity can change the stochastically stable outcome: a greater risk-seeking attitude favors the payoff-dominant equilibrium, while a greater risk-averse attitude favors the maximin equilibrium. Thus, the population's risk attitude may act as a control knob for long-run equilibrium selection. In both population settings, we also identify a robust regime: any super-dominant equilibrium is stochastically stable for all risk attitudes, under both protocols, and across populations. We further extend the single-population analysis to symmetric $k$-action games, which include symmetric $k$-action coordination games as a special case, under risk-sensitive best response with mutations. In this setting, we show that, for sufficiently large populations, sufficiently risk-seeking agents uniquely select the strongly payoff-dominant equilibrium when it exists, whereas sufficiently risk-averse agents uniquely select the strongly maximin equilibrium when it exists. These results show that entropic risk sensitivity may serve as a systematic mechanism for steering equilibrium selection in evolutionary games, beyond the classical risk-neutral benchmark.

eess.SY↗

Self-Synchronized Terahertz and X-Ray Free-Electron Lasers from a Single Pre-Bunched Electron Beam

Ultrafast pump-probe spectroscopy combining intense terahertz (THz) and X-ray pulses is a critical tool for investigating complex structural and electronic dynamics in materials. However, current setups combining THz sources and X-ray free-electron lasers (FELs) often suffer from high system complexity, inherent timing jitter, or limited THz pulse properties. Here, we experimentally demonstrate the generation of intrinsically synchronized, strong-field, narrow-band THz and X-ray FELs from a single pre-bunched electron beam. Sequentially passing the beam through X-ray and THz amplifiers reveals a highly synergistic process: the initial periodic THz density modulation notably boosts the X-ray FEL pulse energy, while robustly surviving the intense X-ray emission to drive high-power, narrow-band THz radiation. Originating from the same electron bunch, the two pulses inherently maintain a precise, constant time delay. This jitter-free scheme establishes a highly reliable platform tailored for both X-ray-pump/THz-probe and THz-pump/X-ray-probe experiments.

physics.acc-ph↗

Biodegradable, Millimeter-Scale Light-Emitting Sensors for Distributed Environmental Monitoring-Functional Pixie Dust

Methods for large-area, precise monitoring across natural environments are of growing interest due to pressing needs for sustainable management of rapidly increasing anthropogenic activities. Established approaches involve sparse spatial sampling and/or sequential measurements, while emerging techniques exploit miniaturized electronics or passive optical methods. Various constraints in scalability, costs, robustness, operational range and other factors create a need for alternatives. Here, we introduce a concept that overcomes many of these limitations through the combined use of chemically induced light emission and chemically responsive optical filter elements in millimeter-scale systems that we refer to as functional pixie dust (fPD) sensors, designed specifically for monitoring natural water systems during nighttime to eliminate background optical interference and to enhance remote analysis. These floating devices act as Lagrangian tracers to follow surface flows and to simultaneously measure the concentrations of key chemical species along their trajectories. Optimized designs exploit environmentally compatible constituent materials that are also degradable through natural processes to benign end products, thereby eliminating the need for recovery. Spatially and spectrally resolved ratiometric measurement schemes ensure robust operation and ability to address practical requirements in range, operational lifetime, time response and sensitivity. Demonstrations include distributed measurements of pH, Hg2+, and NO2-, each of relevance to industrial discharge, toxic metal contamination, and nitrogen-rich runoff, adapted for static concentration gradients, flow-driven transport conditions, and outdoor aquatic settings. The results establish a framework for environmental sensing using degradable, self-powered microsystems capable of scalable deployment and remote readout.

physics.app-ph↗

Decentralized Best-Response-Based Learning in Two-Player Zero-Sum Stochastic Games: A Finite-Sample Analysis

We present a finite-sample analysis of decentralized learning in two-player zero-sum matrix games and stochastic games, with a focus on best-response-based learning algorithms. In matrix games, the learning algorithm is payoff-based and symmetric: each player updates its policy using only its own payoff observations, incrementally moving toward an estimated smoothed best response to the opponent's latest policy. For stochastic games, we build on this matrix-game primitive to develop a learning algorithm called value iteration with smoothed best response (VI-SBR), which combines smoothed-best-response learning in induced matrix games with a decentralized, model-free approximation of minimax value iteration. We establish finite-sample guarantees in both settings. For matrix games, our results imply a sample complexity of $\mathcal{O}(ε^{-1})$ for finding an $ε$-Nash distribution and, with explicit exploration, $\tilde{\mathcal{O}}(ε^{-8})$ for finding an $ε$-Nash equilibrium. For stochastic games, we prove that the exploration-enhanced VI-SBR algorithm achieves a sample complexity of $\tilde{\mathcal{O}}(ε^{-8})$ for finding an $ε$-Nash equilibrium. Technically, our analysis develops a coupled Lyapunov-drift framework. This framework simultaneously handles stochastic iterative algorithms with multiple interacting stochastic iterates, the non-zero-sum auxiliary games generated by independently updated value functions, and the time-inhomogeneous Markovian noise induced by time-varying policies. The resulting tools may be useful more broadly for analyzing learning algorithms with coupled stochastic iterates and nonstationary sampling processes.

cs.LG↗

Regret Minimization with Adaptive Opponents in Repeated Games

In this paper, we study regret minimization in repeated games with \emph{adaptive} opponents who can respond based on histories of play. The standard metric of \emph{external regret} in online learning is known to fail to capture such adaptivity. To account for players' counterfactual reasoning, we introduce {\tt Repeated Policy Regret (RP-Regret)}, a game-theoretic metric that measures the difference between the \emph{realized} and the \emph{best-in-hindsight} accumulated utility when all players can \emph{respond} to the history of play. Compared to existing regret notions in this setting, ours is native to repeated game playing, enabling stronger comparators and opponents with fewer constraints, while maintaining the possibility of finding better equilibria when all players minimize it. We first identify necessary conditions for obtaining {\tt RP-Regret} sublinear in time, on the variation of the player's comparator strategies in the regret definition and on the memories of both the comparator and opponents' strategies. We then study additional conditions and provable algorithms to minimize {\tt RP-Regret}, which is by definition \emph{non-convex} in the strategy space. To address this challenge, we propose three algorithms: (i) one based on an optimization oracle, as assumed in some prior work in online non-convex learning; (ii) one that minimizes a convex and \emph{linearized} surrogate of {\tt RP-Regret} at each iteration; (iii) one that directly minimizes {\tt RP-Regret} when opponents change strategies slowly. Furthermore, when all players can run algorithms to minimize the {\tt RP-Regret} (or its linearized variant), certain subgame perfect equilibria of the repeated game can be learned. We also provide experiments showing that minimizing our regret notions can lead to more cooperative solutions with higher utility in games such as Stag-Hunt.

cs.LG↗

Post-Training LLMs as Better Decision-Making Agents: A Regret-Minimization Approach

Large language models (LLMs) are increasingly deployed as "agents" for decision-making (DM) in interactive and dynamic environments. Yet, since they were not originally designed for DM, recent studies show that LLMs can struggle even in basic online DM problems, failing to achieve low regret or an effective exploration-exploitation tradeoff. To address this, we introduce Iterative Regret-Minimization Fine-Tuning (Iterative RMFT), a post-training procedure that repeatedly distills low-regret decision trajectories back into the base model. At each iteration, the model rolls out multiple decision trajectories, selects the k-lowest regret ones, and fine-tunes itself on them. Unlike prior methods that (a) distill action sequences from known DM algorithms or (b) rely on manually crafted chain-of-thought templates, our approach leverages the regret metric to elicit the model's own DM ability and reasoning rationales. This reliance on model-generated reasoning avoids rigid output engineering and provides more flexible, natural-language training signals. Empirical results show that Iterative RMFT improves LLMs' DM performance across diverse models - from Transformers with numerical input/output, to open-weight LLMs, and advanced closed-weight models like GPT-4o mini. Its flexibility in output and reasoning formats enables generalization across tasks with varying horizons, action spaces, reward processes, and natural-language contexts. Finally, we provide theoretical insight showing that a single-layer Transformer under this paradigm can act as a no-regret learner in a simplified setting. Overall, Iterative RMFT offers a principled and general post-training framework for enhancing LLMs' decision-making capabilities.

cs.AI↗

Fully coherent short wavelength free-electron laser driven by a single sub-microjoule seed

High-repetition-rate, fully coherent extreme-ultraviolet (EUV) and X-ray free-electron lasers (FELs) are essential for advanced time-resolved ultrafast spectroscopies. While external seeding serves as the standard technique to achieve precise temporal coherence, conventional methods demand hundred-megawatt peak-power laser systems. Furthermore, advanced configurations like echo-enabled harmonic generation (EEHG) introduce the severe complexities of dual-laser synchronization. Together, these requirements fundamentally restrict operations to kilohertz repetition rates and compromise overall system stability. Here, we experimentally demonstrate a fully coherent EEHG-FEL driven by a single, sub-microjoule seed laser. By employing a direct-amplification enabled harmonic generation technique, we utilize an initial 0.4 microJ (2 MW peak power) ultraviolet seed to directly drive coherent lasing at nanometer wavelengths. By eliminating the need for extreme peak powers and multiple synchronized lasers, this approach significantly simplifies the seeding architecture and provides a practical and robust pathway toward megahertz-class, fully coherent EUV and X-ray light sources.

physics.acc-ph↗

Principled Learning-to-Communicate with Quasi-Classical Information Structures

Learning-to-communicate (LTC) in partially observable environments has received increasing attention in deep multi-agent reinforcement learning, where the control and communication strategies are jointly learned. Meanwhile, the impact of communication on decision-making has been extensively studied in control theory. In this paper, we seek to formalize and better understand LTC by bridging these two lines of work, through the lens of information structures (ISs). To this end, we formalize LTC in decentralized partially observable Markov decision processes (Dec-POMDPs) under the common-information-based framework from decentralized stochastic control, and classify LTC problems based on the ISs before (additional) information sharing. We first show that non-classical LTCs are computationally intractable in general, and thus focus on quasi-classical (QC) LTCs. We then propose a series of conditions for QC LTCs, under which LTC preserves the QC IS after information sharing, whereas violating them can cause computational hardness in general. Further, we develop provable planning and learning algorithms for QC LTCs, and establish quasi-polynomial time and sample complexities for several QC LTC examples that satisfy the above conditions. Along the way, we also establish new results on a relationship between (strictly) QC IS and the condition of having strategy-independent common-information-based beliefs (SI-CIBs), as well as on solving Dec-POMDPs without computationally intractable oracles but beyond those with SI-CIBs, which may be of independent interest.

eess.SY↗

Can LLM Agents Simulate Dynamic Networks? A Case Study on Email Networks with Phishing Synthesis

While Large Language Model (LLM) multi-agent systems (MAS) offer a transformative approach to simulating human behavior in complex systems, it remains largely unexplored whether these simulations can replicate realistic structural and temporal dynamics from a dynamic network perspective. Our evaluation indicates that existing frameworks excel at generating plausible micro-level interactions but fail to capture the emergent, macroscopic topologies necessary for domains that rely on realistic network dynamics, such as modeling information propagation and cybersecurity threats. To bridge this gap, we introduce two easily integrable extensions to simulation frameworks to ensure they preserve macroscopic network fidelity: 1) augmenting LLM agents with data-driven event triggers to organically sustain long-horizon interactions, and 2) integrating Hawkes processes to accurately model temporal activation dynamics. Our approach allows LLM MAS to capture both plausible micro-level patterns and macroscopic topologies. We further demonstrate the utility of this framework in synthesizing realistic phishing campaigns within evolving communication networks. The study reveals how threats exploit structural vulnerabilities, highlighting the potential of our framework for developing next-generation defenses. Our code is available at https://github.com/Graph-COM/NSL.

cs.SI↗

Online Learning and Equilibrium Computation with Ranking Feedback

Online learning in arbitrary, and possibly adversarial, environments has been extensively studied in sequential decision-making, and it is closely connected to equilibrium computation in game theory. Most existing online learning algorithms rely on \emph{numeric} utility feedback from the environment, which may be unavailable in human-in-the-loop applications and/or may be restricted by privacy concerns. In this paper, we study an online learning model in which the learner only observes a \emph{ranking} over a set of proposed actions at each timestep. We consider two ranking mechanisms: rankings induced by the \emph{instantaneous} utility at the current timestep, and rankings induced by the \emph{time-average} utility up to the current timestep, under both \emph{full-information} and \emph{bandit} feedback settings. Using the standard external-regret metric, we show that sublinear regret is impossible with instantaneous-utility ranking feedback in general. Moreover, when the ranking model is relatively deterministic, \emph{i.e.}, under the Plackett-Luce model with a temperature that is sufficiently small, sublinear regret is also impossible with time-average utility ranking feedback. We then develop new algorithms that achieve sublinear regret under the additional assumption that the utility sequence has sublinear total variation. Notably, for full-information time-average utility ranking feedback, this additional assumption can be removed. As a consequence, when all players in a normal-form game follow our algorithms, repeated play yields an approximate coarse correlated equilibrium. We also demonstrate the effectiveness of our algorithms in an online large-language-model routing task.

cs.LG↗

Partially Observable Multi-Agent Reinforcement Learning with Information Sharing

We study provable multi-agent reinforcement learning (RL) in the general framework of partially observable stochastic games (POSGs). To circumvent the known hardness results and the use of computationally intractable oracles, we advocate leveraging the potential \emph{information-sharing} among agents, a common practice in empirical multi-agent RL, and a standard model for multi-agent control systems with communication. We first establish several computational complexity results to justify the necessity of information-sharing, as well as the observability assumption that has enabled quasi-polynomial time and sample single-agent RL with partial observations, for tractably solving POSGs. Inspired by the inefficiency of planning in the ground-truth model, we then propose to further \emph{approximate} the shared common information to construct an approximate model of the POSG, in which an approximate \emph{equilibrium} (of the original POSG) can be found in quasi-polynomial-time, under the aforementioned assumptions. Furthermore, we develop a partially observable multi-agent RL algorithm whose time and sample complexities are \emph{both} quasi-polynomial. Finally, beyond equilibrium learning, we extend our algorithmic framework to finding the \emph{team-optimal solution} in cooperative POSGs, i.e., decentralized partially observable Markov decision processes, a more challenging goal. We establish concrete computational and sample complexities under several structural assumptions of the model. We hope our study could open up the possibilities of leveraging and even designing different \emph{information structures}, a well-studied notion in control theory, for developing both sample- and computation-efficient partially observable multi-agent RL.

cs.LG↗

Cost-Driven Representation Learning for Linear Quadratic Gaussian Control: Part I

We study the task of learning state representations from potentially high-dimensional observations, with the goal of controlling an unknown partially observable system. We pursue a cost-driven approach, where a dynamic model in some latent state space is learned by predicting the costs without predicting the observations or actions. In particular, we focus on an intuitive cost-driven state representation learning method for solving Linear Quadratic Gaussian (LQG) control, one of the most fundamental partially observable control problems. As our main results, we establish finite-sample guarantees of finding a near-optimal state representation function and a near-optimal controller using the directly learned latent model, for finite-horizon time-varying LQG control problems. To the best of our knowledge, despite various empirical successes, finite-sample guarantees of such a cost-driven approach remain elusive. Our result underscores the value of predicting multi-step costs, an idea that is key to our theory, and notably also an idea that is known to be empirically valuable for learning state representations. A second part of this work, that is to appear as Part II, addresses the infinite-horizon linear time-invariant setting; it also extends the results to an approach that implicitly learns the latent dynamics, inspired by the recent empirical breakthrough of MuZero in model-based reinforcement learning.

cs.LG↗