SearcharxivSearch

arXiv subjects

Siqi Pan

Publications and source records attributed to Siqi Pan.

14 recordsLinked to original sources

Priority Transparency, Admission Chances, and Information Acquisition in School Choice

We study, theoretically and experimentally, how transparency about students' priorities and admission chances shapes their incentives to acquire information about their own preferences in school choice and college admissions. In the model, uninformed students choose schools based on a common prior. When they learn their own preferences, their choices become more heterogeneous, which frees up seats at popular schools. Students who know they have high priority have stronger incentives to learn because they can more readily act on what they learn, whereas students who know they have low priority are discouraged. Full priority disclosure concentrates learning among high-priority students. By pooling priorities, partial disclosure spreads learning incentives to pooled students and yields higher welfare. In the laboratory, however, full disclosure yields the highest welfare instead, followed by partial disclosure, and then no disclosure, because greater transparency improves subjects' understanding of the strategic environment, leading to fewer mistakes. These findings support full disclosure of priorities or admission chances to guide information acquisition. However, deviations in learning remain even under greater priority transparency, partly because subjects respond suboptimally to admission chances when these are provided directly rather than inferred. Students' ability to interpret and use them is therefore itself a policy concern.

econ.GN

Why Your Tokenizer Fails in Information Fusion: A Timing-Aware Pre-Quantization Fusion for Video-Enhanced Audio Tokenization

Audio tokenization has emerged as a critical component in end-to-end audio language models, enabling efficient discrete representation learning for both audio understanding and generation tasks. However, existing audio tokenizers face fundamental limitations in understanding tasks due to single-modality constraints, particularly when audio signals contain ambiguous or incomplete information. While incorporating additional modality information can significantly enhance audio understanding, current multimodal fusion approaches invariably degrade reconstruction quality. This degradation is unacceptable for end-to-end audio systems that require high-fidelity audio generation capabilities. In this work, we investigate the root causes of reconstruction quality degradation in video-enhanced audio tokenization and present three key findings. First, the location of fusion within the tokenizer architecture is crucial for preserving reconstruction quality. Second, we show that contrastive learning, though effective in continuous representation fusion, is unsuitable for discrete tokenizers as it fails to enhance downstream task performance. Third, while feature-dimension fusion approaches achieve moderate success, we discover that fusing along the temporal axis -- guided by the concept of distinctive features -- yields significantly better results. Building on these insights, we introduce the Timing-Aware Pre-Quantization Fusion for Video-Enhanced Audio Tokenization, the first approach to successfully integrate visual information into audio tokenizer architectures while preserving reconstruction fidelity. Our approach not only maintains high-fidelity reconstruction but also achieves superior performance on downstream understanding tasks compared with audio-only tokenizers and established multimodal fusion baselines.

eess.AS

AR&D: A Framework for Retrieving and Describing Concepts for Interpreting AudioLLMs

Despite strong performance in audio perception tasks, large audio-language models (AudioLLMs) remain opaque to interpretation. A major factor behind this lack of interpretability is that individual neurons in these models frequently activate in response to several unrelated concepts. We introduce the first mechanistic interpretability framework for AudioLLMs, leveraging sparse autoencoders (SAEs) to disentangle polysemantic activations into monosemantic features. Our pipeline identifies representative audio clips, assigns meaningful names via automated captioning, and validates concepts through human evaluation and steering. Experiments show that AudioLLMs encode structured and interpretable features, enhancing transparency and control. This work provides a foundation for trustworthy deployment in high-stakes domains and enables future extensions to larger models, multilingual audio, and more fine-grained paralinguistic features. Project URL: https://townim-faisal.github.io/AutoInterpret-AudioLLM/

cs.SD

PROMPTMINER: Black-Box Prompt Stealing against Text-to-Image Generative Models via Reinforcement Learning and Fuzz Optimization

Text-to-image (T2I) generative models such as Stable Diffusion and FLUX can synthesize realistic, high-quality images directly from textual prompts. The resulting image quality depends critically on well-crafted prompts that specify both subjects and stylistic modifiers, which have become valuable digital assets. However, the rising value and ubiquity of high-quality prompts expose them to security and intellectual-property risks. One key threat is the prompt stealing attack, i.e., the task of recovering the textual prompt that generated a given image. Prompt stealing enables unauthorized extraction and reuse of carefully engineered prompts, yet it can also support beneficial applications such as data attribution, model provenance analysis, and watermarking validation. Existing approaches often assume white-box gradient access, require large-scale labeled datasets for supervised training, or rely solely on captioning without explicit optimization, limiting their practicality and adaptability. To address these challenges, we propose PROMPTMINER, a black-box prompt stealing framework that decouples the task into two phases: (1) a reinforcement learning-based optimization phase to reconstruct the primary subject, and (2) a fuzzing-driven search phase to recover stylistic modifiers. Experiments across multiple datasets and diffusion backbones demonstrate that PROMPTMINER achieves superior results, with CLIP similarity up to 0.958 and textual alignment with SBERT up to 0.751, surpassing all baselines. Even when applied to in-the-wild images with unknown generators, it outperforms the strongest baseline by 7.5 percent in CLIP similarity, demonstrating better generalization. Finally, PROMPTMINER maintains strong performance under defensive perturbations, highlighting remarkable robustness. Code: https://github.com/aaFrostnova/PromptMiner

cs.CV

EDITOR: Effective and Interpretable Prompt Inversion for Text-to-Image Diffusion Models

Text-to-image generation models~(e.g., Stable Diffusion) have achieved significant advancements, enabling the creation of high-quality and realistic images based on textual descriptions. Prompt inversion, the task of identifying the textual prompt used to generate a specific artifact, holds significant potential for applications including data attribution, model provenance, and watermarking validation. Recent studies introduced a delayed projection scheme to optimize for prompts representative of the vocabulary space, though challenges in semantic fluency and efficiency remain. Advanced image captioning models or visual large language models can generate highly interpretable prompts, but they often lack in image similarity. In this paper, we propose a prompt inversion technique called \sys for text-to-image diffusion models, which includes initializing embeddings using a pre-trained image captioning model, refining them through reverse-engineering in the latent space, and converting them to texts using an embedding-to-text model. Our experiments on the widely-used datasets, such as MS COCO, LAION, Flickr and DiffusionDB, show that our method outperforms existing methods in terms of image similarity, textual alignment, prompt interpretability and generalizability. We further illustrate the application of our generated prompts in tasks such as cross-concept image synthesis, concept manipulation, evolutionary multi-concept generation and unsupervised segmentation.

cs.CV

Consistency analysis of refined instrumental variable methods for continuous-time system identification in closed-loop

Refined instrumental variable methods have been broadly used for identification of continuous-time systems in both open and closed-loop settings. However, the theoretical properties of these methods are still yet to be fully understood when operating in closed-loop. In this paper, we address the consistency of the simplified refined instrumental variable method for continuous-time systems (SRIVC) and its closed-loop variant CLSRIVC when they are applied on data that is generated from a feedback loop. In particular, we consider feedback loops consisting of continuous-time controllers, as well as the discrete-time control case. This paper proves that the SRIVC and CLSRIVC estimators are not generically consistent when there is a continuous-time controller in the loop, and that generic consistency can be achieved when the controller is implemented in discrete-time. Numerical simulations are presented to support the theoretical results.

eess.SY

A low latency attention module for streaming self-supervised speech representation learning

The transformer is a fundamental building block in deep learning, and the attention mechanism is the transformer's core component. Self-supervised speech representation learning (SSRL) represents a popular use-case for the transformer architecture. Due to transformers' acausal behavior, the use of transformers for SSRL has been predominantly focused on acausal applications. However, several media processing problems, such as speech processing, require real-time solutions. In this paper, we present an implementation of the attention module that enables training of SSRL architectures with low compute and memory requirements, while allowing real-time inference with low and fixed latency. The attention module proposed in this paper includes two components, streaming attention (SA) and low-latency streaming attention (LLSA). The SA represents our proposal for an efficient streaming SSRL implementation, while the LLSA solves the latency build-up problem of other streaming attention architectures, such as the masked acausal attention (MAA), guaranteeing a latency equal to one layer even when multiple layers are stacked. We present a comparative analysis between the vanilla attention, which we will refer here as acausal attention (AA), the SA, and the LLSA, by training a streaming SSRL with automatic speech recognition as downstream task. When training on librispeech-clean-100 and testing on librispeech-test-clean, our low-latency attention module has a word error rate (WER) of 5.84%, which represents a significant improvement over the MAA (WER = 13.82%). Our implementation also reduces the inference latency from 1.92 to 0.16 seconds. The proposed low-latency module preserves many of the benefits of conventional acausal transformers, but also enables latency characteristics that make it applicable to real-time streaming applications.

cs.SD

Spatial HuBERT: Self-supervised Spatial Speech Representation Learning for a Single Talker from Multi-channel Audio

Self-supervised learning has been used to leverage unlabelled data, improving accuracy and generalisation of speech systems through the training of representation models. While many recent works have sought to produce effective representations across a variety of acoustic domains, languages, modalities and even simultaneous speakers, these studies have all been limited to single-channel audio recordings. This paper presents Spatial HuBERT, a self-supervised speech representation model that learns both acoustic and spatial information pertaining to a single speaker in a potentially noisy environment by using multi-channel audio inputs. Spatial HuBERT learns representations that outperform state-of-the-art single-channel speech representations on a variety of spatial downstream tasks, particularly in reverberant and noisy environments. We also demonstrate the utility of the representations learned by Spatial HuBERT on a speech localisation downstream task. Along with this paper, we publicly release a new dataset of 100 000 simulated first-order ambisonics room impulse responses.

cs.CL

On the Relation between Discrete and Continuous-time Refined Instrumental Variable Methods

The Refined Instrumental Variable method for discrete-time systems (RIV) and its variant for continuous-time systems (RIVC) are popular methods for the identification of linear systems in open-loop. The continuous-time equivalent of the transfer function estimate given by the RIV method is commonly used as an initialization point for the RIVC estimator. In this paper, we prove that these estimators share the same converging points for finite sample size when the continuous-time model has relative degree zero or one. This relation does not hold for higher relative degrees. Then, we propose a modification of the RIV method whose continuous-time equivalent is equal to the RIVC estimator for any non-negative relative degree. The implications of the theoretical results are illustrated via a simulation example.

eess.SY

Parsimonious Identification of Continuous-Time Systems: A Block-Coordinate Descent Approach

The identification of electrical, mechanical, and biological systems using data can benefit greatly from prior knowledge extracted from physical modeling. Parametric continuous-time identification methods can naturally incorporate this knowledge, which leads to interpretable and parsimonious models. However, some applications lead to model structures that lack parsimonious descriptions using unfactored transfer functions, which are commonly used in standard direct approaches for continuous-time system identification. In this paper we characterize this parsimony problem, and develop a block-coordinate descent algorithm that delivers parsimonious models by sequentially estimating an additive decomposition of the transfer function of interest. Numerical simulations show the efficacy of the proposed approach.

eess.SY

Consistency Analysis of the Closed-loop SRIVC Estimator

The Consistency of the Closed-Loop Simplified Refined Instrumental Variable method for Continuous-time system (CLSRIVC) is analysed based on sampled data. It is proven that the CLSRIVC estimator is not consistent when a continuous-time controller is used in the closed-loop.

eess.SY

Consistent identification of continuous-time systems under multisine input signal excitation

For many years, the Simplified Refined Instrumental Variable method for Continuous-time systems (SRIVC) has been widely used for identification. The intersample behaviour of the input plays an important role in this method, and it has been shown recently that the SRIVC estimator is not consistent if an incorrect assumption on the intersample behaviour is considered. In this paper, we present an extension of the SRIVC algorithm that is able to deal with continuous-time multisine signals, which cannot be interpolated exactly through hold reconstructions. The proposed estimator is generically consistent for any input reconstructed through zero or first-order-hold devices, and we show that it is generically consistent for continuous-time multisine inputs as well. The statistical performance of the proposed estimator is compared to the standard SRIVC estimator through extensive simulations.

eess.SY

Efficiency Analysis of the Simplified Refined Instrumental Variable Method for Continuous-time Systems

In this paper, we derive the asymptotic Cramér-Rao lower bound for the continuous-time output error model structure and provide an analysis of the statistical efficiency of the Simplified Refined Instrumental Variable method for Continuous-time systems (SRIVC) based on sampled data.It is shown that the asymptotic Cramér-Rao lower bound is independent of the intersample behaviour of the noise-free system output and hence only depends on the intersample behaviour of the system input. We have also shown that, at the converging point of the SRIVC algorithm, the estimates do not depend on the intersample behaviour of the measured output. It is then proven that the SRIVC estimator is asymptotically efficient for the output error model structure under mild conditions. Monte Carlo simulations are performed to verify the asymptotic Cramér-Rao lower bound and the asymptotic covariance of the SRIVC estimates.

eess.SY

Consistency Analysis of the Simplified Refined Instrumental Variable Method for Continuous-time Systems

In this paper, we analyse the consistency of the Simplified Refined Instrumental Variable method for Continuous-time systems (SRIVC). It is well known that the intersample behaviour of the input signal influences the quality and accuracy of the results when estimating and simulating continuous-time models. Here, we present a comprehensive analysis on the consistency of the SRIVC estimator while taking into account the intersample behaviour of the input signal. The main result of the paper shows that, under some mild conditions, the SRIVC estimator is generically consistent. We also describe some conditions when consistency is not achieved, which is important from a practical standpoint. The theoretical results are supported by simulation examples.

eess.SY