arXiv subjects

Wenyi Zhang

Publications and source records attributed to Wenyi Zhang.

At least 37 records · Page 2Linked to original sources

arXiv2026

GRAND for Gaussian Intersymbol Interference Channels

Channel decoding is a challenging task in communication channels exhibiting memory effects. In this work, we apply the recently proposed decoding paradigm of guessing random additive noise decoding (GRAND) to channels with memory, focusing on linear Gaussian intersymbol interference (ISI) channels. For describing error patterns (EPs), we introduce the concept of error burst to account for the memory effect, and define sequence reliability to characterize the likelihood of EP. Based on sequence reliability, we obtain the optimal GRAND algorithm as a generalization of soft GRAND (SGRAND) for linear Gaussian ISI channels, termed SGRAND-ISI, which is equivalent to the maximum-likelihood (ML) decoding algorithm. We then develop order-reliability-bit (ORB) GRAND algorithms based on SGRAND-ISI, to facilitate implementation. In numerical experiments, our proposed algorithms achieve multiple-dB improvements compared to GRAND algorithms which ignore channel memory, and can often attain performance within 0.1--0.2dB of the ML lower bound. We also compare our proposed algorithms with the recently proposed ORBGRAND-Approximate Independence algorithm for handling channel memory, and observe a performance gain of at least 0.5dB at block error rate of $10^{-3}$, meanwhile incurring a substantially lower computational complexity.

cs.IT↗

arXiv2026

Reducing ORBGRAND Latency via Partial Gaussian Elimination

Guessing Random Additive Noise Decoding (GRAND) is a universal framework for decoding all block codes by testing candidate error patterns (EPs). Ordered Reliability Bits GRAND (ORBGRAND) facilitates parallel implementation of GRAND by exploiting log-likelihood ratio (LLR) rankings but still suffers from high tail latency under unfavorable channel conditions, limiting its use in real-time systems. We propose an elimination-aided ORBGRAND scheme that reduces decoding latency by integrating the Rank of the Most Reliable Erroneous (RMRE) bit with a partial Gaussian-elimination (GE) filtering mechanism. The scheme groups and jointly verifies EPs that share the same RMRE, and once a valid EP is identified, the ORBGRAND search is resumed. By leveraging prior GE steps to filter out unnecessary guesses, this approach significantly reduces the number of EPs to be tested, thereby lowering both average and worst-case latency while maintaining error-correction performance. Simulation results show that compared to the original ORBGRAND, the elimination-aided ORBGRAND filters out more than 50\% of EPs and correspondingly reduce overall computational complexity, all with no loss in block error rate. This demonstrates that this approach is suitable for ultra-reliable low-latency communication scenarios.

cs.IT↗

arXiv2026

ProFlow: Zero-Shot Physics-Consistent Sampling via Proximal Flow Guidance

Inferring physical fields from sparse observations while strictly satisfying partial differential equations (PDEs) is a fundamental challenge in computational physics. Recently, deep generative models offer powerful data-driven priors for such inverse problems, yet existing methods struggle to enforce hard physical constraints without costly retraining or disrupting the learned generative prior. Consequently, there is a critical need for a sampling mechanism that can reconcile strict physical consistency and observational fidelity with the statistical structure of the pre-trained prior. To this end, we present ProFlow, a proximal guidance framework for zero-shot physics-consistent sampling, defined as inferring solutions from sparse observations using a fixed generative prior without task-specific retraining. The algorithm employs a rigorous two-step scheme that alternates between: (\romannumeral1) a terminal optimization step, which projects the flow prediction onto the intersection of the physically and observationally consistent sets via proximal minimization; and (\romannumeral2) an interpolation step, which maps the refined state back to the generative trajectory to maintain consistency with the learned flow probability path. This procedure admits a Bayesian interpretation as a sequence of local maximum a posteriori (MAP) updates. Comprehensive benchmarks on Poisson, Helmholtz, Darcy, and viscous Burgers' equations demonstrate that ProFlow achieves superior physical and observational consistency, as well as more accurate distributional statistics, compared to state-of-the-art diffusion- and flow-based baselines.

cs.LG↗

arXiv2025

An Information-Theoretic Framework for Receiver Quantization in Communication

We investigate information-theoretic limits and design of communication under receiver quantization. Unlike most existing studies, this work is more focused on the impact of resolution reduction from high to low. We consider a standard transceiver architecture, which includes i.i.d. complex Gaussian codebook at the transmitter, and a symmetric quantizer cascaded with a nearest neighbor decoder at the receiver. Employing the generalized mutual information (GMI), an achievable rate under general quantization rules is obtained in an analytical form, which shows that the rate loss due to quantization is $\log\left(1+γ\mathsf{SNR}\right)$, where $γ$ is determined by thresholds and levels of the quantizer. Based on this result, the performance under uniform receiver quantization is analyzed comprehensively. We show that the front-end gain control, which determines the loading factor of quantization, has an increasing impact on performance as the resolution decreases. In particular, we prove that the unique loading factor that minimizes the MSE also maximizes the GMI, and the corresponding irreducible rate loss is given by $\log\left(1+\mathsf {mmse}\cdot\mathsf{SNR}\right)$, where mmse is the minimum MSE normalized by the variance of quantizer input, and is equal to the minimum of $γ$. A geometrical interpretation for the optimal uniform quantization at the receiver is further established. Moreover, by asymptotic analysis, we characterize the impact of biased gain control, showing how small rate losses decay to zero and providing rate approximations under large bias. From asymptotic expressions of the optimal loading factor and mmse, approximations and several per-bit rules for performance are also provided. Finally we discuss more types of receiver quantization and show that the consistency between achievable rate maximization and MSE minimization does not hold in general.

cs.IT↗

arXiv2025

Optimizing LVLMs with On-Policy Data for Effective Hallucination Mitigation

Recently, large vision-language models (LVLMs) have risen to be a promising approach for multimodal tasks. However, principled hallucination mitigation remains a critical challenge.In this work, we first analyze the data generation process in LVLM hallucination mitigation and affirm that on-policy data significantly outperforms off-policy data, which thus calls for efficient and reliable preference annotation of on-policy data. We then point out that, existing annotation methods introduce additional hallucination in training samples, which may enhance the model's hallucination patterns, to address this problem, we propose training a hallucination classifier giving binary annotations, which guarantee clean chosen samples for the subsequent alignment. To further harness of the power of on-policy data, we design a robust iterative direct preference optimization (DPO) algorithm adopting a dynamic sample reweighting scheme. We conduct comprehensive experiments on three benchmarks with comparison to 8 state-of-the-art baselines. In particular, our approach reduces the hallucination rate of LLaVA-1.5-7B on MMHalBench by 50.8% and the average hallucination rate on Object HalBench by 79.5%; more significantly, our method fully taps into the potential of open-source models, enabling LLaVA-1.5-13B to even surpass the performance of GPT-4V.

cs.CV↗

arXiv2025

Tree Reward-Aligned Search for TReASURe in Masked Diffusion Language Models

Tree search has recently emerged as a powerful framework for aligning generative models with task-specific rewards at test time. Applying tree search to Masked Diffusion Language Models, however, introduces two key challenges: (i) parallel unmasking yields highly correlated branches, limiting exploration, and (ii) reward evaluation via sampled completions produces high-variance estimates, making pruning unstable. We propose TReASURe, a tree-search test-time alignment method that addresses these issues. It introduces (i) UnmaskBranch, a branching strategy based on first-hitting unmasking that diversifies both token content and reveal order with a single model call per parent node, and (ii) ResubstituteScore, a pruning rule that uses deterministic resubstitution to score partially masked sequences with low-variance proxy completions. Theoretically, we quantify branching efficiency gains in NFEs (number of function evaluations), show that the scoring rule approximates the true reward with error bounded by predictive uncertainty, and prove improvements with larger tree widths. Empirically, TReASURe achieves state-of-the-art results on perplexity, linguistic acceptability, and control of sentiment and toxicity, outperforming prior methods under matched compute budgets, with especially strong gains in low-NFE regimes.

cs.CL↗

arXiv2025

An Optimal Transport-Based Method for Computing LM Rate and Its Convergence Analysis

The mismatch capacity characterizes the highest information rate of the channel under a prescribed decoding metric and serves as a critical performance indicator in numerous practical communication scenarios. Compared to the commonly used Generalized Mutual Information (GMI), the Lower bound on the Mismatch capacity (LM rate) generally provides a tighter lower bound on the mismatch capacity. However, the efficient computation of the LM rate is significantly more challenging than that of the GMI, particularly as the size of the channel input alphabet increases. This growth in complexity renders standard numerical methods (e.g., interior point methods) computationally intensive and, in some cases, impractical. In this work, we reformulate the computation of the LM rate as a special instance of the optimal transport (OT) problem with an additional constraint. Building on this formulation, we develop a novel numerical algorithm based on the Sinkhorn algorithm, which is well known for its efficiency in solving entropy regularized optimization problems. We further provide the convergence analysis of the proposed algorithm, revealing that the algorithm has a sub-linear convergence rate. Numerical experiments demonstrate the feasibility and efficiency of the proposed algorithm for the computation of the LM rate.

cs.IT↗

arXiv2025

Efficient Computation of Marton's Error Exponent via Constraint Decoupling

The error exponent in lossy source coding characterizes the asymptotic decay rate of error probability with respect to blocklength. The Marton's error exponent provides the theoretically optimal bound on this rate. However, computation methods of the Marton's error exponent remain underdeveloped due to its formulation as a non-convex optimization problem with limited efficient solvers. While a recent grid search algorithm can compute its inverse function, it incurs prohibitive computational costs from two-dimensional brute-force parameter grid searches. This paper proposes a composite maximization approach that effectively handles both Marton's error exponent and its inverse function. Through a constraint decoupling technique, the resulting problem formulations admit efficient solvers driven by an alternating maximization algorithm. By fixing one parameter via a one-dimensional line search, the remaining subproblem becomes convex and can be efficiently solved by alternating variable updates, thereby significantly reducing search complexity. Therefore, the global convergence of the algorithm can be guaranteed. Numerical experiments for simple sources and the Ahlswede's counterexample, demonstrates the superior efficiency of our algorithm in contrast to existing methods.

cs.IT↗

arXiv2025

Neural Estimation of the Information Bottleneck Based on a Mapping Approach

The information bottleneck (IB) method is a technique designed to extract meaningful information related to one random variable from another random variable, and has found extensive applications in machine learning problems. In this paper, neural network based estimation of the IB problem solution is studied, through the lens of a novel formulation of the IB problem. Via exploiting the inherent structure of the IB functional and leveraging the mapping approach, the proposed formulation of the IB problem involves only a single variable to be optimized, and subsequently is readily amenable to data-driven estimators based on neural networks. A theoretical analysis is conducted to guarantee that the neural estimator asymptotically solves the IB problem, and the numerical experiments on both synthetic and MNIST datasets demonstrate the effectiveness of the neural estimator.

cs.IT↗

arXiv2025

An Efficient Alternating Minimization Algorithm for Computing Quantum Rate-Distortion Function

We consider the computation of the entanglement-assisted quantum rate-distortion function, which plays a central role in quantum information theory. We propose an efficient alternating minimization algorithm based on the Lagrangian analysis. Instead of fixing the multiplier corresponding to the distortion constraint, we update the multiplier in each iteration. Hence the algorithm solves the original problem itself, rather than the Lagrangian relaxation of it. Moreover, all the other variables are iterated in closed form without solving multi-dimensional nonlinear equations or multivariate optimization problems. Numerical experiments show the accuracy of our proposed algorithm and its improved efficiency over existing methods.

cs.IT↗

arXiv2025

Estimating Rate-Distortion Functions Using the Energy-Based Model

The rate-distortion (RD) theory is one of the key concepts in information theory, providing theoretical limits for compression performance and guiding the source coding design, with both theoretical and practical significance. The Blahut-Arimoto (BA) algorithm, as a classical algorithm to compute RD functions, encounters computational challenges when applied to high-dimensional scenarios. In recent years, many neural methods have attempted to compute high-dimensional RD problems from the perspective of implicit generative models. Nevertheless, these approaches often neglect the reconstruction of the optimal conditional distribution or rely on unreasonable prior assumptions. In face of these issues, we propose an innovative energy-based modeling framework that leverages the connection between the RD dual form and the free energy in statistical physics, achieving effective reconstruction of the optimal conditional distribution.The proposed algorithm requires training only a single neural network and circumvents the challenge of computing the normalization factor in energy-based models using the Markov chain Monte Carlo (MCMC) sampling. Experimental results demonstrate the significant effectiveness of the proposed algorithm in estimating high-dimensional RD functions and reconstructing the optimal conditional distribution.

cs.IT↗

arXiv2025

RDD Function: A Tradeoff Between Rate and Distortion-in-Distortion

In this paper, we propose a novel function named Rate Distortion-in-Distortion (RDD) function as an extension of the classical rate-distortion (RD) function, where the expected distortion constraint is replaced by a Gromov-type distortion. This distortion, integral to the Gromov-Wasserstein (GW) distance, effectively defines the similarity in spaces of possibly different dimensions even without a direct metric between them. While the RDD function qualifies as an informational RD function, encoding theorems substantiate its status as an operational RD function, thereby underscoring its potential applicability in real-world source coding. Due to the high computational complexity associated with Gromov-type distortion, in general, the RDD function cannot be evaluated analytically. Consequently, we develop an alternating mirror descent algorithm that significantly reduces computational complexity by employing decomposition, linearization, and relaxation techniques. Numerical results on classical sources and different grids demonstrate the effectiveness of the developed algorithm. By exploring the relationship between the RDD function and the RD function, we suggest that the RDD function may have potential applications in future scenarios.

cs.IT↗

arXiv2025

Learning-Based Interface for Semantic Communication with Bit Importance Awareness

Joint source-channel coding (JSCC) is an effective approach for semantic communication. However, current JSCC methods are difficult to integrate with existing communication network architectures, where application and network providers are typically different entities. Recently, a novel paradigm termed Split DeepJSCC has been under consideration to address this challenge. Split DeepJSCC employs a bit-level interface that enables separate design of source and channel codes, ensuring compatibility with existing communication networks while preserving the advantages of JSCC in terms of semantic fidelity and channel adaptability. In this paper, we propose a learning-based interface design by treating its parameters as trainable, achieving improved end-to-end performance compared to Split DeepJSCC. In particular, the interface enables specification of bit-level importance at the output of the source code. Furthermore, we propose an Importance-Aware Net that utilizes the interface-derived bit importance information, enabling dynamical adaptation to diverse channel bandwidth ratios and time-varying channel conditions. Experimental results show that our method improves performance in wireless image transmission tasks. This work provides a potential solution for realizing semantic communications in existing wireless networks.

cs.IT↗

arXiv2025

Fine-tuning ORBGRAND with Very Few Channel Soft Values

Guessing random additive noise decoding (GRAND) is a universal decoding paradigm that decodes by repeatedly testing error patterns until identifying a codeword, where the ordering of tests is generated by the received channel values. On one hand, while testing error patterns in a descending order of posterior probabilities leads to maximum likelihood decoding, its implementation complexity is prohibitive. On the other hand, testing error patterns with a prescribed set of error patterns permuted by the ranking among magnitudes of log-likelihood ratios (i.e., ordered reliability bits, ORB) enables efficient implementation, but results in performance loss for finite-length codes. Aiming at harnessing the strengths of these two approaches, this work proposes a fine-tuning method to improve ORBGRAND, adjusting the ordering of tests with the aid of very few exact channel soft values. This method is based on a metric for assessing the ``well-orderedness'' of error patterns. The metric is studied via the lens of the asymptotic theory of integer partitioning, which provides highly accurate estimation in numerical experiments. The metric then leads to an effective identification of fine-tuning to conduct, at the cost of a negligible increment of complexity. Numerical experiments demonstrate that the proposed fine-tuning method achieves a substantial performance enhancement compared with ORBGRAND.

cs.IT↗

arXiv2025

One-Bit Model Aggregation for Differentially Private and Byzantine-Robust Personalized Federated Learning

As the scale of federated learning (FL) systems expands, their inherent performance limitations like communication overhead, Byzantine vulnerability, and privacy leakage have become increasingly critical. This paper considers a personalized FL framework based on model regularization, and proposes a model aggregation algorithm named PRoBit+ to concurrently overcome these limitations. PRoBit+ employs one-bit stochastic quantization and maximum likelihood estimation for parameter aggregation, and dynamically adjusts the step size of parameter updates, improving training stability of deep neural networks under low communication overhead and heterogeneous data distributions. PRoBit+'s statistical analysis is then conducted and its Byzantine robustness is proved. The $(ε,0)$-differential privacy and a convergence upper bound of the PRoBit+ based FL are also theoretically established in heterogeneous contexts. The analysis illustrates the trade-off among transmission accuracy, security guarantees, and convergence rates, and also indicates that the performance degradation caused by transmission errors and privacy protection can be progressively eliminated at a rate of $\mathcal{O}(1/M)$ as the number of uploading clients $M$ increases. Comprehensive numerical experiments are conducted to assess PRoBit+ in comparison to benchmark methods across different Byzantine attacks and varying proportions of malicious clients. The experimental results demonstrate that PRoBit+ exhibits improved Byzantine robustness over existing bit-based transmission schemes, minimal performance degradation related to privacy protection, and nearly identical performance to full-precision FedAvg in a secure environment.

cs.DC↗

arXiv2025

Generalized Nearest Neighbor Decoding: General Input Constellation and a Case Study of Interference Suppression

In this work, generalized nearest neighbor decoding (GNND), a recently proposed receiver architecture, is studied for channels under general input constellations, and multiuser uplink interference suppression is employed as a case study for demonstrating its potential. In essence, GNND generalizes the well-known nearest neighbor decoding, by introducing a symbol-level memoryless processing step, which can be rendered seamlessly compatible with Gaussian channel-based decoders. First, criteria of the optimal GNND are derived for general input constellations, expressed in the form of conditional moments matching, thereby generalizing the prior work which has been confined to Gaussian input. Then, the optimal GNND is applied to the use case of multiuser uplink, for which the optimal GNND is shown to be capable of achieving information rates nearly identical to the channel mutual information. By contrast, the commonly used channel linearization (CL) approach incurs a noticeable rate loss. A coded modulation scheme is subsequently developed, aiming at implementing GNND using off-the-shelf channel codes, without requiring iterative message passing between demodulator and decoder. Through numerical experiments it is validated that the developed scheme significantly outperforms the CL-based scheme.

cs.IT↗

arXiv2025

A High-Resolution Analysis of Receiver Quantization in Communication

We investigate performance limits and design of communication in the presence of uniform output quantization with moderate to high resolution. Under independent and identically distributed (i.i.d.) complex Gaussian codebook and nearest neighbor decoding rule, an achievable rate is derived in an analytical form by the generalized mutual information (GMI). The gain control before quantization is shown to be increasingly important as the resolution decreases, due to the fact that the loading factor (normalized one-sided quantization range) has increasing impact on performance. The impact of imperfect gain control in the high-resolution regime is characterized by two asymptotic results: 1) the rate loss due to overload distortion decays exponentially as the loading factor increases, and 2) the rate loss due to granular distortion decays quadratically as the step size vanishes. For a $2K$-level uniform quantizer, we prove that the optimal loading factor that maximizes the achievable rate scales like $2\sqrt{\ln (2K)}$ as the resolution increases. An asymptotically tight estimate of the optimal loading factor is further given, which is also highly accurate for finite resolutions.

cs.IT↗

arXiv2025

PATFinger: Prompt-Adapted Transferable Fingerprinting against Unauthorized Multimodal Dataset Usage

The multimodal datasets can be leveraged to pre-train large-scale vision-language models by providing cross-modal semantics. Current endeavors for determining the usage of datasets mainly focus on single-modal dataset ownership verification through intrusive methods and non-intrusive techniques, while cross-modal approaches remain under-explored. Intrusive methods can adapt to multimodal datasets but degrade model accuracy, while non-intrusive methods rely on label-driven decision boundaries that fail to guarantee stable behaviors for verification. To address these issues, we propose a novel prompt-adapted transferable fingerprinting scheme from a training-free perspective, called PATFinger, which incorporates the global optimal perturbation (GOP) and the adaptive prompts to capture dataset-specific distribution characteristics. Our scheme utilizes inherent dataset attributes as fingerprints instead of compelling the model to learn triggers. The GOP is derived from the sample distribution to maximize embedding drifts between different modalities. Subsequently, our PATFinger re-aligns the adaptive prompt with GOP samples to capture the cross-modal interactions on the carefully crafted surrogate model. This allows the dataset owner to check the usage of datasets by observing specific prediction behaviors linked to the PATFinger during retrieval queries. Extensive experiments demonstrate the effectiveness of our scheme against unauthorized multimodal dataset usage on various cross-modal retrieval architectures by 30% over state-of-the-art baselines.

cs.IR↗