arXiv subjects

Shao Tang

Publications and source records attributed to Shao Tang.

34 records · Page 2Linked to original sources

arXiv2025

VAOT: Vessel-Aware Optimal Transport for Retinal Fundus Enhancement

Color fundus photography (CFP) is central to diagnosing and monitoring retinal disease, yet its acquisition variability (e.g., illumination changes) often degrades image quality, which motivates robust enhancement methods. Unpaired enhancement pipelines are typically GAN-based, however, they can distort clinically critical vasculature, altering vessel topology and endpoint integrity. Motivated by these structural alterations, we propose Vessel-Aware Optimal Transport (\textbf{VAOT}), a framework that combines an optimal-transport objective with two structure-preserving regularizers: (i) a skeleton-based loss to maintain global vascular connectivity and (ii) an endpoint-aware loss to stabilize local termini. These constraints guide learning in the unpaired setting, reducing noise while preserving vessel structure. Experimental results on synthetic degradation benchmark and downstream evaluations in vessel and lesion segmentation demonstrate the superiority of the proposed methods against several state-of-the art baselines. The code is available at https://github.com/Retinal-Research/VAOT

cs.CV↗

arXiv2025

Scaling Down, Serving Fast: Compressing and Deploying Efficient LLMs for Recommendation Systems

Large language models (LLMs) have demonstrated remarkable performance across a wide range of industrial applications, from search and recommendation systems to generative tasks. Although scaling laws indicate that larger models generally yield better generalization and performance, their substantial computational requirements often render them impractical for many real-world scenarios at scale. In this paper, we present a comprehensive set of insights for training and deploying small language models (SLMs) that deliver high performance for a variety of industry use cases. We focus on two key techniques: (1) knowledge distillation and (2) model compression via structured pruning and quantization. These approaches enable SLMs to retain much of the quality of their larger counterparts while significantly reducing training/serving costs and latency. We detail the impact of these techniques on a variety of use cases in a large professional social network platform and share deployment lessons, including hardware optimization strategies that improve speed and throughput for both predictive and reasoning-based applications in Recommendation Systems.

cs.IR↗

arXiv2025

Planner-R1: Reward Shaping Enables Efficient Agentic RL with Smaller LLMs

We investigated Agentic RL with large language models on the \textsc{TravelPlanner} benchmark. Our approach, \textsc{Planner-R1}, achieved a \textbf{56.9\%} final-pass rate with only 180 training queries, a $2.7\times$ improvement over GPT-5's $21.2\%$ baseline and the strongest agentic result on the public leaderboard. A central finding was that smaller models (8B) were highly responsive to reward shaping: with dense process-level signals, they reached competitive performance while being $3.5\times$ more compute-efficient and $1.5\times$ more memory-efficient than 32B models. Larger models were more robust under sparse rewards but exhibited smaller relative gains from shaping and higher variance across runs. While curriculum learning offered no significant benefit, shaped rewards consistently amplified learning dynamics, making 8B models the most efficient setting for agentic RL. Crucially, these gains did not come at the cost of overfitting: fine-tuned models mostly maintained or exceeded baseline performance on out-of-domain tasks, including \textsc{Multi-IF}, \textsc{NaturalPlan}, and $τ$-\textsc{Bench}. These results establish reward shaping as a decisive lever for scaling agentic RL, highlight the competitive strength of smaller models, and demonstrate that efficiency can be achieved without sacrificing generalization.

cs.AI↗

arXiv2025

EVTP-IVS: Effective Visual Token Pruning For Unifying Instruction Visual Segmentation In Multi-Modal Large Language Models

Instructed Visual Segmentation (IVS) tasks require segmenting objects in images or videos based on natural language instructions. While recent multimodal large language models (MLLMs) have achieved strong performance on IVS, their inference cost remains a major bottleneck, particularly in video. We empirically analyze visual token sampling in MLLMs and observe a strong correlation between subset token coverage and segmentation performance. This motivates our design of a simple and effective token pruning method that selects a compact yet spatially representative subset of tokens to accelerate inference. In this paper, we introduce a novel visual token pruning method for IVS, called EVTP-IV, which builds upon the k-center by integrating spatial information to ensure better coverage. We further provide an information-theoretic analysis to support our design. Experiments on standard IVS benchmarks show that our method achieves up to 5X speed-up on video tasks and 3.5X on image tasks, while maintaining comparable accuracy using only 20% of the tokens. Our method also consistently outperforms state-of-the-art pruning baselines under varying pruning ratios.

cs.CV↗

arXiv2025

AlphaPO: Reward Shape Matters for LLM Alignment

Reinforcement Learning with Human Feedback (RLHF) and its variants have made huge strides toward the effective alignment of large language models (LLMs) to follow instructions and reflect human values. More recently, Direct Alignment Algorithms (DAAs) have emerged in which the reward modeling stage of RLHF is skipped by characterizing the reward directly as a function of the policy being learned. Some popular examples of DAAs include Direct Preference Optimization (DPO) and Simple Preference Optimization (SimPO). These methods often suffer from likelihood displacement, a phenomenon by which the probabilities of preferred responses are often reduced undesirably. In this paper, we argue that, for DAAs the reward (function) shape matters. We introduce \textbf{AlphaPO}, a new DAA method that leverages an $α$-parameter to help change the shape of the reward function beyond the standard log reward. AlphaPO helps maintain fine-grained control over likelihood displacement and over-optimization. Compared to SimPO, one of the best performing DAAs, AlphaPO leads to about 7\% to 10\% relative improvement in alignment performance for the instruct versions of Mistral-7B and Llama3-8B while achieving 15\% to 50\% relative improvement over DPO on the same models. The analysis and results presented highlight the importance of the reward shape and how one can systematically change it to affect training dynamics, as well as improve alignment performance.

cs.CL↗

arXiv2025

On the Robustness of Reward Models for Language Model Alignment

The Bradley-Terry (BT) model is widely practiced in reward modeling for reinforcement learning with human feedback (RLHF). Despite its effectiveness, reward models (RMs) trained with BT model loss are prone to over-optimization, losing generalizability to unseen input distributions. In this paper, we study the cause of over-optimization in RM training and its downstream effects on the RLHF procedure, accentuating the importance of distributional robustness of RMs in unseen data. First, we show that the excessive dispersion of hidden state norms is the main source of over-optimization. Then, we propose batch-wise sum-to-zero regularization (BSR) to enforce zero-centered reward sum per batch, constraining the rewards with extreme magnitudes. We assess the impact of BSR in improving robustness in RMs through four scenarios of over-optimization, where BSR consistently manifests better robustness. Subsequently, we compare the plain BT model and BSR on RLHF training and empirically show that robust RMs better align the policy to the gold preference model. Finally, we apply BSR to high-quality data and models, which surpasses state-of-the-art RMs in the 8B scale by adding more than 5% in complex preference prediction tasks. By conducting RLOO training with 8B RMs, AlpacaEval 2.0 reduces generation length by 40% while adding a 7% increase in win rate, further highlighting that robustness in RMs induces robustness in RLHF training. We release the code, data, and models: https://github.com/LinkedIn-XFACT/RM-Robustness.

cs.CL↗

arXiv2025

Liger Kernel: Efficient Triton Kernels for LLM Training

Training Large Language Models (LLMs) efficiently at scale presents a formidable challenge, driven by their ever-increasing computational demands and the need for enhanced performance. In this work, we introduce Liger-Kernel, an open-sourced set of Triton kernels developed specifically for LLM training. With kernel optimization techniques like kernel operation fusing and input chunking, our kernels achieve on average a 20% increase in training throughput and a 60% reduction in GPU memory usage for popular LLMs compared to HuggingFace implementations. In addition, Liger-Kernel is designed with modularity, accessibility, and adaptability in mind, catering to both casual and expert users. Comprehensive benchmarks and integration tests are built in to ensure compatibility, performance, correctness, and convergence across diverse computing environments and model architectures. The source code is available under a permissive license at: github.com/linkedin/Liger-Kernel.

cs.LG↗

arXiv2023

On Generalization and Computation of Tukey's Depth: Part I

Tukey's depth offers a powerful tool for nonparametric inference and estimation, but also encounters serious computational and methodological difficulties in modern statistical data analysis. This paper studies how to generalize and compute Tukey-type depths in multi-dimensions. A general framework of influence-driven polished subspace depth, which emphasizes the importance of the underlying influence space and discrepancy measure, is introduced. The new matrix formulation enables us to utilize state-of-the-art optimization techniques to develop scalable algorithms with implementation ease and guaranteed fast convergence. In particular, half-space depth as well as regression depth can now be computed much faster than previously possible, with the support from extensive experiments. A companion paper is also offered to the reader in the same issue of this journal.

stat.ME↗

arXiv2023

On Generalization and Computation of Tukey's Depth: Part II

This paper studies how to generalize Tukey's depth to problems defined in a restricted space that may be curved or have boundaries, and to problems with a nondifferentiable objective. First, using a manifold approach, we propose a broad class of Riemannian depth for smooth problems defined on a Riemannian manifold, and showcase its applications in spherical data analysis, principal component analysis, and multivariate orthogonal regression. Moreover, for nonsmooth problems, we introduce additional slack variables and inequality constraints to define a novel slacked data depth, which can perform center-outward rankings of estimators arising from sparse learning and reduced rank regression. Real data examples illustrate the usefulness of some proposed data depths.

stat.ME↗

arXiv2019

Indirect Gaussian Graph Learning beyond Gaussianity

This paper studies how to capture dependency graph structures from real data which may not be Gaussian. Starting from marginal loss functions not necessarily derived from probability distributions, we utilize an additive over-parametrization with shrinkage to incorporate variable dependencies into the criterion. An iterative Gaussian graph learning algorithm is proposed with ease in implementation. Statistical analysis shows that the estimators achieve satisfactory accuracy with the error measured in terms of a proper Bregman divergence. Real-life examples in different settings are given to demonstrate the efficacy of the proposed methodology.

stat.ML↗

arXiv2018

Impurities and Defects in Mott Systems

Disorder has intriguing consequences for correlated electronic materials, which include several families of high-temperature superconductors and resistive switching systems. We address the question of why strongly correlated d-wave superconductors, such as the cuprates, prove to be surprisingly robust against the introduction of non-magnetic impurities. We show that, very generally, both the pair-breaking and the normal state transport scattering rates are significantly suppressed by strong correlations effects arising in the proximity to a Mott insulating state. We also show that the correlation-renormalized scattering amplitude is generically enhanced in the forward direction, an effect which was previously often ascribed to the specific scattering by charged impurities outside the copper-oxide planes. We provide the theoretical insights for resistive switching systems and show how impurities and underlying correlations can play significant roles in practical devices. We report the striking result of a connection between the resistive switching and shock wave formation, a classic topic of non-linear dynamics. We argue that the profile of oxygen vacancies that migrate during the commutation forms a shock wave that propagates through a highly resistive region of the device. We validate the scenario by means of model simulations and experiments in a manganese-oxide based memristor device and we extend our theory to the case of binary oxides. The shock wave scenario brings unprecedented physical insight and enables to rationalize the process of oxygen-vacancy-driven resistive change with direct implications for a key technological aspect- the commutation speed.

cond-mat.str-el↗

arXiv2018

Iterative proportional scaling revisited: a modern optimization perspective

This paper revisits the classic iterative proportional scaling (IPS) from a modern optimization perspective. In contrast to the criticisms made in the literature, we show that based on a coordinate descent characterization, IPS can be slightly modified to deliver coefficient estimates, and from a majorization-minimization standpoint, IPS can be extended to handle log-affine models with features not necessarily binary-valued or nonnegative. Furthermore, some state-of-the-art optimization techniques such as block-wise computation, randomization and momentum-based acceleration can be employed to provide more scalable IPS algorithms, as well as some regularized variants of IPS for concurrent feature selection.

stat.CO↗

arXiv2016

Shock-waves and commutation speed of memristors

Progress of silicon based technology is nearing its physical limit, as minimum feature size of components is reaching a mere 10 nm. The resistive switching behaviour of transition metal oxides and the associated memristor device is emerging as a competitive technology for next generation electronics. Significant progress has already been made in the past decade and devices are beginning to hit the market; however, it has been mainly the result of empirical trial and error. Hence, gaining theoretical insight is of essence. In the present work we report the striking result of a connection between the resistive switching and {\em shock wave} formation, a classic topic of non-linear dynamics. We argue that the profile of oxygen vacancies that migrate during the commutation forms a shock wave that propagates through a highly resistive region of the device. We validate the scenario by means of model simulations and experiments in a manganese-oxide based memristor device. The shock wave scenario brings unprecedented physical insight and enables to rationalize the process of oxygen-vacancy-driven resistive change with direct implications for a key technological aspect -- the commutation speed.

cond-mat.mtrl-sci↗

arXiv2015

Strong correlations generically protect d-wave superconductivity against disorder

We address the question of why strongly correlated d-wave superconductors, such as the cuprates, prove to be surprisingly robust against the introduction of non-magnetic impurities. We show that, very generally, both the pair-breaking and the normal state transport scattering rates are significantly suppressed by strong correlations effects arising in the proximity to a Mott insulating state. We also show that the correlation-renormalized scattering amplitude is generically enhanced in the forward direction, an effect which was previously often ascribed to the specific scattering by charged impurities outside the copper-oxide planes.

cond-mat.str-el↗

arXiv2015

Quantum criticality at the Anderson transition: a TMT perspective

We present a complete analytical and numerical solution of the Typical Medium Theory (TMT) for the Anderson metal-insulator transition. In this theory, we self-consistently calculate the typical amplitude of the electron wave-We present a complete analytical and numerical solution of the Typical Medium Theory (TMT) for the Anderson metal-insulator transition. This approach self-consistently calculates the typical amplitude of the electronic wave-functions, thus representing the conceptually simplest order-parameter theory for the Anderson transition. We identify all possible universality classes for the critical behavior, which can be found within such a mean-field approach. This provides insights into how interaction-induced renormalizations of the disorder potential may produce qualitative modifications of the critical behavior. We also formulate a simplified description of the leading critical behavior, thus obtaining an effective Landau theory for Anderson localization.

cond-mat.str-el↗

arXiv2015

Mottness-induced healing in strongly correlated superconductors

We study impurity healing effects in models of strongly correlated superconductors. We show that in general both the range and the amplitude of the spatial variations caused by nonmagnetic impurities are significantly suppressed in the superconducting as well as in the normal states. We explicitly quantify the weights of the local and the non-local responses to inhomogeneities and show that the former are overwhelmingly dominant over the latter. By quantifying the spatial range of the local response, we show that it is restricted to only a few lattice spacings over a significant range of dopings in the vicinity of the Mott insulating state. We demonstrate that this healing effect is ultimately due to the suppression of charge fluctuations induced by Mottness. We also define and solve analytically a simplified yet accurate model of healing, within which we obtain simple expressions for quantities of direct experimental relevance.

cond-mat.str-el↗