arXiv subjects

Xin Sun

Publications and source records attributed to Xin Sun.

At least 109 records · Page 6Linked to original sources

arXiv2025

Perm: A Parametric Representation for Multi-Style 3D Hair Modeling

We present Perm, a learned parametric representation of human 3D hair designed to facilitate various hair-related applications. Unlike previous work that jointly models the global hair structure and local curl patterns, we propose to disentangle them using a PCA-based strand representation in the frequency domain, thereby allowing more precise editing and output control. Specifically, we leverage our strand representation to fit and decompose hair geometry textures into low- to high-frequency hair structures, termed guide textures and residual textures, respectively. These decomposed textures are later parameterized with different generative models, emulating common stages in the hair grooming process. We conduct extensive experiments to validate the architecture design of Perm, and finally deploy the trained model as a generic prior to solve task-agnostic problems, further showcasing its flexibility and superiority in tasks such as single-view hair reconstruction, hairstyle editing, and hair-conditioned image generation. More details can be found on our project page: https://cs.yale.edu/homes/che/projects/perm/.

cs.CV↗

arXiv2025

Shape-Guided Clothing Warping for Virtual Try-On

Image-based virtual try-on aims to seamlessly fit in-shop clothing to a person image while maintaining pose consistency. Existing methods commonly employ the thin plate spline (TPS) transformation or appearance flow to deform in-shop clothing for aligning with the person's body. Despite their promising performance, these methods often lack precise control over fine details, leading to inconsistencies in shape between clothing and the person's body as well as distortions in exposed limb regions. To tackle these challenges, we propose a novel shape-guided clothing warping method for virtual try-on, dubbed SCW-VTON, which incorporates global shape constraints and additional limb textures to enhance the realism and consistency of the warped clothing and try-on results. To integrate global shape constraints for clothing warping, we devise a dual-path clothing warping module comprising a shape path and a flow path. The former path captures the clothing shape aligned with the person's body, while the latter path leverages the mapping between the pre- and post-deformation of the clothing shape to guide the estimation of appearance flow. Furthermore, to alleviate distortions in limb regions of try-on results, we integrate detailed limb guidance by developing a limb reconstruction network based on masked image modeling. Through the utilization of SCW-VTON, we are able to generate try-on results with enhanced clothing shape consistency and precise control over details. Extensive experiments demonstrate the superiority of our approach over state-of-the-art methods both qualitatively and quantitatively. The code is available at https://github.com/xyhanHIT/SCW-VTON.

cs.CV↗

arXiv2025

Uncertainty Calibration for Counterfactual Propensity Estimation in Recommendation

Post-click conversion rate (CVR) is a reliable indicator of online customers' preferences, making it crucial for developing recommender systems. A major challenge in predicting CVR is severe selection bias, arising from users' inherent self-selection behavior and the system's item selection process. To mitigate this issue, the inverse propensity score (IPS) is employed to weight the prediction error of each observed instance. However, current propensity score estimations are unreliable due to the lack of a quality measure. To address this, we evaluate the quality of propensity scores from the perspective of uncertainty calibration, proposing the use of Expected Calibration Error (ECE) as a measure of propensity-score quality, which quantifies the extent to which predicted probabilities are overconfident by assessing the difference between predicted probabilities and actual observed frequencies. Miscalibrated propensity scores can lead to distorted IPS weights, thereby compromising the debiasing process in CVR prediction. In this paper, we introduce a model-agnostic calibration framework for propensity-based debiasing of CVR predictions. Theoretical analysis on bias and generalization bounds demonstrates the superiority of calibrated propensity estimates over uncalibrated ones. Experiments conducted on the Coat, Yahoo and KuaiRand datasets show improved uncertainty calibration, as evidenced by lower ECE values, leading to enhanced CVR prediction outcomes.

cs.AI↗

arXiv2025

Comprehensive Relighting: Generalizable and Consistent Monocular Human Relighting and Harmonization

This paper introduces Comprehensive Relighting, the first all-in-one approach that can both control and harmonize the lighting from an image or video of humans with arbitrary body parts from any scene. Building such a generalizable model is extremely challenging due to the lack of dataset, restricting existing image-based relighting models to a specific scenario (e.g., face or static human). To address this challenge, we repurpose a pre-trained diffusion model as a general image prior and jointly model the human relighting and background harmonization in the coarse-to-fine framework. To further enhance the temporal coherence of the relighting, we introduce an unsupervised temporal lighting model that learns the lighting cycle consistency from many real-world videos without any ground truth. In inference time, our temporal lighting module is combined with the diffusion models through the spatio-temporal feature blending algorithms without extra training; and we apply a new guided refinement as a post-processing to preserve the high-frequency details from the input image. In the experiments, Comprehensive Relighting shows a strong generalizability and lighting temporal coherence, outperforming existing image-based human relighting and harmonization methods.

cs.CV↗

arXiv2025

COST: Contrastive One-Stage Transformer for Vision-Language Small Object Tracking

Transformer has recently demonstrated great potential in improving vision-language (VL) tracking algorithms. However, most of the existing VL trackers rely on carefully designed mechanisms to perform the multi-stage multi-modal fusion. Additionally, direct multi-modal fusion without alignment ignores distribution discrepancy between modalities in feature space, potentially leading to suboptimal representations. In this work, we propose COST, a contrastive one-stage transformer fusion framework for VL tracking, aiming to learn semantically consistent and unified VL representations. Specifically, we introduce a contrastive alignment strategy that maximizes mutual information (MI) between a video and its corresponding language description. This enables effective cross-modal alignment, yielding semantically consistent features in the representation space. By leveraging a visual-linguistic transformer, we establish an efficient multi-modal fusion and reasoning mechanism, empirically demonstrating that a simple stack of transformer encoders effectively enables unified VL representations. Moreover, we contribute a newly collected VL tracking benchmark dataset for small object tracking, named VL-SOT500, with bounding boxes and language descriptions. Our dataset comprises two challenging subsets, VL-SOT230 and VL-SOT270, dedicated to evaluating generic and high-speed small object tracking, respectively. Small object tracking is notoriously challenging due to weak appearance and limited features, and this dataset is, to the best of our knowledge, the first to explore the usage of language cues to enhance visual representation for small object tracking. Extensive experiments demonstrate that COST achieves state-of-the-art performance on five existing VL tracking datasets, as well as on our proposed VL-SOT500 dataset. Source codes and dataset will be made publicly available.

cs.CV↗

arXiv2025

A Generalist Audio Foundation Model for Comprehensive Body Sound Auscultation

Accurate and efficient auscultation-based diagnostics are vital for early disease detection, especially in resource-limited settings where specialized clinical expertise is scarce. Traditional auscultation, which heavily depends on clinician experience, suffers from significant inter-observer variability, while existing AI models often falter due to the limitations of non-representative training data. In this study, we introduce AuscultaBase, a novel AI-driven diagnostic framework that harnesses self-supervised and contrastive learning techniques alongside large-scale, multi-source data integration to advance body sound analysis. By generating robust feature representations, AuscultaBase markedly enhances performance in abnormality detection, disease classification, and activity recognition tasks. Comprehensive evaluations on our newly established benchmark, AuscultaBench, demonstrate that AuscultaBase consistently outperforms state-of-the-art methods across key performance metrics, underscoring its potential as a scalable and cost-effective tool for clinical screening and early disease intervention. The code and model checkpoint has been released in https://github.com/applewpj/AuscultaBase.

cs.SD↗

arXiv2025

Permutons, meanders, and SLE-decorated Liouville quantum gravity

We study a class of random permutons which can be constructed from a pair of space-filling Schramm-Loewner evolution (SLE) curves on a Liouville quantum gravity (LQG) surface. This class includes the skew Brownian permutons introduced by Borga (2021), which describe the scaling limit of various types of random pattern-avoiding permutations. Another interesting permuton in our class is the meandric permuton, which corresponds to two independent SLE$_8$ curves on a $γ$-LQG surface with $γ= \sqrt{\frac13 \left( 17 - \sqrt{145} \right)}$. Building on work by Di Francesco, Golinelli, and Guitter (2000), we conjecture that the meandric permuton describes the scaling limit of uniform meandric permutations, i.e., the permutations induced by a simple loop in the plane which crosses a line a specified number of times. We show that for any sequence of random permutations which converges to one of the above random permutons, the length of the longest increasing subsequence is sublinear. This proves that the length of the longest increasing subsequence is sublinear for Baxter, strong-Baxter, and semi-Baxter permutations and leads to the conjecture that the same is true for meandric permutations. We also prove that the closed support of each of the random permutons in our class has Hausdorff dimension one. Finally, we prove a re-rooting invariance property for the meandric permuton and write down a formula for its expected pattern densities in terms of LQG correlation functions (which are known explicitly) and the probability that an SLE$_8$ hits a given set of points in numerical order (which is not known explicitly). We conclude with a list of open problems.

math.PR↗

arXiv2025

Path-Adaptive Matting for Efficient Inference Under Various Computational Cost Constraints

In this paper, we explore a novel image matting task aimed at achieving efficient inference under various computational cost constraints, specifically FLOP limitations, using a single matting network. Existing matting methods which have not explored scalable architectures or path-learning strategies, fail to tackle this challenge. To overcome these limitations, we introduce Path-Adaptive Matting (PAM), a framework that dynamically adjusts network paths based on image contexts and computational cost constraints. We formulate the training of the computational cost-constrained matting network as a bilevel optimization problem, jointly optimizing the matting network and the path estimator. Building on this formalization, we design a path-adaptive matting architecture by incorporating path selection layers and learnable connect layers to estimate optimal paths and perform efficient inference within a unified network. Furthermore, we propose a performance-aware path-learning strategy to generate path labels online by evaluating a few paths sampled from the prior distribution of optimal paths and network estimations, enabling robust and efficient online path learning. Experiments on five image matting datasets demonstrate that the proposed PAM framework achieves competitive performance across a range of computational cost constraints.

cs.CV↗

arXiv2025

All in One: Exploring Unified Vision-Language Tracking with Multi-Modal Alignment

Current mainstream vision-language (VL) tracking framework consists of three parts, \ie a visual feature extractor, a language feature extractor, and a fusion model. To pursue better performance, a natural modus operandi for VL tracking is employing customized and heavier unimodal encoders, and multi-modal fusion models. Albeit effective, existing VL trackers separate feature extraction and feature integration, resulting in extracted features that lack semantic guidance and have limited target-aware capability in complex scenarios, \eg similar distractors and extreme illumination. In this work, inspired by the recent success of exploring foundation models with unified architecture for both natural language and computer vision tasks, we propose an All-in-One framework, which learns joint feature extraction and interaction by adopting a unified transformer backbone. Specifically, we mix raw vision and language signals to generate language-injected vision tokens, which we then concatenate before feeding into the unified backbone architecture. This approach achieves feature integration in a unified backbone, removing the need for carefully-designed fusion modules and resulting in a more effective and efficient VL tracking framework. To further improve the learning efficiency, we introduce a multi-modal alignment module based on cross-modal and intra-modal contrastive objectives, providing more reasonable representations for the unified All-in-One transformer backbone. Extensive experiments on five benchmarks, \ie OTB99-L, TNL2K, LaSOT, LaSOT$_{\rm Ext}$ and WebUAV-3M, demonstrate the superiority of the proposed tracker against existing state-of-the-arts on VL tracking. Codes will be made publicly available at https://github.com/983632847/All-in-One.

cs.CV↗

arXiv2025

MegaSynth: Scaling Up 3D Scene Reconstruction with Synthesized Data

We propose scaling up 3D scene reconstruction by training with synthesized data. At the core of our work is MegaSynth, a procedurally generated 3D dataset comprising 700K scenes - over 50 times larger than the prior real dataset DL3DV - dramatically scaling the training data. To enable scalable data generation, our key idea is eliminating semantic information, removing the need to model complex semantic priors such as object affordances and scene composition. Instead, we model scenes with basic spatial structures and geometry primitives, offering scalability. Besides, we control data complexity to facilitate training while loosely aligning it with real-world data distribution to benefit real-world generalization. We explore training LRMs with both MegaSynth and available real data. Experiment results show that joint training or pre-training with MegaSynth improves reconstruction quality by 1.2 to 1.8 dB PSNR across diverse image domains. Moreover, models trained solely on MegaSynth perform comparably to those trained on real data, underscoring the low-level nature of 3D reconstruction. Additionally, we provide an in-depth analysis of MegaSynth's properties for enhancing model capability, training stability, and generalization, as well as application to other tasks.

cs.CV↗

arXiv2025

The moduli of annuli in random conformal geometry

We obtain exact formulae for three basic quantities in random conformal geometry that depend on the modulus of an annulus. The first is for the law of the modulus of the Brownian annulus describing the scaling limit of uniformly sampled planar maps with annular topology, which is as predicted from the ghost partition function in bosonic string theory. The second is for the law of the modulus of the annulus bounded by a loop of a simple conformal loop ensemble (CLE) on a disk and the disk boundary. The formula is as conjectured from the partition function of the O$(n)$ loop model on the annulus derived by Saleur-Bauer (1989) and Cardy (2006). The third is for the annulus partition function of the SLE$_{8/3}$ loop introduced by Werner (2008), {confirming another} prediction of Cardy (2006). The physics principle underlying our proofs is that 2D quantum gravity coupled with conformal matters can be decomposed into three conformal field theories (CFT): the matter CFT, the Liouville CFT, and the ghost CFT. At the technical level, we rely on two types of integrability in Liouville quantum gravity, one from the scaling limit of random planar maps, the other from the Liouville CFT.

math.PR↗

arXiv2025

Annulus crossing formulae for critical planar percolation

We derive exact formulae for three basic annulus crossing events for the critical planar Bernoulli percolation in the continuum limit. The first is for the probability that there is an open path connecting the two boundaries of an annulus of inner radius $r$ and outer radius $R$. The second is for the probability that there are both open and closed paths connecting the two annulus boundaries. These two results were predicted by Cardy based on non-rigorous Coulomb gas arguments. Our third result gives the probability that there are two disjoint open paths connecting the two boundaries. Its leading asymptotic as $r/R\to 0$ is captured by the so-called backbone exponent, a transcendental number recently determined by Nolin, Qian and two of the authors. This exponent is the unique real root to the equation $\frac{\sqrt{36 x +3}}{4} + \sin (\frac{2 π\sqrt{12 x +1}}{3} ) =0$, other than $-\frac{1}{12}$ and $\frac{1}{4}$. Besides these three real roots, this equation has countably many complex roots. Our third result shows that these roots appear exactly as exponents of the subleading terms in the crossing formula. This suggests that the backbone exponent is part of a conformal field theory (CFT) whose bulk spectrum contains this set of roots. Expanding the same crossing probability as $r/R\to 1$, we obtain a series with logarithmic corrections at every order, suggesting that the backbone exponent is related to a logarithmic boundary CFT. Our proofs are based on the coupling between SLE curves and Liouville quantum gravity (LQG). The key is to encode the annulus crossing probabilities by the random moduli of certain LQG surfaces with annular topology, whose law can be extracted from the dependence of the LQG annuli partition function on their boundary lengths.

math.PR↗

arXiv2025

High-Order Exceptional Point-Based Rotation Sensing in Anti-Parity time Symmetric Microresonators

Exceptional points (EPs), which arise from non-Hermitian systems, have been extensively investigated for the development of high-performance gyroscopes. However, operating a non-Hermitian gyroscope at high-order EP (HOEP) to achieve extreme performance requires strict and precise control of parameters. Here, we propose the design of an anti-parity-time (anti-PT) symmetric optical gyroscope operating at a fourth-order EP, achieving both ultra-sensitivity and high robustness. Our configuration exhibits eigenfrequency splitting two orders of magnitude higher than that of anti-PT gyroscopes operating at second-order EP. Furthermore, we demonstrate a significant reduction in angular random walk (ARW) under noise limits, compared to anti-parity symmetric gyroscopes based on second-order EP. Our results provide a novel approach for developing high-sensitivity rotation detection based on HOEPs.

physics.optics↗

arXiv2025

Backbone exponent and annulus crossing probability for planar percolation

We report the recent derivation of the backbone exponent for 2D percolation. In contrast to previously known exactly solved percolation exponents, the backbone exponent is a transcendental number, which is a root of an elementary equation. We also report an exact formula for the probability that there are two disjoint paths of the same color crossing an annulus. The backbone exponent captures the leading asymptotic, while the other roots of the elementary equation capture the asymptotic of the remaining terms. This suggests that the backbone exponent is part of a conformal field theory (CFT) whose bulk spectrum contains this set of roots. Our approach is based on the coupling between SLE curves and Liouville quantum gravity (LQG), and the integrability of Liouville CFT that governs the LQG surfaces.

cond-mat.stat-mech↗

arXiv2025

Rethinking the Upsampling Layer in Hyperspectral Image Super Resolution

Deep learning has achieved significant success in single hyperspectral image super-resolution (SHSR); however, the high spectral dimensionality leads to a heavy computational burden, thus making it difficult to deploy in real-time scenarios. To address this issue, this paper proposes a novel lightweight SHSR network, i.e., LKCA-Net, that incorporates channel attention to calibrate multi-scale channel features of hyperspectral images. Furthermore, we demonstrate, for the first time, that the low-rank property of the learnable upsampling layer is a key bottleneck in lightweight SHSR methods. To address this, we employ the low-rank approximation strategy to optimize the parameter redundancy of the learnable upsampling layer. Additionally, we introduce a knowledge distillation-based feature alignment technique to ensure the low-rank approximated network retains the same feature representation capacity as the original. We conducted extensive experiments on the Chikusei, Houston 2018, and Pavia Center datasets compared to some SOTAs. The results demonstrate that our method is competitive in performance while achieving speedups of several dozen to even hundreds of times compared to other well-performing SHSR methods.

eess.IV↗

arXiv2025

PreciseCam: Precise Camera Control for Text-to-Image Generation

Images as an artistic medium often rely on specific camera angles and lens distortions to convey ideas or emotions; however, such precise control is missing in current text-to-image models. We propose an efficient and general solution that allows precise control over the camera when generating both photographic and artistic images. Unlike prior methods that rely on predefined shots, we rely solely on four simple extrinsic and intrinsic camera parameters, removing the need for pre-existing geometry, reference 3D objects, and multi-view data. We also present a novel dataset with more than 57,000 images, along with their text prompts and ground-truth camera parameters. Our evaluation shows precise camera control in text-to-image generation, surpassing traditional prompt engineering approaches. Our data, model, and code are publicly available at https://graphics.unizar.es/projects/PreciseCam2024.

cs.CV↗

arXiv2025

MPT: A Large-scale Multi-Phytoplankton Tracking Benchmark

Phytoplankton are a crucial component of aquatic ecosystems, and effective monitoring of them can provide valuable insights into ocean environments and ecosystem changes. Traditional phytoplankton monitoring methods are often complex and lack timely analysis. Therefore, deep learning algorithms offer a promising approach for automated phytoplankton monitoring. However, the lack of large-scale, high-quality training samples has become a major bottleneck in advancing phytoplankton tracking. In this paper, we propose a challenging benchmark dataset, Multiple Phytoplankton Tracking (MPT), which covers diverse background information and variations in motion during observation. The dataset includes 27 species of phytoplankton and zooplankton, 14 different backgrounds to simulate diverse and complex underwater environments, and a total of 140 videos. To enable accurate real-time observation of phytoplankton, we introduce a multi-object tracking method, Deviation-Corrected Multi-Scale Feature Fusion Tracker(DSFT), which addresses issues such as focus shifts during tracking and the loss of small target information when computing frame-to-frame similarity. Specifically, we introduce an additional feature extractor to predict the residuals of the standard feature extractor's output, and compute multi-scale frame-to-frame similarity based on features from different layers of the extractor. Extensive experiments on the MPT have demonstrated the validity of the dataset and the superiority of DSFT in tracking phytoplankton, providing an effective solution for phytoplankton monitoring.

cs.CV↗

arXiv2024

Integrability of Conformal Loop Ensemble: Imaginary DOZZ Formula and Beyond

The scaling limit of the probability that $n$ points are on the same cluster for 2D critical percolation is believed to be governed by a conformal field theory (CFT). Although this is not fully understood, Delfino and Viti (2010) made a remarkable prediction on the exact value of a properly normalized three-point probability. It is expressed in terms of the imaginary DOZZ formula of Schomerus, Zamolodchikov and Kostov-Petkova, which extends the structure constants of minimal model CFTs to continuous parameters. Later, similar conjectures were made for scaling limits of random cluster models and O$(n)$ loop models, representing certain three-point observables in terms of the imaginary DOZZ formula. Since the scaling limits of these models can be described by the conformal loop ensemble (CLE), such conjectures can be formulated as exact statements on CLE observables. In this paper, we prove Delfino and Viti's conjecture on percolation as well as a conjecture of Ikhlef, Jacobsen and Saleur (2015) on the nesting loop statistics of CLE. Our proof is based on the coupling between CLE and Liouville quantum gravity on the sphere, and is inspired by the fact that after reparametrization, the imaginary DOZZ formula is the reciprocal of the three-point function of Liouville CFT. Recently, Nivesvivat, Jacobsen and Ribault systematically studied a CFT with a large class of CLE observables as its correlation functions, including the ones from these two conjectures. We believe that our framework admits sufficient flexibility to exactly solve the three-point functions for CLE observables with natural geometric interpretations, including those from this CFT. As a demonstration, we solve the case corresponding to three points lying on the same loop, where the answer is a variant of the imaginary DOZZ formula.

math-ph↗