arXiv subjects

Fang Liu

Publications and source records attributed to Fang Liu.

At least 145 records · Page 8Linked to original sources

arXiv2025

Preference Optimization via Contrastive Divergence: Your Reward Model is Secretly an NLL Estimator

Existing studies on preference optimization (PO) have centered on constructing pairwise preference data following simple heuristics, such as maximizing the margin between preferred and dispreferred completions based on human (or AI) ranked scores. However, none of these heuristics has a full theoretical justification. In this work, we develop a novel PO framework that provides theoretical guidance to effectively sample dispreferred completions. To achieve this, we formulate PO as minimizing the negative log-likelihood (NLL) of a probability model and propose to estimate its normalization constant via a sampling strategy. As we will demonstrate, these estimative samples can act as dispreferred completions in PO. We then select contrastive divergence (CD) as the sampling strategy, and propose a novel MC-PO algorithm that applies the Monte Carlo (MC) kernel from CD to sample hard negatives w.r.t. the parameterized reward model. Finally, we propose the OnMC-PO algorithm, an extension of MC-PO to the online setting. On popular alignment benchmarks, MC-PO outperforms existing SOTA baselines, and OnMC-PO leads to further improvement.

cs.AI↗

arXiv2025

Domain-conditioned and Temporal-guided Diffusion Modeling for Accelerated Dynamic MRI Reconstruction

Purpose: To propose a domain-conditioned and temporal-guided diffusion modeling method, termed dynamic Diffusion Modeling (dDiMo), for accelerated dynamic MRI reconstruction, enabling diffusion process to characterize spatiotemporal information for time-resolved multi-coil Cartesian and non-Cartesian data. Methods: The dDiMo framework integrates temporal information from time-resolved dimensions, allowing for the concurrent capture of intra-frame spatial features and inter-frame temporal dynamics in diffusion modeling. It employs additional spatiotemporal ($x$-$t$) and self-consistent frequency-temporal ($k$-$t$) priors to guide the diffusion process. This approach ensures precise temporal alignment and enhances the recovery of fine image details. To facilitate a smooth diffusion process, the nonlinear conjugate gradient algorithm is utilized during the reverse diffusion steps. The proposed model was tested on two types of MRI data: Cartesian-acquired multi-coil cardiac MRI and Golden-Angle-Radial-acquired multi-coil free-breathing lung MRI, across various undersampling rates. Results: dDiMo achieved high-quality reconstructions at various acceleration factors, demonstrating improved temporal alignment and structural recovery compared to other competitive reconstruction methods, both qualitatively and quantitatively. This proposed diffusion framework exhibited robust performance in handling both Cartesian and non-Cartesian acquisitions, effectively reconstructing dynamic datasets in cardiac and lung MRI under different imaging conditions. Conclusion: This study introduces a novel diffusion modeling method for dynamic MRI reconstruction.

eess.IV↗

arXiv2025

Acc-SpMM: Accelerating General-purpose Sparse Matrix-Matrix Multiplication with GPU Tensor Cores

General-purpose Sparse Matrix-Matrix Multiplication (SpMM) is a fundamental kernel in scientific computing and deep learning. The emergence of new matrix computation units such as Tensor Cores (TCs) brings more opportunities for SpMM acceleration. However, in order to fully unleash the power of hardware performance, systematic optimization is required. In this paper, we propose Acc-SpMM, a high-performance SpMM library on TCs, with multiple optimizations, including data-affinity-based reordering, memory efficient compressed format, high-throughput pipeline, and adaptive sparsity-aware load balancing. In contrast to the state-of-the-art SpMM kernels on various NVIDIA GPU architectures with a diverse range of benchmark matrices, Acc-SpMM achieves significant performance improvements, on average 2.52x (up to 5.11x) speedup on RTX 4090, on average 1.91x (up to 4.68x) speedup on A800, and on average 1.58x (up to 3.60x) speedup on H100 over cuSPARSE.

cs.DC↗

arXiv2025

Few-Shot Adaptation of Training-Free Foundation Model for 3D Medical Image Segmentation

Vision foundation models have achieved remarkable progress across various image analysis tasks. In the image segmentation task, foundation models like the Segment Anything Model (SAM) enable generalizable zero-shot segmentation through user-provided prompts. However, SAM primarily trained on natural images, lacks the domain-specific expertise of medical imaging. This limitation poses challenges when applying SAM to medical image segmentation, including the need for extensive fine-tuning on specialized medical datasets and a dependency on manual prompts, which are both labor-intensive and require intervention from medical experts. This work introduces the Few-shot Adaptation of Training-frEe SAM (FATE-SAM), a novel method designed to adapt the advanced Segment Anything Model 2 (SAM2) for 3D medical image segmentation. FATE-SAM reassembles pre-trained modules of SAM2 to enable few-shot adaptation, leveraging a small number of support examples to capture anatomical knowledge and perform prompt-free segmentation, without requiring model fine-tuning. To handle the volumetric nature of medical images, we incorporate a Volumetric Consistency mechanism that enhances spatial coherence across 3D slices. We evaluate FATE-SAM on multiple medical imaging datasets and compare it with supervised learning methods, zero-shot SAM approaches, and fine-tuned medical SAM methods. Results show that FATE-SAM delivers robust and accurate segmentation while eliminating the need for large annotated datasets and expert intervention. FATE-SAM provides a practical, efficient solution for medical image segmentation, making it more accessible for clinical applications.

cs.CV↗

arXiv2024

Fast and Efficient: Mask Neural Fields for 3D Scene Segmentation

Understanding 3D scenes is a crucial challenge in computer vision research with applications spanning multiple domains. Recent advancements in distilling 2D vision-language foundation models into neural fields, like NeRF and 3DGS, enable open-vocabulary segmentation of 3D scenes from 2D multi-view images without the need for precise 3D annotations. However, while effective, these methods typically rely on the per-pixel distillation of high-dimensional CLIP features, introducing ambiguity and necessitating complex regularization strategies, which adds inefficiency during training. This paper presents MaskField, which enables efficient 3D open-vocabulary segmentation with neural fields from a novel perspective. Unlike previous methods, MaskField decomposes the distillation of mask and semantic features from foundation models by formulating a mask feature field and queries. MaskField overcomes ambiguous object boundaries by naturally introducing SAM segmented object shapes without extra regularization during training. By circumventing the direct handling of dense high-dimensional CLIP features during training, MaskField is particularly compatible with explicit scene representations like 3DGS. Our extensive experiments show that MaskField not only surpasses prior state-of-the-art methods but also achieves remarkably fast convergence. We hope that MaskField will inspire further exploration into how neural fields can be trained to comprehend 3D scenes from 2D models.

cs.CV↗

arXiv2024

Noise-Augmented $\ell_0$ Regularization of Tensor Regression with Tucker Decomposition

Tensor data are multi-dimension arrays. Low-rank decomposition-based regression methods with tensor predictors exploit the structural information in tensor predictors while significantly reducing the number of parameters in tensor regression. We propose a method named NA$_0$CT$^2$ (Noise Augmentation for $\ell_0$ regularization on Core Tensor in Tucker decomposition) to regularize the parameters in tensor regression (TR), coupled with Tucker decomposition. We establish theoretically that NA$_0$CT$^2$ achieves exact $\ell_0$ regularization on the core tensor from the Tucker decomposition in linear TR and generalized linear TR. To our knowledge, NA$_0$CT$^2$ is the first Tucker decomposition-based regularization method in TR to achieve $\ell_0$ in core tensors. NA$_0$CT$^2$ is implemented through an iterative procedure and involves two straightforward steps in each iteration -- generating noisy data based on the core tensor from the Tucker decomposition of the updated parameter estimate and running a regular GLM on noise-augmented data on vectorized predictors. We demonstrate the implementation of NA$_0$CT$^2$ and its $\ell_0$ regularization effect in both simulation studies and real data applications. The results suggest that NA$_0$CT$^2$ can improve predictions compared to other decomposition-based TR approaches, with or without regularization and it identifies important predictors though not designed for that purpose.

stat.ML↗

arXiv2024

Chip-to-chip photonic quantum teleportation over optical fibers of 12.3km

Quantum teleportation is a crucial function in quantum networks. The implementation of photonic quantum teleportation could be highly simplified by quantum photonic circuits. To extend chip-to-chip teleportation distance, more effort is needed on both chip design and system implementation. In this work, we demonstrate a chip-to-chip photonic quantum teleportation over optical fibers under the scenario of star-topology quantum network. Time-bin encoded quantum states are used to achieve a long teleportation distance. Three photonic quantum circuits are designed and fabricated on a single chip, each serving specific functions: heralded single-photon generation at the user node, entangled photon pair generation and Bell state measurement at the relay node, and projective measurement of the teleported photons at the central node. The unbalanced Mach-Zehnder interferometers (UMZI) for time-bin encoding in these quantum photonic circuits are optimized to reduce insertion losses and suppress noise photons generated on the chip. Besides, an active feedback system is employed to suppress the impact of fiber length fluctuation between the circuits, achieving a stable quantum interference for the Bell state measurement in the relay node. As the result, a photonic quantum teleportation over optical fibers of 12.3km is achieved based on these quantum photonic circuits, showing the potential of chip integration on the development of quantum networks.

quant-ph↗

arXiv2024

Spectral Convolutional Neural Network Chip for In-sensor Edge Computing of Incoherent Natural Light

Convolutional neural networks (CNNs) are representative models of artificial neural networks (ANNs). However, the considerable power consumption and limited computing speed of electrical computing platforms restrict further CNN development on edge devices. Optical neural networks are considered next-generation physical implementations of ANNs, but their capabilities are limited by on-chip integration scale and requirement for coherent light sources. This study proposes a spectral convolutional neural network (SCNN) of incoherent natural light by an optical convolutional layer (OCL) and a reconfigurable electrical backend. The OCL is implemented by integrating very large-scale, pixel-aligned spectral filters on a CMOS image sensor on a 12-inch wafer, facilitating highly parallel spectral vector-inner products of incident light. It accepts broadband incoherent natural light containing two spatial and one spectral dimension directly as input with the function of matter meta-imaging. This unique optoelectronic framework empowers in-sensor optical analog computing at extremely high energy efficiency because the OCL is driven by the energy of the information carrier, i. e. natural light. To the best of our knowledge, this is the first integrated optical computing utilizing natural light. We employ the same SCNN chip for completely different real-world complex tasks,and achieve accuracies of over 96% for pathological diagnosis and almost 100% for face anti-spoofing at video rates. The SCNN framework has an unprecedented new function of substance identification, provides a feasible optoelectronic and integrated optical CNN implementation for edge devices or cellphones, providing them with practical and powerful edge computing abilities and facilitating diverse applications, such as intelligent robotics, industrial automation, medical diagnosis, and remote sensing.

physics.optics↗

arXiv2024

Map Optical Properties to Subwavelength Structures Directly via a Diffusion Model

Subwavelength photonic structures and metamaterials provide revolutionary approaches for controlling light. The inverse design methods proposed for these subwavelength structures are vital to the development of new photonic devices. However, most of the existing inverse design methods cannot realize direct mapping from optical properties to photonic structures but instead rely on forward simulation methods to perform iterative optimization. In this work, we exploit the powerful generative abilities of artificial intelligence (AI) and propose a practical inverse design method based on latent diffusion models. Our method maps directly the optical properties to structures without the requirement of forward simulation and iterative optimization. Here, the given optical properties can work as "prompts" and guide the constructed model to correctly "draw" the required photonic structures. Experiments show that our direct mapping-based inverse design method can generate subwavelength photonic structures at high fidelity while following the given optical properties. This may change the method used for optical design and greatly accelerate the research on new photonic devices.

physics.optics↗

arXiv2024

Generalized Design of Basket Trials with P-value Combination Test

The oncology exploratory basket trial design with pruning and pooling (P&P) approach has gained increasing popularity in recent years for its simplicity and efficiency. This method was proposed based on binary endpoint, limiting its wider application. This short communication proposed a generalized framework of using P-value combination test to implement pruning and pooling process in basket trials. Only P-values of any type of statistical testing from each cohort are needed for decision making, which provides great flexibility for basket trial designs with P&P approach.

stat.AP↗

arXiv2024

Boundary conditions dictate frequency dependence of thermal conductivity in silicon

Non-Fourier thermal transports have drawn significant attention for decades. Among them, the frequency dependent thermal conductivity has been extensively explored by pump-probe techniques, such as time-domain thermoreflectance, which is employed to probe the spectra of phonon mean free paths. However, previous studies on silicon have not exhibited apparent frequency dependence despite its broad phonon distribution. Here, we report the frequency dependent thermal transport in Al/Si with an atomically sharp interface, where the matched Debye temperatures preserve non-equilibrium between low- and high-energy phonons in Si. The dependence vanishes in Al/SiO$_2$/Si at room temperature, since the SiO$_2$ interlayer facilitates phonon scattering and destroys thermal non-equilibrium. At 80 K, frequency dependence reemerges in Al/SiO$_2$/Si, due to reduced interfacial phonon scattering. Our findings highlight the significance of boundary conditions in frequency dependent thermal conductivity.

cond-mat.mtrl-sci↗

arXiv2024

Predicting and Enhancing the Fairness of DNNs with the Curvature of Perceptual Manifolds

To address the challenges of long-tailed classification, researchers have proposed several approaches to reduce model bias, most of which assume that classes with few samples are weak classes. However, recent studies have shown that tail classes are not always hard to learn, and model bias has been observed on sample-balanced datasets, suggesting the existence of other factors that affect model bias. In this work, we first establish a geometric perspective for analyzing model fairness and then systematically propose a series of geometric measurements for perceptual manifolds in deep neural networks. Subsequently, we comprehensively explore the effect of the geometric characteristics of perceptual manifolds on classification difficulty and how learning shapes the geometric characteristics of perceptual manifolds. An unanticipated finding is that the correlation between the class accuracy and the separation degree of perceptual manifolds gradually decreases during training, while the negative correlation with the curvature gradually increases, implying that curvature imbalance leads to model bias.Building upon these observations, we propose curvature regularization to facilitate the model to learn curvature-balanced and flatter perceptual manifolds. Evaluations on multiple long-tailed and non-long-tailed datasets show the excellent performance and exciting generality of our approach, especially in achieving significant performance improvements based on current state-of-the-art techniques. Our work opens up a geometric analysis perspective on model bias and reminds researchers to pay attention to model bias on non-long-tailed and even sample-balanced datasets.

cs.CV↗

arXiv2024

Study of the tracking efficiency of charged pions at BESIII

Using $(10087 \pm 44) \times 10^6$ $J/ψ$ events collected with the BESIII detector in 2009, 2012, 2018 and 2019, the tracking efficiency of charged pions is studied using the decay $J/ψ\rightarrow π^+ π^- π^0$. The systematic uncertainty of the tracking efficiency and the corresponding correction factors for charged pions are evaluated, in bins of transverse momentum and polar angle of the charged pions.

hep-ex↗

arXiv2024

Knowledge-aware Evolutionary Graph Neural Architecture Search

Graph neural architecture search (GNAS) can customize high-performance graph neural network architectures for specific graph tasks or datasets. However, existing GNAS methods begin searching for architectures from a zero-knowledge state, ignoring the prior knowledge that may improve the search efficiency. The available knowledge base (e.g. NAS-Bench-Graph) contains many rich architectures and their multiple performance metrics, such as the accuracy (#Acc) and number of parameters (#Params). This study proposes exploiting such prior knowledge to accelerate the multi-objective evolutionary search on a new graph dataset, named knowledge-aware evolutionary GNAS (KEGNAS). KEGNAS employs the knowledge base to train a knowledge model and a deep multi-output Gaussian process (DMOGP) in one go, which generates and evaluates transfer architectures in only a few GPU seconds. The knowledge model first establishes a dataset-to-architecture mapping, which can quickly generate candidate transfer architectures for a new dataset. Subsequently, the DMOGP with architecture and dataset encodings is designed to predict multiple performance metrics for candidate transfer architectures on the new dataset. According to the predicted metrics, non-dominated candidate transfer architectures are selected to warm-start the multi-objective evolutionary algorithm for optimizing the #Acc and #Params on a new dataset. Empirical studies on NAS-Bench-Graph and five real-world datasets show that KEGNAS swiftly generates top-performance architectures, achieving 4.27% higher accuracy than advanced evolutionary baselines and 11.54% higher accuracy than advanced differentiable baselines. In addition, ablation studies demonstrate that the use of prior knowledge significantly improves the search performance.

cs.NE↗

arXiv2024

Analysis Methodology for Age of Information under Sequence Based Scheduling

We focus on the Age of Information (AoI) performance in a system where each user generates packets periodically to send to a common access point (AP) for status updating. To avoid heavy overhead, we assume that channel sensing, feedback information from the AP, and time synchronization are not available in the system. We adopt a multi-access scheme called the sequence scheme, where each user is assigned a periodic binary sequence to schedule their transmissions. In our previous work [18], we have thoroughly studied the AoI performance under sequence scheme when the period of schedule sequences, $L$, is equal to the status generating period, $T$. The results can be extended to the case where $T>L$. However, the case of $T<L$ is not covered by [18]. Therefore, in this paper, we concentrate on analyzing the AoI performance in the case of $T<L$, which is more challenging and requires different approaches. We conduct in-depth analysis on this case and develop a mathematical tool based on integer partitions to facilitate the analysis. We derive low-complexity closed-form expressions for two scenarios under $T<L$. Based on the obtained analytical results, we propose an algorithm to optimize the construction parameters of the sequence scheme. Finally, we compare our proposed sequence scheme with two commonly used baselines, and show that our proposed scheme outperforms the baselines in terms of AoI performance while consuming less energy.

cs.IT↗

arXiv2024

Application of an ANN and LSTM-based Ensemble Model for Stock Market Prediction

Stock trading has always been a key economic indicator in modern society and a primary source of profit for financial giants such as investment banks, quantitative trading firms, and hedge funds. Discovering the underlying patterns within the seemingly volatile yet intrinsically structured economic activities has become a central focus of research for many companies. Our study leverages widely-used modern financial forecasting algorithms, including LSTM, ANN, CNN, and BiLSTM. We begin by comparing the predictive performance of these well-known algorithms on our stock market data, utilizing metrics such as R2, MAE, MSE, RMSE for detailed evaluation. Based on the performance of these models, we then aim to combine their strengths while mitigating their weaknesses, striving to construct a powerful hybrid model that overcomes the performance limitations of individual models.Through rigorous experimentation and exploration, we ultimately developed an LSTM+ANN model that breaks through prior performance bottlenecks, achieving promising and exciting results.

cs.CE↗

arXiv2024

Towards Improved Preference Optimization Pipeline: from Data Generation to Budget-Controlled Regularization

Direct Preference Optimization (DPO) and its variants have become the de facto standards for aligning large language models (LLMs) with human preferences or specific goals. However, DPO requires high-quality preference data and suffers from unstable preference optimization. In this work, we aim to improve the preference optimization pipeline by taking a closer look at preference data generation and training regularization techniques. For preference data generation, we demonstrate that existing scoring-based reward models produce unsatisfactory preference data and perform poorly on out-of-distribution tasks. This significantly impacts the LLM alignment performance when using these data for preference tuning. To ensure high-quality preference data generation, we propose an iterative pairwise ranking mechanism that derives preference ranking of completions using pairwise comparison signals. For training regularization, we observe that preference optimization tends to achieve better convergence when the LLM predicted likelihood of preferred samples gets slightly reduced. However, the widely used supervised next-word prediction regularization strictly prevents any likelihood reduction of preferred samples. This observation motivates our design of a budget-controlled regularization formulation. Empirically we show that combining the two designs leads to aligned models that surpass existing SOTA across two popular benchmarks.

cs.LG↗

arXiv2024

Enhancing Trade-offs in Privacy, Utility, and Computational Efficiency through MUltistage Sampling Technique (MUST)

Applying a randomized algorithm to a subset rather than the entire dataset amplifies privacy guarantees. We propose a class of subsampling methods ``MUltistage Sampling Technique (MUST)'' for privacy amplification (PA) in the context of differential privacy (DP). We conduct comprehensive analyses of the PA effects and utility for several 2-stage MUST procedures through newly introduced concept including strong vs weak PA effects and aligned privacy profile. We provide the privacy loss composition analysis over repeated applications of MUST via the Fourier accountant algorithm. Our theoretical and empirical results suggest that MUST offers stronger PA in $ε$ than the common one-stage sampling procedures including Poisson sampling, sampling without replacement, and sampling with replacement, while the results on $δ$ vary case by case. Our experiments show that MUST is non-inferior in the utility and stability of privacy-preserving (PP) outputs to one-stage subsampling methods at similar privacy loss while enhancing the computational efficiency of algorithms that require complex function calculations on distinct data points. MUST can be seamlessly integrated into stochastic optimization algorithms or procedures that involve parallel or simultaneous subsampling when DP guarantees are necessary.

stat.ML↗