arXiv subjects

Hao Sun

Publications and source records attributed to Hao Sun.

At least 235 records · Page 13Linked to original sources

arXiv2025

PhyMPGN: Physics-encoded Message Passing Graph Network for spatiotemporal PDE systems

Solving partial differential equations (PDEs) serves as a cornerstone for modeling complex dynamical systems. Recent progresses have demonstrated grand benefits of data-driven neural-based models for predicting spatiotemporal dynamics (e.g., tremendous speedup gain compared with classical numerical methods). However, most existing neural models rely on rich training data, have limited extrapolation and generalization abilities, and suffer to produce precise or reliable physical prediction under intricate conditions (e.g., irregular mesh or geometry, complex boundary conditions, diverse PDE parameters, etc.). To this end, we propose a new graph learning approach, namely, Physics-encoded Message Passing Graph Network (PhyMPGN), to model spatiotemporal PDE systems on irregular meshes given small training datasets. Specifically, we incorporate a GNN into a numerical integrator to approximate the temporal marching of spatiotemporal dynamics for a given PDE system. Considering that many physical phenomena are governed by diffusion processes, we further design a learnable Laplace block, which encodes the discrete Laplace-Beltrami operator, to aid and guide the GNN learning in a physically feasible solution space. A boundary condition padding strategy is also designed to improve the model convergence and accuracy. Extensive experiments demonstrate that PhyMPGN is capable of accurately predicting various types of spatiotemporal dynamics on coarse unstructured meshes, consistently achieves the state-of-the-art results, and outperforms other baselines with considerable gains.

cs.LG↗

arXiv2025

Higgs-strahlung at the LHC in the inert doublet model

$pp \to W^{\pm} h, Zh$ processes at the LHC are studied in the framework of the inert doublet model (IDM). To quantify the effects of the IDM and their observability in these processes we revisit the NLO (QCD and EW) predictions in the Standard Model (SM) and their uncertainty. Taking all available current constraints on the parameter space of the IDM, we consider both the case of the IDM providing a good Dark Matter (DM) candidate within the freeze-out mechanism as well as when the DM restrictions are relaxed. In the former, deviations from the SM of only a few per mil in these cross sections at the LHC are found and may not be measured. In the latter, the deviations can reach a few percents and should be observable. Smaller discrepancies in this case require that the theoretical uncertainties be improved, in particular those arising from the parton distribution functions (PDFs). We stress the importance of the photon-induced real corrections and the need for further improvement in the extraction of the photon PDF. The analysis also showcases the development and exploitation of our automated tool for the computation of one-loop electroweak and QCD corrections for a New Physics model with internal tests such as those concerning the soft and collinear parts provided through both dipole subtraction and phase space slicing besides tests for ultra-violet finiteness and gauge-parameter independence.

hep-ph↗

arXiv2025

EyEar: Learning Audio Synchronized Human Gaze Trajectory Based on Physics-Informed Dynamics

Imitating how humans move their gaze in a visual scene is a vital research problem for both visual understanding and psychology, kindling crucial applications such as building alive virtual characters. Previous studies aim to predict gaze trajectories when humans are free-viewing an image, searching for required targets, or looking for clues to answer questions in an image. While these tasks focus on visual-centric scenarios, humans move their gaze also along with audio signal inputs in more common scenarios. To fill this gap, we introduce a new task that predicts human gaze trajectories in a visual scene with synchronized audio inputs and provide a new dataset containing 20k gaze points from 8 subjects. To effectively integrate audio information and simulate the dynamic process of human gaze motion, we propose a novel learning framework called EyEar (Eye moving while Ear listening) based on physics-informed dynamics, which considers three key factors to predict gazes: eye inherent motion tendency, vision salient attraction, and audio semantic attraction. We also propose a probability density score to overcome the high individual variability of gaze trajectories, thereby improving the stabilization of optimization and the reliability of the evaluation. Experimental results show that EyEar outperforms all the baselines in the context of all evaluation metrics, thanks to the proposed components in the learning model.

cs.MM↗

arXiv2025

When Graph meets Multimodal: Benchmarking and Meditating on Multimodal Attributed Graphs Learning

Multimodal Attributed Graphs (MAGs) are ubiquitous in real-world applications, encompassing extensive knowledge through multimodal attributes attached to nodes (e.g., texts and images) and topological structure representing node interactions. Despite its potential to advance diverse research fields like social networks and e-commerce, MAG representation learning (MAGRL) remains underexplored due to the lack of standardized datasets and evaluation frameworks. In this paper, we first propose MAGB, a comprehensive MAG benchmark dataset, featuring curated graphs from various domains with both textual and visual attributes. Based on MAGB dataset, we further systematically evaluate two mainstream MAGRL paradigms: $\textit{GNN-as-Predictor}$, which integrates multimodal attributes via Graph Neural Networks (GNNs), and $\textit{VLM-as-Predictor}$, which harnesses Vision Language Models (VLMs) for zero-shot reasoning. Extensive experiments on MAGB reveal following critical insights: $\textit{(i)}$ Modality significances fluctuate drastically with specific domain characteristics. $\textit{(ii)}$ Multimodal embeddings can elevate the performance ceiling of GNNs. However, intrinsic biases among modalities may impede effective training, particularly in low-data scenarios. $\textit{(iii)}$ VLMs are highly effective at generating multimodal embeddings that alleviate the imbalance between textual and visual attributes. These discoveries, which illuminate the synergy between multimodal attributes and graph topologies, contribute to reliable benchmarks, paving the way for future MAG research. The MAGB dataset and evaluation pipeline are publicly available at https://github.com/sktsherlock/MAGB.

cs.LG↗

arXiv2025

Multimodal Large Language Models for Text-rich Image Understanding: A Comprehensive Review

The recent emergence of Multi-modal Large Language Models (MLLMs) has introduced a new dimension to the Text-rich Image Understanding (TIU) field, with models demonstrating impressive and inspiring performance. However, their rapid evolution and widespread adoption have made it increasingly challenging to keep up with the latest advancements. To address this, we present a systematic and comprehensive survey to facilitate further research on TIU MLLMs. Initially, we outline the timeline, architecture, and pipeline of nearly all TIU MLLMs. Then, we review the performance of selected models on mainstream benchmarks. Finally, we explore promising directions, challenges, and limitations within the field.

cs.CV↗

arXiv2025

UniGO: A Unified Graph Neural Network for Modeling Opinion Dynamics on Graphs

Polarization and fragmentation in social media amplify user biases, making it increasingly important to understand the evolution of opinions. Opinion dynamics provide interpretability for studying opinion evolution, yet incorporating these insights into predictive models remains challenging. This challenge arises due to the inherent complexity of the diversity of opinion fusion rules and the difficulty in capturing equilibrium states while avoiding over-smoothing. This paper constructs a unified opinion dynamics model to integrate different opinion fusion rules and generates corresponding synthetic datasets. To fully leverage the advantages of unified opinion dynamics, we introduces UniGO, a framework for modeling opinion evolution on graphs. Using a coarsen-refine mechanism, UniGO efficiently models opinion dynamics through a graph neural network, mitigating over-smoothing while preserving equilibrium phenomena. UniGO leverages pretraining on synthetic datasets, which enhances its ability to generalize to real-world scenarios, providing a viable paradigm for applications of opinion dynamics. Experimental results on both synthetic and real-world datasets demonstrate UniGO's effectiveness in capturing complex opinion formation processes and predicting future evolution. The pretrained model also shows strong generalization capability, validating the benefits of using synthetic data to boost real-world performance.

cs.SI↗

arXiv2025

Reasoning-Enhanced Object-Centric Learning for Videos

Object-centric learning aims to break down complex visual scenes into more manageable object representations, enhancing the understanding and reasoning abilities of machine learning systems toward the physical world. Recently, slot-based video models have demonstrated remarkable proficiency in segmenting and tracking objects, but they overlook the importance of the effective reasoning module. In the real world, reasoning and predictive abilities play a crucial role in human perception and object tracking; in particular, these abilities are closely related to human intuitive physics. Inspired by this, we designed a novel reasoning module called the Slot-based Time-Space Transformer with Memory buffer (STATM) to enhance the model's perception ability in complex scenes. The memory buffer primarily serves as storage for slot information from upstream modules, the Slot-based Time-Space Transformer makes predictions through slot-based spatiotemporal attention computations and fusion. Our experimental results on various datasets indicate that the STATM module can significantly enhance the capabilities of multiple state-of-the-art object-centric learning models for video. Moreover, as a predictive model, the STATM module also performs well in downstream prediction and Visual Question Answering (VQA) tasks. We will release our codes and data at https://github.com/intell-sci-comput/STATM.

cs.CV↗

arXiv2025

Flavor and CP Symmetries in the Standard Model Effective Field Theory

The CP properties of effective operators are closely related to the spacetime and internal symmetries, such as the flavor symmetry, in the standard model effective field theory (SMEFT). In this work, we utilize the flavor symmetry to organize and reduce numbers of independent Wilson coefficients of the SMEFT operators. We classify the dimension-6 and dimension-8 baryon/lepton-number-conserving operators based on their CP properties. The $U(1)^4$ rephasing symmetry is applied to distinguish CP-odd and CP-violating operators which leads to reduction of the independent CP-violating phases. The $U(3)^5$ and $U(2)^5$ flavor symmetries let us classify the CP-violating phases as the flavor invariants, which can be enumerated by the Hilbert series and obtained explicitly by the Young tensor method. Then after introducing the minimal flavor violation (MFV) hypothesis, we present the flavor structures of the dimension-6 and dimension-8 SMEFT operators under the MFV hypothesis utilizing the spurion method.

hep-ph↗

arXiv2025

Reviving The Classics: Active Reward Modeling in Large Language Model Alignment

Building neural reward models from human preferences is a pivotal component in reinforcement learning from human feedback (RLHF) and large language model alignment research. Given the scarcity and high cost of human annotation, how to select the most informative pairs to annotate is an essential yet challenging open problem. In this work, we highlight the insight that an ideal comparison dataset for reward modeling should balance exploration of the representation space and make informative comparisons between pairs with moderate reward differences. Technically, challenges arise in quantifying the two objectives and efficiently prioritizing the comparisons to be annotated. To address this, we propose the Fisher information-based selection strategies, adapt theories from the classical experimental design literature, and apply them to the final linear layer of the deep neural network-based reward modeling tasks. Empirically, our method demonstrates remarkable performance, high computational efficiency, and stability compared to other selection methods from deep learning and classical statistical literature across multiple open-source LLMs and datasets. Further ablation studies reveal that incorporating cross-prompt comparisons in active reward modeling significantly enhances labeling efficiency, shedding light on the potential for improved annotation strategies in RLHF.

cs.CL↗

arXiv2025

Reusing Embeddings: Reproducible Reward Model Research in Large Language Model Alignment without GPUs

Large Language Models (LLMs) have made substantial strides in structured tasks through Reinforcement Learning (RL), demonstrating proficiency in mathematical reasoning and code generation. However, applying RL in broader domains like chatbots and content generation -- through the process known as Reinforcement Learning from Human Feedback (RLHF) -- presents unique challenges. Reward models in RLHF are critical, acting as proxies that evaluate the alignment of LLM outputs with human intent. Despite advancements, the development of reward models is hindered by challenges such as computational heavy training, costly evaluation, and therefore poor reproducibility. We advocate for using embedding-based input in reward model research as an accelerated solution to those challenges. By leveraging embeddings for reward modeling, we can enhance reproducibility, reduce computational demands on hardware, improve training stability, and significantly reduce training and evaluation costs, hence facilitating fair and efficient comparisons in this active research area. We then show a case study of reproducing existing reward model ensemble research using embedding-based reward models. We discussed future avenues for research, aiming to contribute to safer and more effective LLM deployments.

cs.CL↗

arXiv2025

MultiPDENet: PDE-embedded Learning with Multi-time-stepping for Accelerated Flow Simulation

Solving partial differential equations (PDEs) by numerical methods meet computational cost challenge for getting the accurate solution since fine grids and small time steps are required. Machine learning can accelerate this process, but struggle with weak generalizability, interpretability, and data dependency, as well as suffer in long-term prediction. To this end, we propose a PDE-embedded network with multiscale time stepping (MultiPDENet), which fuses the scheme of numerical methods and machine learning, for accelerated simulation of flows. In particular, we design a convolutional filter based on the structure of finite difference stencils with a small number of parameters to optimize, which estimates the equivalent form of spatial derivative on a coarse grid to minimize the equation's residual. A Physics Block with a 4th-order Runge-Kutta integrator at the fine time scale is established that embeds the structure of PDEs to guide the prediction. To alleviate the curse of temporal error accumulation in long-term prediction, we introduce a multiscale time integration approach, where a neural network is used to correct the prediction error at a coarse time scale. Experiments across various PDE systems, including the Navier-Stokes equations, demonstrate that MultiPDENet can accurately predict long-term spatiotemporal dynamics, even given small and incomplete training data, e.g., spatiotemporally down-sampled datasets. MultiPDENet achieves the state-of-the-art performance compared with other neural baseline models, also with clear speedup compared to classical numerical methods.

math.NA↗

arXiv2025

Rethinking Bradley-Terry Models in Preference-Based Reward Modeling: Foundations, Theory, and Alternatives

The Bradley-Terry (BT) model is a common and successful practice in reward modeling for Large Language Model (LLM) alignment. However, it remains unclear why this model -- originally developed for multi-player stochastic game matching -- can be adopted to convert pairwise response comparisons to reward values and make predictions. Especially given the fact that only a limited number of prompt-response pairs are sparsely compared with others. In this paper, we first revisit the foundations of using BT models in reward modeling, and establish the convergence rate of BT reward models based on deep neural networks using embeddings, providing a theoretical foundation for their use. Despite theoretically sound, we argue that the BT model is not a necessary choice from the perspective of downstream optimization. This is because a reward model only needs to preserve the correct ranking predictions through a monotonic transformation of the true reward. We highlight the critical concept of order consistency in reward modeling and demonstrate that the BT model possesses this property. Consequently, we propose a simple and straightforward upper-bound algorithm, compatible with off-the-shelf binary classifiers, as an alternative order-consistent reward modeling objective. To offer practical insights, we empirically evaluate the performance of these different reward modeling approaches across more than 12,000 experimental setups, using $6$ base LLMs, $2$ datasets, and diverse annotation designs that vary in quantity, quality, and pairing choices in preference annotations.

cs.AI↗

arXiv2025

Inverse-RLignment: Large Language Model Alignment from Demonstrations through Inverse Reinforcement Learning

Aligning Large Language Models (LLMs) is crucial for enhancing their safety and utility. However, existing methods, primarily based on preference datasets, face challenges such as noisy labels, high annotation costs, and privacy concerns. In this work, we introduce Alignment from Demonstrations (AfD), a novel approach leveraging high-quality demonstration data to overcome these challenges. We formalize AfD within a sequential decision-making framework, highlighting its unique challenge of missing reward signals. Drawing insights from forward and inverse reinforcement learning, we introduce divergence minimization objectives for AfD. Analytically, we elucidate the mass-covering and mode-seeking behaviors of various approaches, explaining when and why certain methods are superior. Practically, we propose a computationally efficient algorithm that extrapolates over a tailored reward model for AfD. We validate our key insights through experiments on the Harmless and Helpful tasks, demonstrating their strong empirical performance while maintaining simplicity.

cs.LG↗

arXiv2025

Chiral Effective Field Theories for Strong and Weak Dynamics

The chiral effective field theory (ChEFT) is an extension of the chiral perturbation theory that includes the nuclear forces and weak currents at the hadronic and nuclear scales. We propose a systematic framework of parametrising the pion-nucleon and nucleon-nucleon Lagrangian via the Weinberg power counting rules. We enumerate the operator bases of ChEFT by extending the Hilbert series of the pure meson sector to the nucleon sector with the CP symmetries. The Young tensor method is utilized to obtain the complete sets of the nucleon-nucleon, three-nucleon operators with/without pions ($\notπ$-EFT). Then we use the spurion technique to reconstruct the ChEFT Lagrangian by taking the adjoint spurion and leptonic fields as the building blocks, without the need of external sources. These operators can be applied to both the strong dynamics, and the nucleon/nuclear weak current processes.

hep-ph↗

arXiv2025

Real-time Identity Defenses against Malicious Personalization of Diffusion Models

Personalized generative diffusion models, capable of synthesizing highly realistic images based on a few reference portraits, may pose substantial social, ethical, and legal risks via identity replication. Existing defense mechanisms rely on computationally intensive adversarial perturbations tailored to individual images, rendering them impractical for real-world deployment. This study introduces the Real-time Identity Defender (RID), a neural network designed to generate adversarial perturbations through a single forward pass, bypassing the need for image-specific optimization. RID achieves unprecedented efficiency, with defense times as low as 0.12 seconds on a single NVIDIA A100 80G GPU (4,400 times faster than leading methods) and 1.1 seconds per image on a standard Intel i9 CPU, making it suitable for edge devices such as smartphones. Despite its efficiency, RID achieves promising protection performance across visual and quantitative benchmarks, effectively mitigating identity replication risks. Our analysis reveals that RID's perturbations mimic the efficacy of traditional defenses while exhibiting properties distinct from natural noise, such as Gaussian perturbations. To enhance robustness, we extend RID into an ensemble framework that integrates multiple pre-trained text-to-image diffusion models, ensuring resilience against black-box attacks and post-processing techniques, including image compression and purification. Our model is envisioned to play a crucial role in safeguarding portrait rights, thereby preventing illegal and unethical uses.

cs.CV↗

arXiv2025

Systematic Spurion Matching between Low Energy EFT and Chiral Lagrangian

The hadronic chiral Lagrangian can be matched from the low energy effective field theory (LEFT) operators at the quark level. Traditionally, as the mass dimension of the LEFT operators increases, more and more external sources are necessarily introduced in the chiral perturbation theory (ChPT). In this work, we present a systematic matching procedure with the single spurion field $\mathbf{T}$ of the flavor $SU(3)_V$ octet, without the need of external sources. We present the complete sets of the LEFT operators, which have been reformulated using the flavor $SU(3)_V$ symmetry up to dimension 9. At the same time, the ChPT Lagrangian is also reformulated using the single spurion and lepton fields, instead of external sources. The spurion matching can be performed between LEFT and ChPT operators with the same flavor $SU(3)_V$ and $CP$ structures, and the same leptonic currents, using the naive dimensional analysis at the quark and hadronic levels.

hep-ph↗

arXiv2025

A Comparative Evaluation of Bayesian Model-Assisted Two-Stage Designs for Phase I/II Clinical Trials

The primary goal of a two-stage Phase I/II trial is to identify the optimal dose for the following large-scale Phase III trial. Recently, Phase I dose-finding designs have shifted from identifying the maximum tolerated dose (MTD) to the optimal biological dose (OBD). Typically, several doses are selected as recommended Phase II doses (RP2D) for further evaluation. In Phase II dose optimization trials, each RP2D is evaluated independently to determine its "go/no-go" decision. The optimal RP2D is then chosen from the remaining RP2Ds as the recommended Phase III dose (RP3D). The effectiveness of both dose-finding and dose optimization designs at two stages impacts RP3D selection. This paper reviews and compares fifteen Bayesian model-assisted two-stage designs, combining five Phase I dose-finding designs (BOIN, TITE-BOIN, BF-BOIN, BOIN12, and TITE-BOIN12) with three Phase II dose optimization designs (TS, BOP2, and TOP). We conduct extensive simulation studies to evaluate their performance under different dose-response scenarios, with and without the existence of the OBD. Based on our results, we recommend the TITE-BOIN12 + TOP combination as the optimal two-stage design for Phase I/II trials.

stat.ME↗

arXiv2025

Detailed Object Description with Controllable Dimensions

Object description plays an important role for visually impaired individuals to understand and compare the differences between objects. Recent multimodal large language models(MLLMs) exhibit powerful perceptual abilities and demonstrate impressive potential for generating object-centric descriptions. However, the descriptions generated by such models may still usually contain a lot of content that is not relevant to the user intent or miss some important object dimension details. Under special scenarios, users may only need the details of certain dimensions of an object. In this paper, we propose a training-free object description refinement pipeline, Dimension Tailor, designed to enhance user-specified details in object descriptions. This pipeline includes three steps: dimension extracting, erasing, and supplementing, which decompose the description into user-specified dimensions. Dimension Tailor can not only improve the quality of object details but also offer flexibility in including or excluding specific dimensions based on user preferences. We conducted extensive experiments to demonstrate the effectiveness of Dimension Tailor on controllable object descriptions. Notably, the proposed pipeline can consistently improve the performance of the recent MLLMs. The code is currently accessible at https://github.com/xin-ran-w/ControllableObjectDescription.

cs.CV↗