arXiv subjects

Hao Sun

Publications and source records attributed to Hao Sun.

At least 325 records · Page 18Linked to original sources

arXiv2024

Supervised Fine-Tuning as Inverse Reinforcement Learning

The prevailing approach to aligning Large Language Models (LLMs) typically relies on human or AI feedback and assumes access to specific types of preference datasets. In our work, we question the efficacy of such datasets and explore various scenarios where alignment with expert demonstrations proves more realistic. We build a sequential decision-making framework to formulate the problem of aligning LLMs using demonstration datasets. Drawing insights from inverse reinforcement learning and imitation learning, we introduce various approaches for divergence minimization in the LLM alignment tasks. Our analysis highlights the mass-covering and mode-seeking behaviors of these different approaches. Inclusively, we examine the pros and cons of the classical supervised fine-tuning method, elaborating on scenarios where different methods shine.

cs.LG↗

arXiv2024

Query-Dependent Prompt Evaluation and Optimization with Offline Inverse RL

In this study, we aim to enhance the arithmetic reasoning ability of Large Language Models (LLMs) through zero-shot prompt optimization. We identify a previously overlooked objective of query dependency in such optimization and elucidate two ensuing challenges that impede the successful and economical design of prompt optimization techniques. One primary issue is the absence of an effective method to evaluate prompts during inference when the golden answer is unavailable. Concurrently, learning via interactions with the LLMs to navigate the expansive natural language prompting space proves to be resource-intensive. To address this, we introduce Prompt-OIRL, which harnesses offline inverse reinforcement learning to draw insights from offline prompting demonstration data. Such data exists as by-products when diverse prompts are benchmarked on open-accessible datasets. With Prompt-OIRL, the query-dependent prompt optimization objective is achieved by first learning an offline reward model. This model can evaluate any query-prompt pairs without accessing LLMs. Subsequently, a best-of-N strategy is deployed to recommend the optimal prompt. Our experimental evaluations across various LLM scales and arithmetic reasoning datasets underscore both the efficacy and economic viability of the proposed approach.

cs.CL↗

arXiv2024

Z' Mediated right-handed Neutrinos from Meson Decays at the FASER

We investigate the pair production of right-handed neutrinos mediated by a $Z^\prime$ mainly from the meson decays at the FASER detector of the HL-LHC. The $Z^\prime$ can be the additional gauge boson in either the $U(1)_{B-L}$ or sterile $ν$-specific $U(1)_s$ model. Taking the gauge coupling or the kinetic mixing at the current limits, we analyse the sensitivity to the masses of the heavy neutrinos, $M_N$, and active-sterile mixing, $|V_{lN}|^2$, of the FASER2. In a background free scenario, for the $U(1)_{B-L}$ case, FASER2 is able to probe $|V_{lN}|^2 \approx 10^{-8}$ when $M_N \sim 0.3$ GeV, which is comparable to the current limits from the beam dump experiments. When comes to the $U(1)_s$ model, FASER2 can probe $|V_{lN}|^2 \approx 10^{-10}$, which is better than the current limits for at least one magnitude, in all three flavours. A proposed long-lived particle detector, FACET, is also studied, while no significant difference from FASER2 is derived.

hep-ph↗

arXiv2024

Integrated Interpolation and Block-term Tensor Decomposition for Spectrum Map Construction

This paper addresses the challenge of reconstructing a 3D power spectrum map from sparse, scattered, and incomplete spectrum measurements. It proposes an integrated approach combining interpolation and block-term tensor decomposition (BTD). This approach leverages an interpolation model with the BTD structure to exploit the spatial correlation of power spectrum maps. Additionally, nuclear norm regularization is incorporated to effectively capture the low-rank characteristics. To implement this approach, a novel algorithm that combines alternating regression with singular value thresholding is developed. Analytical justification for the enhancement provided by the BTD structure in interpolating power spectrum maps is provided, yielding several important theoretical insights. The analysis explores the impact of the spectrum on the error in the proposed method and compares it to conventional local polynomial interpolation. Extensive numerical results demonstrate that the proposed method outperforms state-of-the-art methods in terms of signal source separation and power spectrum map construction, and remains stable under off-grid measurements and inhomogeneous measurement topologies.

eess.SP↗

arXiv2024

On the Existence of Gr-semistable Filtrations of Orthogonal/Symplectic $λ$-connections

In this paper, we study the existence of gr-semistable filtrations of orthogonal/symplectic $λ$-connections. It is known that gr-semistable filtrations always exist for flat bundles in arbitrary characteristic. However, we found a counterexample of orthogonal flat bundles of rank 5 in positive characteristic. The central new idea in this example is the notion of quasi gr-semistability for orthogonal/symplectic $λ$-connections. We establish the equivalence between gr-semistability and quasi gr-semistablity for an orthogonal/symplectic $λ$-connection. This provides a way to determine whether an orthogonal/symplectic $λ$-connection is gr-semistable. As an application, we obtain a characterization of gr-semistable orthogonal $λ$-connections of rank $\leq 6$.

math.AG↗

arXiv2024

Dense Reward for Free in Reinforcement Learning from Human Feedback

Reinforcement Learning from Human Feedback (RLHF) has been credited as the key advance that has allowed Large Language Models (LLMs) to effectively follow instructions and produce useful assistance. Classically, this involves generating completions from the LLM in response to a query before using a separate reward model to assign a score to the full completion. As an auto-regressive process, the LLM has to take many "actions" (selecting individual tokens) and only receives a single, sparse reward at the end of an episode, a setup that is known to be difficult to optimise in traditional reinforcement learning. In this work we leverage the fact that the reward model contains more information than just its scalar output, in particular, it calculates an attention map over tokens as part of the transformer architecture. We use these attention weights to redistribute the reward along the whole completion, effectively densifying the signal and highlighting the most important tokens, all without incurring extra computational cost or requiring any additional modelling. We demonstrate that, theoretically, this approach is equivalent to potential-based reward shaping, ensuring that the optimal policy remains unchanged. Empirically, we show that it stabilises training, accelerates the rate of learning, and, in practical cases, may lead to better local optima.

cs.LG↗

arXiv2024

Attosecond two-color x-ray free-electron lasers with dual chirp-taper configuration and bunching inheritance

Attosecond x-ray pulses play a crucial role in the study of ultrafast phenomena occurring within inner and valence electrons. To achieve attosecond time-resolution studies and gain control over electronic wavefunctions, it is crucial to develop techniques capable of generating and synchronizing two-color x-ray pulses at the attosecond scale. In this paper, we present a novel approach for generating attosecond pulse pairs using a dual chirp-taper free-electron laser with bunching inheritance. An electron beam with a sinusoidal energy chirp, introduced by the external seed laser, passes through the main undulator and afterburner, both with tapers. Two-color x-ray pulses are generated from the main undulator and afterburner, respectively, with temporal separations of hundreds of attoseconds to several femtoseconds and energy differences of tens of electron volts. Notably, the afterburner is much shorter than the main undulator due to bunching inheritance, which reduces the distance between two source points and alleviates the beamline focusing requirements of the two-color pulses. A comprehensive stability analysis is conducted in this paper, considering the individual effects of shot noise from self-amplified spontaneous emission and carrier-envelope phase jitter of the few-cycle laser. The results show that the radiation from the afterburner exhibits excellent stability in the proposed scheme, which is beneficial for x-ray pump-probe experiments. The proposed scheme opens up new possibilities for attosecond science enabled by x-ray attosecond pump-probe techniques and coherent control of ultrafast electronic wave packets in quantum systems.

physics.acc-ph↗

arXiv2024

Memory-Inspired Temporal Prompt Interaction for Text-Image Classification

In recent years, large-scale pre-trained multimodal models (LMM) generally emerge to integrate the vision and language modalities, achieving considerable success in various natural language processing and computer vision tasks. The growing size of LMMs, however, results in a significant computational cost for fine-tuning these models for downstream tasks. Hence, prompt-based interaction strategy is studied to align modalities more efficiently. In this contex, we propose a novel prompt-based multimodal interaction strategy inspired by human memory strategy, namely Memory-Inspired Temporal Prompt Interaction (MITP). Our proposed method involves in two stages as in human memory strategy: the acquiring stage, and the consolidation and activation stage. We utilize temporal prompts on intermediate layers to imitate the acquiring stage, leverage similarity-based prompt interaction to imitate memory consolidation, and employ prompt generation strategy to imitate memory activation. The main strength of our paper is that we interact the prompt vectors on intermediate layers to leverage sufficient information exchange between modalities, with compressed trainable parameters and memory usage. We achieve competitive results on several datasets with relatively small memory usage and 2.0M of trainable parameters (about 1% of the pre-trained foundation model).

cs.CV↗

arXiv2024

Multi-resolution partial differential equations preserved learning framework for spatiotemporal dynamics

Traditional data-driven deep learning models often struggle with high training costs, error accumulation, and poor generalizability in complex physical processes. Physics-informed deep learning (PiDL) addresses these challenges by incorporating physical principles into the model. Most PiDL approaches regularize training by embedding governing equations into the loss function, yet this depends heavily on extensive hyperparameter tuning to weigh each loss term. To this end, we propose to leverage physics prior knowledge by ``baking'' the discretized governing equations into the neural network architecture via the connection between the partial differential equations (PDE) operators and network structures, resulting in a PDE-preserved neural network (PPNN). This method, embedding discretized PDEs through convolutional residual networks in a multi-resolution setting, largely improves the generalizability and long-term prediction accuracy, outperforming conventional black-box models. The effectiveness and merit of the proposed methods have been demonstrated across various spatiotemporal dynamical systems governed by spatiotemporal PDEs, including reaction-diffusion, Burgers', and Navier-Stokes equations.

cs.LG↗

arXiv2024

Theoretical study of the $B^+\to D^-D_s^{+}π^+$ reaction

Prompted by the recent discoveries of $T_{c\bar{s}0}(2900)^{++}$ in the $D_s^+π^+$ invariant mass distribution of $B^+\to D^-D_s^+π^+$ process, we present a model that hopes to help us investigate the nature of $T_{c\bar{s}0}(2900)^{++}$ by reproducing the mass distribution of $D^-π^+, D_s^+π^+$ and $D^-D_s^+$ in $B^+ \to D^-D_s^+π^+$ decays. The structure of the triangular singularity peak generated from the $χ_{c1}D^{*+}K^{*+}$ loop near the $D^{*+}K^{*+}$ threshold is considered in our model may be the experimentally discovered resonance-like state structure $T_{c\bar{s}0}(2900)^{++}$. In addition, we employ a coupled-channel approach to describe the dominant contribution of the $Dπ$ $S\text{-wave}$ amplitude, and also consider other excitations. Our model provides a well fit to the invariant mass distributions of $D^-π^+, D_s^+π^+$ and $D^-D_s^+$ simultaneously.

hep-ph↗

arXiv2024

TR-DETR: Task-Reciprocal Transformer for Joint Moment Retrieval and Highlight Detection

Video moment retrieval (MR) and highlight detection (HD) based on natural language queries are two highly related tasks, which aim to obtain relevant moments within videos and highlight scores of each video clip. Recently, several methods have been devoted to building DETR-based networks to solve both MR and HD jointly. These methods simply add two separate task heads after multi-modal feature extraction and feature interaction, achieving good performance. Nevertheless, these approaches underutilize the reciprocal relationship between two tasks. In this paper, we propose a task-reciprocal transformer based on DETR (TR-DETR) that focuses on exploring the inherent reciprocity between MR and HD. Specifically, a local-global multi-modal alignment module is first built to align features from diverse modalities into a shared latent space. Subsequently, a visual feature refinement is designed to eliminate query-irrelevant information from visual features for modal interaction. Finally, a task cooperation module is constructed to refine the retrieval pipeline and the highlight score prediction process by utilizing the reciprocity between MR and HD. Comprehensive experiments on QVHighlights, Charades-STA and TVSum datasets demonstrate that TR-DETR outperforms existing state-of-the-art methods. Codes are available at \url{https://github.com/mingyao1120/TR-DETR}.

cs.CV↗

arXiv2023

PKBOIN-12: A Bayesian optimal interval Phase I/II design incorporating pharmacokinetics outcomes to find the optimal biological dose

Immunotherapies and targeted therapies have gained popularity due to their promising therapeutic effects across multiple treatment areas. The focus of early phase dose-finding clinical trials has shifted from finding the maximum tolerated dose (MTD) to identifying the optimal biological dose (OBD), which aims to balance the toxicity and efficacy outcomes, thereby optimizing the risk-benefit trade-off. These trials often collect multiple pharmacokinetics (PK) outcomes to assess drug exposure, which has shown correlations with toxicity and efficacy outcomes but has not been utilized in the current dose-finding designs for OBD selection. Moreover, PK outcomes are usually available within days after initial treatment, much faster than toxicity and efficacy outcomes. To bridge this gap, we introduce the innovative model-assisted PKBOIN-12 design, which enhances the BOIN12 design by integrating PK information into both the dose-finding algorithm and the final OBD determination process. We further extend PKBOIN-12 to the TITE-PKBOIN-12 design to address the challenges of late-onset toxicity and efficacy outcomes. Simulation results demonstrate that the PKBOIN-12 design more effectively identifies the OBD and allocates a greater number of patients to it than BOIN12. Additionally, PKBOIN-12 decreases the probability of selecting inefficacious doses as the OBD by excluding those with low drug exposure. Comprehensive simulation studies and sensitivity analysis confirm the robustness of both PKBOIN-12 and TITE-PKBOIN-12 designs in various scenarios.

stat.ME↗

arXiv2023

UPRISE: Universal Prompt Retrieval for Improving Zero-Shot Evaluation

Large Language Models (LLMs) are popular for their impressive abilities, but the need for model-specific fine-tuning or task-specific prompt engineering can hinder their generalization. We propose UPRISE (Universal Prompt Retrieval for Improving zero-Shot Evaluation), which tunes a lightweight and versatile retriever that automatically retrieves prompts for a given zero-shot task input. Specifically, we demonstrate universality in a cross-task and cross-model scenario: the retriever is tuned on a diverse set of tasks, but tested on unseen task types; we use a small frozen LLM, GPT-Neo-2.7B, for tuning the retriever, but test the retriever on different LLMs of much larger scales, such as BLOOM-7.1B, OPT-66B and GPT3-175B. Additionally, we show that UPRISE mitigates the hallucination problem in our experiments with ChatGPT, suggesting its potential to improve even the strongest LLMs. Our model and code are available at https://github.com/microsoft/LMOps.

cs.CL↗

arXiv2023

LEAD: Liberal Feature-based Distillation for Dense Retrieval

Knowledge distillation is often used to transfer knowledge from a strong teacher model to a relatively weak student model. Traditional methods include response-based methods and feature-based methods. Response-based methods are widely used but suffer from lower upper limits of performance due to their ignorance of intermediate signals, while feature-based methods have constraints on vocabularies, tokenizers and model architectures. In this paper, we propose a liberal feature-based distillation method (LEAD). LEAD aligns the distribution between the intermediate layers of teacher model and student model, which is effective, extendable, portable and has no requirements on vocabularies, tokenizers, or model architectures. Extensive experiments show the effectiveness of LEAD on widely-used benchmarks, including MS MARCO Passage Ranking, TREC 2019 DL Track, MS MARCO Document Ranking and TREC 2020 DL Track. Our code is available in https://github.com/microsoft/SimXNS/tree/main/LEAD.

cs.IR↗

arXiv2023

Effective Field Theories of Axion, ALP and Dark Photon

With the help of Young tensor technique, we enumerate the complete and independent set of effective operators up to $dim$-8 for the extension of the standard model with a Goldsonte boson by further imposing the Adler's zero condition in the soft momentum limit. Such basis can be reduced to describe the axion or majoron effective Lagrangian if further (symmetry) constraints are imposed. Then reformulating dark photon as combination of Goldstone boson and transverse gauge boson, the effective operators of the Goldstone boson can be extended to effective chiral Lagrangian description of the dark photon. For the first time we obtain 0 (0), 6 (44), 1 (1), 44 (356), 32 (520) operators in Goldstone effective field theory, and 9 (49), 0 (0), 108 (676), 10 (426), 1904 (40783) operators in dark photon effective field theory at the dimension 4, 5, 6, 7, 8 for one (three) generation of fermions.

hep-ph↗

arXiv2023

Investigate the strong coupling of $g_{X J/ψϕ}$ in $X(4500) \to J/ψϕ$ by using the three-point sum rules and the light-cone sum rules

We assign $X(4500)$ as a D-wave tetraquark state and study the decay of $X(4500)$ $\to$ $J/ψϕ$. The mass and the decay constant of $X(4500)$ are calculated by using the SVZ sum rules. For the decay width of $X(4500)$ $\to$ $J/ψϕ$, we present the calculation within the framework of both the three-point sum rules and the light-cone sum rules. The strong coupling $g_{X J/ψϕ}$ is obtained by considering the soft-meson approximation when we use the light-cone sum rules calculation. Both calculations show that the decay of $X(4500)$ $\to$ $J/ψϕ$ close to the total width of $X(4500)$ if we assign $X$(4500) as a D-wave tetraquark. In this paper, only the hidden-charm decay channel is considered. With these results and those from the open-charm decay channel, we are able to give a more rational conclusion when comparing with the total width of $X(4500)$. In the future, experiments will be more helpful in determining whether or not this structure of $X(4500)$ is appropriate.

hep-ph↗

arXiv2023

The strong decay of $Y(4230)\to J/ψf_0(980)$ in light cone sum rules

In this work, we assign the tetraquark state for $Y(4230)$ resonance, and investigate the mass and decay constant of $Y(4230)$ in the framework of SVZ sum rules through a different calculation technique. Then we calculate the strong coupling $g_{Y J/ψf_0}$ by considering soft-meson approximation techniques, within the framework of light cone sum rules. And we use strong coupling $g_{Y J/ψf_0}$ to obtain the width of the decay $Y(4230)\to J/ψf_0(980)$. Our prediction for the mass is in agreement with the experimental measurement, and that for the decay width of $Y(4230)\to J/ψf_0(980)$ is within the upper limit.

hep-ph↗

arXiv2023

Towards Safer Generative Language Models: A Survey on Safety Risks, Evaluations, and Improvements

As generative large model capabilities advance, safety concerns become more pronounced in their outputs. To ensure the sustainable growth of the AI ecosystem, it's imperative to undertake a holistic evaluation and refinement of associated safety risks. This survey presents a framework for safety research pertaining to large models, delineating the landscape of safety risks as well as safety evaluation and improvement methods. We begin by introducing safety issues of wide concern, then delve into safety evaluation methods for large models, encompassing preference-based testing, adversarial attack approaches, issues detection, and other advanced evaluation methods. Additionally, we explore the strategies for enhancing large model safety from training to deployment, highlighting cutting-edge safety approaches for each stage in building large models. Finally, we discuss the core challenges in advancing towards more responsible AI, including the interpretability of safety mechanisms, ongoing safety issues, and robustness against malicious attacks. Through this survey, we aim to provide clear technical guidance for safety researchers and encourage further study on the safety of large models.

cs.AI↗