arXiv subjects

Yikun Wang

Publications and source records attributed to Yikun Wang.

50 records · Page 3Linked to original sources

arXiv2024

Rescue: Ranking LLM Responses with Partial Ordering to Improve Response Generation

Customizing LLMs for a specific task involves separating high-quality responses from lower-quality ones. This skill can be developed using supervised fine-tuning with extensive human preference data. However, obtaining a large volume of expert-annotated data is costly for most tasks. In this paper, we explore a novel method to optimize LLMs using ranking metrics. This method trains the model to prioritize the best responses from a pool of candidates created for a particular task. Rather than a traditional full ordering, we advocate for a partial ordering, as achieving consensus on the perfect order of candidate responses can be challenging. Our partial ordering is more robust, less sensitive to noise, and can be achieved with limited human annotations or through heuristic methods. We test our system's improved response generation ability using benchmark datasets, including textual entailment and multi-document question answering. We conduct ablation studies to understand crucial factors, such as how to gather candidate responses for a specific task, determine their most suitable order, and balance supervised fine-tuning with ranking metrics. Our approach, named Rescue, offers a promising avenue for enhancing the response generation and task accuracy of LLMs.

cs.CL↗

arXiv2024

Uncertainty Aware Learning for Language Model Alignment

As instruction-tuned large language models (LLMs) evolve, aligning pretrained foundation models presents increasing challenges. Existing alignment strategies, which typically leverage diverse and high-quality data sources, often overlook the intrinsic uncertainty of tasks, learning all data samples equally. This may lead to suboptimal data efficiency and model performance. In response, we propose uncertainty-aware learning (UAL) to improve the model alignment of different task scenarios, by introducing the sample uncertainty (elicited from more capable LLMs). We implement UAL in a simple fashion -- adaptively setting the label smoothing value of training according to the uncertainty of individual samples. Analysis shows that our UAL indeed facilitates better token clustering in the feature space, validating our hypothesis. Extensive experiments on widely used benchmarks demonstrate that our UAL significantly and consistently outperforms standard supervised fine-tuning. Notably, LLMs aligned in a mixed scenario have achieved an average improvement of 10.62\% on high-entropy tasks (i.e., AlpacaEval leaderboard), and 1.81\% on complex low-entropy tasks (i.e., MetaMath and GSM8K).

cs.CL↗

arXiv2024

Towards Efficient SRAM-PIM Architecture Design by Exploiting Unstructured Bit-Level Sparsity

Bit-level sparsity in neural network models harbors immense untapped potential. Eliminating redundant calculations of randomly distributed zero-bits significantly boosts computational efficiency. Yet, traditional digital SRAM-PIM architecture, limited by rigid crossbar architecture, struggles to effectively exploit this unstructured sparsity. To address this challenge, we propose Dyadic Block PIM (DB-PIM), a groundbreaking algorithm-architecture co-design framework. First, we propose an algorithm coupled with a distinctive sparsity pattern, termed a dyadic block (DB), that preserves the random distribution of non-zero bits to maintain accuracy while restricting the number of these bits in each weight to improve regularity. Architecturally, we develop a custom PIM macro that includes dyadic block multiplication units (DBMUs) and Canonical Signed Digit (CSD)-based adder trees, specifically tailored for Multiply-Accumulate (MAC) operations. An input pre-processing unit (IPU) further refines performance and efficiency by capitalizing on block-wise input sparsity. Results show that our proposed co-design framework achieves a remarkable speedup of up to 7.69x and energy savings of 83.43%.

cs.AR↗

arXiv2024

LLM-DA: Data Augmentation via Large Language Models for Few-Shot Named Entity Recognition

Despite the impressive capabilities of large language models (LLMs), their performance on information extraction tasks is still not entirely satisfactory. However, their remarkable rewriting capabilities and extensive world knowledge offer valuable insights to improve these tasks. In this paper, we propose $LLM-DA$, a novel data augmentation technique based on LLMs for the few-shot NER task. To overcome the limitations of existing data augmentation methods that compromise semantic integrity and address the uncertainty inherent in LLM-generated text, we leverage the distinctive characteristics of the NER task by augmenting the original data at both the contextual and entity levels. Our approach involves employing 14 contextual rewriting strategies, designing entity replacements of the same type, and incorporating noise injection to enhance robustness. Extensive experiments demonstrate the effectiveness of our approach in enhancing NER model performance with limited data. Furthermore, additional analyses provide further evidence supporting the assertion that the quality of the data we generate surpasses that of other existing methods.

cs.CL↗

arXiv2023

Macroscopic Dark Matter Detection with Gravitational Wave Experiments

We study signatures of macroscopic dark matter (DM) in current and future gravitational wave (GW) experiments. Transiting DM with a mass of $\sim10^5-10^{15}$ kg that saturates the local DM density can be potentially detectable by GW detectors, depending on the baseline of the detector and the strength of the force mediating the interaction. In the context of laser interferometers, we derive the gauge invariant observable due to a transiting DM, including the Shapiro effect (gravitational time delay accumulated during the photon propagation), and adequately account for the finite photon travel time within an interferometer arm. In particular, we find that the Shapiro effect can be dominant for short-baseline interferometers such as Holometer and GQuEST. We also find that proposed experiments such as Cosmic Explorer and Einstein Telescope can constrain a fifth force between DM and baryons, at the level of strength $\sim 10^3$ times stronger than gravity for, e.g., kg mass DM with a fifth-force range of $10^6$ m.

astro-ph.CO↗

arXiv2023

DDC-PIM: Efficient Algorithm/Architecture Co-design for Doubling Data Capacity of SRAM-based Processing-In-Memory

Processing-in-memory (PIM), as a novel computing paradigm, provides significant performance benefits from the aspect of effective data movement reduction. SRAM-based PIM has been demonstrated as one of the most promising candidates due to its endurance and compatibility. However, the integration density of SRAM-based PIM is much lower than other non-volatile memory-based ones, due to its inherent 6T structure for storing a single bit. Within comparable area constraints, SRAM-based PIM exhibits notably lower capacity. Thus, aiming to unleash its capacity potential, we propose DDC-PIM, an efficient algorithm/architecture co-design methodology that effectively doubles the equivalent data capacity. At the algorithmic level, we propose a filter-wise complementary correlation (FCC) algorithm to obtain a bitwise complementary pair. At the architecture level, we exploit the intrinsic cross-coupled structure of 6T SRAM to store the bitwise complementary pair in their complementary states ($Q/\overline{Q}$), thereby maximizing the data capacity of each SRAM cell. The dual-broadcast input structure and reconfigurable unit support both depthwise and pointwise convolution, adhering to the requirements of various neural networks. Evaluation results show that DDC-PIM yields about $2.84\times$ speedup on MobileNetV2 and $2.69\times$ on EfficientNet-B0 with negligible accuracy loss compared with PIM baseline implementation. Compared with state-of-the-art SRAM-based PIM macros, DDC-PIM achieves up to $8.41\times$ and $2.75\times$ improvement in weight density and area efficiency, respectively.

cs.AR↗

arXiv2023

Axion Detection with Optomechanical Cavities

We propose a novel technique to search for axions with an optomechanical cavity filled with a material such as superfluid helium. Axion absorption converts a pump laser photon to a photon plus a phonon. The axion absorption rate is enhanced by the high occupation number of coherent photons or phonons in the cavity, allowing our proposal to largely overcome the extremely small axion coupling. The axion mass probed is set by the relative frequency of the photon produced in the final state and the Stokes mode. Because neither the axion mass nor momentum need to be matched to the physical size of the cavity, we can scale up the cavity size while maintaining access to a wide range of axion masses (up to a meV) complementary to other cavity proposals.

hep-ph↗

arXiv2022

Anatomy of the Electroweak Phase Transition for Dark Sector induced Baryogenesis

We investigate the electroweak phase transition patterns for a recently proposed baryogenesis model with CP violation originated in the dark sector. The model includes a complex scalar singlet-Higgs boson portal, a $U(1)_l$ gauge lepton symmetry with a $Z^\prime$ gauge boson portal and a fermionic dark matter particle. We find a novel thermal history of the scalar sector, featuring a $Z_2$ breaking singlet vacuum in the early Universe driven by a dark Yukawa coupling, that induces a one-step strongly first order electroweak phase transition. We explore the parameter space that generates the observed matter-antimatter asymmetry and dark matter relic abundance, while being consistent with constraints from electric dipole moment and collider searches, and dark matter direct detection bounds. The complex singlet can be produced via the Higgs portal and decays into Standard Model particles after traveling a certain distance. We explore the reach for long-lived singlet scalars at the {\unit[13]{TeV}} Large Hadron Collider with $\mathcal{L}=\unit[139]{fb^{-1}}$ and show its impact on the parameter space of the model. Setting aside currently unresolved theoretical uncertainties, we estimate the gravitational wave signatures detectable at future observatories.

hep-ph↗

arXiv2022

Probing the Electroweak Phase Transition with Exotic Higgs Decays

An essential goal of the Higgs physics program at the LHC and beyond is to explore the nature of the Higgs potential and shed light on the mechanism of electroweak symmetry breaking. An important class of models defining the strength and order of the electroweak phase transition is driven by the Higgs boson coupling to a light new state. This Snowmass white paper points out the existence of a region of parameter space where a strongly first order electroweak phase transition is compatible with exotic decays of the SM-like Higgs boson. A dedicated search for exotic Higgs decays can actively explore this framework at the Large Hadron Collider (LHC), while future exotic Higgs decay searches at the high-luminosity LHC and future Higgs factories will be vital to conclusively probe the scenario.

hep-ph↗

arXiv2021

A New Approach to Electroweak Symmetry Non-Restoration

Electroweak symmetry non-restoration up to high temperatures well above the electroweak scale offers new alternatives for baryogenesis. We propose a new approach for electroweak symmetry non-restoration via an inert Higgs sector that couples to the Standard Model Higgs as well as an extended scalar singlet sector. We implement renormalization group improvements and thermal resummation, necessary to evaluate the effective potential spanning over a broad range of energy scales and temperatures. We present examples of benchmark scenarios that allow for electroweak symmetry non-restoration all the way up to hundreds of TeV temperatures, and also feature suppressed sphaleron washout factors down to the electroweak scale. Our method for transmitting the Standard Model broken electroweak symmetry to an inert Higgs sector has several intriguing implications for (electroweak) baryogenesis, early universe thermal histories, and can be scrutinized through Higgs physics phenomenology and electroweak precision measurements at the HL-LHC.

hep-ph↗

arXiv2021

Towards an all-orders calculation of the electroweak bubble wall velocity

We analyze Higgs condensate bubble expansion during a first-order electroweak phase transition in the early Universe. The interaction of particles with the bubble wall can be accompanied by the emission of multiple soft gauge bosons. When computed at fixed order in perturbation theory, this process exhibits large logarithmic enhancements which must be resummed to all orders when the wall velocity is large. We perform this resummation both analytically and numerically at leading logarithmic accuracy. The numerical simulation is achieved by means of a particle shower in the broken phase of the electroweak theory. The two approaches agree to the 10\% level. For fast-moving walls, we find the scaling of the thermal pressure exerted against the wall to be $P\sim γ^2T^4$, independent of the particle masses, implying a significantly slower terminal velocity than previously suggested.

hep-ph↗

arXiv2021

Nucleation is More than Critical -- A Case Study of the Electroweak Phase Transition in the NMSSM

Electroweak baryogenesis is an attractive mechanism to generate the baryon asymmetry of the Universe via a strong first order electroweak phase transition. We compare the phase transition patterns suggested by the vacuum structure at the critical temperatures, at which local minima are degenerate, with those obtained from computing the probability for nucleation via tunneling through the barrier separating local minima. Heuristically, nucleation becomes difficult if the barrier between the local minima is too high, or if the distance (in field space) between the minima is too large. As an example of a model exhibiting such behavior, we study the Next-to-Minimal Supersymmetric Standard Model, whose scalar sector contains two SU(2) doublets and one gauge singlet. We find that the calculation of the nucleation probabilities prefers different regions of parameter space for a strong first order electroweak phase transition than the calculation based solely on the critical temperatures. Our results demonstrate that analyzing only the vacuum structure via the critical temperatures can provide a misleading picture of the phase transition patterns, and, in turn, of the parameter space suitable for electroweak baryogenesis.

hep-ph↗

arXiv2020

Electroweak $\!$phase $\!$transition $\!$with $\!$spontaneous $\!$$Z_2$-breaking

This work investigates a simple, representative extension of the Standard Model with a real scalar singlet and spontaneous $Z_2$ breaking, which allows for a strongly first-order phase transition, as required by electroweak baryogenesis. We perform analytical and numerical calculations that systematically include one-loop thermal effects, Coleman-Weinberg corrections, and daisy resummation, as well as evaluation of bubble nucleation. We study the rich thermal history and identify the conditions for a strongly first-order electroweak phase transition with nearly degenerate extrema at zero temperature. This requires a light scalar with mass below 50 GeV. Exotic Higgs decays, as well as Higgs coupling precision measurements at the LHC and future collider facilities, will test this model. Additional information may be obtained from future collider constraints on the Higgs self-coupling. Gravitational-wave signals are typically too low to be probed by future gravitational wave experiments.

hep-ph↗

arXiv2014

Commutators of lepton mass matrices associated with seesaw and leptogenesis

The origin of tiny neutrino masses and the baryon number asymmetry of the Universe are naturally interpreted by the canonical seesaw and leptogenesis mechanisms, in which there are the heavy Majorana neutrino mass matrix $M_{\rm R}$, the Dirac neutrino mass matrix $M_{\rm D}$, the charged-lepton mass matrix $M_\ell$ and the effective (light) neutrino mass matrix $M_ν$. We find that ${\rm Im}\left(\det\left[ M^\dagger_{\rm D} M_{\rm D}, M^\dagger_{\rm R} M_{\rm R} \right]\right)$, ${\rm Im}\left(\det\left[ M_\ell M^\dagger_\ell, M_νM^\dagger_ν\right]\right)$ and ${\rm Im}\left(\det\left[ M_\ell M^\dagger_\ell, M_{\rm D} M^\dagger_{\rm D} \right]\right)$ can serve for a basis-independent measure of CP violation associated with lepton-number-violating decays of heavy neutrinos, flavor oscillations of light neutrinos and lepton-flavor-violating decays of charged leptons, respectively. We first calculate these quantities with the help of a standard parametrization of the $6\times 6$ flavor mixing matrix, and then discuss their implications on both leptogenesis and CP violation at low energy scales. A comparison with the weak-basis invariants of leptogenesis as proposed by Branco {\it et al} is also made.

hep-ph↗