arXiv subjects

Wei Xiong

Publications and source records attributed to Wei Xiong.

At least 109 records · Page 6Linked to original sources

arXiv2024

Harnessing metastability for grain size control in multiprincipal element alloys during additive manufacturing

Controlling microstructure in fusion-based metal additive manufacturing (AM) remains a challenge due to numerous parameters directly impacting solidification conditions. Multiprincipal element alloys (MPEAs) offer a vast compositional design space for microstructural engineering due to their chemical complexity and exceptional properties. Here, we establish a novel alloy design paradigm in MPEAs for AM using the FeMnCoCr system. By exploiting the decreasing phase stability with increasing Mn content, we achieve notable grain refinement and breakdown of columnar grain growth. We combine thermodynamic modeling, operando synchrotron X-ray diffraction, multiscale microstructural characterization, and mechanical testing to gain insight into the solidification physics and its ramifications on the resulting microstructure. This work paves way for tailoring grain sizes through targeted manipulation of phase stability, thereby advancing microstructure control in AM.

cond-mat.mtrl-sci↗

arXiv2024

LMFlow: An Extensible Toolkit for Finetuning and Inference of Large Foundation Models

Foundation models have demonstrated a great ability to achieve general human-level intelligence far beyond traditional approaches. As the technique keeps attracting attention from the AI community, an increasing number of foundation models are becoming publicly accessible. However, a significant shortcoming of most of these models lies in their performance in specialized-domain and task-specific applications, necessitating domain- and task-aware fine-tuning to develop effective scientific language models. As the number of available foundation models and specialized tasks keeps growing, the job of training scientific language models becomes highly nontrivial. In this paper, we initiate steps to tackle this issue. We introduce an extensible and lightweight toolkit, LMFlow, which aims to simplify the domain- and task-aware finetuning of general foundation models. LMFlow offers a complete finetuning workflow for a foundation model to support specialized training with limited computing resources. Furthermore, it supports continuous pretraining, instruction tuning, parameter-efficient finetuning, alignment tuning, inference acceleration, long context generalization, model customization, and even multimodal finetuning, along with carefully designed and extensible APIs. This toolkit has been thoroughly tested and is available at https://github.com/OptimalScale/LMFlow.

cs.CL↗

arXiv2024

Iterative Preference Learning from Human Feedback: Bridging Theory and Practice for RLHF under KL-Constraint

This paper studies the alignment process of generative models with Reinforcement Learning from Human Feedback (RLHF). We first identify the primary challenges of existing popular methods like offline PPO and offline DPO as lacking in strategical exploration of the environment. Then, to understand the mathematical principle of RLHF, we consider a standard mathematical formulation, the reverse-KL regularized contextual bandit for RLHF. Despite its widespread practical application, a rigorous theoretical analysis of this formulation remains open. We investigate its behavior in three distinct settings -- offline, online, and hybrid -- and propose efficient algorithms with finite-sample theoretical guarantees. Moving towards practical applications, our framework, with a robust approximation of the information-theoretical policy improvement oracle, naturally gives rise to several novel RLHF algorithms. This includes an iterative version of the Direct Preference Optimization (DPO) algorithm for online settings, and a multi-step rejection sampling strategy for offline scenarios. Our empirical evaluations on real-world alignment experiment of large language model demonstrate that these proposed methods significantly surpass existing strong baselines, such as DPO and Rejection Sampling Optimization (RSO), showcasing the connections between solid theoretical foundations and their potent practical implementations.

cs.LG↗

arXiv2024

Nonreciprocal Photon-Phonon Entanglement in Kerr-Modified Spinning Cavity Magnomechanics

Cavity magnomechanics has shown great potential in studying macroscopic quantum effects, especially for quantum entanglement, which is a key resource for quantum information science. Here we propose to realize magnon mediated nonreciprocal photon-phonon entanglement, which exhibits asymmetry when opposite magnetic or driving fields are respectively applied to the magnons with the Kerr effect or the photons with the Sagnac effect. We find that the mean magnon number can selectively exhibit nonreciprocal linear or nonlinear (bistable) behavior with the strength of the strong driving field on the cavity. Assisted by this driving field, the magnon-phonon coupling is greatly enhanced, leading to the nonreciprocal photon-phonon entanglement via the swapping interaction between the magnons and photons. This nonreciprocal entanglement can be significantly enhanced with the magnon Kerr and Sagnac effects. Given the available parameters, the nonreciprocal photon-phonon entanglement can be preserved at $\sim3$ K, showing remarkable resilience against the bath temperature. The result reveals that our paper holds promise in developing various nonreciprocal devices with both the magnon Kerr and Sagnac effects in cavity magnomechanics.

quant-ph↗

arXiv2024

Relightful Harmonization: Lighting-aware Portrait Background Replacement

Portrait harmonization aims to composite a subject into a new background, adjusting its lighting and color to ensure harmony with the background scene. Existing harmonization techniques often only focus on adjusting the global color and brightness of the foreground and ignore crucial illumination cues from the background such as apparent lighting direction, leading to unrealistic compositions. We introduce Relightful Harmonization, a lighting-aware diffusion model designed to seamlessly harmonize sophisticated lighting effect for the foreground portrait using any background image. Our approach unfolds in three stages. First, we introduce a lighting representation module that allows our diffusion model to encode lighting information from target image background. Second, we introduce an alignment network that aligns lighting features learned from image background with lighting features learned from panorama environment maps, which is a complete representation for scene illumination. Last, to further boost the photorealism of the proposed method, we introduce a novel data simulation pipeline that generates synthetic training pairs from a diverse range of natural images, which are used to refine the model. Our method outperforms existing benchmarks in visual fidelity and lighting coherence, showing superior generalization in real-world testing scenarios, highlighting its versatility and practicality.

cs.CV↗

arXiv2024

Strengthening Multimodal Large Language Model with Bootstrapped Preference Optimization

Multimodal Large Language Models (MLLMs) excel in generating responses based on visual inputs. However, they often suffer from a bias towards generating responses similar to their pretraining corpus, overshadowing the importance of visual information. We treat this bias as a "preference" for pretraining statistics, which hinders the model's grounding in visual input. To mitigate this issue, we propose Bootstrapped Preference Optimization (BPO), which conducts preference learning with datasets containing negative responses bootstrapped from the model itself. Specifically, we propose the following two strategies: 1) using distorted image inputs to the MLLM for eliciting responses that contain signified pretraining bias; 2) leveraging text-based LLM to explicitly inject erroneous but common elements into the original response. Those undesirable responses are paired with original annotated responses from the datasets to construct the preference dataset, which is subsequently utilized to perform preference learning. Our approach effectively suppresses pretrained LLM bias, enabling enhanced grounding in visual inputs. Extensive experimentation demonstrates significant performance improvements across multiple benchmarks, advancing the state-of-the-art in multimodal conversational systems.

cs.CL↗

arXiv2024

IMPRINT: Generative Object Compositing by Learning Identity-Preserving Representation

Generative object compositing emerges as a promising new avenue for compositional image editing. However, the requirement of object identity preservation poses a significant challenge, limiting practical usage of most existing methods. In response, this paper introduces IMPRINT, a novel diffusion-based generative model trained with a two-stage learning framework that decouples learning of identity preservation from that of compositing. The first stage is targeted for context-agnostic, identity-preserving pretraining of the object encoder, enabling the encoder to learn an embedding that is both view-invariant and conducive to enhanced detail preservation. The subsequent stage leverages this representation to learn seamless harmonization of the object composited to the background. In addition, IMPRINT incorporates a shape-guidance mechanism offering user-directed control over the compositing process. Extensive experiments demonstrate that IMPRINT significantly outperforms existing methods and various baselines on identity preservation and composition quality.

cs.CV↗

arXiv2024

Coherent competition and control between three-wave mixing and four-wave mixing in superconducting circuits

Exploring intermixing and interplay between different frequency-mixing processes has always been one of the interesting subjects at the interface of nonlinear optics with quantum optics. Here we investigate coherent competition and control between three-wave mixing (TWM) and four-wave mixing (FWM) in a cyclic three-level superconducting quantum system. In the weak control-field regime, strong competition leads to an alternating oscillation between TWM and FWM signals and this oscillation is a signature of strong energy exchange between these two nonlinear processes. In particular, such oscillation is absent from conventional multi-wave mixing in atomic systems. Surprisingly, synchronous TWM and FWM processes are demonstrated in the strong control-field regime and, at the same time, their efficiencies can be as high as 40% and 45%, respectively. Our study shows that these competitive behaviors between TWM and FWM can be manipulated by tuning the control-field intensity.

quant-ph↗

arXiv2024

Exploring global symmetry-breaking superradiant phase via phase competition

Superradiant phase transitions play a fundamental role in understanding the mechanism of collective light-matter interaction at the quantum level. Here we investigate multiple superradiant phases and phase transitions with different symmetry-breaking patterns in a two-mode V-type Dicke model. Interestingly, we show that there exists a quadruple point where one normal phase, one global symmetry-breaking superradiant phase and two local symmetry-breaking superradiant phases meet. Such a global phase results from the phase competition between two local superradiant phases and can not occur in the standard $Λ$- and $Ξ$-type three-level configurations in quantum optics. Moreover, we exhibit a sequential first-order quantum phase transition from one local to the global again to the other local superradiant phase. Our study opens up a perspective of exploring multi-level quantum critical phenomena with global symmetry breaking.

quant-ph↗

arXiv2024

Arithmetic Control of LLMs for Diverse User Preferences: Directional Preference Alignment with Multi-Objective Rewards

Fine-grained control over large language models (LLMs) remains a significant challenge, hindering their adaptability to diverse user needs. While Reinforcement Learning from Human Feedback (RLHF) shows promise in aligning LLMs, its reliance on scalar rewards often limits its ability to capture diverse user preferences in real-world applications. To address this limitation, we introduce the Directional Preference Alignment (DPA) framework. Unlike the scalar-reward RLHF, DPA incorporates multi-objective reward modeling to represent diverse preference profiles. Additionally, DPA models user preferences as directions (i.e., unit vectors) in the reward space to achieve user-dependent preference control. Our method involves training a multi-objective reward model and then fine-tuning the LLM with a preference-conditioned variant of Rejection Sampling Finetuning (RSF), an RLHF method adopted by Llama 2. This method enjoys a better performance trade-off across various reward objectives. In comparison with the scalar-reward RLHF, DPA offers users intuitive control over LLM generation: they can arithmetically specify their desired trade-offs (e.g., more helpfulness with less verbosity). We also validate the effectiveness of DPA with real-world alignment experiments on Mistral-7B. Our method provides straightforward arithmetic control over the trade-off between helpfulness and verbosity while maintaining competitive performance with strong baselines such as Direct Preference Optimization (DPO).

cs.LG↗

arXiv2024

Giant enhancement of higher-order harmonics of an optical-tweezer phonon laser

Phonon lasers, as mechanical analogues of optical lasers, are unique tools for not only fundamental studies of phononics but also diverse applications such as acoustic imaging and force sensing. Very recently, by levitating a micro-size sphere in an optical tweezer, higher-order mechanical harmonics were observed in the phonon-lasing regime, as the first step towards nonlinear levitated optomechanics [Nat. Phys. 19, 414 (2023)]. However, both the lasing strengths and the quality factors of the observed harmonics are typically very low, thus severely hindering their applications. Here we show that, by applying a simple but powerful electronic control to such a levitated micro-sphere, three orders of magnitude enhancement are achievable in the brightness of the phonon lasers, including both the fundamental mode and all its higher-order harmonics. Also, giant improvements of their linewidth and frequency stability are realized in such an electro-optomechanical system, together with further improved higher-order phonon coherence. These results, as a significant step forward for enhancing and controlling micro-object phonon lasers, can be readily used for a wide range of applications involving nonlinear phonon lasers, such as acoustic frequency comb, ultra-sound sensing, atmospherical monitoring, and even bio-medical diagnosis of levitated micro-size objects.

physics.optics↗

arXiv2024

Corruption-Robust Algorithms with Uncertainty Weighting for Nonlinear Contextual Bandits and Markov Decision Processes

Despite the significant interest and progress in reinforcement learning (RL) problems with adversarial corruption, current works are either confined to the linear setting or lead to an undesired $\tilde{O}(\sqrt{T}ζ)$ regret bound, where $T$ is the number of rounds and $ζ$ is the total amount of corruption. In this paper, we consider the contextual bandit with general function approximation and propose a computationally efficient algorithm to achieve a regret of $\tilde{O}(\sqrt{T}+ζ)$. The proposed algorithm relies on the recently developed uncertainty-weighted least-squares regression from linear contextual bandit and a new weighted estimator of uncertainty for the general function class. In contrast to the existing analysis that heavily relies on the linear structure, we develop a novel technique to control the sum of weighted uncertainty, thus establishing the final regret bounds. We then generalize our algorithm to the episodic MDP setting and first achieve an additive dependence on the corruption level $ζ$ in the scenario of general function approximation. Notably, our algorithms achieve regret bounds either nearly match the performance lower bound or improve the existing methods for all the corruption levels and in both known and unknown $ζ$ cases.

stat.ML↗

arXiv2024

Unitary and efficient spin squeezing in cavity optomechanics

We propose an approach to produce spin squeezed states of a large number of nitrogen-vacancy centers in diamond nanostructures coupled to an optical cavity. Unlike the previous squeezing method proposed by Bennett et al. [Phys. Rev. Lett. 110, 156402 (2013)], which is limited by phonon number fluctuations due to the existence of phonon-spin entanglement, our proposal can completely erase the entanglement between spins and hybrid phonon-photon mode mediating the effective spin-spin interaction, and thus achieves unitary one-axis-twisting interactions between nitrogen-vacancy centres, yielding a squeezing scaling $J^{-2/3}$, where J is the total angular momentum. We found that, under certain conditions, our method has the potential to enhance the spin-spin nonlinear interactions. We also proposed a scheme utilizing repeatedly applying the one-axis-twisting evolution to two orthogonal spin directions, which enables the transformation of the one-axis-twisting interactions into two-axis-twisting type, and therefore leads to the spin squeezing with Heisenberg-limited scaling $J^{-1}$. Taking into account the noise effects of spin dephasing and relaxtion, we found that the proposed approaches are robust against imperfections.

quant-ph↗

arXiv2024

The rotating solutions beyond the spontaneous scalarization in Einstein-Maxwell-scalar theory

The Einstein-Maxwell-scalar (EMS) theory with a quartic coupling function features three branches of fundamental black hole (BH) solutions, labeled as cold, hot, and bald black holes. The static bald black holes (the Reissner-Nordström BH) exhibit an intriguing nonlinear instability beyond the spontaneous scalarization. We study the rotating scalarized black hole solutions in the EMS model with a quartic coupling function through the spectral method numerically. The domain of existence for the scalarized BHs is presented in the spin-charge region. We found that the rotating solutions for both the two scalarized branches possess similar thermodynamic behavior compared to the static case while varying the electric charge. The BH spin enlarges the thermodynamic differences between the cold and hot branches. The profile of the metric function and the scalar field for the scalarized BHs is depicted, which demonstrates that the scalar field concentrates more on the equatorial plane in contrast to the axisymmetric region as the spin increases.

gr-qc↗

arXiv2024

Unraveling collisional energy loss of a heavy quark in quark-gluon plasma

At leading order in QCD coupling constant, we compute the energy loss per traveling distance of a heavy quark $dE/dz$ from elastic scattering off thermal quarks and gluons at a temperature $T$, including the thermal perturbative description of soft scatterings ($-t<-t^{\ast}$) and a perturbative QCD-based calculation for hard collisions ($-t>-t^{\ast}$). Within this soft-hard factorization model, we find that the full results of $dE/dz$ behaves a mild sensitivity to the intermediate cutoff $t^{\ast}$, supporting the validity of the soft-hard approach within the temperature region of interest. We re-derive the analytic formula for $dE/dz$ in the high-energy approximation, $E_{1}\gg m^{2}_{1}/T$, where $E_{1}$ is the injected heavy quark energy and $m_{1}$ is its mass. It is realized that the soft logarithmic contribution, $dE/dz\propto ln(-t^{\ast}/m^{2}_{D})$, arises from the $t$-channel scattering off thermal partons, while the hard logarithmic term, $dE/dz\propto ln[E_{1}T/(-t^{\ast})]$, stems from the $t$-channel scattering off thermal partons, and the one $dE/dz\propto ln(E_{1}T/m^{2}_{1})$ comes from the $s$- and $u$-channel scattering off gluons. The sum of these contributions cancels the $t^{\ast}$-dependence as observed in the full result. The mass hierarchy is observed $dE/dz(charm)>dE/dz(bottom)$. Our full results are crucial for a better description of heavy quark transport in QCD medium, in particular at low and moderate energy. We also calculate the energy loss by imposing the Einstein's relationship. The related results appear to be systematically larger than that without imposing the Einstein's relationship.

hep-ph↗

arXiv2023

Measuring the Sense of Presence and Learning Efficacy in Immersive Virtual Assembly Training

With the rapid progress in virtual reality (VR) technology, the scope of VR applications has greatly expanded across various domains. However, the superiority of VR training over traditional methods and its impact on learning efficacy are still uncertain. To investigate whether VR training is more effective than traditional methods, we designed virtual training systems for mechanical assembly on both VR and desktop platforms, subsequently conducting pre-test and post-test experiments. A cohort of 53 students, all enrolled in engineering drawing course without prior knowledge distinctions, was randomly divided into three groups: physical training, desktop virtual training, and immersive VR training. Our investigation utilized analysis of covariance (ANCOVA) to examine the differences in post-test scores among the three groups while controlling for pre-test scores. The group that received VR training showed the highest scores on the post-test. Another facet of our study delved into the presence of the virtual system. We developed a specialized scale to assess this aspect for our research objectives. Our findings indicate that VR training can enhance the sense of presence, particularly in terms of sensory factors and realism factors. Moreover, correlation analysis uncovers connections between the various dimensions of presence. This study confirms that using VR training can improve learning efficacy and the presence in the context of mechanical assembly, surpassing traditional training methods. Furthermore, it provides empirical evidence supporting the integration of VR technology in higher education and engineering training. This serves as a reference for the practical application of VR technology in different fields.

cs.HC↗

arXiv2023

RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignment

Generative foundation models are susceptible to implicit biases that can arise from extensive unsupervised training data. Such biases can produce suboptimal samples, skewed outcomes, and unfairness, with potentially serious consequences. Consequently, aligning these models with human ethics and preferences is an essential step toward ensuring their responsible and effective deployment in real-world applications. Prior research has primarily employed Reinforcement Learning from Human Feedback (RLHF) to address this problem, where generative models are fine-tuned with RL algorithms guided by a human-feedback-informed reward model. However, the inefficiencies and instabilities associated with RL algorithms frequently present substantial obstacles to the successful alignment, necessitating the development of a more robust and streamlined approach. To this end, we introduce a new framework, Reward rAnked FineTuning (RAFT), designed to align generative models effectively. Utilizing a reward model and a sufficient number of samples, our approach selects the high-quality samples, discarding those that exhibit undesired behavior, and subsequently enhancing the model by fine-tuning on these filtered samples. Our studies show that RAFT can effectively improve the model performance in both reward learning and other automated metrics in both large language models and diffusion models.

cs.LG↗

arXiv2023

Windformer:Bi-Directional Long-Distance Spatio-Temporal Network For Wind Speed Prediction

Wind speed prediction is critical to the management of wind power generation. Due to the large range of wind speed fluctuations and wake effect, there may also be strong correlations between long-distance wind turbines. This difficult-to-extract feature has become a bottleneck for improving accuracy. History and future time information includes the trend of airflow changes, whether this dynamic information can be utilized will also affect the prediction effect. In response to the above problems, this paper proposes Windformer. First, Windformer divides the wind turbine cluster into multiple non-overlapping windows and calculates correlations inside the windows, then shifts the windows partially to provide connectivity between windows, and finally fuses multi-channel features based on detailed and global information. To dynamically model the change process of wind speed, this paper extracts time series in both history and future directions simultaneously. Compared with other current-advanced methods, the Mean Square Error (MSE) of Windformer is reduced by 0.5\% to 15\% on two datasets from NERL.

eess.SP↗