Searcharxiv⌕ Search

arXiv subjects

Bo Chen

Publications and source records attributed to Bo Chen.

At least 253 records · Page 14Linked to original sources
arXiv2023

YAYI 2: Multilingual Open-Source Large Language Models

As the latest advancements in natural language processing, large language models (LLMs) have achieved human-level language understanding and generation abilities in many real-world tasks, and even have been regarded as a potential path to the artificial general intelligence. To better facilitate research on LLMs, many open-source LLMs, such as Llama 2 and Falcon, have recently been proposed and gained comparable performances to proprietary models. However, these models are primarily designed for English scenarios and exhibit poor performances in Chinese contexts. In this technical report, we propose YAYI 2, including both base and chat models, with 30 billion parameters. YAYI 2 is pre-trained from scratch on a multilingual corpus which contains 2.65 trillion tokens filtered by our pre-training data processing pipeline. The base model is aligned with human values through supervised fine-tuning with millions of instructions and reinforcement learning from human feedback. Extensive experiments on multiple benchmarks, such as MMLU and CMMLU, consistently demonstrate that the proposed YAYI 2 outperforms other similar sized open-source models.

cs.CL↗
arXiv2023

CTRL: Connect Collaborative and Language Model for CTR Prediction

Traditional click-through rate (CTR) prediction models convert the tabular data into one-hot vectors and leverage the collaborative relations among features for inferring the user's preference over items. This modeling paradigm discards essential semantic information. Though some works like P5 and CTR-BERT have explored the potential of using Pre-trained Language Models (PLMs) to extract semantic signals for CTR prediction, they are computationally expensive and suffer from low efficiency. Besides, the beneficial collaborative relations are not considered, hindering the recommendation performance. To solve these problems, in this paper, we propose a novel framework \textbf{CTRL}, which is industrial-friendly and model-agnostic with superior inference efficiency. Specifically, the original tabular data is first converted into textual data. Both tabular data and converted textual data are regarded as two different modalities and are separately fed into the collaborative CTR model and pre-trained language model. A cross-modal knowledge alignment procedure is performed to fine-grained align and integrate the collaborative and semantic signals, and the lightweight collaborative model can be deployed online for efficient serving after fine-tuned with supervised signals. Experimental results on three public datasets show that CTRL outperforms the state-of-the-art (SOTA) CTR models significantly. Moreover, we further verify its effectiveness on a large-scale industrial recommender system.

cs.IR↗
arXiv2023

Towards Open-World Recommendation with Knowledge Augmentation from Large Language Models

Recommender systems play a vital role in various online services. However, the insulated nature of training and deploying separately within a specific domain limits their access to open-world knowledge. Recently, the emergence of large language models (LLMs) has shown promise in bridging this gap by encoding extensive world knowledge and demonstrating reasoning capability. Nevertheless, previous attempts to directly use LLMs as recommenders have not achieved satisfactory results. In this work, we propose an Open-World Knowledge Augmented Recommendation Framework with Large Language Models, dubbed KAR, to acquire two types of external knowledge from LLMs -- the reasoning knowledge on user preferences and the factual knowledge on items. We introduce factorization prompting to elicit accurate reasoning on user preferences. The generated reasoning and factual knowledge are effectively transformed and condensed into augmented vectors by a hybrid-expert adaptor in order to be compatible with the recommendation task. The obtained vectors can then be directly used to enhance the performance of any recommendation model. We also ensure efficient inference by preprocessing and prestoring the knowledge from the LLM. Extensive experiments show that KAR significantly outperforms the state-of-the-art baselines and is compatible with a wide range of recommendation algorithms. We deploy KAR to Huawei's news and music recommendation platforms and gain a 7\% and 1.7\% improvement in the online A/B test, respectively.

cs.IR↗
arXiv2023

Full-scale ab initio simulations of laser-driven atomistic dynamics

The coupling of excited states and ionic dynamics is the basic and challenging point for the materials response at extreme conditions. In laboratory, the intense laser produces transient nature and complexity with highly nonequilibrium states, making it extremely difficult and interesting for both experimental measurements and theoretical methods. With the inclusion of laser-excited states, we extended ab initio method into the direct simulations of whole laser-driven microscopic dynamics from solid to liquid. We constructed the framework of combining the electron-temperaturedependent deep neural network potential energy surface with hybrid atomistic-continuum approach, controlling non-adiabatic energy exchange and atomistic dynamics, which enables consistent interpretation of experimental data. By large scale ab inito simulations, we demonstrate that the nonthermal effects introduced by hot electrons play a dominant role in modulating the lattice dynamics, thermodynamic pathway, and structural transformation. We highlight that the present work provides a path to realistic computational studies of laser-driven processes, thus bridging the gap between experiments and simulations.

physics.comp-ph↗
arXiv2023

Tuning Multi-mode Token-level Prompt Alignment across Modalities

Advancements in prompt tuning of vision-language models have underscored their potential in enhancing open-world visual concept comprehension. However, prior works only primarily focus on single-mode (only one prompt for each modality) and holistic level (image or sentence) semantic alignment, which fails to capture the sample diversity, leading to sub-optimal prompt discovery. To address the limitation, we propose a multi-mode token-level tuning framework that leverages the optimal transportation to learn and align a set of prompt tokens across modalities. Specifically, we rely on two essential factors: 1) multi-mode prompts discovery, which guarantees diverse semantic representations, and 2) token-level alignment, which helps explore fine-grained similarity. Consequently, the similarity can be calculated as a hierarchical transportation problem between the modality-specific sets. Extensive experiments on popular image recognition benchmarks show the superior generalization and few-shot abilities of our approach. The qualitative analysis demonstrates that the learned prompt tokens have the ability to capture diverse visual concepts.

cs.CV↗
arXiv2023

Hierarchical Vector Quantized Transformer for Multi-class Unsupervised Anomaly Detection

Unsupervised image Anomaly Detection (UAD) aims to learn robust and discriminative representations of normal samples. While separate solutions per class endow expensive computation and limited generalizability, this paper focuses on building a unified framework for multiple classes. Under such a challenging setting, popular reconstruction-based networks with continuous latent representation assumption always suffer from the "identical shortcut" issue, where both normal and abnormal samples can be well recovered and difficult to distinguish. To address this pivotal issue, we propose a hierarchical vector quantized prototype-oriented Transformer under a probabilistic framework. First, instead of learning the continuous representations, we preserve the typical normal patterns as discrete iconic prototypes, and confirm the importance of Vector Quantization in preventing the model from falling into the shortcut. The vector quantized iconic prototype is integrated into the Transformer for reconstruction, such that the abnormal data point is flipped to a normal data point.Second, we investigate an exquisite hierarchical framework to relieve the codebook collapse issue and replenish frail normal patterns. Third, a prototype-oriented optimal transport method is proposed to better regulate the prototypes and hierarchically evaluate the abnormal score. By evaluating on MVTec-AD and VisA datasets, our model surpasses the state-of-the-art alternatives and possesses good interpretability. The code is available at https://github.com/RuiyingLu/HVQ-Trans.

cs.CV↗
arXiv2023

Experimental observation of the significant difference between surface and bulk Kondo processes in Kondo lattice YbCu$_2$Si$_2$

Synchrotron-based angle-resolved photoemission spectroscopy was employed to investigate the temperature evolution of the Yb 4f spectral for surface and bulk in the Kondo lattice YbCu$_2$Si$_2$.Our study quantitatively distinguishes between the surface and bulk hybridization processes, revealing that the onset temperatures for both surface and bulk hybridization processes are significantly higher than the Kondo temperature. Additionally, we found that the effective surface Kondo temperature is much lower than that of the bulk. These findings offer valuable insights into the understanding of heavy fermion physics.

cond-mat.str-el↗
arXiv2023

Differentially Private Computation of Basic Reproduction Numbers in Networked Epidemic Models

The basic reproduction number of a networked epidemic model, denoted $R_0$, can be computed from a network's topology to quantify epidemic spread. However, disclosure of $R_0$ risks revealing sensitive information about the underlying network, such as an individual's relationships within a social network. Therefore, we propose a framework to compute and release $R_0$ in a differentially private way. First, we provide a new result that shows how $R_0$ can be used to bound the level of penetration of an epidemic within a single community as a motivation for the need of privacy, which may also be of independent interest. We next develop a privacy mechanism to formally safeguard the edge weights in the underlying network when computing $R_0$. Then we formalize tradeoffs between the level of privacy and the accuracy of values of the privatized $R_0$. To show the utility of the private $R_0$ in practice, we use it to bound this level of penetration under privacy, and concentration bounds on these analyses show they remain accurate with privacy implemented. We apply our results to real travel data gathered during the spread of COVID-19, and we show that, under real-world conditions, we can compute $R_0$ in a differentially private way while incurring errors as low as $7.6\%$ on average.

cs.SI↗
arXiv2023

The triggering process of an X-class solar flare on a small quadrupolar active region

The occurrence of X-class solar flares and their potential impact on the space weather often receive great attention than other flares. But predicting when and where an X-class flare will occur is still a challenge. With the multi-wavelength observation from the Solar Dynamics Observatory and FengYun- 3E satellite, we investigate the triggering of a GOES X1.0 flare occurring in the NOAA active region (AR) 12887. Our results show that this unique X-class flare is bred in a relatively small but complex quadrupolar AR. Before the X-class flare, two filaments (F1 and F2) exist below a null-point topology of the quadrupolar AR. Magnetic field extrapolation and observation reveal that F1 and F2 correspond to two magnetic flux ropes with the same chirality and their adjacent feet rooted at nonconjugated opposite polarities, respectively. Interestingly, these two polarities collide rapidly, accompanied by photospheric magnetic flux emergence, cancellation and shear motion in the AR center. Above this site, F1 and F2 subsequently intersect and merge to a longer filament (F3) via a tether-cutting-like reconnection process. As a result, the F3 rises and erupts, involving the large-scale arcades overlying filament and the quadrupolar magnetic field above the AR, and eventually leads to the eruption of the X-class flare with a quasi-X-shaped flare ribbon and a coronal mass ejection. It suggests that the rapid collision of nonconjugated opposite polarities provides a key condition for the triggering of this X-class flare, and also provides a featured case for flare trigger mechanism and space weather forecasting.

astro-ph.SR↗
arXiv2023

Scenario-Aware Hierarchical Dynamic Network for Multi-Scenario Recommendation

Click-Through Rate (CTR) prediction is a fundamental technique in recommendation and advertising systems. Recent studies have shown that implementing multi-scenario recommendations contributes to strengthening information sharing and improving overall performance. However, existing multi-scenario models only consider coarse-grained explicit scenario modeling that depends on pre-defined scenario identification from manual prior rules, which is biased and sub-optimal. To address these limitations, we propose a Scenario-Aware Hierarchical Dynamic Network for Multi-Scenario Recommendations (HierRec), which perceives implicit patterns adaptively and conducts explicit and implicit scenario modeling jointly. In particular, HierRec designs a basic scenario-oriented module based on the dynamic weight to capture scenario-specific information. Then the hierarchical explicit and implicit scenario-aware modules are proposed to model hybrid-grained scenario information. The multi-head implicit modeling design contributes to perceiving distinctive patterns from different perspectives. Our experiments on two public datasets and real-world industrial applications on a mainstream online advertising platform demonstrate that our HierRec outperforms existing models significantly.

cs.IR↗
arXiv2023

PatchCT: Aligning Patch Set and Label Set with Conditional Transport for Multi-Label Image Classification

Multi-label image classification is a prediction task that aims to identify more than one label from a given image. This paper considers the semantic consistency of the latent space between the visual patch and linguistic label domains and introduces the conditional transport (CT) theory to bridge the acknowledged gap. While recent cross-modal attention-based studies have attempted to align such two representations and achieved impressive performance, they required carefully-designed alignment modules and extra complex operations in the attention computation. We find that by formulating the multi-label classification as a CT problem, we can exploit the interactions between the image and label efficiently by minimizing the bidirectional CT cost. Specifically, after feeding the images and textual labels into the modality-specific encoders, we view each image as a mixture of patch embeddings and a mixture of label embeddings, which capture the local region features and the class prototypes, respectively. CT is then employed to learn and align those two semantic sets by defining the forward and backward navigators. Importantly, the defined navigators in CT distance model the similarities between patches and labels, which provides an interpretable tool to visualize the learned prototypes. Extensive experiments on three public image benchmarks show that the proposed model consistently outperforms the previous methods.

cs.CV↗
arXiv2023

A hybrid Decoder-DeepONet operator regression framework for unaligned observation data

Deep neural operators (DNOs) have been utilized to approximate nonlinear mappings between function spaces. However, DNOs face the challenge of increased dimensionality and computational cost associated with unaligned observation data. In this study, we propose a hybrid Decoder-DeepONet operator regression framework to handle unaligned data effectively. Additionally, we introduce a Multi-Decoder-DeepONet, which utilizes an average field of training data as input augmentation. The consistencies of the frameworks with the operator approximation theory are provided, on the basis of the universal approximation theorem. Two numerical experiments, Darcy problem and flow-field around an airfoil, are conducted to validate the efficiency and accuracy of the proposed methods. Results illustrate the advantages of Decoder-DeepONet and Multi-Decoder-DeepONet in handling unaligned observation data and showcase their potentials in improving prediction accuracy.

cs.LG↗
arXiv2023

AutoAssign+: Automatic Shared Embedding Assignment in Streaming Recommendation

In the domain of streaming recommender systems, conventional methods for addressing new user IDs or item IDs typically involve assigning initial ID embeddings randomly. However, this practice results in two practical challenges: (i) Items or users with limited interactive data may yield suboptimal prediction performance. (ii) Embedding new IDs or low-frequency IDs necessitates consistently expanding the embedding table, leading to unnecessary memory consumption. In light of these concerns, we introduce a reinforcement learning-driven framework, namely AutoAssign+, that facilitates Automatic Shared Embedding Assignment Plus. To be specific, AutoAssign+ utilizes an Identity Agent as an actor network, which plays a dual role: (i) Representing low-frequency IDs field-wise with a small set of shared embeddings to enhance the embedding initialization, and (ii) Dynamically determining which ID features should be retained or eliminated in the embedding table. The policy of the agent is optimized with the guidance of a critic network. To evaluate the effectiveness of our approach, we perform extensive experiments on three commonly used benchmark datasets. Our experiment results demonstrate that AutoAssign+ is capable of significantly enhancing recommendation performance by mitigating the cold-start problem. Furthermore, our framework yields a reduction in memory usage of approximately 20-30%, verifying its practical effectiveness and efficiency for streaming recommender systems.

cs.IR↗
arXiv2023

A novel sampling method for time domain acoustic inverse source problems

This paper is concerned with the inverse acoustic scattering problems of reconstructing time-dependent multiple point sources and sources on a curve $L$ of the form $λ(t)τ(x)δ_L(x)$. A direct sampling method with a novel indicator function is proposed to reconstruct the sources. The sampling method is easy to implement and low in calculation cost. Based on the sampling method, numerical algorithms are provided to get the positions and intensities of the sources. Both theoretical analysis of the indicator function and numerical experiments are provided to show the effectiveness of the proposed method.

math.NA↗
arXiv2023

A Homotopy Invariant Based on Convex Dissection Topology and a Distance Optimal Path Planning Algorithm

The concept of path homotopy has received widely attention in the field of path planning in recent years. In this article, a homotopy invariant based on convex dissection for a two-dimensional bounded Euclidean space is developed, which can efficiently encode all homotopy path classes between any two points. Thereafter, the optimal path planning task consists of two steps: (i) search for the homotopy path class that may contain the optimal path, and (ii) obtain the shortest homotopy path in this class. Furthermore, an optimal path planning algorithm called CDT-RRT* (Rapidly-exploring Random Tree Star based on Convex Division Topology) is proposed. We designed an efficient sampling formula for CDT-RRT*, which gives it a tendency to actively explore unknown homotopy classes, and incorporated the principles of the Elastic Band algorithm to obtain the shortest path in each class. Through a series of experiments, it was determined that the performance of the proposed algorithm is comparable with state-of-the-art path planning algorithms. Hence, the application significance of the developed homotopy invariant in the field of path planning was verified.

cs.RO↗
arXiv2023

CDT-Dijkstra: Fast Planning of Globally Optimal Paths for All Points in 2D Continuous Space

The Dijkstra algorithm is a classic path planning method, which in a discrete graph space, can start from a specified source node and find the shortest path between the source node and all other nodes in the graph. However, to the best of our knowledge, there is no effective method that achieves a function similar to that of the Dijkstra's algorithm in a continuous space. In this study, an optimal path planning algorithm called convex dissection topology (CDT)-Dijkstra is developed, which can quickly compute the global optimal path from one point to all other points in a 2D continuous space. CDT-Dijkstra is mainly divided into two stages: SetInit and GetGoal. In SetInit, the algorithm can quickly obtain the optimal CDT encoding set of all the cut lines based on the initial point x_{init}. In GetGoal, the algorithm can return the global optimal path of any goal point at an extremely high speed. In this study, we propose and prove the planning principle of considering only the points on the cutlines, thus reducing the state space of the distance optimal path planning task from 2D to 1D. In addition, we propose a fast method to find the optimal path in a homogeneous class and theoretically prove the correctness of the method. Finally, by testing in a series of environments, the experimental results demonstrate that CDT-Dijkstra not only plans the optimal path from all points at once, but also has a significant advantage over advanced algorithms considering certain complex tasks.

cs.RO↗
arXiv2023

Prototypes-oriented Transductive Few-shot Learning with Conditional Transport

Transductive Few-Shot Learning (TFSL) has recently attracted increasing attention since it typically outperforms its inductive peer by leveraging statistics of query samples. However, previous TFSL methods usually encode uniform prior that all the classes within query samples are equally likely, which is biased in imbalanced TFSL and causes severe performance degradation. Given this pivotal issue, in this work, we propose a novel Conditional Transport (CT) based imbalanced TFSL model called {\textbf P}rototypes-oriented {\textbf U}nbiased {\textbf T}ransfer {\textbf M}odel (PUTM) to fully exploit unbiased statistics of imbalanced query samples, which employs forward and backward navigators as transport matrices to balance the prior of query samples per class between uniform and adaptive data-driven distributions. For efficiently transferring statistics learned by CT, we further derive a closed form solution to refine prototypes based on MAP given the learned navigators. The above two steps of discovering and transferring unbiased statistics follow an iterative manner, formulating our EM-based solver. Experimental results on four standard benchmarks including miniImageNet, tieredImageNet, CUB, and CIFAR-FS demonstrate superiority of our model in class-imbalanced generalization.

cs.CV↗
arXiv2023

Coupling of optical phonons with Kondo effect and magnetic orders in antiferromagnetic Kondo lattice CeAuSb$_2$

Ultrafast optical spectroscopy was used to investigate the ultrafast quasiparticle dynamics of antiferromagnetic Kondo lattice CeAuSb$_2$ as a function of temperature and fluence. Our results reveal (i) the opening of a narrow hybridization gap ($Δ$ $\sim$ 4.5 meV) near the Fermi level below the coherence temperature $T^*$ $\approx$ 100 K, (ii) the coupling of coherent phonon modes with Kondo effect and magnetic orders, leading to the frequencies anomaly at the characteristic temperatures, and (iii) a possible photoinduced nonthermal phase transition. Our observations thus shed light on the hybridization dynamics and magnetic orders in heavy fermion systems.

cond-mat.str-el↗