arXiv subjects

Liu Yang

Publications and source records attributed to Liu Yang.

At least 109 records · Page 6Linked to original sources

arXiv2024

EasyTime: Time Series Forecasting Made Easy

Time series forecasting has important applications across diverse domains. EasyTime, the system we demonstrate, facilitates easy use of time-series forecasting methods by researchers and practitioners alike. First, EasyTime enables one-click evaluation, enabling researchers to evaluate new forecasting methods using the suite of diverse time series datasets collected in the preexisting time series forecasting benchmark (TFB). This is achieved by leveraging TFB's flexible and consistent evaluation pipeline. Second, when practitioners must perform forecasting on a new dataset, a nontrivial first step is often to find an appropriate forecasting method. EasyTime provides an Automated Ensemble module that combines the promising forecasting methods to yield superior forecasting accuracy compared to individual methods. Third, EasyTime offers a natural language Q&A module leveraging large language models. Given a question like "Which method is best for long term forecasting on time series with strong seasonality?", EasyTime converts the question into SQL queries on the database of results obtained by TFB and then returns an answer in natural language and charts. By demonstrating EasyTime, we intend to show how it is possible to simplify the use of time series forecasting and to offer better support for the development of new generations of time series forecasting methods.

cs.LG↗

arXiv2024

Research on high-frequency quasi-periodic oscillations in black bounce-type spacetime

This paper investigates the high frequency quasi-periodic oscillations (HFQPOs) phenomenon around the black bounce-type (BBT) spacetime using the resonance models. We calculated the location of the innermost stable circular orbit (ISCO) for different types of celestial bodies, and derived the expression for the epicyclic frequencies of test particles. The results show that the BBT spacetime possesses unique observational characteristics, where the ordering of epicyclic frequencies varies with the regularization parameter $a$, enabling the excitation of low-order resonances and producing stronger observational signals. Using parametric and forced resonance models, we compared theoretical results with the observed 3:2 twin-peak HFQPOs in microquasars (GRO 1655-40, XTE 1550-564, GRS $1915+105$ ), analyzed the formation mechanisms of HFQPOs, constrained the parameters of the BBT model, and explored the possible types of celestial objects corresponding to microquasars. The study indicates that, certain parametric resonance conditions (e.g., $n=1, 2$) lead to traversable wormhole models in BBT that closely align with observations. And forced resonance corresponding to BH or wormhole models can be verified through observations. These results deviate from the data fits of the original black-bounce model. It is found that the oscillatory behavior of three types of microquasars can also be explained by particle oscillations generated in BBT theory, providing evidence for exploring the existence of wormholes, under the assumptions of parametric resonance and forced resonance.

gr-qc↗

arXiv2024

Unifying Generative and Dense Retrieval for Sequential Recommendation

Sequential dense retrieval models utilize advanced sequence learning techniques to compute item and user representations, which are then used to rank relevant items for a user through inner product computation between the user and all item representations. However, this approach requires storing a unique representation for each item, resulting in significant memory requirements as the number of items grow. In contrast, the recently proposed generative retrieval paradigm offers a promising alternative by directly predicting item indices using a generative model trained on semantic IDs that encapsulate items' semantic information. Despite its potential for large-scale applications, a comprehensive comparison between generative retrieval and sequential dense retrieval under fair conditions is still lacking, leaving open questions regarding performance, and computation trade-offs. To address this, we compare these two approaches under controlled conditions on academic benchmarks and propose LIGER (LeveragIng dense retrieval for GEnerative Retrieval), a hybrid model that combines the strengths of these two widely used methods. LIGER integrates sequential dense retrieval into generative retrieval, mitigating performance differences and enhancing cold-start item recommendation in the datasets evaluated. This hybrid approach provides insights into the trade-offs between these approaches and demonstrates improvements in efficiency and effectiveness for recommendation systems in small-scale benchmarks.

cs.IR↗

arXiv2024

Xmodel-1.5: An 1B-scale Multilingual LLM

We introduce Xmodel-1.5, a 1-billion-parameter multilingual large language model pretrained on 2 trillion tokens, designed for balanced performance and scalability. Unlike most large models that use the BPE tokenizer, Xmodel-1.5 employs a custom unigram tokenizer with 65,280 tokens, optimizing both efficiency and accuracy. The model delivers competitive results across multiple languages, including Thai, Arabic, French, Chinese, and English, outperforming Alibaba's PolyLM-1.7B on respective evaluation datasets. Xmodel-1.5 excels in benchmarks like mMMLU and PIQA, and achieves state-of-the-art results in Thai. To support low-resource language research, we release Xdata_Thai, a Thai-specific evaluation dataset featuring unique linguistic challenges such as gendered particles and idioms. While the model demonstrates strong performance, there is still room for improvement in handling culturally specific nuances. We hope this work contributes to advancements in multilingual AI research. Models and code are publicly available on GitHub at https://github.com/XiaoduoAILab/XmodelLM-1.5

cs.CL↗

arXiv2024

Learning Dynamics from Multicellular Graphs with Deep Neural Networks

Multicellular self-assembly into functional structures is a dynamic process that is critical in the development and diseases, including embryo development, organ formation, tumor invasion, and others. Being able to infer collective cell migratory dynamics from their static configuration is valuable for both understanding and predicting these complex processes. However, the identification of structural features that can indicate multicellular motion has been difficult, and existing metrics largely rely on physical instincts. Here we show that using a graph neural network (GNN), the motion of multicellular collectives can be inferred from a static snapshot of cell positions, in both experimental and synthetic datasets.

physics.bio-ph↗

arXiv2024

Domain Generalization for Cross-Receiver Radio Frequency Fingerprint Identification

Radio Frequency Fingerprint Identification (RFFI) technology uniquely identifies emitters by analyzing unique distortions in the transmitted signal caused by non-ideal hardware. Recently, RFFI based on deep learning methods has gained popularity and is seen as a promising way to address the device authentication problem for Internet of Things (IoT) systems. However, in cross-receiver scenarios, where the RFFI model is trained over RF signals from some receivers but deployed at a new receiver, the alteration of receivers' characteristics would lead to data distribution shift and cause significant performance degradation at the new receiver. To address this problem, we first perform a theoretical analysis of the cross-receiver generalization error bound and propose a sufficient condition, named Separable Condition (SC), to minimize the classification error probability on the new receiver. Guided by the SC, a Receiver-Independent Emitter Identification (RIEI)model is devised to decouple the received signals into emitter-related features and receiver-related features and only the emitter-related features are used for identification. Furthermore, by leveraging federated learning, we also develop a FedRIEI model to eliminate the need for centralized collection of raw data from multiple receivers. Experiments on two real-world datasets demonstrate the superiority of our proposed methods over some baseline methods.

eess.SP↗

arXiv2024

MultiBalance: Multi-Objective Gradient Balancing in Industrial-Scale Multi-Task Recommendation System

In industrial recommendation systems, multi-task learning (learning multiple tasks simultaneously on a single model) is a predominant approach to save training/serving resources and improve recommendation performance via knowledge transfer between the joint learning tasks. However, multi-task learning often suffers from negative transfer: one or several tasks are less optimized than training them separately. To carefully balance the optimization, we propose a gradient balancing approach called MultiBalance, which is suitable for industrial-scale multi-task recommendation systems. It balances the per-task gradients to alleviate the negative transfer, while saving the huge cost for grid search or manual explorations for appropriate task weights. Moreover, compared with prior work that normally balance the per-task gradients of shared parameters, MultiBalance is more efficient since only requiring to access per-task gradients with respect to the shared feature representations. We conduct experiments on Meta's large-scale ads and feeds multi-task recommendation system, and observe that MultiBalance achieves significant gains (e.g., 0.738% improvement for normalized entropy (NE)) with neutral training cost in Queries Per Second (QPS), which is significantly more efficient than prior methods that balance per-task gradients of shared parameters with 70~80% QPS degradation.

cs.IR↗

arXiv2024

Everything Everywhere All at Once: LLMs can In-Context Learn Multiple Tasks in Superposition

Large Language Models (LLMs) have demonstrated remarkable in-context learning (ICL) capabilities. In this study, we explore a surprising phenomenon related to ICL: LLMs can perform multiple, computationally distinct ICL tasks simultaneously, during a single inference call, a capability we term "task superposition". We provide empirical evidence of this phenomenon across various LLM families and scales and show that this phenomenon emerges even if we train the model to in-context learn one task at a time. We offer theoretical explanations that this capability is well within the expressive power of transformers. We also explore how LLMs internally compose task vectors during superposition. Furthermore, we show that larger models can solve more ICL tasks in parallel, and better calibrate their output distribution. Our findings offer insights into the latent capabilities of LLMs, further substantiate the perspective of "LLMs as superposition of simulators", and raise questions about the mechanisms enabling simultaneous task execution.

cs.LG↗

arXiv2024

Two-dimensional Topological Ferroelectric Metal with Giant Shift Current

The pursuit for "ferroelectric metal" which combines seemingly incompatible spontaneous electric polarization and metallicity, has been assiduously ongoing but remains elusive. Unlike traditional ferroelectrics with a wide band gap, ferroelectric (FE) metals can naturally incorporate nontrivial band topology near the Fermi level, endowing them with additional exotic properties. Here, we show first-principles evidence that the metallic PtBi2 monolayer is an intrinsic two-dimensional (2D) topological FE metal, characterized by out-of-plane polarization and a moderate switching barrier. Moreover, it exhibits a topologically nontrivial electronic structure with Z2 invariant equal to 1, leading to a significant FE bulk photovoltaic effect. A slight strain can further enhance this effect to a remarkable level, which far surpass that of previously reported 2D/3D FE materials. Our work provides an important step towards realizing intrinsic monolayer topological FE metals and paves a promising way for future nonlinear optical devices.

cond-mat.mtrl-sci↗

arXiv2024

The Non-reciprocity of Multi-mode Optical Directional Amplifier Realized by Non-Hermitian Resonator Arrays

In the present paper, a multi-frequency optical non-reciprocal transmission is first realized by using a non-Hermitian multi-mode resonator array.We find that the non-reciprocity can be used to route optical signals, to prevent the reverse flow of noise, and find that the multi-frequency can be used to enhance information processing. In terms of the Scully-Lamb model and gain saturation effect, we accomplish a dual-frequency non-reciprocal transmission by introducing nonlinearity into a linear array of four-mode resonators. For example, a directional cyclic amplifier is constructed with non-reciprocal units. As potential applications, the non-reciprocity optical systems can be employed in dual-frequency control, parallel information processing, photonic integrated circuits, optical devices and so on.

physics.optics↗

arXiv2024

Robust approximation of chance constrained optimization with polynomial perturbation

This paper proposes a robust approximation method for solving chance constrained optimization (CCO) of polynomials. Assume the CCO is defined with an individual chance constraint that is affine in the decision variables. We construct a robust approximation by replacing the chance constraint with a robust constraint over an uncertainty set. When the objective function is linear or SOS-convex, the robust approximation can be equivalently transformed into linear conic optimization. Semidefinite relaxation algorithms are proposed to solve these linear conic transformations globally and their convergent properties are studied. We also introduce a heuristic method to find efficient uncertainty sets such that optimizers of the robust approximation are feasible to the original problem. Numerical experiments are given to show the efficiency of our method.

math.OC↗

arXiv2024

Multiple sliding ferroelectricity of rhombohedral-stacked InSe for reconfigurable photovoltaics and imaging applications

Through stacking engineering of two-dimensional (2D) materials, a switchable interface polarization can be generated through interlayer sliding, so called sliding ferroelectricity, which is advantageous over the traditional ferroelectricity due to ultra-thin thickness, high switching speed and low fatigue. However, 2D materials with intrinsic sliding ferroelectricity are still rare, with the exception of rhombohedral-stacked MoS2, which limits sliding ferroelectricity for practical applications such as high-speed storage, photovoltaic, and neuromorphic computing. Here, we reported the observation of sliding ferroelectricity with multiple states in undoped rhombohedral-stacked InSe (γ-InSe) via dual-frequency resonance tracking piezoresponse force microscopy, scanning Kelvin probe microscopy and conductive atomic force microscopy. The tunable bulk photovoltaic effect via the electric field is achieved in the graphene/γ-InSe/graphene tunneling device with a photovoltaic current density of ~15 mA/cm2, which is attributed to the multiple sliding steps in γ-InSe according to our theoretical calculations. The vdw tunneling device also features a high photo responsivity of ~255 A/W and a fast response time for real-time imaging. Our work not only enriches rhombohedral-stacked 2D materials for sliding ferroelectricity, but also sheds light on their potential for tunable photovoltaics and imaging applications.

cond-mat.mtrl-sci↗

arXiv2024

Skywork-Math: Data Scaling Laws for Mathematical Reasoning in Large Language Models -- The Story Goes On

In this paper, we investigate the underlying factors that potentially enhance the mathematical reasoning capabilities of large language models (LLMs). We argue that the data scaling law for math reasoning capabilities in modern LLMs is far from being saturated, highlighting how the model's quality improves with increases in data quantity. To support this claim, we introduce the Skywork-Math model series, supervised fine-tuned (SFT) on common 7B LLMs using our proposed 2.5M-instance Skywork-MathQA dataset. Skywork-Math 7B has achieved impressive accuracies of 51.2% on the competition-level MATH benchmark and 83.9% on the GSM8K benchmark using only SFT data, outperforming an early version of GPT-4 on MATH. The superior performance of Skywork-Math models contributes to our novel two-stage data synthesis and model SFT pipelines, which include three different augmentation methods and a diverse seed problem set, ensuring both the quantity and quality of Skywork-MathQA dataset across varying difficulty levels. Most importantly, we provide several practical takeaways to enhance math reasoning abilities in LLMs for both research and industry applications.

cs.AI↗

arXiv2024

LongSkywork: A Training Recipe for Efficiently Extending Context Length in Large Language Models

We introduce LongSkywork, a long-context Large Language Model (LLM) capable of processing up to 200,000 tokens. We provide a training recipe for efficiently extending context length of LLMs. We identify that the critical element in enhancing long-context processing capability is to incorporate a long-context SFT stage following the standard SFT stage. A mere 200 iterations can convert the standard SFT model into a long-context model. To reduce the effort in collecting and annotating data for long-context language modeling, we develop two novel methods for creating synthetic data. These methods are applied during the continual pretraining phase as well as the Supervised Fine-Tuning (SFT) phase, greatly enhancing the training efficiency of our long-context LLMs. Our findings suggest that synthetic long-context SFT data can surpass the performance of data curated by humans to some extent. LongSkywork achieves outstanding performance on a variety of long-context benchmarks. In the Needle test, a benchmark for long-context information retrieval, our models achieved perfect accuracy across multiple context spans. Moreover, in realistic application scenarios, LongSkywork-13B demonstrates performance on par with Claude2.1, the leading long-context model, underscoring the effectiveness of our proposed methods.

cs.CL↗

arXiv2024

Nonvolatile optical control of interlayer stacking order in 1T-TaS2

Nonvolatile optical manipulation of material properties on demand is a highly sought-after feature in the advancement of future optoelectronic applications. While the discovery of such metastable transition in various materials holds good promise for achieving this goal, their practical implementation is still in the nascent stage. Here, we unravel the nature of the ultrafast laser-induced hidden state in 1T-TaS2 by systematically characterizing the electronic structure evolution throughout the reversible transition cycle. We identify it as a mixed-stacking state involving two similarly low-energy interlayer orders, which is manifested as the charge density wave phase disruption. Furthermore, our comparative experiments utilizing the single-pulse writing, pulse-train erasing and pulse-pair control explicitly reveal the distinct mechanism of the bidirectional transformations -- the ultrafast formation of the hidden state is initiated by a coherent phonon which triggers a competition of interlayer stacking orders, while its recovery to the initial state is governed by the progressive domain coarsening. Our work highlights the deterministic role of the competing interlayer orders in the nonvolatile phase transition in the layered material 1T-TaS2, and promises the coherent control of the phase transition and switching speed. More importantly, these results establish all-optical engineering of stacking orders in low-dimensional materials as a viable strategy for achieving desirable nonvolatile electronic devices.

cond-mat.mtrl-sci↗

arXiv2024

PackVFL: Efficient HE Packing for Vertical Federated Learning

As an essential tool of secure distributed machine learning, vertical federated learning (VFL) based on homomorphic encryption (HE) suffers from severe efficiency problems due to data inflation and time-consuming operations. To this core, we propose PackVFL, an efficient VFL framework based on packed HE (PackedHE), to accelerate the existing HE-based VFL algorithms. PackVFL packs multiple cleartexts into one ciphertext and supports single-instruction-multiple-data (SIMD)-style parallelism. We focus on designing a high-performant matrix multiplication (MatMult) method since it takes up most of the ciphertext computation time in HE-based VFL. Besides, devising the MatMult method is also challenging for PackedHE because a slight difference in the packing way could predominantly affect its computation and communication costs. Without domain-specific design, directly applying SOTA MatMult methods is hard to achieve optimal. Therefore, we make a three-fold design: 1) we systematically explore the current design space of MatMult and quantify the complexity of existing approaches to provide guidance; 2) we propose a hybrid MatMult method according to the unique characteristics of VFL; 3) we adaptively apply our hybrid method in representative VFL algorithms, leveraging distinctive algorithmic properties to further improve efficiency. As the batch size, feature dimension and model size of VFL scale up to large sizes, PackVFL consistently delivers enhanced performance. Empirically, PackVFL propels existing VFL algorithms to new heights, achieving up to a 51.52X end-to-end speedup. This represents a substantial 34.51X greater speedup compared to the direct application of SOTA MatMult methods.

cs.CR↗

arXiv2024

Mitigating Receiver Impact on Radio Frequency Fingerprint Identification via Domain Adaptation

Radio Frequency Fingerprint Identification (RFFI), which exploits non-ideal hardware-induced unique distortion resident in the transmit signals to identify an emitter, is emerging as a means to enhance the security of communication systems. Recently, machine learning has achieved great success in developing state-of-the-art RFFI models. However, few works consider cross-receiver RFFI problems, where the RFFI model is trained and deployed on different receivers. Due to altered receiver characteristics, direct deployment of RFFI model on a new receiver leads to significant performance degradation. To address this issue, we formulate the cross-receiver RFFI as a model adaptation problem, which adapts the trained model to unlabeled signals from a new receiver. We first develop a theoretical generalization error bound for the adaptation model. Motivated by the bound, we propose a novel method to solve the cross-receiver RFFI problem, which includes domain alignment and adaptive pseudo-labeling. The former aims at finding a feature space where both domains exhibit similar distributions, effectively reducing the domain discrepancy. Meanwhile, the latter employs a dynamic pseudo-labeling scheme to implicitly transfer the label information from the labeled receiver to the new receiver. Experimental results indicate that the proposed method can effectively mitigate the receiver impact and improve the cross-receiver RFFI performance.

eess.SP↗

arXiv2024

AIGCOIQA2024: Perceptual Quality Assessment of AI Generated Omnidirectional Images

In recent years, the rapid advancement of Artificial Intelligence Generated Content (AIGC) has attracted widespread attention. Among the AIGC, AI generated omnidirectional images hold significant potential for Virtual Reality (VR) and Augmented Reality (AR) applications, hence omnidirectional AIGC techniques have also been widely studied. AI-generated omnidirectional images exhibit unique distortions compared to natural omnidirectional images, however, there is no dedicated Image Quality Assessment (IQA) criteria for assessing them. This study addresses this gap by establishing a large-scale AI generated omnidirectional image IQA database named AIGCOIQA2024 and constructing a comprehensive benchmark. We first generate 300 omnidirectional images based on 5 AIGC models utilizing 25 text prompts. A subjective IQA experiment is conducted subsequently to assess human visual preferences from three perspectives including quality, comfortability, and correspondence. Finally, we conduct a benchmark experiment to evaluate the performance of state-of-the-art IQA models on our database. The database will be released to facilitate future research.

cs.CV↗