Searcharxiv⌕ Search

arXiv subjects

Zheng Liu

Publications and source records attributed to Zheng Liu.

At least 253 records · Page 14Linked to original sources
arXiv2024

Enhancing Weak magnetic field sensing of cavity-magnon system with dual frequency modulation

The crucial limitation of improving the sensitivity of the detection of weak magnetic fields is the unavoidable measurement noise. In this paper, we propose a scheme to achieve precise sensing robust against additional noise by employing a dual-frequency bias field modulation within a cavity magnon system. We find that the anti-rotating wave term can amplify the signal of the detected magnetic field, but this amplification effect must coexist with the rotating wave term. In particular, by the bias field modulation, we find the robustness against cavity field thermal noise is substantially enhanced, quantum noise and cavity field thermal noise is greatly reduced, and the external magnetic field signal is amplified, thereby improving the weak magnetic field sensing system's sensitivity. Compared with the previous scheme, our scheme requires neither an ultra- or deep-strong coupling mechanism nor the suppression of the additional noise by increasing the electromagnetic cooperativity. Our scheme could provide a valuable candidate for weak magnetic field sensing.

quant-ph↗
arXiv2024

Incremental Learning of Stock Trends via Meta-Learning with Dynamic Adaptation

Forecasting the trend of stock prices is an enduring topic at the intersection of finance and computer science. Periodical updates to forecasters have proven effective in handling concept drifts arising from non-stationary markets. However, the existing methods neglect either emerging patterns in recent data or recurring patterns in historical data, both of which are empirically advantageous for future forecasting. To address this issue, we propose meta-learning with dynamic adaptation (MetaDA) for the incremental learning of stock trends, which periodically performs dynamic model adaptation utilizing the emerging and recurring patterns simultaneously. We initially organize the stock trend forecasting into meta-learning tasks and train a forecasting model following meta-learning protocols. During model adaptation, MetaDA efficiently adapts the forecasting model with the latest data and a selected portion of historical data, which is dynamically identified by a task inference module. The task inference module first extracts task-level embeddings from the historical tasks, and then identifies the informative data with a task inference network. MetaDA has been evaluated on real-world stock datasets, achieving state-of-the-art performance with satisfactory efficiency.

cs.CE↗
arXiv2024

Flexibly Scaling Large Language Models Contexts Through Extensible Tokenization

Large language models (LLMs) are in need of sufficient contexts to handle many critical applications, such as retrieval augmented generation and few-shot learning. However, due to the constrained window size, the LLMs can only access to the information within a limited context. Although the size of context window can be extended by fine-tuning, it will result in a substantial cost in both training and inference stage. In this paper, we present Extensible Tokenization as an alternative method which realizes the flexible scaling of LLMs' context. Extensible Tokenization stands as a midware in between of the tokenized context and the LLM, which transforms the raw token embeddings into the extensible embeddings. Such embeddings provide a more compact representation for the long context, on top of which the LLM is able to perceive more information with the same context window. Extensible Tokenization is also featured by its flexibility: the scaling factor can be flexibly determined within a feasible scope, leading to the extension of an arbitrary context length at the inference time. Besides, Extensible Tokenization is introduced as a drop-in component, which can be seamlessly plugged into not only the LLM itself and but also its fine-tuned derivatives, bringing in the extended contextual information while fully preserving the LLM's existing capabilities. We perform comprehensive experiments on long-context language modeling and understanding tasks, which verify Extensible Tokenization as an effective, efficient, flexible, and compatible method to extend LLM's context. Our model and source code will be made publicly available.

cs.CL↗
arXiv2024

Quantum heat valve and diode of strongly coupled defects in amorphous material

The mechanical strain can control the frequency of two-level atoms in amorphous material. In this work, we would like to employ two coupled two-level atoms to manipulate the magnitude and direction of heat transport by controlling mechanical strain to realize the function of a thermal switch and valve. It is found that a high-performance heat diode can be realized in the wide Piezo voltage range at different temperatures. We also discuss the dependence of the rectification factor on temperatures and couplings of heat reservoirs. We find that the higher temperature differences correspond to the larger rectification effect. The asymmetry system-reservoir coupling strength can enhance the magnitude of heat transfer, and the impact of asymmetric and symmetric coupling strength on the performance of the heat diode is complementary. It may provide an efficient way to modulate and control heat transport's magnitude and flow preference. This work may give insight into designing and tuning quantum heat machines.

quant-ph↗
arXiv2024

Magnetically controlled quantum thermal devices via three nearest-neighbor coupled spin-1/2 systems

A quantum thermal device based on three nearest-neighbor coupled spin-1/2 systems controlled by the magnetic field is proposed. We systematically study the steady-state thermal behaviors of the system. When the two terminals of our system are in contact with two thermal reservoirs, respectively, the system behaves as a perfect thermal modulator that can manipulate heat current from zero to specific values by adjusting magnetic field direction over different parameter ranges, since the longitudinal magnetic field can completely block the heat transport. Significantly, the modulator can also be achieved when a third thermal reservoir perturbs the middle spin. We also find that the transverse field can induce the system to separate into two subspaces in which neither steady-state heat current vanishes, thus providing an extra level of control over the heat current through the manipulation of the initial state. In addition, the performance of this device as a transistor can be enhanced by controlling the magnetic field, achieving versatile amplification behaviors, in particular substantial amplification factors.

quant-ph↗
arXiv2023

LM-Cocktail: Resilient Tuning of Language Models via Model Merging

The pre-trained language models are continually fine-tuned to better support downstream applications. However, this operation may result in significant performance degeneration on general tasks beyond the targeted domain. To overcome this problem, we propose LM-Cocktail which enables the fine-tuned model to stay resilient in general perspectives. Our method is conducted in the form of model merging, where the fine-tuned language model is merged with the pre-trained base model or the peer models from other domains through weighted average. Despite simplicity, LM-Cocktail is surprisingly effective: the resulted model is able to achieve a strong empirical performance in the whole scope of general tasks while preserving a superior capacity in its targeted domain. We conduct comprehensive experiments with LLama and BGE model on popular benchmarks, including FLAN, MMLU, MTEB, whose results validate the efficacy of our proposed method. The code and checkpoints are available at https://github.com/FlagOpen/FlagEmbedding/tree/master/LM_Cocktail.

cs.CL↗
arXiv2023

Non-cuspidal Hida theory for Siegel modular forms and trivial zeros of $p$-adic $L$-functions

We study the derivative of the standard $p$-adic $L$-function associated with a $P$-ordinary Siegel modular form (for $P$ a parabolic subgroup of $\mathrm{GL}(n)$) when it presents a semi-stable trivial zero. This implies part of Greenberg's conjecture on the order and leading coefficient of $p$-adic $L$-functions at such trivial zero. We use the method of Greenberg-Stevens. For the construction of the improved $p$-adic $L$-function we develop Hida theory for non-cuspidal Siegel modular forms.

math.NT↗
arXiv2023

A Diffusion Model Based Quality Enhancement Method for HEVC Compressed Video

Video post-processing methods can improve the quality of compressed videos at the decoder side. Most of the existing methods need to train corresponding models for compressed videos with different quantization parameters to improve the quality of compressed videos. However, in most cases, the quantization parameters of the decoded video are unknown. This makes existing methods have their limitations in improving video quality. To tackle this problem, this work proposes a diffusion model based post-processing method for compressed videos. The proposed method first estimates the feature vectors of the compressed video and then uses the estimated feature vectors as the prior information for the quality enhancement model to adaptively enhance the quality of compressed video with different quantization parameters. Experimental results show that the quality enhancement results of our proposed method on mixed datasets are superior to existing methods.

eess.IV↗
arXiv2023

Infrared Imaging of Magnetic Octupole Domains in Non-collinear Antiferromagnets

Magnetic structure plays a pivotal role in the functionality of antiferromagnets (AFMs), which not only can be employed to encode digital data but also yields novel phenomena. Despite its growing significance, visualizing the antiferromagnetic domain structure remains a challenge, particularly for non-collinear AFMs. Currently, the observation of magnetic domains in non-collinear antiferromagnetic materials is feasible only in Mn$_{3}$Sn, underscoring the limitations of existing techniques that necessitate distinct methods for in-plane and out-of-plane magnetic domain imaging. In this study, we present a versatile method for imaging the antiferromagnetic domain structure in a series of non-collinear antiferromagnetic materials by utilizing the anomalous Ettingshausen effect (AEE), which resolves both the magnetic octupole moments parallel and perpendicular to the sample surface. Temperature modulation due to the AEE originating from different magnetic domains is measured by the lock-in thermography, revealing distinct behaviors of octupole domains in different antiferromagnets. This work delivers an efficient technique for the visualization of magnetic domains in non-collinear AFMs, which enables comprehensive study of the magnetization process at the microscopic level and paves the way for potential advancements in applications.

cond-mat.mtrl-sci↗
arXiv2023

Hybrid Inverted Index Is a Robust Accelerator for Dense Retrieval

Inverted file structure is a common technique for accelerating dense retrieval. It clusters documents based on their embeddings; during searching, it probes nearby clusters w.r.t. an input query and only evaluates documents within them by subsequent codecs, thus avoiding the expensive cost of exhaustive traversal. However, the clustering is always lossy, which results in the miss of relevant documents in the probed clusters and hence degrades retrieval quality. In contrast, lexical matching, such as overlaps of salient terms, tends to be strong feature for identifying relevant documents. In this work, we present the Hybrid Inverted Index (HI$^2$), where the embedding clusters and salient terms work collaboratively to accelerate dense retrieval. To make best of both effectiveness and efficiency, we devise a cluster selector and a term selector, to construct compact inverted lists and efficiently searching through them. Moreover, we leverage simple unsupervised algorithms as well as end-to-end knowledge distillation to learn these two modules, with the latter further boosting the effectiveness. Based on comprehensive experiments on popular retrieval benchmarks, we verify that clusters and terms indeed complement each other, enabling HI$^2$ to achieve lossless retrieval quality with competitive efficiency across various index settings. Our code and checkpoint are publicly available at https://github.com/namespace-Pt/Adon/tree/HI2.

cs.IR↗
arXiv2023

GraphFormers: GNN-nested Transformers for Representation Learning on Textual Graph

The representation learning on textual graph is to generate low-dimensional embeddings for the nodes based on the individual textual features and the neighbourhood information. Recent breakthroughs on pretrained language models and graph neural networks push forward the development of corresponding techniques. The existing works mainly rely on the cascaded model architecture: the textual features of nodes are independently encoded by language models at first; the textual embeddings are aggregated by graph neural networks afterwards. However, the above architecture is limited due to the independent modeling of textual features. In this work, we propose GraphFormers, where layerwise GNN components are nested alongside the transformer blocks of language models. With the proposed architecture, the text encoding and the graph aggregation are fused into an iterative workflow, {making} each node's semantic accurately comprehended from the global perspective. In addition, a {progressive} learning strategy is introduced, where the model is successively trained on manipulated data and original data to reinforce its capability of integrating information on graph. Extensive evaluations are conducted on three large-scale benchmark datasets, where GraphFormers outperform the SOTA baselines with comparable running efficiency.

cs.CL↗
arXiv2023

A Counterfactual Fair Model for Longitudinal Electronic Health Records via Deconfounder

The fairness issue of clinical data modeling, especially on Electronic Health Records (EHRs), is of utmost importance due to EHR's complex latent structure and potential selection bias. It is frequently necessary to mitigate health disparity while keeping the model's overall accuracy in practice. However, traditional methods often encounter the trade-off between accuracy and fairness, as they fail to capture the underlying factors beyond observed data. To tackle this challenge, we propose a novel model called Fair Longitudinal Medical Deconfounder (FLMD) that aims to achieve both fairness and accuracy in longitudinal Electronic Health Records (EHR) modeling. Drawing inspiration from the deconfounder theory, FLMD employs a two-stage training process. In the first stage, FLMD captures unobserved confounders for each encounter, which effectively represents underlying medical factors beyond observed EHR, such as patient genotypes and lifestyle habits. This unobserved confounder is crucial for addressing the accuracy/fairness dilemma. In the second stage, FLMD combines the learned latent representation with other relevant features to make predictions. By incorporating appropriate fairness criteria, such as counterfactual fairness, FLMD ensures that it maintains high prediction accuracy while simultaneously minimizing health disparities. We conducted comprehensive experiments on two real-world EHR datasets to demonstrate the effectiveness of FLMD. Apart from the comparison of baseline methods and FLMD variants in terms of fairness and accuracy, we assessed the performance of all models on disturbed/imbalanced and synthetic datasets to showcase the superiority of FLMD across different settings and provide valuable insights into its capabilities.

cs.LG↗
arXiv2023

M3FGM:a node masking and multi-granularity message passing-based federated graph model for spatial-temporal data prediction

Researchers are solving the challenges of spatial-temporal prediction by combining Federated Learning (FL) and graph models with respect to the constrain of privacy and security. In order to make better use of the power of graph model, some researchs also combine split learning(SL). However, there are still several issues left unattended: 1) Clients might not be able to access the server during inference phase; 2) The graph of clients designed manually in the server model may not reveal the proper relationship between clients. This paper proposes a new GNN-oriented split federated learning method, named node {\bfseries M}asking and {\bfseries M}ulti-granularity {\bfseries M}essage passing-based Federated Graph Model (M$^3$FGM) for the above issues. For the first issue, the server model of M$^3$FGM employs a MaskNode layer to simulate the case of clients being offline. We also redesign the decoder of the client model using a dual-sub-decoders structure so that each client model can use its local data to predict independently when offline. As for the second issue, a new GNN layer named Multi-Granularity Message Passing (MGMP) layer enables each client node to perceive global and local information. We conducted extensive experiments in two different scenarios on two real traffic datasets. Results show that M$^3$FGM outperforms the baselines and variant models, achieves the best results in both datasets and scenarios.

cs.LG↗
arXiv2023

A Consumer-tier based Visual-Brain Machine Interface for Augmented Reality Glasses Interactions

Objective.Visual-Brain Machine Interface(V-BMI) has provide a novel interaction technique for Augmented Reality (AR) industries. Several state-of-arts work has demonstates its high accuracy and real-time interaction capbilities. However, most of the studies employ EEGs devices that are rigid and difficult to apply in real-life AR glasseses application sceniraros. Here we develop a consumer-tier Visual-Brain Machine Inteface(V-BMI) system specialized for Augmented Reality(AR) glasses interactions. Approach. The developed system consists of a wearable hardware which takes advantages of fast set-up, reliable recording and comfortable wearable experience that specificized for AR glasses applications. Complementing this hardware, we have devised a software framework that facilitates real-time interactions within the system while accommodating a modular configuration to enhance scalability. Main results. The developed hardware is only 110g and 120x85x23 mm, which with 1 Tohm and peak to peak voltage is less than 1.5 uV, and a V-BMI based angry bird game and an Internet of Thing (IoT) AR applications are deisgned, we demonstrated such technology merits of intuitive experience and efficiency interaction. The real-time interaction accuracy is between 85 and 96 percentages in a commercial AR glasses (DTI is 2.24s and ITR 65 bits-min ). Significance. Our study indicates the developed system can provide an essential hardware-software framework for consumer based V-BMI AR glasses. Also, we derive several pivotal design factors for a consumer-grade V-BMI-based AR system: 1) Dynamic adaptation of stimulation patterns-classification methods via computer vision algorithms is necessary for AR glasses applications; and 2) Algorithmic localization to foster system stability and latency reduction.

cs.CV↗
arXiv2023

Exciton-exciton Interaction in Monolayer MoSe$_2$ from Mutual Screening of Coulomb Binding

The potential for low-threshold optical nonlinearity has received significant attention in the fields of photonics and conceptual optical neuron networks. Excitons in two-dimensional (2D) semiconductors are particularly promising in this regard as reduced screening and dimensional confinement foster their pronounced many-body interactions towards nonlinearity. However, experimental determination of the interactions remains ambiguous, as optical pumping in general creates a mixture of excitons and unbound carriers, where the impacts of band gap renormalization and carrier screening on exciton energy counteract each other. Here by comparing the influences on exciton ground and excited states energies in the photoluminescence spectroscopy of monolayer MoSe$_2$, we are able to identify separately the screening of Coulomb binding by the neutral excitons and by charge carriers. The energy difference between exciton ground state (A-1s) and excited state (A-2s) red-shifts by 5.5 meV when the neutral exciton density increases from 0 to $4\times 10^{11}$ cm$^{-2}$, in contrast to the blue shifts with the increase of either electron or hole density. This energy difference change is attributed to the mutual screening of Coulomb binding of neutral excitons, from which we extract an exciton polarizability of $α_{2D}^{\rm exciton} = 2.55\times 10^{-17}$ eV(m/V)$^2$. Our finding uncovers a new mechanism that dominates the repulsive part of many-body interaction between neutral excitons.

cond-mat.mtrl-sci↗
arXiv2023

Joint Intrinsic and Extrinsic LiDAR-Camera Calibration in Targetless Environments Using Plane-Constrained Bundle Adjustment

This paper introduces a novel targetless method for joint intrinsic and extrinsic calibration of LiDAR-camera systems using plane-constrained bundle adjustment (BA). Our method leverages LiDAR point cloud measurements from planes in the scene, alongside visual points derived from those planes. The core novelty of our method lies in the integration of visual BA with the registration between visual points and LiDAR point cloud planes, which is formulated as a unified optimization problem. This formulation achieves concurrent intrinsic and extrinsic calibration, while also imparting depth constraints to the visual points to enhance the accuracy of intrinsic calibration. Experiments are conducted on both public data sequences and self-collected dataset. The results showcase that our approach not only surpasses other state-of-the-art (SOTA) methods but also maintains remarkable calibration accuracy even within challenging environments. For the benefits of the robotics community, we have open sourced our codes.

cs.RO↗
arXiv2023

$p$-adic $L$-functions for ${\rm GSp}(4)\times{\rm GL}(2)$

We construct $p$-adic $L$-functions interpolating the critical values of the degree eight $L$-functions of ${\rm GSp}(4)\times {\rm GL}(2)$ for cuspidal automorphic representations generated by $p$-ordinary Siegel modular forms of genus two and $p$-ordinary modular forms.

math.NT↗
arXiv2023

When Large Language Models Meet Personalization: Perspectives of Challenges and Opportunities

The advent of large language models marks a revolutionary breakthrough in artificial intelligence. With the unprecedented scale of training and model parameters, the capability of large language models has been dramatically improved, leading to human-like performances in understanding, language synthesizing, and common-sense reasoning, etc. Such a major leap-forward in general AI capacity will change the pattern of how personalization is conducted. For one thing, it will reform the way of interaction between humans and personalization systems. Instead of being a passive medium of information filtering, large language models present the foundation for active user engagement. On top of such a new foundation, user requests can be proactively explored, and user's required information can be delivered in a natural and explainable way. For another thing, it will also considerably expand the scope of personalization, making it grow from the sole function of collecting personalized information to the compound function of providing personalized services. By leveraging large language models as general-purpose interface, the personalization systems may compile user requests into plans, calls the functions of external tools to execute the plans, and integrate the tools' outputs to complete the end-to-end personalization tasks. Today, large language models are still being developed, whereas the application in personalization is largely unexplored. Therefore, we consider it to be the right time to review the challenges in personalization and the opportunities to address them with LLMs. In particular, we dedicate this perspective paper to the discussion of the following aspects: the development and challenges for the existing personalization system, the newly emerged capabilities of large language models, and the potential ways of making use of large language models for personalization.

cs.IR↗