arXiv subjects

Zheng Li

Publications and source records attributed to Zheng Li.

At least 271 records · Page 15Linked to original sources

arXiv2024

Enc2DB: A Hybrid and Adaptive Encrypted Query Processing Framework

As cloud computing gains traction, data owners are outsourcing their data to cloud service providers (CSPs) for Database Service (DBaaS), bringing in a deviation of data ownership and usage, and intensifying privacy concerns, especially with potential breaches by hackers or CSP insiders. To address that, encrypted database services propose encrypting every tuple and query statement before submitting to the CSP, ensuring data confidentiality when the CSP is honest-but-curious, or even compromised. Existing solutions either employ property preserving cryptography schemes, which can perform certain operations over ciphertext without decrypting the data over the CSP, or utilize trusted execution environment (TEE) to safeguard data and computations from the CSP. Based on these efforts, we introduce Enc2DB, a novel secure database system, following a hybrid strategy on PostgreSQL and openGauss. We present a micro-benchmarking test and self-adaptive mode switch strategy that can dynamically choose the best execution path (cryptography or TEE) to answer a given query. Besides, we also design and implement a ciphertext index compatible with native cost model and query optimizers to accelerate query processing. Empirical study over TPC-C test justifies that Enc2DB outperforms pure TEE and cryptography solutions, and our ciphertext index implementation also outperforms the state-of-the-art cryptographic-based system.

cs.CR↗

arXiv2024

Intelligent Reflecting Surface Assisted Integrated Sensing and Communications for mmWave Channels

This paper proposes an intelligent reflecting surface (IRS) assisted integrated sensing and communication (ISAC) system operating at the millimeter-wave (mmWave) band. Specifically, the ISAC system combines communication and radar operations and performs, detecting and communicating simultaneously with multiple targets and users. The IRS dynamically controls the amplitude or phase of the radio signal via reflecting elements to reconfigure the radio propagation environment and enhance the transmission rate of the ISAC system. By jointly designing the radar signal covariance (RSC) matrix, the beamforming vector of the communication system, and the IRS phase shift, the ISAC system transmission rate can be improved while matching the desired waveform for radar. The problem is non-convex due to multivariate coupling, and thus we decompose it into two separate subproblems. First, a closed-form solution of the RSC matrix is derived from the desired radar waveform. Next, the quadratic transformation (QT) technique is applied to the subproblem, and then alternating optimization (AO) is employed to determine the communication beamforming vector and the IRS phase shift. For computing the IRS phase shift, we adopt both the majorization minimization (MM) and the manifold optimization (MO). Also, we derive a closed-form solution for the formulated problem, effectively decreasing computational complexity. Furthermore, a trade-off factor is introduced to balance the performance of communication and sensing. Finally, the simulations verify the effectiveness of the algorithm and demonstrate that the IRS can improve the performance of the ISAC system.

eess.SP↗

arXiv2024

Fusion Dynamical Systems with Machine Learning in Imitation Learning: A Comprehensive Overview

Imitation Learning (IL), also referred to as Learning from Demonstration (LfD), holds significant promise for capturing expert motor skills through efficient imitation, facilitating adept navigation of complex scenarios. A persistent challenge in IL lies in extending generalization from historical demonstrations, enabling the acquisition of new skills without re-teaching. Dynamical system-based IL (DSIL) emerges as a significant subset of IL methodologies, offering the ability to learn trajectories via movement primitives and policy learning based on experiential abstraction. This paper emphasizes the fusion of theoretical paradigms, integrating control theory principles inherent in dynamical systems into IL. This integration notably enhances robustness, adaptability, and convergence in the face of novel scenarios. This survey aims to present a comprehensive overview of DSIL methods, spanning from classical approaches to recent advanced approaches. We categorize DSIL into autonomous dynamical systems and non-autonomous dynamical systems, surveying traditional IL methods with low-dimensional input and advanced deep IL methods with high-dimensional input. Additionally, we present and analyze three main stability methods for IL: Lyapunov stability, contraction theory, and diffeomorphism mapping. Our exploration also extends to popular policy improvement methods for DSIL, encompassing reinforcement learning, deep reinforcement learning, and evolutionary strategies.

cs.RO↗

arXiv2024

Reinforcement Learning in Agent-Based Market Simulation: Unveiling Realistic Stylized Facts and Behavior

Investors and regulators can greatly benefit from a realistic market simulator that enables them to anticipate the consequences of their decisions in real markets. However, traditional rule-based market simulators often fall short in accurately capturing the dynamic behavior of market participants, particularly in response to external market impact events or changes in the behavior of other participants. In this study, we explore an agent-based simulation framework employing reinforcement learning (RL) agents. We present the implementation details of these RL agents and demonstrate that the simulated market exhibits realistic stylized facts observed in real-world markets. Furthermore, we investigate the behavior of RL agents when confronted with external market impacts, such as a flash crash. Our findings shed light on the effectiveness and adaptability of RL-based agents within the simulation, offering insights into their response to significant market events.

q-fin.TR↗

arXiv2024

IterAlign: Iterative Constitutional Alignment of Large Language Models

With the rapid development of large language models (LLMs), aligning LLMs with human values and societal norms to ensure their reliability and safety has become crucial. Reinforcement learning with human feedback (RLHF) and Constitutional AI (CAI) have been proposed for LLM alignment. However, these methods require either heavy human annotations or explicitly pre-defined constitutions, which are labor-intensive and resource-consuming. To overcome these drawbacks, we study constitution-based LLM alignment and propose a data-driven constitution discovery and self-alignment framework called IterAlign. IterAlign leverages red teaming to unveil the weaknesses of an LLM and automatically discovers new constitutions using a stronger LLM. These constitutions are then used to guide self-correction of the base LLM. Such a constitution discovery pipeline can be run iteratively and automatically to discover new constitutions that specifically target the alignment gaps in the current LLM. Empirical results on several safety benchmark datasets and multiple base LLMs show that IterAlign successfully improves truthfulness, helpfulness, harmlessness and honesty, improving the LLM alignment by up to $13.5\%$ in harmlessness.

cs.CL↗

arXiv2024

Hierarchical Query Classification in E-commerce Search

E-commerce platforms typically store and structure product information and search data in a hierarchy. Efficiently categorizing user search queries into a similar hierarchical structure is paramount in enhancing user experience on e-commerce platforms as well as news curation and academic research. The significance of this task is amplified when dealing with sensitive query categorization or critical information dissemination, where inaccuracies can lead to considerable negative impacts. The inherent complexity of hierarchical query classification is compounded by two primary challenges: (1) the pronounced class imbalance that skews towards dominant categories, and (2) the inherent brevity and ambiguity of search queries that hinder accurate classification. To address these challenges, we introduce a novel framework that leverages hierarchical information through (i) enhanced representation learning that utilizes the contrastive loss to discern fine-grained instance relationships within the hierarchy, called ''instance hierarchy'', and (ii) a nuanced hierarchical classification loss that attends to the intrinsic label taxonomy, named ''label hierarchy''. Additionally, based on our observation that certain unlabeled queries share typographical similarities with labeled queries, we propose a neighborhood-aware sampling technique to intelligently select these unlabeled queries to boost the classification performance. Extensive experiments demonstrate that our proposed method is better than state-of-the-art (SOTA) on the proprietary Amazon dataset, and comparable to SOTA on the public datasets of Web of Science and RCV1-V2. These results underscore the efficacy of our proposed solution, and pave the path toward the next generation of hierarchy-aware query classification systems.

cs.IR↗

arXiv2024

Quantum tomography of molecules using ultrafast electron diffraction

We propose a quantum tomography (QT) approach to retrieve the temporally evolving reduced density matrix in elecotronic state basis, where the populations and coherence between ground state and excited state are reconstructed from the ultrafast electron diffraction signal. In order to showcase the capability of the proposed QT approach, we simulate the nuclear wavepacket dynamics and ultrafast electron diffraction of photoexcited pyrrole molecules using ab initio quantum chemical CASSCF method. From simulated time-resolved diffraction data, we retrieve the evolving density matrix in a crude diabatic representation basis and reveal the symmetry of the excited pyrrole wavepacket. Our QT approach opens the route to make quantum version of "molecular movie" that covers the electronic degree of freedom, and equips ultrafast electron diffraction with the power to reveal the coherence between electronic states, relaxation and dynamics of population transfer.

physics.chem-ph↗

arXiv2024

WikiTableEdit: A Benchmark for Table Editing by Natural Language Instruction

Tabular data, as a crucial form of data representation, exists in diverse formats on the Web. When confronted with complex and irregular tables, manual modification becomes a laborious task. This paper investigates the performance of Large Language Models (LLMs) in the context of table editing tasks. Existing research mainly focuses on regular-shaped tables, wherein instructions are used to generate code in SQL, Python, or Excel Office-script for manipulating the tables. Nevertheless, editing tables with irregular structures, particularly those containing merged cells spanning multiple rows, poses a challenge when using code. To address this, we introduce the WikiTableEdit dataset. Leveraging 26,531 tables from the WikiSQL dataset, we automatically generate natural language instructions for six distinct basic operations and the corresponding outcomes, resulting in over 200,000 instances. Subsequently, we evaluate several representative large language models on the WikiTableEdit dataset to demonstrate the challenge of this task. The dataset will be released to the community to promote related researches.

cs.AI↗

arXiv2024

Augmenting Prototype Network with TransMix for Few-shot Hyperspectral Image Classification

Few-shot hyperspectral image classification aims to identify the classes of each pixel in the images by only marking few of these pixels. And in order to obtain the spatial-spectral joint features of each pixel, the fixed-size patches centering around each pixel are often used for classification. However, observing the classification results of existing methods, we found that boundary patches corresponding to the pixels which are located at the boundary of the objects in the hyperspectral images, are hard to classify. These boundary patchs are mixed with multi-class spectral information. Inspired by this, we propose to augment the prototype network with TransMix for few-shot hyperspectrial image classification(APNT). While taking the prototype network as the backbone, it adopts the transformer as feature extractor to learn the pixel-to-pixel relation and pay different attentions to different pixels. At the same time, instead of directly using the patches which are cut from the hyperspectral images for training, it randomly mixs up two patches to imitate the boundary patches and uses the synthetic patches to train the model, with the aim to enlarge the number of hard training samples and enhance their diversity. And by following the data agumentation technique TransMix, the attention returned by the transformer is also used to mix up the labels of two patches to generate better labels for synthetic patches. Compared with existing methods, the proposed method has demonstrated sate of the art performance and better robustness for few-shot hyperspectral image classification in our experiments.

cs.CV↗

arXiv2024

Understanding how off-stoichiometry promotes cation mixing in LiNiO$_2$

Although LiNiO$_2$ is chemically similar to LiCoO$_2$ and offers a nearly identical theoretical capacity, LiNiO$_2$ and related Co-free Ni-rich cathode materials suffer from degradation during electrochemical cycling that has prevented practical use in Li-ion batteries. The observed capacity decay of LiNiO$_2$ has been attributed to the formation of structural defects via Li/Ni cation mixing that reduces cyclability and leads to poor capacity retention. Herein, we investigate the kinetics and thermodynamics of Li/Ni mixing in ideal LiNiO$_2$ and off-stoichiometric Li$_{1-z}$Ni$_{1+z}$O$_2$. We find that ideal LiNiO$_2$ is stable against cation mixing with similar characteristics as LiCoO$_2$. Li/Ni mixing is promoted by extra Ni in the Li layers that cannot be avoided in synthesis. Our study elucidates the crucial role of extra Ni atoms on Li sites in the cation mixing mechanism, an insight that can inform the development of Co-free cathode materials.

cond-mat.mtrl-sci↗

arXiv2024

CodeFuse-13B: A Pretrained Multi-lingual Code Large Language Model

Code Large Language Models (Code LLMs) have gained significant attention in the industry due to their wide applications in the full lifecycle of software engineering. However, the effectiveness of existing models in understanding non-English inputs for multi-lingual code-related tasks is still far from well studied. This paper introduces CodeFuse-13B, an open-sourced pre-trained code LLM. It is specifically designed for code-related tasks with both English and Chinese prompts and supports over 40 programming languages. CodeFuse achieves its effectiveness by utilizing a high quality pre-training dataset that is carefully filtered by program analyzers and optimized during the training process. Extensive experiments are conducted using real-world usage scenarios, the industry-standard benchmark HumanEval-x, and the specially designed CodeFuseEval for Chinese prompts. To assess the effectiveness of CodeFuse, we actively collected valuable human feedback from the AntGroup's software development process where CodeFuse has been successfully deployed. The results demonstrate that CodeFuse-13B achieves a HumanEval pass@1 score of 37.10%, positioning it as one of the top multi-lingual code LLMs with similar parameter sizes. In practical scenarios, such as code generation, code translation, code comments, and testcase generation, CodeFuse performs better than other models when confronted with Chinese prompts.

cs.SE↗

arXiv2024

Deep Learning-Based Knowledge Injection for Metaphor Detection: A Comprehensive Review

Metaphor as an advanced cognitive modality works by extracting familiar concepts in the target domain in order to understand vague and abstract concepts in the source domain. This helps humans to quickly understand and master new domains and thus adapt to changing environments. With the continuous development of metaphor research in the natural language community, many studies using knowledge-assisted models to detect textual metaphors have emerged in recent years. Compared to not using knowledge, systems that introduce various kinds of knowledge achieve greater performance gains and reach SOTA in a recent study. Based on this, the goal of this paper is to provide a comprehensive review of research advances in the application of deep learning for knowledge injection in metaphor detection tasks. We will first systematically summarize and generalize the mainstream knowledge and knowledge injection principles. Then, the datasets, evaluation metrics, and benchmark models used in metaphor detection tasks are examined. Finally, we explore the current issues facing knowledge injection methods and provide an outlook on future research directions.

cs.CL↗

arXiv2023

Ultrafast X-ray Diffraction Probe of Coherent Spin-state Dynamics in Molecules

We propose an approach to probe coherent spin-state dynamics of molecules using circularly polarized hard x-ray pulses. For the dynamically aligned nitric oxide molecules in a coherent superposition spin-orbit coupled electronic state that can be prepared through stimulated Raman scattering, we demonstrate the capability of ultrafast x-ray diffraction to not only reveal the quantum beating of the coherent spin-state wave packet, but also image the spatial spin density of the molecule. With circularly polarized ultrafast x-ray diffraction signal, we show that the electronic density matrix can be retrieved. The spatio-temporal resolving power of ultrafast x-ray diffraction paves the way for tracking transient spatial wave function in molecular dynamics involving spin degree of freedom.

physics.optics↗

arXiv2023

Multi-level Relation Learning for Cross-domain Few-shot Hyperspectral Image Classification

Cross-domain few-shot hyperspectral image classification focuses on learning prior knowledge from a large number of labeled samples from source domains and then transferring the knowledge to the tasks which contain few labeled samples in target domains. Following the metric-based manner, many current methods first extract the features of the query and support samples, and then directly predict the classes of query samples according to their distance to the support samples or prototypes. The relations between samples have not been fully explored and utilized. Different from current works, this paper proposes to learn sample relations on different levels and take them into the model learning process, to improve the cross-domain few-shot hyperspectral image classification. Building on current method of "Deep Cross-Domain Few-Shot Learning for Hyperspectral Image Classification" which adopts a domain discriminator to deal with domain-level distribution difference, the proposed method applies contrastive learning to learn the class-level sample relations to obtain more discriminable sample features. In addition, it adopts a transformer based cross-attention learning module to learn the set-level sample relations and acquire the attention from query samples to support samples. Our experimental results have demonstrated the contribution of the multi-level relation learning mechanism for few-shot hyperspectral image classification when compared with the state of the art methods.

cs.CV↗

arXiv2023

Enhancing User Intent Capture in Session-Based Recommendation with Attribute Patterns

The goal of session-based recommendation in E-commerce is to predict the next item that an anonymous user will purchase based on the browsing and purchase history. However, constructing global or local transition graphs to supplement session data can lead to noisy correlations and user intent vanishing. In this work, we propose the Frequent Attribute Pattern Augmented Transformer (FAPAT) that characterizes user intents by building attribute transition graphs and matching attribute patterns. Specifically, the frequent and compact attribute patterns are served as memory to augment session representations, followed by a gate and a transformer block to fuse the whole session information. Through extensive experiments on two public benchmarks and 100 million industrial data in three domains, we demonstrate that FAPAT consistently outperforms state-of-the-art methods by an average of 4.5% across various evaluation metrics (Hits, NDCG, MRR). Besides evaluating the next-item prediction, we estimate the models' capabilities to capture user intents via predicting items' attributes and period-item recommendations.

cs.IR↗

arXiv2023

Third order nonlinear transport properties in topological chiral antiferromagnetic semimetal CoNb3S6

The topology between Bloch states in reciprocal space has attracted tremendous attention in recent years. The quantum geometry of the band structure is composed of quantum metric as real part and berry curvature as imaginary part. While the Berry curvature, the Berry curvature dipole and Berry connection polarizability have been recently revealed by the first order anomalous hall, second order and third order nonlinear Hall effect respectively, the quantum metric induced second order nonlinear transverse and longitudinal response in topological antiferromagnetic material MnBi2Te4 was only very recently reported. Here we demonstrate the similar third order nonlinear transport properties in the topological antiferromagnetic CoNb3S6. We observed that the third order nonlinear longitudinal V3ω xx increase significantly at the antiferromagnetic transition temperature TN ~ 29 K, which was probably induced by the quantum metric without time-reversal symmetry or inversion symmetry. Besides, temperature-dependent nonlinear behaviour was observed in the first order I-V curve below the Neel temperature TN, which was not reported in MnBi2Te4 and FeSn. Such nonlinear I-V behaviour hints for the possible existence of Charge Density Wave (CDW) state, which has been discovered in its sister material FeNb3S6. Simultaneously, two plateaus in the third order nonlinear longitudinal V3ω xx~ I^ω curve are observed, which is also speculated to be related with the possible CDW state. However, the genuine mechanism for the first order nonlinear I-V and its relation with the third order nonlinear transport call for more experimental investigations and theoretical interpretation. Our work provides a way to explore third harmonic nonlinear transport and interaction with magnetic order and CDW.

cond-mat.mtrl-sci↗

arXiv2023

Intelligent Reflective Surface Assisted Integrated Sensing and Wireless Power Transfer

Wireless sensing and wireless energy are enablers to pave the way for smart transportation and a greener future. In this paper, an intelligent reflecting surface (IRS) assisted integrated sensing and wireless power transfer (ISWPT) system is investigated, where the transmitter in transportation infrastructure networks sends signals to sense multiple targets and simultaneously to multiple energy harvesting devices (EHDs) to power them. In light of the performance tradeoff between energy harvesting and sensing, we propose to jointly optimize the system performance via optimizing the beamforming and IRS phase shift. However, the coupling of optimization variables makes the formulated problem non-convex. Thus, an alternative optimization approach is introduced and based on which two algorithms are proposed to solve the problem. Specifically, the first one involves a semi-definite program technique, while the second one features a low-complexity optimization algorithm based on successive convex approximation and majorization minimization. Our simulation results validate the proposed algorithms and demonstrate the advantages of using IRS to assist wireless power transfer in ISWPT systems.

eess.SP↗

arXiv2023

RemovalNet: DNN Fingerprint Removal Attacks

With the performance of deep neural networks (DNNs) remarkably improving, DNNs have been widely used in many areas. Consequently, the DNN model has become a valuable asset, and its intellectual property is safeguarded by ownership verification techniques (e.g., DNN fingerprinting). However, the feasibility of the DNN fingerprint removal attack and its potential influence remains an open problem. In this paper, we perform the first comprehensive investigation of DNN fingerprint removal attacks. Generally, the knowledge contained in a DNN model can be categorized into general semantic and fingerprint-specific knowledge. To this end, we propose a min-max bilevel optimization-based DNN fingerprint removal attack named RemovalNet, to evade model ownership verification. The lower-level optimization is designed to remove fingerprint-specific knowledge. While in the upper-level optimization, we distill the victim model's general semantic knowledge to maintain the surrogate model's performance. We conduct extensive experiments to evaluate the fidelity, effectiveness, and efficiency of the RemovalNet against four advanced defense methods on six metrics. The empirical results demonstrate that (1) the RemovalNet is effective. After our DNN fingerprint removal attack, the model distance between the target and surrogate models is x100 times higher than that of the baseline attacks, (2) the RemovalNet is efficient. It uses only 0.2% (400 samples) of the substitute dataset and 1,000 iterations to conduct our attack. Besides, compared with advanced model stealing attacks, the RemovalNet saves nearly 85% of computational resources at most, (3) the RemovalNet achieves high fidelity that the created surrogate model maintains high accuracy after the DNN fingerprint removal process. Our code is available at: https://github.com/grasses/RemovalNet.

cs.CV↗