arXiv subjects

Qi Hu

Publications and source records attributed to Qi Hu.

At least 37 records · Page 2Linked to original sources

arXiv2024

Privacy in Large Language Models: Attacks, Defenses and Future Directions

The advancement of large language models (LLMs) has significantly enhanced the ability to effectively tackle various downstream NLP tasks and unify these tasks into generative pipelines. On the one hand, powerful language models, trained on massive textual data, have brought unparalleled accessibility and usability for both models and users. On the other hand, unrestricted access to these models can also introduce potential malicious and unintentional privacy risks. Despite ongoing efforts to address the safety and privacy concerns associated with LLMs, the problem remains unresolved. In this paper, we provide a comprehensive analysis of the current privacy attacks targeting LLMs and categorize them according to the adversary's assumed capabilities to shed light on the potential vulnerabilities present in LLMs. Then, we present a detailed overview of prominent defense strategies that have been developed to counter these privacy attacks. Beyond existing works, we identify upcoming privacy concerns as LLMs evolve. Lastly, we point out several potential avenues for future exploration.

cs.CL↗

arXiv2024

Node Level Graph Autoencoder: Unified Pretraining for Textual Graph Learning

Textual graphs are ubiquitous in real-world applications, featuring rich text information with complex relationships, which enables advanced research across various fields. Textual graph representation learning aims to generate low-dimensional feature embeddings from textual graphs that can improve the performance of downstream tasks. A high-quality feature embedding should effectively capture both the structural and the textual information in a textual graph. However, most textual graph dataset benchmarks rely on word2vec techniques to generate feature embeddings, which inherently limits their capabilities. Recent works on textual graph representation learning can be categorized into two folds: supervised and unsupervised methods. Supervised methods finetune a language model on labeled nodes, which have limited capabilities when labeled data is scarce. Unsupervised methods, on the other hand, extract feature embeddings by developing complex training pipelines. To address these limitations, we propose a novel unified unsupervised learning autoencoder framework, named Node Level Graph AutoEncoder (NodeGAE). We employ language models as the backbone of the autoencoder, with pretraining on text reconstruction. Additionally, we add an auxiliary loss term to make the feature embeddings aware of the local graph structure. Our method maintains simplicity in the training process and demonstrates generalizability across diverse textual graphs and downstream tasks. We evaluate our method on two core graph representation learning downstream tasks: node classification and link prediction. Comprehensive experiments demonstrate that our approach substantially enhances the performance of diverse graph neural networks (GNNs) across multiple textual graph datasets.

cs.LG↗

arXiv2024

Privacy-Preserved Neural Graph Databases

In the era of large language models (LLMs), efficient and accurate data retrieval has become increasingly crucial for the use of domain-specific or private data in the retrieval augmented generation (RAG). Neural graph databases (NGDBs) have emerged as a powerful paradigm that combines the strengths of graph databases (GDBs) and neural networks to enable efficient storage, retrieval, and analysis of graph-structured data which can be adaptively trained with LLMs. The usage of neural embedding storage and Complex neural logical Query Answering (CQA) provides NGDBs with generalization ability. When the graph is incomplete, by extracting latent patterns and representations, neural graph databases can fill gaps in the graph structure, revealing hidden relationships and enabling accurate query answering. Nevertheless, this capability comes with inherent trade-offs, as it introduces additional privacy risks to the domain-specific or private databases. Malicious attackers can infer more sensitive information in the database using well-designed queries such as from the answer sets of where Turing Award winners born before 1950 and after 1940 lived, the living places of Turing Award winner Hinton are probably exposed, although the living places may have been deleted in the training stage due to the privacy concerns. In this work, we propose a privacy-preserved neural graph database (P-NGDB) framework to alleviate the risks of privacy leakage in NGDBs. We introduce adversarial training techniques in the training stage to enforce the NGDBs to generate indistinguishable answers when queried with private information, enhancing the difficulty of inferring sensitive information through combinations of multiple innocuous queries.

cs.DB↗

arXiv2024

PrivLM-Bench: A Multi-level Privacy Evaluation Benchmark for Language Models

The rapid development of language models (LMs) brings unprecedented accessibility and usage for both models and users. On the one hand, powerful LMs achieve state-of-the-art performance over numerous downstream NLP tasks. On the other hand, more and more attention is paid to unrestricted model accesses that may bring malicious privacy risks of data leakage. To address these issues, many recent works propose privacy-preserving language models (PPLMs) with differential privacy (DP). Unfortunately, different DP implementations make it challenging for a fair comparison among existing PPLMs. In this paper, we present PrivLM-Bench, a multi-perspective privacy evaluation benchmark to empirically and intuitively quantify the privacy leakage of LMs. Instead of only reporting DP parameters, PrivLM-Bench sheds light on the neglected inference data privacy during actual usage. PrivLM-Bench first clearly defines multi-faceted privacy objectives. Then, PrivLM-Bench constructs a unified pipeline to perform private fine-tuning. Lastly, PrivLM-Bench performs existing privacy attacks on LMs with pre-defined privacy objectives as the empirical evaluation results. The empirical attack results are used to fairly and intuitively evaluate the privacy leakage of various PPLMs. We conduct extensive experiments on three datasets of GLUE for mainstream LMs.

cs.CL↗

arXiv2023

User Consented Federated Recommender System Against Personalized Attribute Inference Attack

Recommender systems can be privacy-sensitive. To protect users' private historical interactions, federated learning has been proposed in distributed learning for user representations. Using federated recommender (FedRec) systems, users can train a shared recommendation model on local devices and prevent raw data transmissions and collections. However, the recommendation model learned by a common FedRec may still be vulnerable to private information leakage risks, particularly attribute inference attacks, which means that the attacker can easily infer users' personal attributes from the learned model. Additionally, traditional FedRecs seldom consider the diverse privacy preference of users, leading to difficulties in balancing the recommendation utility and privacy preservation. Consequently, FedRecs may suffer from unnecessary recommendation performance loss due to over-protection and private information leakage simultaneously. In this work, we propose a novel user-consented federated recommendation system (UC-FedRec) to flexibly satisfy the different privacy needs of users by paying a minimum recommendation accuracy price. UC-FedRec allows users to self-define their privacy preferences to meet various demands and makes recommendations with user consent. Experiments conducted on different real-world datasets demonstrate that our framework is more efficient and flexible compared to baselines.

cs.IR↗

arXiv2023

Independent Distribution Regularization for Private Graph Embedding

Learning graph embeddings is a crucial task in graph mining tasks. An effective graph embedding model can learn low-dimensional representations from graph-structured data for data publishing benefiting various downstream applications such as node classification, link prediction, etc. However, recent studies have revealed that graph embeddings are susceptible to attribute inference attacks, which allow attackers to infer private node attributes from the learned graph embeddings. To address these concerns, privacy-preserving graph embedding methods have emerged, aiming to simultaneously consider primary learning and privacy protection through adversarial learning. However, most existing methods assume that representation models have access to all sensitive attributes in advance during the training stage, which is not always the case due to diverse privacy preferences. Furthermore, the commonly used adversarial learning technique in privacy-preserving representation learning suffers from unstable training issues. In this paper, we propose a novel approach called Private Variational Graph AutoEncoders (PVGAE) with the aid of independent distribution penalty as a regularization term. Specifically, we split the original variational graph autoencoder (VGAE) to learn sensitive and non-sensitive latent representations using two sets of encoders. Additionally, we introduce a novel regularization to enforce the independence of the encoders. We prove the theoretical effectiveness of regularization from the perspective of mutual information. Experimental results on three real-world datasets demonstrate that PVGAE outperforms other baselines in private embedding learning regarding utility performance and privacy protection.

cs.LG↗

arXiv2023

THLNet: two-stage heterogeneous lightweight network for monaural speech enhancement

In this paper, we propose a two-stage heterogeneous lightweight network for monaural speech enhancement. Specifically, we design a novel two-stage framework consisting of a coarse-grained full-band mask estimation stage and a fine-grained low-frequency refinement stage. Instead of using a hand-designed real-valued filter, we use a novel learnable complex-valued rectangular bandwidth (LCRB) filter bank as an extractor of compact features. Furthermore, considering the respective characteristics of the proposed two-stage task, we used a heterogeneous structure, i.e., a U-shaped subnetwork as the backbone of CoarseNet and a single-scale subnetwork as the backbone of FineNet. We conducted experiments on the VoiceBank + DEMAND and DNS datasets to evaluate the proposed approach. The experimental results show that the proposed method outperforms the current state-of-the-art methods, while maintaining relatively small model size and low computational complexity.

cs.SD↗

arXiv2023

ForkNet: Simultaneous Time and Time-Frequency Domain Modeling for Speech Enhancement

Previous research in speech enhancement has mostly focused on modeling time or time-frequency domain information alone, with little consideration given to the potential benefits of simultaneously modeling both domains. Since these domains contain complementary information, combining them may improve the performance of the model. In this letter, we propose a new approach to simultaneously model time and time-frequency domain information in a single model. We begin with the DPT-FSNet (causal version) model as a baseline and modify the encoder structure by replacing the original encoder with three separate encoders, each dedicated to modeling time-domain, real-imaginary, and magnitude information, respectively. Additionally, we introduce a feature fusion module both before and after the dual-path processing blocks to better leverage information from the different domains. The outcomes of our experiments reveal that the proposed approach achieves superior performance compared to existing state-of-the-art causal models, while preserving a relatively compact model size and low computational complexity.

cs.SD↗

arXiv2023

Universal adaptive optics for microscopy through embedded neural network control

The resolution and contrast of microscope imaging is often affected by aberrations introduced by imperfect optical systems and inhomogeneous refractive structures in specimens. Adaptive optics (AO) compensates these aberrations and restores diffraction limited performance. A wide range of AO solutions have been introduced, often tailored to a specific microscope type or application. Until now, a universal AO solution -- one that can be readily transferred between microscope modalities -- has not been deployed. We propose versatile and fast aberration correction using a physics-based machine learning (ML) assisted wavefront-sensorless AO control method. Unlike previous ML methods, we used a bespoke neural network (NN) architecture, designed using physical understanding of image formation, that was embedded in the control loop of the microscope. The approach means that not only is the resulting NN orders of magnitude simpler than previous NN methods, but the concept is translatable across microscope modalities. We demonstrated the method on a two-photon, a three-photon and a widefield three-dimensional (3D) structured illumination microscope. Results showed that the method outperformed commonly-used modal-based sensorless AO methods. We also showed that our ML-based method was robust in a range of challenging imaging conditions, such as extended 3D sample structures, specimen motion, low signal to noise ratio and activity-induced fluorescence fluctuations. Moreover, as the bespoke architecture encapsulated physical understanding of the imaging process, the internal NN configuration was no-longer a ``black box'', but provided physical insights on internal workings, which could influence future designs.

eess.IV↗

arXiv2023

Single-parameter variational wavefunctions for quantum Hall bilayers

Bilayer quantum Hall states have been shown to be described by a BCS-paired state of composite fermions. However, finding a qualitatively accurate model state valid across all values of the bilayer separation is challenging. Here, we introduce two variational wavefunctions, each with a single variational parameter, which can be thought of as a proxy for the BCS order parameter. Studying systems of up to 9+9 electrons in a spherical geometry using Monte Carlo methods, we show that the ground state can be accurately described by these single-parameter variational states. In addition, for the first time we provide a numerically exact wavefunction for the Halperin-111 state in terms of composite fermions.

cond-mat.mes-hall↗

arXiv2021

SOUP: Spatial-Temporal Demand Forecasting and Competitive Supply

We consider a setting with an evolving set of requests for transportation from an origin to a destination before a deadline and a set of agents capable of servicing the requests. In this setting, an assignment authority is to assign agents to requests such that the average idle time of the agents is minimized. An example is the scheduling of taxis (agents) to meet incoming requests for trips while ensuring that the taxis are empty as little as possible. In this paper, we study the problem of spatial-temporal demand forecasting and competitive supply (SOUP). We address the problem in two steps. First, we build a granular model that provides spatial-temporal predictions of requests. Specifically, we propose a Spatial-Temporal Graph Convolutional Sequential Learning (ST-GCSL) algorithm that predicts the service requests across locations and time slots. Second, we provide means of routing agents to request origins while avoiding competition among the agents. In particular, we develop a demand-aware route planning (DROP) algorithm that considers both the spatial-temporal predictions and the supplydemand state. We report on extensive experiments with realworld and synthetic data that offer insight into the performance of the solution and show that it is capable of outperforming the state-of-the-art proposals.

cs.DB↗

arXiv2020

Emergent universality in critical quantum spin chains: entanglement Virasoro algebra

Entanglement entropy and entanglement spectrum have been widely used to characterize quantum entanglement in extended many-body systems. Given a pure state of the system and a division into regions $A$ and $B$, they can be obtained in terms of the $Schmidt~ values$, or eigenvalues $λ_α$ of the reduced density matrix $ρ_A$ for region $A$. In this paper we draw attention instead to the $Schmidt~ vectors$, or eigenvectors $|v_α\rangle$ of $ρ_A$. We consider the ground state of critical quantum spin chains whose low energy/long distance physics is described by an emergent conformal field theory (CFT). We show that the Schmidt vectors $|v_α\rangle$ display an emergent universal structure, corresponding to a realization of the Virasoro algebra of a boundary CFT (a chiral version of the original CFT). Indeed, we build weighted sums $H_n$ of the lattice Hamiltonian density $h_{j,j+1}$ over region $A$ and show that the matrix elements $\langle v_αH_n |v_{α'}\rangle$ are universal, up to finite-size corrections. More concretely, these matrix elements are given by an analogous expression for $H_n^{\tiny \text{CFT}} = \frac 1 2 (L_n + L_{-n})$ in the boundary CFT, where $L_n$'s are (one copy of) the Virasoro generators. We numerically confirm our results using the critical Ising quantum spin chain and other (free-fermion equivalent) models.

quant-ph↗

arXiv2020

Practical sensorless aberration estimation for 3D microscopy with deep learning

Estimation of optical aberrations from volumetric intensity images is a key step in sensorless adaptive optics for 3D microscopy. Recent approaches based on deep learning promise accurate results at fast processing speeds. However, collecting ground truth microscopy data for training the network is typically very difficult or even impossible thereby limiting this approach in practice. Here, we demonstrate that neural networks trained only on simulated data yield accurate predictions for real experimental images. We validate our approach on simulated and experimental datasets acquired with two different microscopy modalities, and also compare the results to non-learned methods. Additionally, we study the predictability of individual aberrations with respect to their data requirements and find that the symmetry of the wavefront plays a crucial role. Finally, we make our implementation freely available as open source software in Python.

eess.IV↗

arXiv2019

Active compensation of extrinsic polarization errors using adaptive optics

We present a scheme for active compensation of complex extrinsic polarization perturbations introduced into an optical system. Imaging polarimeter is used to measure the polarization state across a beam profile and a liquid crystal spatial light modulator controls the polarization of the input beam. A sequence of measurements permits determination of the birefringence properties of a perturbing specimen. The necessary correction is calculated and fed back to the polarization modulator to compensate for the polarization perturbation. The system capabilities are demonstrated on a range of birefringent specimens.

physics.optics↗

arXiv2019

Modeling and Performance Analysis in Cache-enabled Millimeter Wave HetNets with Access and Backhaul Integration

Recently, a mmWave-based access and backhaul integration HetNetsarchitecture (mABHetNets) has been envisioned to provide the high link capacity in both access and backhaul links. In mABHetNets, the access link and the backhaul link will share the same mm-wave spectral resources to serve cellular users as well as to maintain the backhaul capacity of the SBSs. However, a large spectrum bandwidth are occupied by the backhaul link to maintain the backhaul link capacity. Up to 50\% mmWave spectrum resource will be used in backhaul link to satisfy the high speed data traffic. Such "\emph{spectrum occupation}" of backhaul link has restricted network performance to achieve a better possible improvement. To overcome spectrum occupation problem in the existing mABHetNets, involving caches in the mABHetNets brings a opportunity. When the backhaul traffic is offloaded by caching popular fils in the SBSs and the corresponding mmWave spectrum can be transfered to the access link by the bandwidth partition. Then with more spectrum for access link, both the network throughout and the coverage ability are increased significantly. However, the caching power consumption will be increased and the EE will be decreased. Therefore, we analyze the througput and EE in mABHetNets by stochastic geometry tool. Extensive numerical and simulations results are carried out to validate our work.

cs.NI↗

arXiv2019

Complex vectorial optics through gradient index lens cascades

Graded index (GRIN) lenses are commonly used for compact imaging systems. It is not widely appreciated that the ion-exchange process that creates the rotationally symmetric GRIN lens index profile also causes a symmetric birefringence variation. This property is usually considered a nuisance, such that manufacturing processes are optimized to keep it to a minimum. Here, rather than avoiding this birefringence, we understand and harness it by using GRIN lenses in cascade with other optical components to enable extra functionality in commonplace GRIN lens systems. We show how birefringence in the GRIN cascades can generate vector vortex beams and foci, and how it can be used advantageously to improve axial resolution. Through using the birefringence for analysis, we show that the GRIN cascades form the basis of a new single-shot Mueller matrix polarimeter with potential for endoscopic label-free cancer diagnostics. The versatility of these cascades opens up new technological directions.

physics.optics↗

arXiv2018

Continuous tensor network renormalization for quantum fields

On the lattice, a renormalization group (RG) flow for two-dimensional partition functions expressed as a tensor network can be obtained using the tensor network renormalization (TNR) algorithm [G. Evenbly, G. Vidal, Phys. Rev. Lett. 115 (18), 180405 (2015)]. In this work we explain how to extend TNR to field theories in the continuum. First, a short-distance length scale $1/Λ$ is introduced in the continuum partition function by smearing the fields. The resulting object is still defined in the continuum but has no fluctuations at distances shorter than $1/Λ$. An infinitesimal coarse-graining step is then generated by the combined action of a $rescaling$ operator $L$ and a $disentangling$ operator $K$ that implements a quasi-local field redefinition. As demonstrated for a free boson in two dimensions, continuous TNR exactly preserves translation and rotation symmetries and can generate a proper RG flow. Moreover, from a critical fixed point of this RG flow one can then extract the conformal data of the underlying conformal field theory.

hep-th↗

arXiv2018

An End-to-end Approach for Handling Unknown Slot Values in Dialogue State Tracking

We highlight a practical yet rarely discussed problem in dialogue state tracking (DST), namely handling unknown slot values. Previous approaches generally assume predefined candidate lists and thus are not designed to output unknown values, especially when the spoken language understanding (SLU) module is absent as in many end-to-end (E2E) systems. We describe in this paper an E2E architecture based on the pointer network (PtrNet) that can effectively extract unknown slot values while still obtains state-of-the-art accuracy on the standard DSTC2 benchmark. We also provide extensive empirical evidence to show that tracking unknown values can be challenging and our approach can bring significant improvement with the help of an effective feature dropout technique.

cs.CL↗