Searcharxiv⌕ Search

arXiv subjects

Huan Li

Publications and source records attributed to Huan Li.

At least 91 records · Page 5Linked to original sources

Preventing the Popular Item Embedding Based Attack in Federated Recommendations

Privacy concerns have led to the rise of federated recommender systems (FRS), which can create personalized models across distributed clients. However, FRS is vulnerable to poisoning attacks, where malicious users manipulate gradients to promote their target items intentionally. Existing attacks against FRS have limitations, as they depend on specific models and prior knowledge, restricting their real-world applicability. In our exploration of practical FRS vulnerabilities, we devise a model-agnostic and prior-knowledge-free attack, named PIECK (Popular Item Embedding based Attack). The core module of PIECK is popular item mining, which leverages embedding changes during FRS training to effectively identify the popular items. Built upon the core module, PIECK branches into two diverse solutions: The PIECKIPE solution employs an item popularity enhancement module, which aligns the embeddings of targeted items with the mined popular items to increase item exposure. The PIECKUEA further enhances the robustness of the attack by using a user embedding approximation module, which approximates private user embeddings using mined popular items. Upon identifying PIECK, we evaluate existing federated defense methods and find them ineffective against PIECK, as poisonous gradients inevitably overwhelm the cold target items. We then propose a novel defense method by introducing two regularization terms during user training, which constrain item popularity enhancement and user embedding approximation while preserving FRS performance. We evaluate PIECK and its defense across two base models, three real datasets, four top-tier attacks, and six general defense methods, affirming the efficacy of both PIECK and its defense.

cs.CR↗

KcMF: A Knowledge-compliant Framework for Schema and Entity Matching with Fine-tuning-free LLMs

Schema matching (SM) and entity matching (EM) tasks are crucial for data integration. While large language models (LLMs) have shown promising results in these tasks, they suffer from hallucinations and confusion about task instructions. This study presents the Knowledge-Compliant Matching Framework (KcMF), an LLM-based approach that addresses these issues without the need for domain-specific fine-tuning. KcMF employs a once-and-for-all pseudo-code-based task decomposition strategy to adopt natural language statements that guide LLM reasoning and reduce confusion across various task types. We also propose two mechanisms, Dataset as Knowledge (DaK) and Example as Knowledge (EaK), to build domain knowledge sets when unstructured domain knowledge is lacking. Moreover, we introduce a result-ensemble strategy to leverage multiple knowledge sources and suppress badly formatted outputs. Extensive evaluations confirm that KcMF clearly enhances five LLM backbones in both SM and EM tasks while outperforming the non-LLM competitors by an average F1-score of 17.93%.

cs.CL↗

Near-optimal Linear Sketches and Fully-Dynamic Algorithms for Hypergraph Spectral Sparsification

A hypergraph spectral sparsifier of a hypergraph $G$ is a weighted subgraph $H$ that approximates the Laplacian of $G$ to a specified precision. Recent work has shown that similar to ordinary graphs, there exist $\widetilde{O}(n)$-size hypergraph spectral sparsifiers. However, the task of computing such sparsifiers turns out to be much more involved, and all known algorithms rely on the notion of balanced weight assignments, whose computation inherently relies on repeated, complete access to the underlying hypergraph. We introduce a significantly simpler framework for hypergraph spectral sparsification which bypasses the need to compute such weight assignments, essentially reducing hypergraph sparsification to repeated effective resistance sampling in \textit{ordinary graphs}, which are obtained by \textit{oblivious vertex-sampling} of the original hypergraph. Our framework immediately yields a simple, new nearly-linear time algorithm for nearly-linear size spectral hypergraph sparsification. Furthermore, as a direct consequence of our framework, we obtain the first nearly-optimal algorithms in several other models of computation, namely the linear sketching, fully dynamic, and online settings.

cs.DS↗

Mixture of neural fields for heterogeneous reconstruction in cryo-EM

Cryo-electron microscopy (cryo-EM) is an experimental technique for protein structure determination that images an ensemble of macromolecules in near-physiological contexts. While recent advances enable the reconstruction of dynamic conformations of a single biomolecular complex, current methods do not adequately model samples with mixed conformational and compositional heterogeneity. In particular, datasets containing mixtures of multiple proteins require the joint inference of structure, pose, compositional class, and conformational states for 3D reconstruction. Here, we present Hydra, an approach that models both conformational and compositional heterogeneity fully ab initio by parameterizing structures as arising from one of K neural fields. We employ a new likelihood-based loss function and demonstrate the effectiveness of our approach on synthetic datasets composed of mixtures of proteins with large degrees of conformational variability. We additionally demonstrate Hydra on an experimental dataset of a cellular lysate containing a mixture of different protein complexes. Hydra expands the expressivity of heterogeneous reconstruction methods and thus broadens the scope of cryo-EM to increasingly complex samples.

cs.LG↗

Silicon photonic MEMS switches based on split waveguide crossings

The continuous push for high-performance photonic switches is one of the most crucial premises for the sustainable scaling of programmable and reconfigurable photonic circuits for a wide spectrum of applications. Conventional optical switches rely on the perturbative mechanisms of mode coupling or mode interference, resulting in inherent bottlenecks in their switching performance concerning size, power consumption and bandwidth. Here we propose and realize a silicon photonic 2x2 elementary switch based on a split waveguide crossing (SWX) consisting of two halves. The propagation direction of the incident light is manipulated to implement the OFF/ON states by splitting/combining the two halves of the SWX, showing excellent performance with low excess loss and low crosstalk over an ultrawide bandwidth. Both elementary switch and a 64x64 switch array based on Benes topology are fabricated and characterized, demonstrating great potential for practical scenarios such as photonic interconnect/routing, Lidar and spectroscopy, photonic computing, as well as microwave photonics.

physics.optics↗

Accelerated Gradient Tracking over Time-varying Graphs for Decentralized Optimization

Decentralized optimization over time-varying graphs has been increasingly common in modern machine learning with massive data stored on millions of mobile devices, such as in federated learning. This paper revisits the widely used accelerated gradient tracking and extends it to time-varying graphs. We prove that the practical single loop accelerated gradient tracking needs $O((\fracγ{1-σ_γ})^2\sqrt{\frac{L}ε})$ and $O((\fracγ{1-σ_γ})^{1.5}\sqrt{\frac{L}μ}\log\frac{1}ε)$ iterations to reach an $ε$-optimal solution over time-varying graphs when the problems are nonstrongly convex and strongly convex, respectively, where $γ$ and $σ_γ$ are two common constants charactering the network connectivity, $L$ and $μ$ are the smoothness and strong convexity constants, respectively, and one iteration corresponds to one gradient oracle call and one communication round. Our convergence rates improve significantly over the ones of $O(\frac{1}{ε^{5/7}})$ and $O((\frac{L}μ)^{5/7}\frac{1}{(1-σ)^{1.5}}\log\frac{1}ε)$, respectively, which were proved in the original literature of accelerated gradient tracking only for static graphs, where $\fracγ{1-σ_γ}$ equals $\frac{1}{1-σ}$ when the network is time-invariant. When combining with a multiple consensus subroutine, the dependence on the network connectivity constants can be further improved to $O(1)$ and $O(\fracγ{1-σ_γ})$ for the gradient oracle and communication round complexities, respectively. When the network is static, by employing the Chebyshev acceleration, our complexities exactly match the lower bounds without hiding any poly-logarithmic factor for both nonstrongly convex and strongly convex problems.

math.OC↗

Adan: Adaptive Nesterov Momentum Algorithm for Faster Optimizing Deep Models

In deep learning, different kinds of deep networks typically need different optimizers, which have to be chosen after multiple trials, making the training process inefficient. To relieve this issue and consistently improve the model training speed across deep networks, we propose the ADAptive Nesterov momentum algorithm, Adan for short. Adan first reformulates the vanilla Nesterov acceleration to develop a new Nesterov momentum estimation (NME) method, which avoids the extra overhead of computing gradient at the extrapolation point. Then, Adan adopts NME to estimate the gradient's first- and second-order moments in adaptive gradient algorithms for convergence acceleration. Besides, we prove that Adan finds an $ε$-approximate first-order stationary point within $\mathcal{O}(ε^{-3.5})$ stochastic gradient complexity on the non-convex stochastic problems (e.g., deep learning problems), matching the best-known lower bound. Extensive experimental results show that Adan consistently surpasses the corresponding SoTA optimizers on vision, language, and RL tasks and sets new SoTAs for many popular networks and frameworks, e.g., ResNet, ConvNext, ViT, Swin, MAE, DETR, GPT-2, Transformer-XL, and BERT. More surprisingly, Adan can use half of the training cost (epochs) of SoTA optimizers to achieve higher or comparable performance on ViT, GPT-2, MAE, etc., and also shows great tolerance to a large range of minibatch size, e.g., from 1k to 32k. Code is released at https://github.com/sail-sg/Adan, and has been used in multiple popular deep learning frameworks or projects.

cs.LG↗

Detecting Low-Degree Truncation

We consider the following basic, and very broad, statistical problem: Given a known high-dimensional distribution ${\cal D}$ over $\mathbb{R}^n$ and a collection of data points in $\mathbb{R}^n$, distinguish between the two possibilities that (i) the data was drawn from ${\cal D}$, versus (ii) the data was drawn from ${\cal D}|_S$, i.e. from ${\cal D}$ subject to truncation by an unknown truncation set $S \subseteq \mathbb{R}^n$. We study this problem in the setting where ${\cal D}$ is a high-dimensional i.i.d. product distribution and $S$ is an unknown degree-$d$ polynomial threshold function (one of the most well-studied types of Boolean-valued function over $\mathbb{R}^n$). Our main results are an efficient algorithm when ${\cal D}$ is a hypercontractive distribution, and a matching lower bound: $\bullet$ For any constant $d$, we give a polynomial-time algorithm which successfully distinguishes ${\cal D}$ from ${\cal D}|_S$ using $O(n^{d/2})$ samples (subject to mild technical conditions on ${\cal D}$ and $S$); $\bullet$ Even for the simplest case of ${\cal D}$ being the uniform distribution over $\{+1, -1\}^n$, we show that for any constant $d$, any distinguishing algorithm for degree-$d$ polynomial threshold functions must use $Ω(n^{d/2})$ samples.

cs.CC↗

PZT Optical Memristors

Optical memristors represent a monumental leap in the fusion of photonics and electronics, heralding a new era of applications from neuromorphic computing to artificial intelligence. However, current technologies are hindered by complex fabrication, limited endurance, high optical loss or low modulation depth. For the first time, we reveal optical non-volatility in thin-film Lead Zirconate Titanate (PZT) by electrically manipulating the ferroelectric domains to control the refractive index, providing a brand-new routine for optical memristors. The developed PZT optical memristors offer unprecedented advantages more than exceptional performance metrics like low loss of <2 dB/cm, high precision exceeding 6-bits, large modulation depth with an index change as large as 4.6x10-3. Additionally, these devices offer impressive stability, maintaining minimal wavelength variation for over three weeks and enduring more than 10,000 cycles, and require a mere 0.8 pJ of energy for non-volatile operation. The wafer-scale sol-gel fabrication process also ensures compatible with standardized mass fabrication processes and high scalability for photonic integration. Specially, these devices also demonstrate unique functional duality: setting above a threshold voltage enables non-volatile behaviors, below this threshold allows volatile high-speed optical modulation. This marks the first-ever optical memristor capable of performing high-speed (48 Gbps) and energy-efficient (450 fJ/bit) signal processing and non-volatile retention on a single platform, and is also the inaugural demonstration of scalable functional systems. The PZT optical memristors developed here facilitate the realization of novel paradigms for high-speed and energy-efficient optical interconnects, programmable PICs, quantum computing, neural networks, in-memory computing and brain-like architecture.

physics.optics↗

Revisiting CNNs for Trajectory Similarity Learning

Similarity search is a fundamental but expensive operator in querying trajectory data, due to its quadratic complexity of distance computation. To mitigate the computational burden for long trajectories, neural networks have been widely employed for similarity learning and each trajectory is encoded as a high-dimensional vector for similarity search with linear complexity. Given the sequential nature of trajectory data, previous efforts have been primarily devoted to the utilization of RNNs or Transformers. In this paper, we argue that the common practice of treating trajectory as sequential data results in excessive attention to capturing long-term global dependency between two sequences. Instead, our investigation reveals the pivotal role of local similarity, prompting a revisit of simple CNNs for trajectory similarity learning. We introduce ConvTraj, incorporating both 1D and 2D convolutions to capture sequential and geo-distribution features of trajectories, respectively. In addition, we conduct a series of theoretical analyses to justify the effectiveness of ConvTraj. Experimental results on four real-world large-scale datasets demonstrate that ConvTraj achieves state-of-the-art accuracy in trajectory similarity search. Owing to the simple network structure of ConvTraj, the training and inference speed on the Porto dataset with 1.6 million trajectories are increased by at least $240$x and $2.16$x, respectively. The source code and dataset can be found at \textit{\url{https://github.com/Proudc/ConvTraj}}.

cs.AI↗

Modeling and Monitoring of Indoor Populations using Sparse Positioning Data (Extension)

In large venues like shopping malls and airports, knowledge on the indoor populations fuels applications such as business analytics, venue management, and safety control. In this work, we provide means of modeling populations in partitions of indoor space offline and of monitoring indoor populations continuously, by using indoor positioning data. However, the low-sampling rates of indoor positioning render the data temporally and spatially sparse, which in turn renders the offline capture of indoor populations challenging. It is even more challenging to continuously monitor indoor populations, as positioning data may be missing or not ready yet at the current moment. To address these challenges, we first enable probabilistic modeling of populations in indoor space partitions as Normal distributions. Based on that, we propose two learning-based estimators for on-the-fly prediction of population distributions. Leveraging the prediction-based schemes, we provide a unified continuous query processing framework for a type of query that enables continuous monitoring of populated partitions. The framework encompasses caching and result validity mechanisms to reduce cost and maintain monitoring effectiveness. Extensive experiments on two real data sets show that the proposed estimators are able to outperform the state-of-the-art alternatives and that the query processing framework is effective and efficient.

cs.DB↗

Fast Outflow in the Host Galaxy of the Luminous z $=$ 7.5 Quasar J1007$+$2115

James Webb Space Telescope opens a new window to directly probe luminous quasars powered by billion solar mass black holes in the epoch of reionization and their co-evolution with massive galaxies with unprecedented details. In this paper, we report the first results from the deep NIRSpec integral field spectroscopy study of a quasar at $z = 7.5$. We obtain a bolometric luminosity of $\sim$$1.8\times10^{47}$ erg s$^{-1}$ and a black hole mass of $\sim$0.7--2.5$\times10^{9}$ M$_{\odot}$ based on H$β$ emission line from the quasar spectrum. We discover $\sim$2 kpc scale, highly blueshifted ($\sim$$-$870 km/s) and broad ($\sim$1400 km/s) [O III] line emission after the quasar PSF has been subtracted. Such line emission most likely originates from a fast, quasar-driven outflow, the earliest one on galactic-scale known so far. The dynamical properties of this outflow fall within the typical ranges of quasar-driven outflows at lower redshift, and the outflow may be fast enough to reach the circumgalactic medium. Combining both the extended and nuclear outflow together, the mass outflow rate, $\sim$300 M$_{\odot}$yr, is $\sim$60%--380% of the star formation rate of the quasar host galaxy, suggesting that the outflow may expel a significant amount of gas from the inner region of the galaxy. The kinetic energy outflow rate, $\sim$3.6$\times10^{44}$ erg s$^{-1}$, is $\sim$0.2% of the quasar bolometric luminosity, which is comparable to the minimum value required for negative feedback based on simulation predictions. The dynamical timescale of the extended outflow is $\sim$1.7 Myr, consistent with the typical quasar lifetime in this era.

astro-ph.GA↗

Nonvolatile Silicon Photonic MEMS Switch Based on Centrally-Clamped Stepped Bistable Mechanical Beams

High-performance photonic switches are essential for large-scale optical routing for AI large models and Internet of things. Realizing nonvolatility can further reduce power consumption and expand application scenarios. We propose a nonvolatile 2*2 silicon photonic micro-electromechanical system (MEMS) switch compatible with standard silicon photonic foundry processes. The switch employs electrostatic comb actuator to change the air gap of the compact horizontal adiabatic coupler and achieves nonvolatility with centrally-clamped stepped bistable mechanical beams. The photonic switch features a 10s us-scale switching speed and a 10s fJ-scale simulated switching energy within a 100*100 um2 footprint, with <=12 V driving voltages. This 2*2 switch can be used in a variety of topologies for large-scale photonic switches, and its nonvolatility can potentially support future photonic FPGA designs.

physics.app-ph↗

An End-to-End Model for Time Series Classification In the Presence of Missing Values

Time series classification with missing data is a prevalent issue in time series analysis, as temporal data often contain missing values in practical applications. The traditional two-stage approach, which handles imputation and classification separately, can result in sub-optimal performance as label information is not utilized in the imputation process. On the other hand, a one-stage approach can learn features under missing information, but feature representation is limited as imputed errors are propagated in the classification process. To overcome these challenges, this study proposes an end-to-end neural network that unifies data imputation and representation learning within a single framework, allowing the imputation process to take advantage of label information. Differing from previous methods, our approach places less emphasis on the accuracy of imputation data and instead prioritizes classification performance. A specifically designed multi-scale feature learning module is implemented to extract useful information from the noise-imputation data. The proposed model is evaluated on 68 univariate time series datasets from the UCR archive, as well as a multivariate time series dataset with various missing data ratios and 4 real-world datasets with missing information. The results indicate that the proposed model outperforms state-of-the-art approaches for incomplete time series classification, particularly in scenarios with high levels of missing data.

cs.LG↗

Tabular Data Augmentation for Machine Learning: Progress and Prospects of Embracing Generative AI

Machine learning (ML) on tabular data is ubiquitous, yet obtaining abundant high-quality tabular data for model training remains a significant obstacle. Numerous works have focused on tabular data augmentation (TDA) to enhance the original table with additional data, thereby improving downstream ML tasks. Recently, there has been a growing interest in leveraging the capabilities of generative AI for TDA. Therefore, we believe it is time to provide a comprehensive review of the progress and future prospects of TDA, with a particular emphasis on the trending generative AI. Specifically, we present an architectural view of the TDA pipeline, comprising three main procedures: pre-augmentation, augmentation, and post-augmentation. Pre-augmentation encompasses preparation tasks that facilitate subsequent TDA, including error handling, table annotation, table simplification, table representation, table indexing, table navigation, schema matching, and entity matching. Augmentation systematically analyzes current TDA methods, categorized into retrieval-based methods, which retrieve external data, and generation-based methods, which generate synthetic data. We further subdivide these methods based on the granularity of the augmentation process at the row, column, cell, and table levels. Post-augmentation focuses on the datasets, evaluation and optimization aspects of TDA. We also summarize current trends and future directions for TDA, highlighting promising opportunities in the era of generative AI. In addition, the accompanying papers and related resources are continuously updated and maintained in the GitHub repository at https://github.com/SuDIS-ZJU/awesome-tabular-data-augmentation to reflect ongoing advancements in the field.

cs.LG↗

E2USD: Efficient-yet-effective Unsupervised State Detection for Multivariate Time Series

Cyber-physical system sensors emit multivariate time series (MTS) that monitor physical system processes. Such time series generally capture unknown numbers of states, each with a different duration, that correspond to specific conditions, e.g., "walking" or "running" in human-activity monitoring. Unsupervised identification of such states facilitates storage and processing in subsequent data analyses, as well as enhances result interpretability. Existing state-detection proposals face three challenges. First, they introduce substantial computational overhead, rendering them impractical in resourceconstrained or streaming settings. Second, although state-of-the-art (SOTA) proposals employ contrastive learning for representation, insufficient attention to false negatives hampers model convergence and accuracy. Third, SOTA proposals predominantly only emphasize offline non-streaming deployment, we highlight an urgent need to optimize online streaming scenarios. We propose E2Usd that enables efficient-yet-accurate unsupervised MTS state detection. E2Usd exploits a Fast Fourier Transform-based Time Series Compressor (fftCompress) and a Decomposed Dual-view Embedding Module (ddEM) that together encode input MTSs at low computational overhead. Additionally, we propose a False Negative Cancellation Contrastive Learning method (fnccLearning) to counteract the effects of false negatives and to achieve more cluster-friendly embedding spaces. To reduce computational overhead further in streaming settings, we introduce Adaptive Threshold Detection (adaTD). Comprehensive experiments with six baselines and six datasets offer evidence that E2Usd is capable of SOTA accuracy at significantly reduced computational overhead.

cs.LG↗

Draft & Verify: Lossless Large Language Model Acceleration via Self-Speculative Decoding

We present a novel inference scheme, self-speculative decoding, for accelerating Large Language Models (LLMs) without the need for an auxiliary model. This approach is characterized by a two-stage process: drafting and verification. The drafting stage generates draft tokens at a slightly lower quality but more quickly, which is achieved by selectively skipping certain intermediate layers during drafting. Subsequently, the verification stage employs the original LLM to validate those draft output tokens in one forward pass. This process ensures the final output remains identical to that produced by the unaltered LLM. Moreover, the proposed method requires no additional neural network training and no extra memory footprint, making it a plug-and-play and cost-effective solution for inference acceleration. Benchmarks with LLaMA-2 and its variants demonstrated a speedup up to 1.99$\times$.

cs.CL↗

Hybrid thin-film lithium niobate micro-ring acousto-optic modulator for microwave-to-optical conversion

Highly efficient acousto-optic modulation plays a vital role in the microwave-to-optical conversion. Herein, we demonstrate a hybrid thin-film lithium niobate (TFLN) racetrack micro-ring acousto-optic modulator (AOM) implemented with low-loss chalcogenide (ChG) waveguide. By engineering the electrode configuration of the interdigital transducer, the double-arm micro-ring acousto-optic modulation is experimentally confirmed in nonsuspended ChG loaded TFLN waveguide platform. Varying the position of blue-detuned bias point, the half-wave-voltage-length product VpaiL of the hybrid TFLN micro-ring AOM is as small as 9 mVcm. Accordingly, the acousto-optic coupling strength is estimated to be 0.48 Hz s1/2 at acoustic frequency of 0.84 GHz. By analyzing the generation of phonon number from the piezoelectric transducer, the microwave-to-optical conversion efficiency is calculated to be 0.05%, approximately one order of magnitude larger than that of the state-of-the-art suspended counterpart. Efficient microwave-to-optical conversion thus provides new opportunities for low-power-consumption quantum information transduction using the TFLN-ChG hybrid piezo-optomechanical devices.

physics.optics↗