arXiv subjects

Hao Zhang

Publications and source records attributed to Hao Zhang.

At least 613 records · Page 34Linked to original sources

arXiv2024

Cavity dark mode mediated by atom array without atomic scattering loss

We realize a ring cavity strongly interacting with an atom array with configurable spatial structures. By preparing the atom array with a maximized structure factor, we observe the emergence of a cavity dark mode, where the standing-wave nodes are dynamically locked to the positions of the atoms. The dark mode is decoupled from the atoms, protecting the system from dissipation through atomic scattering, but still mediates strong coupling and enables efficient conversion between two optical modes. Moreover, we impart an arbitrary large phase shift on the converted optical fields by translating the atom array. This strongly interacting ring cavity system with single-atom addressability opens ways to quantum optical engineering and the generation of photonic quantum states based on the geometrical structure of atom arrays.

quant-ph↗

arXiv2024

Neuroevolution Neural Architecture Search for Evolving RNNs in Stock Return Prediction and Portfolio Trading

Stock return forecasting is a major component of numerous finance applications. Predicted stock returns can be incorporated into portfolio trading algorithms to make informed buy or sell decisions which can optimize returns. In such portfolio trading applications, the predictive performance of a time series forecasting model is crucial. In this work, we propose the use of the Evolutionary eXploration of Augmenting Memory Models (EXAMM) algorithm to progressively evolve recurrent neural networks (RNNs) for stock return predictions. RNNs are evolved independently for each stocks and portfolio trading decisions are made based on the predicted stock returns. The portfolio used for testing consists of the 30 companies in the Dow-Jones Index (DJI) with each stock have the same weight. Results show that using these evolved RNNs and a simple daily long-short strategy can generate higher returns than both the DJI index and the S&P 500 Index for both 2022 (bear market) and 2023 (bull market).

q-fin.PM↗

arXiv2024

Learning to Generate and Evaluate Fact-checking Explanations with Transformers

In an era increasingly dominated by digital platforms, the spread of misinformation poses a significant challenge, highlighting the need for solutions capable of assessing information veracity. Our research contributes to the field of Explainable Artificial Antelligence (XAI) by developing transformer-based fact-checking models that contextualise and justify their decisions by generating human-accessible explanations. Importantly, we also develop models for automatic evaluation of explanations for fact-checking verdicts across different dimensions such as \texttt{(self)-contradiction}, \texttt{hallucination}, \texttt{convincingness} and \texttt{overall quality}. By introducing human-centred evaluation methods and developing specialised datasets, we emphasise the need for aligning Artificial Intelligence (AI)-generated explanations with human judgements. This approach not only advances theoretical knowledge in XAI but also holds practical implications by enhancing the transparency, reliability and users' trust in AI-driven fact-checking systems. Furthermore, the development of our metric learning models is a first step towards potentially increasing efficiency and reducing reliance on extensive manual assessment. Based on experimental results, our best performing generative model \textsc{ROUGE-1} score of 47.77, demonstrating superior performance in generating fact-checking explanations, particularly when provided with high-quality evidence. Additionally, the best performing metric learning model showed a moderately strong correlation with human judgements on objective dimensions such as \texttt{(self)-contradiction and \texttt{hallucination}, achieving a Matthews Correlation Coefficient (MCC) of around 0.7.}

cs.CL↗

arXiv2024

The Generation of All Regular Rational Orthogonal Matrices

A \emph{rational orthogonal matrix} $Q$ is an orthogonal matrix with rational entries, and $Q$ is called \emph{regular} if each of its row sum equals one, i.e., $Qe = e$ where $e$ is the all-one vector. This paper presents a method for generating all regular rational orthogonal matrices using the classic Cayley transformation. Specifically, we demonstrate that for any regular rational orthogonal matrix $Q$, there exists a permutation matrix $P$ such that $QP$ does not possess an eigenvalue of $-1$. Consequently, $Q$ can be expressed in the form $Q = (I_n + S)^{-1}(I_n - S)P$, where $I_n$ is the identity matrix of order $n$, $S$ is a rational skew-symmetric matrix satisfying $Se = 0$, and $P$ is a permutation matrix. Central to our approach is a pivotal intermediate result, which holds independent interest: given a square matrix $M$, then $MP$ has $-1$ as an eigenvalue for every permutation matrix $P$ if and only if either every row sum of $M$ is $-1$ or every column sum of $M$ is $-1$.

math.CO↗

arXiv2024

AdaMoE: Token-Adaptive Routing with Null Experts for Mixture-of-Experts Language Models

Mixture of experts (MoE) has become the standard for constructing production-level large language models (LLMs) due to its promise to boost model capacity without causing significant overheads. Nevertheless, existing MoE methods usually enforce a constant top-k routing for all tokens, which is arguably restrictive because various tokens (e.g., " " vs. "apple") may require various numbers of experts for feature abstraction. Lifting such a constraint can help make the most of limited resources and unleash the potential of the model for downstream tasks. In this sense, we introduce AdaMoE to realize token-adaptive routing for MoE, where different tokens are permitted to select a various number of experts. AdaMoE makes minimal modifications to the vanilla MoE with top-k routing -- it simply introduces a fixed number of null experts, which do not consume any FLOPs, to the expert set and increases the value of k. AdaMoE does not force each token to occupy a fixed number of null experts but ensures the average usage of the null experts with a load-balancing loss, leading to an adaptive number of null/true experts used by each token. AdaMoE exhibits a strong resemblance to MoEs with expert choice routing while allowing for trivial auto-regressive modeling. AdaMoE is easy to implement and can be effectively applied to pre-trained (MoE-)LLMs. Extensive studies show that AdaMoE can reduce average expert load (FLOPs) while achieving superior performance. For example, on the ARC-C dataset, applying our method to fine-tuning Mixtral-8x7B can reduce FLOPs by 14.5% while increasing accuracy by 1.69%.

cs.AI↗

arXiv2024

Scalable Weibull Graph Attention Autoencoder for Modeling Document Networks

Although existing variational graph autoencoders (VGAEs) have been widely used for modeling and generating graph-structured data, most of them are still not flexible enough to approximate the sparse and skewed latent node representations, especially those of document relational networks (DRNs) with discrete observations. To analyze a collection of interconnected documents, a typical branch of Bayesian models, specifically relational topic models (RTMs), has proven their efficacy in describing both link structures and document contents of DRNs, which motives us to incorporate RTMs with existing VGAEs to alleviate their potential issues when modeling the generation of DRNs. In this paper, moving beyond the sophisticated approximate assumptions of traditional RTMs, we develop a graph Poisson factor analysis (GPFA), which provides analytic conditional posteriors to improve the inference accuracy, and extend GPFA to a multi-stochastic-layer version named graph Poisson gamma belief network (GPGBN) to capture the hierarchical document relationships at multiple semantic levels. Then, taking GPGBN as the decoder, we combine it with various Weibull-based graph inference networks, resulting in two variants of Weibull graph auto-encoder (WGAE), equipped with model inference algorithms. Experimental results demonstrate that our models can extract high-quality hierarchical latent document representations and achieve promising performance on various graph analytic tasks.

cs.LG↗

arXiv2024

A new criterion for oriented graphs to be determined by their generalized skew spectrum

Spectral characterizations of graphs is an important topic in spectral graph theory which has been studied extensively by researchers in recent years. The study of oriented graphs, however, has received less attention so far. In Qiu et al.~\cite{QWW} (Linear Algebra Appl. 622 (2021) 316-332), the authors gave an arithmetic criterion for an oriented graph to be determined by its \emph{generalized skew spectrum} (DGSS for short). More precisely, let $Σ$ be an $n$-vertex oriented graph with skew adjacency matrix $S$ and $W(Σ)=[e,Se,\ldots,S^{n-1}e]$ be the \emph{walk-matrix} of $Σ$, where $e$ is the all-one vector. A theorem of Qiu et al.~\cite{QWW} shows that a self-converse oriented graph $Σ$ is DGSS, provided that the Smith normal form of $W(Σ)$ is ${\rm diag}(1,\ldots,1,2,\ldots,2,2d)$, where $d$ is an odd and square-free integer and the number of $1$'s appeared in the diagonal is precisely $\lceil \frac{n}{2}\rceil$. In this paper, we show that the above square-freeness assumptions on $d$ can actually be removed, which significantly improves upon the above theorem. Our new ingredient is a key intermediate result, which is of independent interest: for a self-converse oriented graphs $Σ$ and an odd prime $p$, if the rank of $W(Σ)$ is $n-1$ over $\mathbb{F}_p$, then the kernel of $W(Σ)^{\rm T}$ over $\mathbb{F}_p$ is \emph{anisotropic}, i.e., $v^{\rm T}v\neq 0$ for any $0\ne v\in{{\rm ker}\,W(Σ)^{\rm T}}$ over $\mathbb{F}_p$.

math.CO↗

arXiv2024

Annihilating polynomial, Jordan canonical from, and generalized spectral characterizations of Eulerian graphs

Let $G$ be an Eulerian graph on $n$ vertices with adjacency matrix $A$ and characteristic polynomial $ϕ(x)$. We show that when $n$ is even (resp. odd), the square-root of $ϕ(x)$ (resp. $xϕ(x)$) is an annihilating polynomial of $A$, over $\mathbb{F}_2$. The result was achieved by applying the Jordan canonical form of $A$ over the algebraic closure $\bar{\mathbb{F}}_2$. Based on this, we show a family of Eulerian graphs are determined by their generalized spectrum among all Eulerian graphs, which significantly simplifies and strengthens the previous result.

math.CO↗

arXiv2024

UnSeg: One Universal Unlearnable Example Generator is Enough against All Image Segmentation

Image segmentation is a crucial vision task that groups pixels within an image into semantically meaningful segments, which is pivotal in obtaining a fine-grained understanding of real-world scenes. However, an increasing privacy concern exists regarding training large-scale image segmentation models on unauthorized private data. In this work, we exploit the concept of unlearnable examples to make images unusable to model training by generating and adding unlearnable noise into the original images. Particularly, we propose a novel Unlearnable Segmentation (UnSeg) framework to train a universal unlearnable noise generator that is capable of transforming any downstream images into their unlearnable version. The unlearnable noise generator is finetuned from the Segment Anything Model (SAM) via bilevel optimization on an interactive segmentation dataset towards minimizing the training error of a surrogate model that shares the same architecture with SAM but is trained from scratch. We empirically verify the effectiveness of UnSeg across 6 mainstream image segmentation tasks, 10 widely used datasets, and 7 different network architectures, and show that the unlearnable images can reduce the segmentation performance by a large margin. Our work provides useful insights into how to leverage foundation models in a data-efficient and computationally affordable manner to protect images against image segmentation models.

cs.CV↗

arXiv2024

GALA: Geometry-Aware Local Adaptive Grids for Detailed 3D Generation

We propose GALA, a novel representation of 3D shapes that (i) excels at capturing and reproducing complex geometry and surface details, (ii) is computationally efficient, and (iii) lends itself to 3D generative modelling with modern, diffusion-based schemes. The key idea of GALA is to exploit both the global sparsity of surfaces within a 3D volume and their local surface properties. Sparsity is promoted by covering only the 3D object boundaries, not empty space, with an ensemble of tree root voxels. Each voxel contains an octree to further limit storage and compute to regions that contain surfaces. Adaptivity is achieved by fitting one local and geometry-aware coordinate frame in each non-empty leaf node. Adjusting the orientation of the local grid, as well as the anisotropic scales of its axes, to the local surface shape greatly increases the amount of detail that can be stored in a given amount of memory, which in turn allows for quantization without loss of quality. With our optimized C++/CUDA implementation, GALA can be fitted to an object in less than 10 seconds. Moreover, the representation can efficiently be flattened and manipulated with transformer networks. We provide a cascaded generation pipeline capable of generating 3D shapes with great geometric detail.

cs.CV↗

arXiv2024

Training Overhead Ratio: A Practical Reliability Metric for Large Language Model Training Systems

Large Language Models (LLMs) are revolutionizing the AI industry with their superior capabilities. Training these models requires large-scale GPU clusters and significant computing time, leading to frequent failures that significantly increase training costs. Despite its significance, this field lacks a metric for evaluating reliability. In this work, we introduce a novel reliability metric called \emph{Training Overhead Ratio} (TOR) to evaluate the reliability of fault-tolerant LLM training systems. TOR is defined as the ratio of optimal training time to the observed training time of a system, serving as a practical tool for users to estimate the actual time required to train an LLM on a given system. Furthermore, our investigation identifies the key factor for enhancing reliability and present TOR equations for various types of failures encountered in practice.

cs.DC↗

arXiv2024

Learning Recommender Systems with Soft Target: A Decoupled Perspective

Learning recommender systems with multi-class optimization objective is a prevalent setting in recommendation. However, as observed user feedback often accounts for a tiny fraction of the entire item pool, the standard Softmax loss tends to ignore the difference between potential positive feedback and truly negative feedback. To address this challenge, we propose a novel decoupled soft label optimization framework to consider the objectives as two aspects by leveraging soft labels, including target confidence and the latent interest distribution of non-target items. Futhermore, based on our carefully theoretical analysis, we design a decoupled loss function to flexibly adjust the importance of these two aspects. To maximize the performance of the proposed method, we additionally present a sensible soft-label generation algorithm that models a label propagation algorithm to explore users' latent interests in unobserved feedback via neighbors. We conduct extensive experiments on various recommendation system models and public datasets, the results demonstrate the effectiveness and generality of the proposed method.

cs.IR↗

arXiv2024

Towards Natural Image Matting in the Wild via Real-Scenario Prior

Recent approaches attempt to adapt powerful interactive segmentation models, such as SAM, to interactive matting and fine-tune the models based on synthetic matting datasets. However, models trained on synthetic data fail to generalize to complex and occlusion scenes. We address this challenge by proposing a new matting dataset based on the COCO dataset, namely COCO-Matting. Specifically, the construction of our COCO-Matting includes accessory fusion and mask-to-matte, which selects real-world complex images from COCO and converts semantic segmentation masks to matting labels. The built COCO-Matting comprises an extensive collection of 38,251 human instance-level alpha mattes in complex natural scenarios. Furthermore, existing SAM-based matting methods extract intermediate features and masks from a frozen SAM and only train a lightweight matting decoder by end-to-end matting losses, which do not fully exploit the potential of the pre-trained SAM. Thus, we propose SEMat which revamps the network architecture and training objectives. For network architecture, the proposed feature-aligned transformer learns to extract fine-grained edge and transparency features. The proposed matte-aligned decoder aims to segment matting-specific objects and convert coarse masks into high-precision mattes. For training objectives, the proposed regularization and trimap loss aim to retain the prior from the pre-trained model and push the matting logits extracted from the mask decoder to contain trimap-based semantic information. Extensive experiments across seven diverse datasets demonstrate the superior performance of our method, proving its efficacy in interactive natural image matting. We open-source our code, models, and dataset at https://github.com/XiaRho/SEMat.

cs.CV↗

arXiv2024

Coulomb blockade in open superconducting islands on InAs nanowires

Electrons in closed systems can exhibit Coulomb blockade (CB) oscillations due to charge quantization. Here, we report CB oscillations in aluminum superconducting islands on InAs nanowires in the open regime. The Al island is connected to the source/drain leads through two contacts: One is fully transmitting while the other is tuned into the tunneling regime. This device configuration is typical for tunneling spectroscopy where charging energy is generally considered negligible. The oscillation periods are 2$e$ or 1$e$, depending on the gate settings. A magnetic field can induce the 2$e$ to 1$e$ transition. Our result is reminiscent of the "mesoscopic Coulomb blockade" in open quantum dots caused by electron interference.

cond-mat.mes-hall↗

arXiv2024

Electric field driven spin textures in heavy fermion van der Waals magnets

The recently discovered van der Waals material CeSiI exhibits both heavy fermion behavior and spiral order with strong magnetic anisotropy which makes it a potential host for topological spin textures such as skyrmions through electrical gating. A monolayer of CeSiI consists of two layers of Ce atoms on triangular lattices that sandwich a silicene layer. Motivated by the experiments, we explore magnetic phase diagram in van der Waals heavy fermion materials as a function of anisotropy and applied magnetic field using an effective spin model. We demonstrate that application of an external electric field can tune the Kondo coupling on each Ce layer differently, in turn allowing for controlling the intra- and interlayer magnetic couplings. Our analysis indicates that this fine-tuning leads to the coexistence of different magnetic orders in a single monolayer. In particular, we show that a novel vortex phase can be stabilized only in the presence of an external electric field. Our results highlight the unique advantages and the tunability of van der Waals heavy fermion materials for manipulation of chiral magnetic phases.

cond-mat.mes-hall↗

arXiv2024

MFE-ETP: A Comprehensive Evaluation Benchmark for Multi-modal Foundation Models on Embodied Task Planning

In recent years, Multi-modal Foundation Models (MFMs) and Embodied Artificial Intelligence (EAI) have been advancing side by side at an unprecedented pace. The integration of the two has garnered significant attention from the AI research community. In this work, we attempt to provide an in-depth and comprehensive evaluation of the performance of MFM s on embodied task planning, aiming to shed light on their capabilities and limitations in this domain. To this end, based on the characteristics of embodied task planning, we first develop a systematic evaluation framework, which encapsulates four crucial capabilities of MFMs: object understanding, spatio-temporal perception, task understanding, and embodied reasoning. Following this, we propose a new benchmark, named MFE-ETP, characterized its complex and variable task scenarios, typical yet diverse task types, task instances of varying difficulties, and rich test case types ranging from multiple embodied question answering to embodied task reasoning. Finally, we offer a simple and easy-to-use automatic evaluation platform that enables the automated testing of multiple MFMs on the proposed benchmark. Using the benchmark and evaluation platform, we evaluated several state-of-the-art MFMs and found that they significantly lag behind human-level performance. The MFE-ETP is a high-quality, large-scale, and challenging benchmark relevant to real-world tasks.

cs.AI↗

arXiv2024

CtrlA: Adaptive Retrieval-Augmented Generation via Inherent Control

Retrieval-augmented generation (RAG) has emerged as a promising solution for mitigating hallucinations of large language models (LLMs) with retrieved external knowledge. Adaptive RAG enhances this approach by enabling dynamic retrieval during generation, activating retrieval only when the query exceeds LLM's internal knowledge. Existing methods primarily focus on detecting LLM's confidence via statistical uncertainty. Instead, we present the first attempts to solve adaptive RAG from a representation perspective and develop an inherent control-based framework, termed \name. Specifically, we extract the features that represent the honesty and confidence directions of LLM and adopt them to control LLM behavior and guide retrieval timing decisions. We also design a simple yet effective query formulation strategy to support adaptive retrieval. Experiments show that \name is superior to existing adaptive RAG methods on a diverse set of tasks, the honesty steering can effectively make LLMs more honest and confidence monitoring is a promising indicator of retrieval trigger.Our code is available at \url{https://github.com/HSLiu-Initial/CtrlA}.

cs.CL↗

arXiv2024

Retrieval-Oriented Knowledge for Click-Through Rate Prediction

Click-through rate (CTR) prediction is crucial for personalized online services. Sample-level retrieval-based models, such as RIM, have demonstrated remarkable performance. However, they face challenges including inference inefficiency and high resource consumption due to the retrieval process, which hinder their practical application in industrial settings. To address this, we propose a universal plug-and-play \underline{r}etrieval-\underline{o}riented \underline{k}nowledge (\textbf{\name}) framework that bypasses the real retrieval process. The framework features a knowledge base that preserves and imitates the retrieved \& aggregated representations using a decomposition-reconstruction paradigm. Knowledge distillation and contrastive learning optimize the knowledge base, enabling the integration of retrieval-enhanced representations with various CTR models. Experiments on three large-scale datasets demonstrate \name's exceptional compatibility and performance, with the neural knowledge base serving as an effective surrogate for the retrieval pool. \name surpasses the teacher model while maintaining superior inference efficiency and demonstrates the feasibility of distilling knowledge from non-parametric methods using a parametric approach. These results highlight \name's strong potential for real-world applications and its ability to transform retrieval-based methods into practical solutions. Our implementation code is available to support reproducibility in \url{https://github.com/HSLiu-Initial/ROK.git}.

cs.IR↗