arXiv subjects

Jie Wang

Publications and source records attributed to Jie Wang.

At least 289 records · Page 16Linked to original sources

arXiv2024

On the Cosmic Variance of the Merger Rate Density of Binary Neutron Stars

The cosmic variance on the star formation history may lead to bias to the merger rate density estimation of binary neutron star (BNS) mergers by the compact binary population synthesis. In this paper, we take the advantage of the large boxsize of the Millennium Simulation combined with the semi-analytic galaxy formation model GABE, and the parameterized population binary star evolution (BSE) model to examine how much effect will the cosmic variance introduce on the estimation of merger rate density of BNS mergers. We find that for sub-box size of $100\rm Mpc$ and $200\rm Mpc$, the variance of merger rate density $σ_{\rm R}/\rm R$ at different redshift is about $23\%-35\%$ and $13\%-20\%$ respectively. On one hand, as for the variance of the detection rate on BNS mergers with current LIGO-Virgo-KAGRA (LVK) detector network, this value is very small $\lesssim 10\%$, which indicates ignoring the cosmic variance is reasonable for estimating the merger rate density from current LVK observation. On the other hand, with next-generation gravitational wave detectors, it is possible to localize BNS mergers within sub-boxes possessing length of $\rm 40 Mpc$ for source redshift $z_{s}<0.2$. In such a small box, the cosmic variance of the merger rate density is significant, i.e., the value of $σ_{\rm R}/\rm R$ is about $\sim 55\%$. This hints that estimating the merger rate density of BNS in different sky areas may provide useful information on the cosmic variance.

astro-ph.HE↗

arXiv2024

Regularization for Adversarial Robust Learning

Despite the growing prevalence of artificial neural networks in real-world applications, their vulnerability to adversarial attacks remains a significant concern, which motivates us to investigate the robustness of machine learning models. While various heuristics aim to optimize the distributionally robust risk using the $\infty$-Wasserstein metric, such a notion of robustness frequently encounters computation intractability. To tackle the computational challenge, we develop a novel approach to adversarial training that integrates $ϕ$-divergence regularization into the distributionally robust risk function. This regularization brings a notable improvement in computation compared with the original formulation. We develop stochastic gradient methods with biased oracles to solve this problem efficiently, achieving the near-optimal sample complexity. Moreover, we establish its regularization effects and demonstrate it is asymptotic equivalence to a regularized empirical risk minimization framework, by considering various scaling regimes of the regularization parameter and robustness level. These regimes yield gradient norm regularization, variance regularization, or a smoothed gradient norm regularization that interpolates between these extremes. We numerically validate our proposed method in supervised learning, reinforcement learning, and contextual learning and showcase its state-of-the-art performance against various adversarial attacks.

cs.LG↗

arXiv2024

Layer skyrmions for ideal Chern bands and twisted bilayer graphene

Ideal $C=1$ Chern bands exhibit a Landau level correspondence: they factorize as a lowest Landau levels and a spinor wavefunction that spans the layer index. We demonstrate that, in single Dirac moiré models, the spinor develops generally a Skyrme texture in real space with an associated Berry phase which compensates exactly the magnetic phase of the Landau level. For ideal bands with higher Chern numbers $C>1$, we find that $C$ color Landau levels are carried by $C$ spinors with Skyrme textures. We identify a SU(C) gauge symmetry in the color space of spinors and an emergent non-Abelian connection in real space intimately linked to the Pontryagin winding index of the layer skyrmions. They result in a total real-space Chern number of $-1$, screening the magnetic phase, irrespective of $C$ and of the number of layers. The topologically robust Skyrme texture remains remarkably intact in twisted bilayer graphene, even far from the chiral limit, and for realistic values of corrugation, making it an experimentally testable feature. We verify our predictions at the first magic angle of twisted bilayer, trilayer, and monolayer-bilayer graphene.

cond-mat.mes-hall↗

arXiv2024

Provably Convergent Subgraph-wise Sampling for Fast GNN Training

Subgraph-wise sampling -- a promising class of mini-batch training techniques for graph neural networks (GNNs -- is critical for real-world applications. During the message passing (MP) in GNNs, subgraph-wise sampling methods discard messages outside the mini-batches in backward passes to avoid the well-known neighbor explosion problem, i.e., the exponentially increasing dependencies of nodes with the number of MP iterations. However, discarding messages may sacrifice the gradient estimation accuracy, posing significant challenges to their convergence analysis and convergence speeds. To address this challenge, we propose a novel subgraph-wise sampling method with a convergence guarantee, namely Local Message Compensation (LMC). To the best of our knowledge, LMC is the first subgraph-wise sampling method with provable convergence. The key idea is to retrieve the discarded messages in backward passes based on a message passing formulation of backward passes. By efficient and effective compensations for the discarded messages in both forward and backward passes, LMC computes accurate mini-batch gradients and thus accelerates convergence. Moreover, LMC is applicable to various MP-based GNN architectures, including convolutional GNNs (finite message passing iterations with different layers) and recurrent GNNs (infinite message passing iterations with a shared layer). Experiments on large-scale benchmarks demonstrate that LMC is significantly faster than state-of-the-art subgraph-wise sampling methods.

cs.LG↗

arXiv2024

Learning Complete Topology-Aware Correlations Between Relations for Inductive Link Prediction

Inductive link prediction -- where entities during training and inference stages can be different -- has shown great potential for completing evolving knowledge graphs in an entity-independent manner. Many popular methods mainly focus on modeling graph-level features, while the edge-level interactions -- especially the semantic correlations between relations -- have been less explored. However, we notice a desirable property of semantic correlations between relations is that they are inherently edge-level and entity-independent. This implies the great potential of the semantic correlations for the entity-independent inductive link prediction task. Inspired by this observation, we propose a novel subgraph-based method, namely TACO, to model Topology-Aware COrrelations between relations that are highly correlated to their topological structures within subgraphs. Specifically, we prove that semantic correlations between any two relations can be categorized into seven topological patterns, and then proposes Relational Correlation Network (RCN) to learn the importance of each pattern. To further exploit the potential of RCN, we propose Complete Common Neighbor induced subgraph that can effectively preserve complete topological patterns within the subgraph. Extensive experiments demonstrate that TACO effectively unifies the graph-level information and edge-level interactions to jointly perform reasoning, leading to a superior performance over existing state-of-the-art methods for the inductive link prediction task.

cs.AI↗

arXiv2024

Multi-agent Multi-armed Bandits with Stochastic Sharable Arm Capacities

Motivated by distributed selection problems, we formulate a new variant of multi-player multi-armed bandit (MAB) model, which captures stochastic arrival of requests to each arm, as well as the policy of allocating requests to players. The challenge is how to design a distributed learning algorithm such that players select arms according to the optimal arm pulling profile (an arm pulling profile prescribes the number of players at each arm) without communicating to each other. We first design a greedy algorithm, which locates one of the optimal arm pulling profiles with a polynomial computational complexity. We also design an iterative distributed algorithm for players to commit to an optimal arm pulling profile with a constant number of rounds in expectation. We apply the explore then commit (ETC) framework to address the online setting when model parameters are unknown. We design an exploration strategy for players to estimate the optimal arm pulling profile. Since such estimates can be different across different players, it is challenging for players to commit. We then design an iterative distributed algorithm, which guarantees that players can arrive at a consensus on the optimal arm pulling profile in only M rounds. We conduct experiments to validate our algorithm.

cs.AI↗

arXiv2024

Multi-level Monte-Carlo Gradient Methods for Stochastic Optimization with Biased Oracles

We consider stochastic optimization when one only has access to biased stochastic oracles of the objective and the gradient, and obtaining stochastic gradients with low biases comes at high costs. This setting captures various optimization paradigms, such as conditional stochastic optimization, distributionally robust optimization, shortfall risk optimization, and machine learning paradigms, such as contrastive learning. We examine a family of multi-level Monte Carlo (MLMC) gradient methods that exploit a delicate tradeoff among bias, variance, and oracle cost. We systematically study their total sample and computational complexities for strongly convex, convex, and nonconvex objectives and demonstrate their superiority over the widely used biased stochastic gradient method. When combined with the variance reduction techniques like SPIDER, these MLMC gradient methods can further reduce the complexity in the nonconvex regime. Our results imply that a series of stochastic optimization problems with biased oracles, previously considered to be more challenging, is fundamentally no harder than the classical stochastic optimization with unbiased oracles. We also delineate the boundary conditions under which these problems become more difficult. Moreover, MLMC gradient methods significantly improve the best-known complexities in the literature for conditional stochastic optimization and shortfall risk optimization. Our extensive numerical experiments on distributionally robust optimization, pricing and staffing scheduling problems, and contrastive learning demonstrate the superior performance of MLMC gradient methods.

math.OC↗

arXiv2024

Learning Rule-Induced Subgraph Representations for Inductive Relation Prediction

Inductive relation prediction (IRP) -- where entities can be different during training and inference -- has shown great power for completing evolving knowledge graphs. Existing works mainly focus on using graph neural networks (GNNs) to learn the representation of the subgraph induced from the target link, which can be seen as an implicit rule-mining process to measure the plausibility of the target link. However, these methods cannot differentiate the target link and other links during message passing, hence the final subgraph representation will contain irrelevant rule information to the target link, which reduces the reasoning performance and severely hinders the applications for real-world scenarios. To tackle this problem, we propose a novel \textit{single-source edge-wise} GNN model to learn the \textbf{R}ule-induc\textbf{E}d \textbf{S}ubgraph represen\textbf{T}ations (\textbf{REST}), which encodes relevant rules and eliminates irrelevant rules within the subgraph. Specifically, we propose a \textit{single-source} initialization approach to initialize edge features only for the target link, which guarantees the relevance of mined rules and target link. Then we propose several RNN-based functions for \textit{edge-wise} message passing to model the sequential property of mined rules. REST is a simple and effective approach with theoretical support to learn the \textit{rule-induced subgraph representation}. Moreover, REST does not need node labeling, which significantly accelerates the subgraph preprocessing time by up to \textbf{11.66$\times$}. Experiments on inductive relation prediction benchmarks demonstrate the effectiveness of our REST. Our code is available at https://github.com/smart-lty/REST.

cs.LG↗

arXiv2024

Chinese Metaphor Recognition Using a Multi-stage Prompting Large Language Model

Metaphors are common in everyday language, and the identification and understanding of metaphors are facilitated by models to achieve a better understanding of the text. Metaphors are mainly identified and generated by pre-trained models in existing research, but situations, where tenors or vehicles are not included in the metaphor, cannot be handled. The problem can be effectively solved by using Large Language Models (LLMs), but significant room for exploration remains in this early-stage research area. A multi-stage generative heuristic-enhanced prompt framework is proposed in this study to enhance the ability of LLMs to recognize tenors, vehicles, and grounds in Chinese metaphors. In the first stage, a small model is trained to obtain the required confidence score for answer candidate generation. In the second stage, questions are clustered and sampled according to specific rules. Finally, the heuristic-enhanced prompt needed is formed by combining the generated answer candidates and demonstrations. The proposed model achieved 3rd place in Track 1 of Subtask 1, 1st place in Track 2 of Subtask 1, and 1st place in both tracks of Subtask 2 at the NLPCC-2024 Shared Task 9.

cs.CL↗

arXiv2024

Assembly History and Internal Structure of Cluster Cold Dark Matter Haloes

We use the Phoenix simulations to study the mass assembly history and internal structures of cluster dark matter haloes ($M_{200} \gtrsim 5\times 10^{14} h^{-1}{\rm M}_\odot$). We confirm that cluster haloes grow inside-out, similar to galactic haloes. Major merger events dominate the growth of the internal region and minor mergers/diffuse accretion shape the outskirts. However, compared to galactic haloes, cluster haloes tend to have a younger and more actively evolving inner region. On average, the majority of mass (> 80%) in the inner region ($R< 0.1 r_{200}$) of Phoenix haloes is accreted after $z = 3$, while for galactic haloes, most mass in the central region has already been accreted before $z=6$. The density profiles of cluster haloes are less stable than those of galactic haloes over different radii. The enclosed mass within $50$ or $150$ kpc of all Phoenix haloes evolves substantially in the past ${\sim} 7$ Gyr, while galactic haloes remained stable during the same period. We suggest that the relatively younger and more active state explains the various observations of cluster haloes, especially in central regions.

astro-ph.GA↗

arXiv2024

The influence of baryons on low-mass haloes

The Voids-within-Voids-within-Voids (VVV) project used dark-matter-only simulations to study the abundance and structure of dark matter haloes over the full mass range populated in the standard $Λ\mathrm{CDM}$ cosmology. Here we explore how baryonic effects modify these results for $z=0$ halo masses in the range $10^4$ to $10^7~\mathrm{M_\odot}$, below the threshold for galaxy formation. Our main study focuses on three simulations from identical initial conditions at $z=127$, one following dark matter only, one including non-radiative gas, and one additionally including the baryonic physics relevant in this halo mass range (cooling and photoheating). In the non-radiative simulation, above $10^{5.5}~\mathrm{M_\odot}$, halo abundance and internal structure are very similar to the dark-matter-only simulation, and the baryon to dark matter ratio is everywhere close to the cosmic value. At lower mass, this ratio drops and haloes are less concentrated and less massive in the non-radiative case. Test simulations at higher resolution show this to be mainly a resolution effect; the expected drop in baryon content due to residual pressure effects only becomes substantial for $z=0$ haloes below $\sim 10^{2.7}~\mathrm{M_\odot}$. However, gas is heated by reionization at $z=6$ in our "full physics" run, and this results in almost complete expulsion of gas from all haloes in our simulated mass range. This suppresses the halo mass function by $\sim 30 \%$, lowers halo concentration, and consequently weakens the dark matter annihilation signal by $\sim 40-60 \%$.

astro-ph.GA↗

arXiv2024

SAM 2 in Robotic Surgery: An Empirical Evaluation for Robustness and Generalization in Surgical Video Segmentation

The recent Segment Anything Model (SAM) 2 has demonstrated remarkable foundational competence in semantic segmentation, with its memory mechanism and mask decoder further addressing challenges in video tracking and object occlusion, thereby achieving superior results in interactive segmentation for both images and videos. Building upon our previous empirical studies, we further explore the zero-shot segmentation performance of SAM 2 in robot-assisted surgery based on prompts, alongside its robustness against real-world corruption. For static images, we employ two forms of prompts: 1-point and bounding box, while for video sequences, the 1-point prompt is applied to the initial frame. Through extensive experimentation on the MICCAI EndoVis 2017 and EndoVis 2018 benchmarks, SAM 2, when utilizing bounding box prompts, outperforms state-of-the-art (SOTA) methods in comparative evaluations. The results with point prompts also exhibit a substantial enhancement over SAM's capabilities, nearing or even surpassing existing unprompted SOTA methodologies. Besides, SAM 2 demonstrates improved inference speed and less performance degradation against various image corruption. Although slightly unsatisfactory results remain in specific edges or regions, SAM 2's robust adaptability to 1-point prompts underscores its potential for downstream surgical tasks with limited prompt requirements.

cs.CV↗

arXiv2024

Squeezing-enhanced quantum sensing with quadratic optomechanics

Cavity optomechanical (COM) sensors, enhanced by quantum squeezing or entanglement, have become powerful tools for measuring ultra-weak forces with high precision and sensitivity. However, these sensors usually rely on linear COM couplings, a fundamental limitation when measurements of the mechanical energy are desired. Very recently, a giant enhancement of the signal-to-noise ratio was predicted in a quadratic COM system. Here we show that the performance of such a system can be further improved surpassing the standard quantum limit by using quantum squeezed light. Our approach is compatible with available engineering techniques of advanced COM sensors and provides new opportunities for using COM sensors in tests of fundamental laws of physics and quantum metrology applications.

quant-ph↗

arXiv2024

A Fan-type condition for cycles in $1$-tough and $k$-connected $(P_2\cup kP_1)$-free graphs

For a graph $G$, let $μ_k(G):=\min~\{\max_{x\in S}d_G(x):~S\in \mathcal{S}_k\}$, where $\mathcal{S}_k$ is the set consisting of all independent sets $\{u_1,\ldots,u_k\}$ of $G$ such that some vertex, say $u_i$ ($1\leq i\leq k$), is at distance two from every other vertex in it. A graph $G$ is $1$-tough if for each cut set $S\subseteq V(G)$, $G-S$ has at most $|S|$ components. Recently, Shi and Shan \cite{Shi} conjectured that for each integer $k\geq 4$, being $2k$-connected is sufficient for $1$-tough $(P_2\cup kP_1)$-free graphs to be hamiltonian, which was confirmed by Xu et al. \cite{Xu} and Ota and Sanka \cite{Ota2}, respectively. In this article, we generalize the above results through the following Fan-type theorem: Let $k$ be an integer with $k\geq 2$ and let $G$ be a $1$-tough and $k$-connected $(P_2\cup kP_1)$-free graph with $μ_{k+1}(G)\geq\frac{7k-6}{5}$, then $G$ is hamiltonian or the Petersen graph.

math.CO↗

arXiv2024

IISAN: Efficiently Adapting Multimodal Representation for Sequential Recommendation with Decoupled PEFT

Multimodal foundation models are transformative in sequential recommender systems, leveraging powerful representation learning capabilities. While Parameter-efficient Fine-tuning (PEFT) is commonly used to adapt foundation models for recommendation tasks, most research prioritizes parameter efficiency, often overlooking critical factors like GPU memory efficiency and training speed. Addressing this gap, our paper introduces IISAN (Intra- and Inter-modal Side Adapted Network for Multimodal Representation), a simple plug-and-play architecture using a Decoupled PEFT structure and exploiting both intra- and inter-modal adaptation. IISAN matches the performance of full fine-tuning (FFT) and state-of-the-art PEFT. More importantly, it significantly reduces GPU memory usage - from 47GB to just 3GB for multimodal sequential recommendation tasks. Additionally, it accelerates training time per epoch from 443s to 22s compared to FFT. This is also a notable improvement over the Adapter and LoRA, which require 37-39 GB GPU memory and 350-380 seconds per epoch for training. Furthermore, we propose a new composite efficiency metric, TPME (Training-time, Parameter, and GPU Memory Efficiency) to alleviate the prevalent misconception that "parameter efficiency represents overall efficiency". TPME provides more comprehensive insights into practical efficiency comparisons between different methods. Besides, we give an accessible efficiency analysis of all PEFT and FFT approaches, which demonstrate the superiority of IISAN. We release our codes and other materials at https://github.com/GAIR-Lab/IISAN.

cs.IR↗

arXiv2024

Certifying ground-state properties of quantum many-body systems

A ubiquitous problem in quantum physics is to understand the ground-state properties of many-body systems. Confronted with the fact that exact diagonalisation quickly becomes impossible when increasing the system size, variational approaches are typically employed as a scalable alternative: energy is minimised over a subset of all possible states and then different physical quantities are computed over the solution state. Despite remarkable success, rigorously speaking, all what variational methods offer are upper bounds on the ground-state energy. On the other hand, so-called relaxations of the ground-state problem based on semidefinite programming represent a complementary approach, providing lower bounds to the ground-state energy. However, in their current implementation, neither variational nor relaxation methods offer provable bound on other observables in the ground state beyond the energy. In this work, we show that the combination of the two classes of approaches can be used to derive certifiable bounds on the value of any observable in the ground state, such as correlation functions of arbitrary order, structure factors, or order parameters. We illustrate the power of this approach in paradigmatic examples of 1D and 2D spin-one-half Heisenberg models. To improve the scalability of the method, we exploit the symmetries and sparsity of the considered systems to reach sizes of hundreds of particles at much higher precision than previous works. Our analysis therefore shows how to obtain certifiable bounds on many-body ground-state properties beyond energy in a scalable way.

quant-ph↗

arXiv2024

Dual-stage Hyperspectral Image Classification Model with Spectral Supertoken

Hyperspectral image classification, a task that assigns pre-defined classes to each pixel in a hyperspectral image of remote sensing scenes, often faces challenges due to the neglect of correlations between spectrally similar pixels. This oversight can lead to inaccurate edge definitions and difficulties in managing minor spectral variations in contiguous areas. To address these issues, we introduce the novel Dual-stage Spectral Supertoken Classifier (DSTC), inspired by superpixel concepts. DSTC employs spectrum-derivative-based pixel clustering to group pixels with similar spectral characteristics into spectral supertokens. By projecting the classification of these tokens onto the image space, we achieve pixel-level results that maintain regional classification consistency and precise boundary. Moreover, recognizing the diversity within tokens, we propose a class-proportion-based soft label. This label adaptively assigns weights to different categories based on their prevalence, effectively managing data distribution imbalances and enhancing classification performance. Comprehensive experiments on WHU-OHS, IP, KSC, and UP datasets corroborate the robust classification capabilities of DSTC and the effectiveness of its individual components. Code will be publicly available at https://github.com/laprf/DSTC.

cs.CV↗

arXiv2024

Deep Symbolic Optimization for Combinatorial Optimization: Accelerating Node Selection by Discovering Potential Heuristics

Combinatorial optimization (CO) is one of the most fundamental mathematical models in real-world applications. Traditional CO solvers, such as Branch-and-Bound (B&B) solvers, heavily rely on expert-designed heuristics, which are reliable but require substantial manual tuning. Recent studies have leveraged deep learning (DL) models as an alternative to capture rich feature patterns for improved performance on GPU machines. Nonetheless, the drawbacks of high training and inference costs, as well as limited interpretability, severely hinder the adoption of DL methods in real-world applications. To address these challenges, we propose a novel deep symbolic optimization learning framework that combines their advantages. Specifically, we focus on the node selection module within B&B solvers -- namely, deep symbolic optimization for node selection (Dso4NS). With data-driven approaches, Dso4NS guides the search for mathematical expressions within the high-dimensional discrete symbolic space and then incorporates the highest-performing mathematical expressions into a solver. The data-driven model captures the rich feature information in the input data and generates symbolic expressions, while the expressions deployed in solvers enable fast inference with high interpretability. Experiments demonstrate the effectiveness of Dso4NS in learning high-quality expressions, outperforming existing approaches on a CPU machine. Encouragingly, the learned CPU-based policies consistently achieve performance comparable to state-of-the-art GPU-based approaches.

cs.LG↗