Searcharxiv⌕ Search

arXiv subjects

Di Wang

Publications and source records attributed to Di Wang.

At least 433 records · Page 24Linked to original sources
arXiv2023

Generalization Guarantees of Gradient Descent for Multi-Layer Neural Networks

Recently, significant progress has been made in understanding the generalization of neural networks (NNs) trained by gradient descent (GD) using the algorithmic stability approach. However, most of the existing research has focused on one-hidden-layer NNs and has not addressed the impact of different network scaling parameters. In this paper, we greatly extend the previous work \cite{lei2022stability,richards2021stability} by conducting a comprehensive stability and generalization analysis of GD for multi-layer NNs. For two-layer NNs, our results are established under general network scaling parameters, relaxing previous conditions. In the case of three-layer NNs, our technical contribution lies in demonstrating its nearly co-coercive property by utilizing a novel induction strategy that thoroughly explores the effects of over-parameterization. As a direct application of our general findings, we derive the excess risk rate of $O(1/\sqrt{n})$ for GD algorithms in both two-layer and three-layer NNs. This sheds light on sufficient or necessary conditions for under-parameterized and over-parameterized NNs trained by GD to attain the desired risk rate of $O(1/\sqrt{n})$. Moreover, we demonstrate that as the scaling parameter increases or the network complexity decreases, less over-parameterization is required for GD to achieve the desired error rates. Additionally, under a low-noise condition, we obtain a fast risk rate of $O(1/n)$ for GD in both two-layer and three-layer NNs.

cs.LG↗
arXiv2023

Coupled Active Perception and Manipulation Planning for a Mobile Manipulator in Precision Agriculture Applications

A mobile manipulator often finds itself in an application where it needs to take a close-up view before performing a manipulation task. Named this as a coupled active perception and manipulation (CAPM) problem, we model the uncertainty in the perception process and devise a key state/task planning approach that considers reachability conditions as task constraints of both perception and manipulation tasks for the mobile platform. By minimizing the expected energy usage in the body key state planning while satisfying task constraints, our algorithm achieves the best balance between the task success rate and energy usage. We have implemented the algorithm and tested it in both simulation and physical experiments. The results have confirmed that our algorithm has a lower energy consumption compared to a two-stage decoupled approach, while still maintaining a success rate of 100\% for the task.

cs.RO↗
arXiv2023

High-Dimensional Low-Rank Tensor Autoregressive Time Series Modeling

Modern technological advances have enabled an unprecedented amount of structured data with complex temporal dependence, urging the need for new methods to efficiently model and forecast high-dimensional tensor-valued time series. This paper provides a new modeling framework to accomplish this task via autoregression (AR). By considering a low-rank Tucker decomposition for the transition tensor, the proposed tensor AR can flexibly capture the underlying low-dimensional tensor dynamics, providing both substantial dimension reduction and meaningful multi-dimensional dynamic factor interpretations. For this model, we first study several nuclear-norm-regularized estimation methods and derive their non-asymptotic properties under the approximate low-rank setting. In particular, by leveraging the special balanced structure of the transition tensor, a novel convex regularization approach based on the sum of nuclear norms of square matricizations is proposed to efficiently encourage low-rankness of the coefficient tensor. To further improve the estimation efficiency under exact low-rankness, a non-convex estimator is proposed with a gradient descent algorithm, and its computational and statistical convergence guarantees are established. Simulation studies and an empirical analysis of tensor-valued time series data from multi-category import-export networks demonstrate the advantages of the proposed approach.

stat.ME↗
arXiv2023

Fake News Detectors are Biased against Texts Generated by Large Language Models

The spread of fake news has emerged as a critical challenge, undermining trust and posing threats to society. In the era of Large Language Models (LLMs), the capability to generate believable fake content has intensified these concerns. In this study, we present a novel paradigm to evaluate fake news detectors in scenarios involving both human-written and LLM-generated misinformation. Intriguingly, our findings reveal a significant bias in many existing detectors: they are more prone to flagging LLM-generated content as fake news while often misclassifying human-written fake news as genuine. This unexpected bias appears to arise from distinct linguistic patterns inherent to LLM outputs. To address this, we introduce a mitigation strategy that leverages adversarial training with LLM-paraphrased genuine news. The resulting model yielded marked improvements in detection accuracy for both human and LLM-generated news. To further catalyze research in this domain, we release two comprehensive datasets, \texttt{GossipCop++} and \texttt{PolitiFact++}, thus amalgamating human-validated articles with LLM-generated fake and real news.

cs.CL↗
arXiv2023

Incorporating Surprisingly Popular Algorithm and Euclidean Distance-based Adaptive Topology into PSO

While many Particle Swarm Optimization (PSO) algorithms only use fitness to assess the performance of particles, in this work, we adopt Surprisingly Popular Algorithm (SPA) as a complementary metric in addition to fitness. Consequently, particles that are not widely known also have the opportunity to be selected as the learning exemplars. In addition, we propose a Euclidean distance-based adaptive topology to cooperate with SPA, where each particle only connects to k number of particles with the shortest Euclidean distance during each iteration. We also introduce the adaptive topology into heterogeneous populations to better solve large-scale problems. Specifically, the exploration sub-population better preserves the diversity of the population while the exploitation sub-population achieves fast convergence. Therefore, large-scale problems can be solved in a collaborative manner to elevate the overall performance. To evaluate the performance of our method, we conduct extensive experiments on various optimization problems, including three benchmark suites and two real-world optimization problems. The results demonstrate that our Euclidean distance-based adaptive topology outperforms the other widely adopted topologies and further suggest that our method performs significantly better than state-of-the-art PSO variants on small, medium, and large-scale problems.

cs.NE↗
arXiv2023

Giant non-volatile electric field control of proximity induced magnetism in the spin-orbit semimetal SrIrO3

With its potential for drastically reduced operation power of information processing devices, electric field control of magnetism has generated huge research interest. Recently, novel perspectives offered by the inherently large spin-orbit coupling of 5d transition metals have emerged. Here, we demonstrate non-volatile electrical control of the proximity induced magnetism in SrIrO3 based back-gated heterostructures. We report up to a 700 % variation of the anomalous Hall conductivity σ_AHE and Hall angle θ_AHE as function of the applied gate voltage Vg. In contrast, the Curie temperature TC = 100K and magnetic anisotropy of the system remain essentially unaffected by Vg indicating a robust ferromagnetic state in SrIrO3 which strongly hints to gating-induced changes of the anomalous Berry curvature. The electric-field induced ferroelectric-like state of SrTiO3 enables non-volatile switching behavior of σ_AHE and θ_AHE below 60 K. The large tunability of this system, opens new avenues towards efficient electric-field manipulation of magnetism.

cond-mat.mtrl-sci↗
arXiv2023

Adaptive Distributed Kernel Ridge Regression: A Feasible Distributed Learning Scheme for Data Silos

Data silos, mainly caused by privacy and interoperability, significantly constrain collaborations among different organizations with similar data for the same purpose. Distributed learning based on divide-and-conquer provides a promising way to settle the data silos, but it suffers from several challenges, including autonomy, privacy guarantees, and the necessity of collaborations. This paper focuses on developing an adaptive distributed kernel ridge regression (AdaDKRR) by taking autonomy in parameter selection, privacy in communicating non-sensitive information, and the necessity of collaborations in performance improvement into account. We provide both solid theoretical verification and comprehensive experiments for AdaDKRR to demonstrate its feasibility and effectiveness. Theoretically, we prove that under some mild conditions, AdaDKRR performs similarly to running the optimal learning algorithms on the whole data, verifying the necessity of collaborations and showing that no other distributed learning scheme can essentially beat AdaDKRR under the same conditions. Numerically, we test AdaDKRR on both toy simulations and two real-world applications to show that AdaDKRR is superior to other existing distributed learning schemes. All these results show that AdaDKRR is a feasible scheme to defend against data silos, which are highly desired in numerous application regions such as intelligent decision-making, pricing forecasting, and performance prediction for products.

cs.LG↗
arXiv2023

Improving Misaligned Multi-modality Image Fusion with One-stage Progressive Dense Registration

Misalignments between multi-modality images pose challenges in image fusion, manifesting as structural distortions and edge ghosts. Existing efforts commonly resort to registering first and fusing later, typically employing two cascaded stages for registration,i.e., coarse registration and fine registration. Both stages directly estimate the respective target deformation fields. In this paper, we argue that the separated two-stage registration is not compact, and the direct estimation of the target deformation fields is not accurate enough. To address these challenges, we propose a Cross-modality Multi-scale Progressive Dense Registration (C-MPDR) scheme, which accomplishes the coarse-to-fine registration exclusively using a one-stage optimization, thus improving the fusion performance of misaligned multi-modality images. Specifically, two pivotal components are involved, a dense Deformation Field Fusion (DFF) module and a Progressive Feature Fine (PFF) module. The DFF aggregates the predicted multi-scale deformation sub-fields at the current scale, while the PFF progressively refines the remaining misaligned features. Both work together to accurately estimate the final deformation fields. In addition, we develop a Transformer-Conv-based Fusion (TCF) subnetwork that considers local and long-range feature dependencies, allowing us to capture more informative features from the registered infrared and visible images for the generation of high-quality fused images. Extensive experimental analysis demonstrates the superiority of the proposed method in the fusion of misaligned cross-modality images.

cs.CV↗
arXiv2023

First Direct Observation of Collider Neutrinos with FASER at the LHC

We report the first direct observation of neutrino interactions at a particle collider experiment. Neutrino candidate events are identified in a 13.6 TeV center-of-mass energy $pp$ collision data set of 35.4 fb${}^{-1}$ using the active electronic components of the FASER detector at the Large Hadron Collider. The candidates are required to have a track propagating through the entire length of the FASER detector and be consistent with a muon neutrino charged-current interaction. We infer $153^{+12}_{-13}$ neutrino interactions with a significance of 16 standard deviations above the background-only hypothesis. These events are consistent with the characteristics expected from neutrino interactions in terms of secondary particle production and spatial distribution, and they imply the observation of both neutrinos and anti-neutrinos with an incident neutrino energy of significantly above 200 GeV.

hep-ex↗
arXiv2023

Quantizing Heavy-tailed Data in Statistical Estimation: (Near) Minimax Rates, Covariate Quantization, and Uniform Recovery

This paper studies the quantization of heavy-tailed data in some fundamental statistical estimation problems, where the underlying distributions have bounded moments of some order. We propose to truncate and properly dither the data prior to a uniform quantization. Our major standpoint is that (near) minimax rates of estimation error are achievable merely from the quantized data produced by the proposed scheme. In particular, concrete results are worked out for covariance estimation, compressed sensing, and matrix completion, all agreeing that the quantization only slightly worsens the multiplicative factor. Besides, we study compressed sensing where both covariate (i.e., sensing vector) and response are quantized. Under covariate quantization, although our recovery program is non-convex because the covariance matrix estimator lacks positive semi-definiteness, all local minimizers are proved to enjoy near optimal error bound. Moreover, by the concentration inequality of product process and covering argument, we establish near minimax uniform recovery guarantee for quantized compressed sensing with heavy-tailed noise.

math.ST↗
arXiv2023

$\mathcal{A}$-theory: A brane world-volume theory with manifest U-duality

In this paper, the ${\cal A}$-theory, an extension of F-theory, is described as a fully U-duality covariant brane theory. This theory has some distinguishing features not known from world-sheet models. In particular, seen as a sigma model, both world-volume and target space coordinates are specific representations of the same group (the U-duality group). The U-duality group in question is an exceptional group (a split form of the $E_d$ series). The structure of this group allows it to encompass both the T-duality group of string theory as well as the general linear symmetry group of ${\cal M}$-theory. ${\cal A}$-theory is defined by the current algebras in Hamiltonian formalism, or by world-volume actions in Lagrangian formalism. The spacetime coordinates are selfdual gauge fields on the world-volume, requiring the Gauss law constraints tying the world-volume to spacetime. Solving the Gauss law constraints/the Virasoro constraints gives the world-volume/spacetime sectioning from ${\cal A}$-theory to ${\cal T}$-theory/ ${\cal M}$-theory respectively. The ${\cal A}$-theory Lagrangian admits extended symmetry which has not been observed previously in the literature, where the background fields include both the spacetime and the world-volume gravitational fields. We also constructed the four-point amplitude of ${\cal A}$-theory in the low energy limit. The amplitude is written in a way that the U-duality symmetry is manifest, but after solving the section condition, it reduces to the usual four-graviton amplitude. In the previous papers, we have referred to this model as F-theory, however, F-theory initiated by Vafa is now a big branch of string theory as the study of elliptic fibrations, so we refer to these constructions as generalized models of theory for all dimensions with all duality symmetries as ${\cal A}$-theory.

hep-th↗
arXiv2023

Online Bidding Algorithms for Return-on-Spend Constrained Advertisers

Online advertising has recently grown into a highly competitive and complex multi-billion-dollar industry, with advertisers bidding for ad slots at large scales and high frequencies. This has resulted in a growing need for efficient "auto-bidding" algorithms that determine the bids for incoming queries to maximize advertisers' targets subject to their specified constraints. This work explores efficient online algorithms for a single value-maximizing advertiser under an increasingly popular constraint: Return-on-Spend (RoS). We quantify efficiency in terms of regret relative to the optimal algorithm, which knows all queries a priori. We contribute a simple online algorithm that achieves near-optimal regret in expectation while always respecting the specified RoS constraint when the input sequence of queries are i.i.d. samples from some distribution. We also integrate our results with the previous work of Balseiro, Lu, and Mirrokni [BLM20] to achieve near-optimal regret while respecting both RoS and fixed budget constraints. Our algorithm follows the primal-dual framework and uses online mirror descent (OMD) for the dual updates. However, we need to use a non-canonical setup of OMD, and therefore the classic low-regret guarantee of OMD, which is for the adversarial setting in online learning, no longer holds. Nonetheless, in our case and more generally where low-regret dynamics are applied in algorithm design, the gradients encountered by OMD can be far from adversarial but influenced by our algorithmic choices. We exploit this key insight to show our OMD setup achieves low regret in the realm of our algorithm.

cs.LG↗
arXiv2023

On the Global Convergence of Natural Actor-Critic with Two-layer Neural Network Parametrization

Actor-critic algorithms have shown remarkable success in solving state-of-the-art decision-making problems. However, despite their empirical effectiveness, their theoretical underpinnings remain relatively unexplored, especially with neural network parametrization. In this paper, we delve into the study of a natural actor-critic algorithm that utilizes neural networks to represent the critic. Our aim is to establish sample complexity guarantees for this algorithm, achieving a deeper understanding of its performance characteristics. To achieve that, we propose a Natural Actor-Critic algorithm with 2-Layer critic parametrization (NAC2L). Our approach involves estimating the $Q$-function in each iteration through a convex optimization problem. We establish that our proposed approach attains a sample complexity of $\tilde{\mathcal{O}}\left(\frac{1}{ε^{4}(1-γ)^{4}}\right)$. In contrast, the existing sample complexity results in the literature only hold for a tabular or linear MDP. Our result, on the other hand, holds for countable state spaces and does not require a linear or low-rank structure on the MDP.

cs.LG↗
arXiv2023

Star formation concentration as a tracer of environmental quenching in action: a study of the Eagle and C-Eagle simulations

We study environmental quenching in the Eagle}/C-Eagle cosmological hydrodynamic simulations over the last 11 Gyr (i.e. $z=0-2$). The simulations are compared with observations from the SAMI Galaxy Survey at $z=0$. We focus on satellite galaxies in galaxy groups and clusters ($10^{12}\,\rm M_{\odot}$ $\lesssim$ $M_{200}$ < $3 \times 10^{15}\, \rm M_{\odot}$). A star-formation concentration index [$C$-index $= \log_{10}(r_\mathrm{50,SFR} / r_\mathrm{50,rband})$] is defined, which measures how concentrated star formation is relative to the stellar distribution. Both Eagle/C-Eagle and SAMI show a higher fraction of galaxies with low $C$-index in denser environments at $z=0-0.5$. Low $C$-index galaxies are found below the SFR-$M_{\star}$ main sequence (MS), and display a declining specific star formation rate (sSFR) with increasing radii, consistent with ``outside-in'' environmental quenching. Additionally, we show that $C$-index can be used as a proxy for how long galaxies have been satellites. These trends become weaker at increasing redshift and are absent by $z=1-2$. We define a quenching timescale $t_{\rm quench}$ as how long it takes satellites to transition from the MS to the quenched population. We find that simulated galaxies experiencing ``outside-in'' environmental quenching at low redshift ($z=0\sim0.5$) have a long quenching timescale (median $t_{\rm quench}$ > 2 Gyr). The simulated galaxies at higher redshift ($z=0.7\sim2$) experience faster quenching (median $t_{\rm quench}$ < 2Gyr). At $z\gtrsim 1-2$ galaxies undergoing environmental quenching have decreased sSFR across the entire galaxy with no ``outside-in'' quenching signatures and a narrow range of $C$-index, showing that on average environmental quenching acts differently than at $z\lesssim 1$.

astro-ph.GA↗
arXiv2023

High-Dimensional Vector Autoregression with Common Response and Predictor Factors

The reduced-rank vector autoregressive (VAR) model can be interpreted as a supervised factor model, where two factor modelings are simultaneously applied to response and predictor spaces. This article introduces a new model, called vector autoregression with common response and predictor factors, to explore further the common structure between the response and predictors in the VAR framework. The new model can provide better physical interpretations and improve estimation efficiency. In conjunction with the tensor operation, the model can easily be extended to any finite-order VAR model. A regularization-based method is considered for the high-dimensional estimation with the gradient descent algorithm, and its computational and statistical convergence guarantees are established. For data with pervasive cross-sectional dependence, a transformation for responses is developed to alleviate the diverging eigenvalue effect. Moreover, we consider additional sparsity structure in factor loading for the case of ultra-high dimension. Simulation experiments confirm our theoretical findings and a macroeconomic application showcases the appealing properties of the proposed model in structural analysis and forecasting.

stat.ME↗
arXiv2023

Differentially Private Episodic Reinforcement Learning with Heavy-tailed Rewards

In this paper, we study the problem of (finite horizon tabular) Markov decision processes (MDPs) with heavy-tailed rewards under the constraint of differential privacy (DP). Compared with the previous studies for private reinforcement learning that typically assume rewards are sampled from some bounded or sub-Gaussian distributions to ensure DP, we consider the setting where reward distributions have only finite $(1+v)$-th moments with some $v \in (0,1]$. By resorting to robust mean estimators for rewards, we first propose two frameworks for heavy-tailed MDPs, i.e., one is for value iteration and another is for policy optimization. Under each framework, we consider both joint differential privacy (JDP) and local differential privacy (LDP) models. Based on our frameworks, we provide regret upper bounds for both JDP and LDP cases and show that the moment of distribution and privacy budget both have significant impacts on regrets. Finally, we establish a lower bound of regret minimization for heavy-tailed MDPs in JDP model by reducing it to the instance-independent lower bound of heavy-tailed multi-armed bandits in DP model. We also show the lower bound for the problem in LDP by adopting some private minimax methods. Our results reveal that there are fundamental differences between the problem of private RL with sub-Gaussian and that with heavy-tailed rewards.

cs.LG↗
arXiv2023

Hybrid organic-inorganic two-dimensional metal carbide MXenes with amido- and imido-terminated surfaces

Two-dimensional (2D) transition-metal carbides and nitrides (MXenes) show impressive performance in applications, such as supercapacitors, batteries, electromagnetic interference shielding, or electrocatalysis. These materials combine the electronic and mechanical properties of 2D inorganic crystals with chemically modifiable surfaces, and surface-engineered MXenes represent an ideal platform for fundamental and applied studies of interfaces in 2D functional materials. A natural step in structural engineering of MXene compounds is the development and understanding of MXenes with various organic functional groups covalently bound to inorganic 2D sheets. Such hybrid structures have the potential to unite the tailorability of organic molecules with the unique electronic properties of inorganic 2D solids. Here, we introduce a new family of hybrid MXenes (h-MXenes) with amido- and imido-bonding between organic and inorganic parts. The description of h-MXene structure requires an intricate mix of concepts from the fields of coordination chemistry, self-assembled monolayers (SAMs) and surface science. The optical properties of h-MXenes reveal coherent coupling between the organic and inorganic components. h-MXenes also show superior stability against hydrolysis in aqueous solutions.

cond-mat.mtrl-sci↗
arXiv2023

Direct synthesis and chemical vapor deposition of 2D carbide and nitride MXenes

Two-dimensional (2D) transition metal carbides and nitrides (MXenes) are a large family of materials actively studied for various applications, especially in the field of energy storage. To date, MXenes are commonly synthesized by etching the layered ternary compounds, MAX phases. Here we demonstrate a direct synthetic route for scalable and atom-economic synthesis of MXenes, including phases that have not been synthesized from MAX phases, by the reactions of metals and metal halides with graphite, methane or nitrogen. These directly synthesized MXenes showed excellent energy storage capacity for Li-ion intercalation. The direct synthesis enables chemical vapor deposition (CVD) growth of MXene carpets and complex spherulite-like morphologies. The latter form in a process resembling the evolution of cellular membranes during endocytosis.

cond-mat.mtrl-sci↗