Searcharxiv⌕ Search

arXiv subjects

Yi Xu

Publications and source records attributed to Yi Xu.

At least 289 records · Page 16Linked to original sources
arXiv2023

Phase diagram of the chiral SU(3) antiferromagnet on the kagome lattice

Motivated by the search for chiral spin liquids (CSL), we consider a simple model defined on the kagome lattice of interacting SU(3) spins (in the fundamental representation) including two-site and three-site permutations between nearest neighbor sites and on triangles, respectively. By combining analytical developments and various numerical techniques, namely exact Lanczos diagonalizations and tensor network variational approaches, we find a rich phase diagram with non-topological (``trivial") and topological (possibly chiral) gapped spin liquids (SLs). Trivial spin liquids include an Affleck-Kennedy-Lieb-Tasaki (AKLT)-like phase and a trimerized phase, the latter breaking the inversion center between the up and down triangles of the kagome lattice. A topological SL is stabilized in a restricted part of the phase diagram by the time-reversal symmetry breaking (complex) 3-site permutation term. Analyzing the chiral edge modes of this topological SL on long cylinders or on finite disks, we have come up with two competing scenarios, either a CSL or a double Chern-Simon SL characterized by a single or by two counter-propagating Wess-Zumino-Witten SU(3)$_1$ chiral mode(s), respectively. In the vicinity of the extended ferromagnetic region we have found a magnetic phase corresponding either to a modulated canted ferromagnet or to a uniform partially magnetized ferromagnet.

cond-mat.str-el↗
arXiv2023

Attentional-Biased Stochastic Gradient Descent

In this paper, we present a simple yet effective provable method (named ABSGD) for addressing the data imbalance or label noise problem in deep learning. Our method is a simple modification to momentum SGD where we assign an individual importance weight to each sample in the mini-batch. The individual-level weight of sampled data is systematically proportional to the exponential of a scaled loss value of the data, where the scaling factor is interpreted as the regularization parameter in the framework of distributionally robust optimization (DRO). Depending on whether the scaling factor is positive or negative, ABSGD is guaranteed to converge to a stationary point of an information-regularized min-max or min-min DRO problem, respectively. Compared with existing class-level weighting schemes, our method can capture the diversity between individual examples within each class. Compared with existing individual-level weighting methods using meta-learning that require three backward propagations for computing mini-batch stochastic gradients, our method is more efficient with only one backward propagation at each iteration as in standard deep learning methods. ABSGD is flexible enough to combine with other robust losses without any additional cost. Our empirical studies on several benchmark datasets demonstrate the effectiveness of the proposed method.\footnote{Code is available at:\url{https://github.com/qiqi-helloworld/ABSGD/}}

cs.LG↗
arXiv2023

Graph-Aware Language Model Pre-Training on a Large Graph Corpus Can Help Multiple Graph Applications

Model pre-training on large text corpora has been demonstrated effective for various downstream applications in the NLP domain. In the graph mining domain, a similar analogy can be drawn for pre-training graph models on large graphs in the hope of benefiting downstream graph applications, which has also been explored by several recent studies. However, no existing study has ever investigated the pre-training of text plus graph models on large heterogeneous graphs with abundant textual information (a.k.a. large graph corpora) and then fine-tuning the model on different related downstream applications with different graph schemas. To address this problem, we propose a framework of graph-aware language model pre-training (GALM) on a large graph corpus, which incorporates large language models and graph neural networks, and a variety of fine-tuning methods on downstream applications. We conduct extensive experiments on Amazon's real internal datasets and large public datasets. Comprehensive empirical results and in-depth analysis demonstrate the effectiveness of our proposed methods along with lessons learned.

cs.CL↗
arXiv2023

Exploring and Verbalizing Academic Ideas by Concept Co-occurrence

Researchers usually come up with new ideas only after thoroughly comprehending vast quantities of literature. The difficulty of this procedure is exacerbated by the fact that the number of academic publications is growing exponentially. In this study, we devise a framework based on concept co-occurrence for academic idea inspiration, which has been integrated into a research assistant system. From our perspective, the fusion of two concepts that co-occur in an academic paper can be regarded as an important way of the emergence of a new idea. We construct evolving concept graphs according to the co-occurrence relationship of concepts from 20 disciplines or topics. Then we design a temporal link prediction method based on masked language model to explore potential connections between different concepts. To verbalize the newly discovered connections, we also utilize the pretrained language model to generate a description of an idea based on a new data structure called co-occurrence citation quintuple. We evaluate our proposed system using both automatic metrics and human assessment. The results demonstrate that our system has broad prospects and can assist researchers in expediting the process of discovering new ideas.

cs.CL↗
arXiv2023

EMPRESS. XI. SDSS and JWST Search for Local and z~4-5 Extremely Metal-Poor Galaxies (EMPGs): Clustering and Chemical Properties of Local EMPGs

We search for local extremely metal-poor galaxies (EMPGs), selecting photometric candidates by broadband color excess and machine-learning techniques with the SDSS photometric data. After removing stellar contaminants by shallow spectroscopy with Seimei and Nayuta telescopes, we confirm that three candidates are EMPGs with 0.05--0.1 $Z_\odot$ by deep Magellan/MagE spectroscopy for faint {\sc[Oiii]}$λ$4363 lines. Using a statistical sample consisting of 105 spectroscopically-confirmed EMPGs taken from our study and the literature, we calculate cross-correlation function (CCF) of the EMPGs and all SDSS galaxies to quantify environments of EMPGs. Comparing another CCF of all SDSS galaxies and comparison SDSS galaxies in the same stellar mass range ($10^{7.0}-10^{8.4} M_\odot$), we find no significant ($>1σ$) difference between these two CCFs. We also compare mass-metallicity relations (MZRs) of the EMPGs and those of galaxies at $z\sim$ 0--4 with a steady chemical evolution model and find that the EMPG MZR is comparable with the model prediction on average. These clustering and chemical properties of EMPGs are explained by a scenario of stochastic metal-poor gas accretion on metal-rich galaxies showing metal-poor star formation. Extending the broadband color-excess technique to a high-$z$ EMPG search, we select 17 candidates of $z\sim$ 4--5 EMPGs with the deep ($\simeq30$ mag) near-infrared JWST/NIRCam images obtained by ERO and ERS programs. We find galaxy candidates with negligible {\sc[Oiii]}$λλ$4959,5007 emission weaker than the local EMPGs and known high-$z$ galaxies, suggesting that some of these candidates may fall in 0--0.01 $Z_\odot$, which potentially break the lowest metallicity limit known to date.

astro-ph.GA↗
arXiv2023

EMPRESS. IX. Extremely Metal-Poor Galaxies are Very Gas-Rich Dispersion-Dominated Systems: Will JWST Witness Gaseous Turbulent High-z Primordial Galaxies?

We present kinematics of 6 local extremely metal-poor galaxies (EMPGs) with low metallicities ($0.016-0.098\ Z_{\odot}$) and low stellar masses ($10^{4.7}-10^{7.6} M_{\odot}$). Taking deep medium-high resolution ($R\sim7500$) integral-field spectra with 8.2-m Subaru, we resolve the small inner velocity gradients and dispersions of the EMPGs with H$α$ emission. Carefully masking out sub-structures originated by inflow and/or outflow, we fit 3-dimensional disk models to the observed H$α$ flux, velocity, and velocity-dispersion maps. All the EMPGs show rotational velocities ($v_{\rm rot}$) of 5--23 km s$^{-1}$ smaller than the velocity dispersions ($σ_{0}$) of 17--31 km s$^{-1}$, indicating dispersion-dominated ($v_{\rm rot}/σ_{0}=0.29-0.80<1$) systems affected by inflow and/or outflow. Except for two EMPGs with large uncertainties, we find that the EMPGs have very large gas-mass fractions of $f_{\rm gas}\simeq 0.9-1.0$. Comparing our results with other H$α$ kinematics studies, we find that $v_{\rm rot}/σ_{0}$ decreases and $f_{\rm gas}$ increases with decreasing metallicity, decreasing stellar mass, and increasing specific star-formation rate. We also find that simulated high-$z$ ($z\sim 7$) forming galaxies have gas fractions and dynamics similar to the observed EMPGs. Our EMPG observations and the simulations suggest that primordial galaxies are gas-rich dispersion-dominated systems, which would be identified by the forthcoming James Webb Space Telescope (JWST) observations at $z\sim 7$.

astro-ph.GA↗
arXiv2023

Dynamic Voxel Grid Optimization for High-Fidelity RGB-D Supervised Surface Reconstruction

Direct optimization of interpolated features on multi-resolution voxel grids has emerged as a more efficient alternative to MLP-like modules. However, this approach is constrained by higher memory expenses and limited representation capabilities. In this paper, we introduce a novel dynamic grid optimization method for high-fidelity 3D surface reconstruction that incorporates both RGB and depth observations. Rather than treating each voxel equally, we optimize the process by dynamically modifying the grid and assigning more finer-scale voxels to regions with higher complexity, allowing us to capture more intricate details. Furthermore, we develop a scheme to quantify the dynamic subdivision of voxel grid during optimization without requiring any priors. The proposed approach is able to generate high-quality 3D reconstructions with fine details on both synthetic and real-world data, while maintaining computational efficiency, which is substantially faster than the baseline method NeuralRGBD.

cs.CV↗
arXiv2023

Uncovering the Missing Pattern: Unified Framework Towards Trajectory Imputation and Prediction

Trajectory prediction is a crucial undertaking in understanding entity movement or human behavior from observed sequences. However, current methods often assume that the observed sequences are complete while ignoring the potential for missing values caused by object occlusion, scope limitation, sensor failure, etc. This limitation inevitably hinders the accuracy of trajectory prediction. To address this issue, our paper presents a unified framework, the Graph-based Conditional Variational Recurrent Neural Network (GC-VRNN), which can perform trajectory imputation and prediction simultaneously. Specifically, we introduce a novel Multi-Space Graph Neural Network (MS-GNN) that can extract spatial features from incomplete observations and leverage missing patterns. Additionally, we employ a Conditional VRNN with a specifically designed Temporal Decay (TD) module to capture temporal dependencies and temporal missing patterns in incomplete trajectories. The inclusion of the TD module allows for valuable information to be conveyed through the temporal flow. We also curate and benchmark three practical datasets for the joint problem of trajectory imputation and prediction. Extensive experiments verify the exceptional performance of our proposed method. As far as we know, this is the first work to address the lack of benchmarks and techniques for trajectory imputation and prediction in a unified manner.

cs.CV↗
arXiv2023

TWINS: A Fine-Tuning Framework for Improved Transferability of Adversarial Robustness and Generalization

Recent years have seen the ever-increasing importance of pre-trained models and their downstream training in deep learning research and applications. At the same time, the defense for adversarial examples has been mainly investigated in the context of training from random initialization on simple classification tasks. To better exploit the potential of pre-trained models in adversarial robustness, this paper focuses on the fine-tuning of an adversarially pre-trained model in various classification tasks. Existing research has shown that since the robust pre-trained model has already learned a robust feature extractor, the crucial question is how to maintain the robustness in the pre-trained model when learning the downstream task. We study the model-based and data-based approaches for this goal and find that the two common approaches cannot achieve the objective of improving both generalization and adversarial robustness. Thus, we propose a novel statistics-based approach, Two-WIng NormliSation (TWINS) fine-tuning framework, which consists of two neural networks where one of them keeps the population means and variances of pre-training data in the batch normalization layers. Besides the robust information transfer, TWINS increases the effective learning rate without hurting the training stability since the relationship between a weight norm and its gradient norm in standard batch normalization layer is broken, resulting in a faster escape from the sub-optimal initialization and alleviating the robust overfitting. Finally, TWINS is shown to be effective on a wide range of image classification datasets in terms of both generalization and robustness. Our code is available at https://github.com/ziquanliu/CVPR2023-TWINS.

cs.LG↗
arXiv2023

Harnessing Low-Frequency Neural Fields for Few-Shot View Synthesis

Neural Radiance Fields (NeRF) have led to breakthroughs in the novel view synthesis problem. Positional Encoding (P.E.) is a critical factor that brings the impressive performance of NeRF, where low-dimensional coordinates are mapped to high-dimensional space to better recover scene details. However, blindly increasing the frequency of P.E. leads to overfitting when the reconstruction problem is highly underconstrained, \eg, few-shot images for training. We harness low-frequency neural fields to regularize high-frequency neural fields from overfitting to better address the problem of few-shot view synthesis. We propose reconstructing with a low-frequency only field and then finishing details with a high-frequency equipped field. Unlike most existing solutions that regularize the output space (\ie, rendered images), our regularization is conducted in the input space (\ie, signal frequency). We further propose a simple-yet-effective strategy for tuning the frequency to avoid overfitting few-shot inputs: enforcing consistency among the frequency domain of rendered 2D images. Thanks to the input space regularizing scheme, our method readily applies to inputs beyond spatial locations, such as the time dimension in dynamic scenes. Comparisons with state-of-the-art on both synthetic and natural datasets validate the effectiveness of our proposed solution for few-shot view synthesis. Code is available at \href{https://github.com/lsongx/halo}{https://github.com/lsongx/halo}.

cs.CV↗
arXiv2023

EgoViT: Pyramid Video Transformer for Egocentric Action Recognition

Capturing interaction of hands with objects is important to autonomously detect human actions from egocentric videos. In this work, we present a pyramid video transformer with a dynamic class token generator for egocentric action recognition. Different from previous video transformers, which use the same static embedding as the class token for diverse inputs, we propose a dynamic class token generator that produces a class token for each input video by analyzing the hand-object interaction and the related motion information. The dynamic class token can diffuse such information to the entire model by communicating with other informative tokens in the subsequent transformer layers. With the dynamic class token, dissimilarity between videos can be more prominent, which helps the model distinguish various inputs. In addition, traditional video transformers explore temporal features globally, which requires large amounts of computation. However, egocentric videos often have a large amount of background scene transition, which causes discontinuities across distant frames. In this case, blindly reducing the temporal sampling rate will risk losing crucial information. Hence, we also propose a pyramid architecture to hierarchically process the video from short-term high rate to long-term low rate. With the proposed architecture, we significantly reduce the computational cost as well as the memory requirement without sacrificing from the model performance. We perform comparisons with different baseline video transformers on the EPIC-KITCHENS-100 and EGTEA Gaze+ datasets. Both quantitative and qualitative results show that the proposed model can efficiently improve the performance for egocentric action recognition.

cs.CV↗
arXiv2023

Interventional Bag Multi-Instance Learning On Whole-Slide Pathological Images

Multi-instance learning (MIL) is an effective paradigm for whole-slide pathological images (WSIs) classification to handle the gigapixel resolution and slide-level label. Prevailing MIL methods primarily focus on improving the feature extractor and aggregator. However, one deficiency of these methods is that the bag contextual prior may trick the model into capturing spurious correlations between bags and labels. This deficiency is a confounder that limits the performance of existing MIL methods. In this paper, we propose a novel scheme, Interventional Bag Multi-Instance Learning (IBMIL), to achieve deconfounded bag-level prediction. Unlike traditional likelihood-based strategies, the proposed scheme is based on the backdoor adjustment to achieve the interventional training, thus is capable of suppressing the bias caused by the bag contextual prior. Note that the principle of IBMIL is orthogonal to existing bag MIL methods. Therefore, IBMIL is able to bring consistent performance boosting to existing schemes, achieving new state-of-the-art performance. Code is available at https://github.com/HHHedo/IBMIL.

cs.CV↗
arXiv2023

Understanding and Constructing Latent Modality Structures in Multi-modal Representation Learning

Contrastive loss has been increasingly used in learning representations from multiple modalities. In the limit, the nature of the contrastive loss encourages modalities to exactly match each other in the latent space. Yet it remains an open question how the modality alignment affects the downstream task performance. In this paper, based on an information-theoretic argument, we first prove that exact modality alignment is sub-optimal in general for downstream prediction tasks. Hence we advocate that the key of better performance lies in meaningful latent modality structures instead of perfect modality alignment. To this end, we propose three general approaches to construct latent modality structures. Specifically, we design 1) a deep feature separation loss for intra-modality regularization; 2) a Brownian-bridge loss for inter-modality regularization; and 3) a geometric consistency loss for both intra- and inter-modality regularization. Extensive experiments are conducted on two popular multi-modal representation learning frameworks: the CLIP-based two-tower model and the ALBEF-based fusion model. We test our model on a variety of tasks including zero/few-shot image classification, image-text retrieval, visual question answering, visual reasoning, and visual entailment. Our method achieves consistent improvements over existing methods, demonstrating the effectiveness and generalizability of our proposed approach on latent modality structure regularization.

cs.LG↗
arXiv2023

Super quasi-bound state in the continuum

Avoided crossing of resonances and merging multiple bound states in the continuum (BICs) are parallel means for tailoring the physical properties of BICs. Herein, we introduce a new concept of super quasi-BIC for photonic crystal (PhC) systems where its quality ($Q$) factor is boosted in both parametric and momentum spaces. A super quasi-BIC with substantial enhancement of $Q$ factor can be achieved in a finite PhC by combining avoiding crossing of two symmetry protected (SP) quasi-BICs in parametric space and merging BICs in momentum space simultaneously. More importantly, analytical theory shows that the proposed mechanism results in the transition of asymptotic behavior of the $Q$ factor over the numbers of resonators from $N^2$ to exclusive $N^3$ for SP-BICs, which is of vital importance for realizing quasi-BICs in a compact PhC. Microwave experiments are performed to validate the theoretical results. Our results provide a paradigm shift for manipulating the physical properties quasi-BICs in finite PhC structures, which would facilitate various applications, including but not limited to low threshold lasing, wireless power transfer and high figure of merit sensing etc.

physics.optics↗
arXiv2023

NeRFPlayer: A Streamable Dynamic Scene Representation with Decomposed Neural Radiance Fields

Visually exploring in a real-world 4D spatiotemporal space freely in VR has been a long-term quest. The task is especially appealing when only a few or even single RGB cameras are used for capturing the dynamic scene. To this end, we present an efficient framework capable of fast reconstruction, compact modeling, and streamable rendering. First, we propose to decompose the 4D spatiotemporal space according to temporal characteristics. Points in the 4D space are associated with probabilities of belonging to three categories: static, deforming, and new areas. Each area is represented and regularized by a separate neural field. Second, we propose a hybrid representations based feature streaming scheme for efficiently modeling the neural fields. Our approach, coined NeRFPlayer, is evaluated on dynamic scenes captured by single hand-held cameras and multi-camera arrays, achieving comparable or superior rendering performance in terms of quality and speed comparable to recent state-of-the-art methods, achieving reconstruction in 10 seconds per frame and interactive rendering.

cs.CV↗
arXiv2023

Making Reconstruction-based Method Great Again for Video Anomaly Detection

Anomaly detection in videos is a significant yet challenging problem. Previous approaches based on deep neural networks employ either reconstruction-based or prediction-based approaches. Nevertheless, existing reconstruction-based methods 1) rely on old-fashioned convolutional autoencoders and are poor at modeling temporal dependency; 2) are prone to overfit the training samples, leading to indistinguishable reconstruction errors of normal and abnormal frames during the inference phase. To address such issues, firstly, we get inspiration from transformer and propose ${\textbf S}$patio-${\textbf T}$emporal ${\textbf A}$uto-${\textbf T}$rans-${\textbf E}$ncoder, dubbed as $\textbf{STATE}$, as a new autoencoder model for enhanced consecutive frame reconstruction. Our STATE is equipped with a specifically designed learnable convolutional attention module for efficient temporal learning and reasoning. Secondly, we put forward a novel reconstruction-based input perturbation technique during testing to further differentiate anomalous frames. With the same perturbation magnitude, the testing reconstruction error of the normal frames lowers more than that of the abnormal frames, which contributes to mitigating the overfitting problem of reconstruction. Owing to the high relevance of the frame abnormality and the objects in the frame, we conduct object-level reconstruction using both the raw frame and the corresponding optical flow patches. Finally, the anomaly score is designed based on the combination of the raw and motion reconstruction errors using perturbed inputs. Extensive experiments on benchmark video anomaly detection datasets demonstrate that our approach outperforms previous reconstruction-based methods by a notable margin, and achieves state-of-the-art anomaly detection performance consistently. The code is available at https://github.com/wyzjack/MRMGA4VAD.

cs.CV↗
arXiv2022

Efficient Few-Shot Object Detection via Knowledge Inheritance

Few-shot object detection (FSOD), which aims at learning a generic detector that can adapt to unseen tasks with scarce training samples, has witnessed consistent improvement recently. However, most existing methods ignore the efficiency issues, e.g., high computational complexity and slow adaptation speed. Notably, efficiency has become an increasingly important evaluation metric for few-shot techniques due to an emerging trend toward embedded AI. To this end, we present an efficient pretrain-transfer framework (PTF) baseline with no computational increment, which achieves comparable results with previous state-of-the-art (SOTA) methods. Upon this baseline, we devise an initializer named knowledge inheritance (KI) to reliably initialize the novel weights for the box classifier, which effectively facilitates the knowledge transfer process and boosts the adaptation speed. Within the KI initializer, we propose an adaptive length re-scaling (ALR) strategy to alleviate the vector length inconsistency between the predicted novel weights and the pretrained base weights. Finally, our approach not only achieves the SOTA results across three public benchmarks, i.e., PASCAL VOC, COCO and LVIS, but also exhibits high efficiency with 1.8-100x faster adaptation speed against the other methods on COCO/LVIS benchmark during few-shot transfer. To our best knowledge, this is the first work to consider the efficiency problem in FSOD. We hope to motivate a trend toward powerful yet efficient few-shot technique development. The codes are publicly available at https://github.com/Ze-Yang/Efficient-FSOD.

cs.CV↗
arXiv2022

CLIP-FLow: Contrastive Learning by semi-supervised Iterative Pseudo labeling for Optical Flow Estimation

Synthetic datasets are often used to pretrain end-to-end optical flow networks, due to the lack of a large amount of labeled, real-scene data. But major drops in accuracy occur when moving from synthetic to real scenes. How do we better transfer the knowledge learned from synthetic to real domains? To this end, we propose CLIP-FLow, a semi-supervised iterative pseudo-labeling framework to transfer the pretraining knowledge to the target real domain. We leverage large-scale, unlabeled real data to facilitate transfer learning with the supervision of iteratively updated pseudo-ground truth labels, bridging the domain gap between the synthetic and the real. In addition, we propose a contrastive flow loss on reference features and the warped features by pseudo ground truth flows, to further boost the accurate matching and dampen the mismatching due to motion, occlusion, or noisy pseudo labels. We adopt RAFT as the backbone and obtain an F1-all error of 4.11%, i.e. a 19% error reduction from RAFT (5.10%) and ranking 2$^{nd}$ place at submission on the KITTI 2015 benchmark. Our framework can also be extended to other models, e.g. CRAFT, reducing the F1-all error from 4.79% to 4.66% on KITTI 2015 benchmark.

cs.CV↗