arXiv subjects

Shiwei Zhang

Publications and source records attributed to Shiwei Zhang.

At least 145 records · Page 8Linked to original sources

arXiv2021

OadTR: Online Action Detection with Transformers

Most recent approaches for online action detection tend to apply Recurrent Neural Network (RNN) to capture long-range temporal structure. However, RNN suffers from non-parallelism and gradient vanishing, hence it is hard to be optimized. In this paper, we propose a new encoder-decoder framework based on Transformers, named OadTR, to tackle these problems. The encoder attached with a task token aims to capture the relationships and global interactions between historical observations. The decoder extracts auxiliary information by aggregating anticipated future clip representations. Therefore, OadTR can recognize current actions by encoding historical information and predicting future context simultaneously. We extensively evaluate the proposed OadTR on three challenging datasets: HDD, TVSeries, and THUMOS14. The experimental results show that OadTR achieves higher training and inference speeds than current RNN based approaches, and significantly outperforms the state-of-the-art methods in terms of both mAP and mcAP. Code is available at https://github.com/wangxiang1230/OadTR.

cs.CV↗

arXiv2021

Weakly-Supervised Temporal Action Localization Through Local-Global Background Modeling

Weakly-Supervised Temporal Action Localization (WS-TAL) task aims to recognize and localize temporal starts and ends of action instances in an untrimmed video with only video-level label supervision. Due to lack of negative samples of background category, it is difficult for the network to separate foreground and background, resulting in poor detection performance. In this report, we present our 2021 HACS Challenge - Weakly-supervised Learning Track solution that based on BaSNet to address above problem. Specifically, we first adopt pre-trained CSN, Slowfast, TDN, and ViViT as feature extractors to get feature sequences. Then our proposed Local-Global Background Modeling Network (LGBM-Net) is trained to localize instances by using only video-level labels based on Multi-Instance Learning (MIL). Finally, we ensemble multiple models to get the final detection results and reach 22.45% mAP on the test set

cs.CV↗

arXiv2021

Proposal Relation Network for Temporal Action Detection

This technical report presents our solution for temporal action detection task in AcitivityNet Challenge 2021. The purpose of this task is to locate and identify actions of interest in long untrimmed videos. The crucial challenge of the task comes from that the temporal duration of action varies dramatically, and the target actions are typically embedded in a background of irrelevant activities. Our solution builds on BMN, and mainly contains three steps: 1) action classification and feature encoding by Slowfast, CSN and ViViT; 2) proposal generation. We improve BMN by embedding the proposed Proposal Relation Network (PRN), by which we can generate proposals of high quality; 3) action detection. We calculate the detection results by assigning the proposals with corresponding classification results. Finally, we ensemble the results under different settings and achieve 44.7% on the test set, which improves the champion result in ActivityNet 2020 by 1.9% in terms of average mAP.

cs.CV↗

arXiv2021

Relation Modeling in Spatio-Temporal Action Localization

This paper presents our solution to the AVA-Kinetics Crossover Challenge of ActivityNet workshop at CVPR 2021. Our solution utilizes multiple types of relation modeling methods for spatio-temporal action detection and adopts a training strategy to integrate multiple relation modeling in end-to-end training over the two large-scale video datasets. Learning with memory bank and finetuning for long-tailed distribution are also investigated to further improve the performance. In this paper, we detail the implementations of our solution and provide experiments results and corresponding discussions. We finally achieve 40.67 mAP on the test set of AVA-Kinetics.

cs.CV↗

arXiv2021

A Stronger Baseline for Ego-Centric Action Detection

This technical report analyzes an egocentric video action detection method we used in the 2021 EPIC-KITCHENS-100 competition hosted in CVPR2021 Workshop. The goal of our task is to locate the start time and the end time of the action in the long untrimmed video, and predict action category. We adopt sliding window strategy to generate proposals, which can better adapt to short-duration actions. In addition, we show that classification and proposals are conflict in the same network. The separation of the two tasks boost the detection performance with high efficiency. By simply employing these strategy, we achieved 16.10\% performance on the test set of EPIC-KITCHENS-100 Action Detection challenge using a single model, surpassing the baseline method by 11.7\% in terms of average mAP.

cs.CV↗

arXiv2021

Towards Training Stronger Video Vision Transformers for EPIC-KITCHENS-100 Action Recognition

With the recent surge in the research of vision transformers, they have demonstrated remarkable potential for various challenging computer vision applications, such as image recognition, point cloud classification as well as video understanding. In this paper, we present empirical results for training a stronger video vision transformer on the EPIC-KITCHENS-100 Action Recognition dataset. Specifically, we explore training techniques for video vision transformers, such as augmentations, resolutions as well as initialization, etc. With our training recipe, a single ViViT model achieves the performance of 47.4\% on the validation set of EPIC-KITCHENS-100 dataset, outperforming what is reported in the original paper by 3.4%. We found that video transformers are especially good at predicting the noun in the verb-noun action prediction task. This makes the overall action prediction accuracy of video transformers notably higher than convolutional ones. Surprisingly, even the best video transformers underperform the convolutional networks on the verb prediction. Therefore, we combine the video vision transformers and some of the convolutional video networks and present our solution to the EPIC-KITCHENS-100 Action Recognition competition.

cs.CV↗

arXiv2021

Muonium hydride: the lowest density crystal

A muonium hydride molecule is a bound state of muonium and hydrogen atoms. It has half the mass of a parahydrogen molecule and very similar electronic properties in its ground state. The phase diagram of an assembly of such particles is investigated by first principle quantum simulations. In the bulk limit, the low-temperature equilibrium phase is a crystal of extraordinarily low density, lower than that of any other known atomic or molecular crystal. Despite the low density and particle mass, the melting temperature is surprisingly high (close to 9 K). No (metastable) supersolid phase is observed. We investigated the physical properties of nanoscale clusters (up to 200 particles) of muonium hydride and found the superfluid response to be greatly enhanced compared to that of parahydrogen clusters. The possible experimental realization of these systems is discussed.

cond-mat.stat-mech↗

arXiv2021

Self-Supervised Learning for Semi-Supervised Temporal Action Proposal

Self-supervised learning presents a remarkable performance to utilize unlabeled data for various video tasks. In this paper, we focus on applying the power of self-supervised methods to improve semi-supervised action proposal generation. Particularly, we design an effective Self-supervised Semi-supervised Temporal Action Proposal (SSTAP) framework. The SSTAP contains two crucial branches, i.e., temporal-aware semi-supervised branch and relation-aware self-supervised branch. The semi-supervised branch improves the proposal model by introducing two temporal perturbations, i.e., temporal feature shift and temporal feature flip, in the mean teacher framework. The self-supervised branch defines two pretext tasks, including masked feature reconstruction and clip-order prediction, to learn the relation of temporal clues. By this means, SSTAP can better explore unlabeled videos, and improve the discriminative abilities of learned action features. We extensively evaluate the proposed SSTAP on THUMOS14 and ActivityNet v1.3 datasets. The experimental results demonstrate that SSTAP significantly outperforms state-of-the-art semi-supervised methods and even matches fully-supervised methods. Code is available at https://github.com/wangxiang1230/SSTAP.

cs.CV↗

arXiv2021

Self-supervised Motion Learning from Static Images

Motions are reflected in videos as the movement of pixels, and actions are essentially patterns of inconsistent motions between the foreground and the background. To well distinguish the actions, especially those with complicated spatio-temporal interactions, correctly locating the prominent motion areas is of crucial importance. However, most motion information in existing videos are difficult to label and training a model with good motion representations with supervision will thus require a large amount of human labour for annotation. In this paper, we address this problem by self-supervised learning. Specifically, we propose to learn Motion from Static Images (MoSI). The model learns to encode motion information by classifying pseudo motions generated by MoSI. We furthermore introduce a static mask in pseudo motions to create local motion patterns, which forces the model to additionally locate notable motion areas for the correct classification.We demonstrate that MoSI can discover regions with large motion even without fine-tuning on the downstream datasets. As a result, the learned motion representations boost the performance of tasks requiring understanding of complex scenes and motions, i.e., action recognition. Extensive experiments show the consistent and transferable improvements achieved by MoSI. Codes will be soon released.

cs.CV↗

arXiv2021

Tracking the Footprints of Spin Fluctuations: A MultiMethod, MultiMessenger Study of the Two-Dimensional Hubbard Model

The Hubbard model represents the fundamental model for interacting quantum systems and electronic correlations. Using the two-dimensional half-filled Hubbard model at weak coupling as a testing ground, we perform a comparative study of a comprehensive set of state of the art quantum many-body methods. Upon cooling into its insulating antiferromagnetic ground-state, the model hosts a rich sequence of distinct physical regimes with crossovers between a high-temperature incoherent regime, an intermediate temperature metallic regime and a low-temperature insulating regime with a pseudogap created by antiferromagnetic fluctuations. We assess the ability of each method to properly address these physical regimes and crossovers through the computation of several observables probing both quasiparticle properties and magnetic correlations, with two numerically exact methods (diagrammatic and determinantal quantum Monte Carlo) serving as a benchmark. By combining computational results and analytical insights, we elucidate the nature and role of spin fluctuations in each of these regimes. Based on this analysis, we explain how quasiparticles can coexist with increasingly long-range antiferromagnetic correlations, and why dynamical mean-field theory is found to provide a remarkably accurate approximation of local quantities in the metallic regime. We also critically discuss whether imaginary time methods are able to capture the non-Fermi liquid singularities of this fully nested system.

cond-mat.str-el↗

arXiv2021

Ab initio electronic density in solids by many-body plane-wave auxiliary-field quantum Monte Carlo calculations

We present accurate many-body results of the electronic densities in several solid materials, including Si, NaCl, and Cu. These results are obtained using the ab initio auxiliary-field quantum Monte Carlo (AFQMC) method working in a plane-wave basis with norm-conserving, multiple-projector pseudopotentials. AFQMC has been shown to be an excellent many-body total energy method. Computation of observables and correlation functions other than the ground-state energy requires back-propagation, whose adaption and implementation in the plane-wave basis AFQMC framework are discussed in the present paper. This development allows us to compute correlation functions, electronic densities and interatomic forces, paving the way for geometry optimizations and calculations of thermodynamic properties in solids. Finite supercell size effects are considerably more subtle in the many-body framework than in independent-electron calculations. We analyze the convergence of the electronic density, and obtain best estimates for the thermodynamic limit. The densities from several typical density functionals are benchmarked against our near-exact results. The electronic densities we have obtained can also be used to help construct improved density functionals.

cond-mat.mtrl-sci↗

arXiv2020

Constrained-Path Auxiliary-Field Quantum Monte Carlo for Coupled Electrons and Phonons

We present an extension of constrained-path auxiliary-field quantum Monte Carlo (CP-AFQMC) for the treatment of correlated electronic systems coupled to phonons. The algorithm follows the standard CP-AFQMC approach for description of the electronic degrees of freedom while phonons are described in first quantization and propagated via a diffusion Monte Carlo approach. Our method is tested on the one- and two-dimensional Holstein and Hubbard-Holstein models. With a simple semiclassical trial wavefunction, our approach is remarkably accurate for $ω/(2\text{d}tλ) < 1$ for all parameters in the Holstein model considered in this study. In addition, we empirically show that the autocorrelation time scales as $1/ω$ for $ω/t \lesssim 1$, which is an improvement over the $1/ω^2$ scaling of the conventional determinant quantum Monte Carlo algorithm. In the Hubbard-Holstein model, the accuracy of our algorithm is found to be consistent with that of standard CP-AFQMC for the Hubbard model when the Hubbard $U$ term dominates the physics of the model, and is nearly exact when the ground state is dominated by the electron-phonon coupling scale $λ$. The approach developed in this work should be valuable for understanding the complex physics arising from the interplay between electrons and phonons in both model lattice problems and ab-initio systems.

cond-mat.str-el↗

arXiv2020

Tuning the quantumness of simple Bose systems: A universal phase diagram

We present a comprehensive theoretical study of the phase diagram of a system of many Bose particles interacting with a two-body central potential of the so-called Lennard-Jones form. First-principles path-integral computations are carried out, providing essentially exact numerical results on the thermodynamic properties. The theoretical model used here provides a realistic and remarkably general framework for describing simple Bose systems ranging from crystals to normal fluids to superfluids and gases. The interplay between particle interactions on the one hand, quantum indistinguishability and delocalization on the other, is characterized by a single quantumness parameter, which can be tuned to engineer and explore different regimes. Taking advantage of the rare combination of the versatility of the many-body Hamiltonian and the possibility for exact computations, we systematically investigate the phases of the systems as a function of pressure (P) and temperature (T), as well as the quantumness parameter. We show how the topology of the phase diagram evolves from the known case of He-4, as the system is made more (and less) quantum, and compare our predictions with available results from mean-field theory. Possible realization and observation of the phases and physical regimes predicted here are discussed in various experimental systems, including hypothetical muonic matter.

cond-mat.stat-mech↗

arXiv2020

A Pseudo-BCS Wavefunction from Density Matrix Decomposition:Application in Auxiliary-Field Quantum Monte Carlo

We present a method to construct pseudo-BCS wave functions from the one-body density matrix. The resulting many-body wave function, which can be produced for any fermion systems, including those with purely repulsive interactions, has the form of a number-projected BCS form, or antisymmetrized germinal power (AGP). Such wave functions provide a better ansatz for correlated fermion systems than a single Slater determinant, and often better than a linear combination of Slater determinants (for example from a truncated active space calculation). We describe a procedure to build such a wave function conveniently from a given reduced density matrix of the system, rather than from a mean-field solution (which gives a Slater determinant for repulsive interactions). The pseudo-BCS wave function thus obtained reproduces the density matrix or minimizes the difference between the input and resulting density matrices. One application of the pseudo-BCS wave function is in auxiliary-field quantum Monte Carlo (AFQMC) calculations as the trial wave function to control the sign/phase problem. AFQMC is often among the most accurate general methods for correlated fermion systems. We show that the pseudo-BCS form further reduces the constraint bias and leads to improved accuracy compared to the usual Slater determinant trial wave functions, using the two-dimensional Hubbard model as an example. Furthermore, the pseudo-BCS trial wave function allows a new systematically improvable self-consistent approach, with pseudo-BCS trial wave function iteratively generated by AFQMC via the one-body density matrix.

cond-mat.str-el↗

arXiv2020

Some Recent Developments in Auxiliary-Field Quantum Monte Carlo for Real Materials

The auxiliary-field quantum Monte Carlo (AFQMC) method is a general numerical method for correlated many-electron systems, which is being increasingly applied in lattice models, atoms, molecules, and solids. Here we introduce the theory and algorithm of the method specialized for real materials, and present several recent developments. We give a systematic exposition of the key steps of AFQMC, closely tracking the framework of a modern software library we are developing. The building of a Monte Carlo Hamiltonian, projecting to the ground state, sampling two-body operators, phaseless approximation, and measuring ground state properties are discussed in details. An advanced implementation for multi-determinant trial wave functions is described which dramatically speeds up the algorithm and reduces the memory cost. We propose a self-consistent constraint for real materials, and discuss two flavors for its realization, either by coupling the AFQMC calculation to an effective independent-electron calculation, or via the natural orbitals of the computed one-body density matrix.

physics.comp-ph↗

arXiv2020

Magnetic and charge orders in the ground state of the Emery model -- accurate numerical results

We perform extensive auxiliary-field quantum Monte Carlo (AFQMC) calculations for the three-band Hubbard (Emery) model in order to study the ground-state properties of Copper-Oxygen planes in the cuprates. Employing cutting-edge AFQMC techniques with a self-consistent gauge constraint in auxiliary-field space to control the sign problem, we reach supercells containing around 500 atoms to capture collective modes in the charge and spin orders and characterize the behavior in the thermodynamic limit. The self-consistency scheme interfacing with generalized Hartree-Fock calculations allows high accuracy in AFQMC to resolve small energy scales, which is crucial for determining the complex candidate orders in such a system. We present detailed information on the charge order, spin order, momentum distribution, and localization properties as a function of charge-transfer energy for the the under-doped regime. In contrast with the stripe and spiral orders under hole-doping, we find that the corresponding 1/8 electron-doped system exhibits purely antiferromagnetic order in the three-band model, consistent with the asymmetry between electron and hole-doping in the phase diagram of cuprates.

cond-mat.supr-con↗

arXiv2020

Multi-Level Temporal Pyramid Network for Action Detection

Currently, one-stage frameworks have been widely applied for temporal action detection, but they still suffer from the challenge that the action instances span a wide range of time. The reason is that these one-stage detectors, e.g., Single Shot Multi-Box Detector (SSD), extract temporal features only applying a single-level layer for each head, which is not discriminative enough to perform classification and regression. In this paper, we propose a Multi-Level Temporal Pyramid Network (MLTPN) to improve the discrimination of the features. Specially, we first fuse the features from multiple layers with different temporal resolutions, to encode multi-layer temporal information. We then apply a multi-level feature pyramid architecture on the features to enhance their discriminative abilities. Finally, we design a simple yet effective feature fusion module to fuse the multi-level multi-scale features. By this means, the proposed MLTPN can learn rich and discriminative features for different action instances with different durations. We evaluate MLTPN on two challenging datasets: THUMOS'14 and Activitynet v1.3, and the experimental results show that MLTPN obtains competitive performance on Activitynet v1.3 and outperforms the state-of-the-art approaches on THUMOS'14 significantly.

cs.CV↗

arXiv2020

Absence of superconductivity in the pure two-dimensional Hubbard model

We study the superconducting pairing correlations in the ground state of the doped Hubbard model -- in its original form without hopping beyond nearest neighbor or other perturbing parameters -- in two dimensions at intermediate to strong coupling and near optimal doping. The nature of such correlations has been a central question ever since the discovery of cuprate high-temperature superconductors. Despite unprecedented effort and tremendous progress in understanding the properties of this fundamental model, a definitive answer to whether the ground state is superconducting in the parameter regime most relevant to cuprates has proved exceedingly difficult to establish. In this work, we employ two complementary, state-of-the-art many-body computational methods, constrained path (CP) auxiliary-field quantum Monte Carlo (AFQMC) and density matrix renormalization group (DMRG) methods, deploying the most recent algorithmic advances in each. Systematic and detailed comparisons between the two methods are performed. The DMRG is extremely reliable on small width cylinders, where we use it to validate the AFQMC. The AFQMC is then used to study wide systems as well as fully periodic systems, to establish that we have reached the thermodynamic limit. The ground state is found to be non-superconducting in the moderate to strong coupling regime in the vicinity of optimal hole doping.

cond-mat.str-el↗