Searcharxiv⌕ Search

arXiv subjects

Tao Chen

Publications and source records attributed to Tao Chen.

At least 343 records · Page 19Linked to original sources
arXiv2023

Attention Consistency Refined Masked Frequency Forgery Representation for Generalizing Face Forgery Detection

Due to the successful development of deep image generation technology, visual data forgery detection would play a more important role in social and economic security. Existing forgery detection methods suffer from unsatisfactory generalization ability to determine the authenticity in the unseen domain. In this paper, we propose a novel Attention Consistency Refined masked frequency forgery representation model toward generalizing face forgery detection algorithm (ACMF). Most forgery technologies always bring in high-frequency aware cues, which make it easy to distinguish source authenticity but difficult to generalize to unseen artifact types. The masked frequency forgery representation module is designed to explore robust forgery cues by randomly discarding high-frequency information. In addition, we find that the forgery attention map inconsistency through the detection network could affect the generalizability. Thus, the forgery attention consistency is introduced to force detectors to focus on similar attention regions for better generalization ability. Experiment results on several public face forgery datasets (FaceForensic++, DFD, Celeb-DF, and WDF datasets) demonstrate the superior performance of the proposed method compared with the state-of-the-art methods.

cs.CV↗
arXiv2023

MotionGPT: Human Motion as a Foreign Language

Though the advancement of pre-trained large language models unfolds, the exploration of building a unified model for language and other multi-modal data, such as motion, remains challenging and untouched so far. Fortunately, human motion displays a semantic coupling akin to human language, often perceived as a form of body language. By fusing language data with large-scale motion models, motion-language pre-training that can enhance the performance of motion-related tasks becomes feasible. Driven by this insight, we propose MotionGPT, a unified, versatile, and user-friendly motion-language model to handle multiple motion-relevant tasks. Specifically, we employ the discrete vector quantization for human motion and transfer 3D motion into motion tokens, similar to the generation process of word tokens. Building upon this "motion vocabulary", we perform language modeling on both motion and text in a unified manner, treating human motion as a specific language. Moreover, inspired by prompt learning, we pre-train MotionGPT with a mixture of motion-language data and fine-tune it on prompt-based question-and-answer tasks. Extensive experiments demonstrate that MotionGPT achieves state-of-the-art performances on multiple motion tasks including text-driven motion generation, motion captioning, motion prediction, and motion in-between.

cs.CV↗
arXiv2023

Breadcrumbs to the Goal: Goal-Conditioned Exploration from Human-in-the-Loop Feedback

Exploration and reward specification are fundamental and intertwined challenges for reinforcement learning. Solving sequential decision-making tasks requiring expansive exploration requires either careful design of reward functions or the use of novelty-seeking exploration bonuses. Human supervisors can provide effective guidance in the loop to direct the exploration process, but prior methods to leverage this guidance require constant synchronous high-quality human feedback, which is expensive and impractical to obtain. In this work, we present a technique called Human Guided Exploration (HuGE), which uses low-quality feedback from non-expert users that may be sporadic, asynchronous, and noisy. HuGE guides exploration for reinforcement learning not only in simulation but also in the real world, all without meticulous reward specification. The key concept involves bifurcating human feedback and policy learning: human feedback steers exploration, while self-supervised learning from the exploration data yields unbiased policies. This procedure can leverage noisy, asynchronous human feedback to learn policies with no hand-crafted reward design or exploration bonuses. HuGE is able to learn a variety of challenging multi-stage robotic navigation and manipulation tasks in simulation using crowdsourced feedback from non-expert users. Moreover, this paradigm can be scaled to learning directly on real-world robots, using occasional, asynchronous feedback from human supervisors.

cs.LG↗
arXiv2023

Holistic Prototype Attention Network for Few-Shot VOS

Few-shot video object segmentation (FSVOS) aims to segment dynamic objects of unseen classes by resorting to a small set of support images that contain pixel-level object annotations. Existing methods have demonstrated that the domain agent-based attention mechanism is effective in FSVOS by learning the correlation between support images and query frames. However, the agent frame contains redundant pixel information and background noise, resulting in inferior segmentation performance. Moreover, existing methods tend to ignore inter-frame correlations in query videos. To alleviate the above dilemma, we propose a holistic prototype attention network (HPAN) for advancing FSVOS. Specifically, HPAN introduces a prototype graph attention module (PGAM) and a bidirectional prototype attention module (BPAM), transferring informative knowledge from seen to unseen classes. PGAM generates local prototypes from all foreground features and then utilizes their internal correlations to enhance the representation of the holistic prototypes. BPAM exploits the holistic information from support images and video frames by fusing co-attention and self-attention to achieve support-query semantic consistency and inner-frame temporal consistency. Extensive experiments on YouTube-FSVOS have been provided to demonstrate the effectiveness and superiority of our proposed HPAN method.

cs.CV↗
arXiv2023

Michelangelo: Conditional 3D Shape Generation based on Shape-Image-Text Aligned Latent Representation

We present a novel alignment-before-generation approach to tackle the challenging task of generating general 3D shapes based on 2D images or texts. Directly learning a conditional generative model from images or texts to 3D shapes is prone to producing inconsistent results with the conditions because 3D shapes have an additional dimension whose distribution significantly differs from that of 2D images and texts. To bridge the domain gap among the three modalities and facilitate multi-modal-conditioned 3D shape generation, we explore representing 3D shapes in a shape-image-text-aligned space. Our framework comprises two models: a Shape-Image-Text-Aligned Variational Auto-Encoder (SITA-VAE) and a conditional Aligned Shape Latent Diffusion Model (ASLDM). The former model encodes the 3D shapes into the shape latent space aligned to the image and text and reconstructs the fine-grained 3D neural fields corresponding to given shape embeddings via the transformer-based decoder. The latter model learns a probabilistic mapping function from the image or text space to the latent shape space. Our extensive experiments demonstrate that our proposed approach can generate higher-quality and more diverse 3D shapes that better semantically conform to the visual or textural conditional inputs, validating the effectiveness of the shape-image-text-aligned space for cross-modality 3D shape generation.

cs.CV↗
arXiv2023

Nonadiabatic Holonomic Quantum Computation and Its Optimal Control

Geometric phase has the intrinsic property of being resistant to some types of local noises as it only depends on global properties of the evolution path. Meanwhile, the non-Abelian geometric phase is in the matrix form, and thus can naturally be used to implement high performance quantum gates, i.e., the so-called holonomic quantum computation. This article reviews recent advances in nonadiabatic holonomic quantum computation, and focuses on various optimal control approaches that can improve the gate performance, in terms of the gate fidelity and robustness. Besides, we also pay special attention to its possible physical realizations and some concrete examples of experimental realizations. Finally, with all these efforts, within state-of-the-art technology, the performance of the implemented holonomic quantum gates can outperform the conventional dynamical ones, under certain conditions.

quant-ph↗
arXiv2023

Topological Experience Replay

State-of-the-art deep Q-learning methods update Q-values using state transition tuples sampled from the experience replay buffer. This strategy often uniformly and randomly samples or prioritizes data sampling based on measures such as the temporal difference (TD) error. Such sampling strategies can be inefficient at learning Q-function because a state's Q-value depends on the Q-value of successor states. If the data sampling strategy ignores the precision of the Q-value estimate of the next state, it can lead to useless and often incorrect updates to the Q-values. To mitigate this issue, we organize the agent's experience into a graph that explicitly tracks the dependency between Q-values of states. Each edge in the graph represents a transition between two states by executing a single action. We perform value backups via a breadth-first search starting from that expands vertices in the graph starting from the set of terminal states and successively moving backward. We empirically show that our method is substantially more data-efficient than several baselines on a diverse range of goal-reaching tasks. Notably, the proposed method also outperforms baselines that consume more batches of training experience and operates from high-dimensional observational data such as images.

cs.LG↗
arXiv2023

QuadSwarm: A Modular Multi-Quadrotor Simulator for Deep Reinforcement Learning with Direct Thrust Control

Reinforcement learning (RL) has shown promise in creating robust policies for robotics tasks. However, contemporary RL algorithms are data-hungry, often requiring billions of environment transitions to train successful policies. This necessitates the use of fast and highly-parallelizable simulators. In addition to speed, such simulators need to model the physics of the robots and their interaction with the environment to a level acceptable for transferring policies learned in simulation to reality. We present QuadSwarm, a fast, reliable simulator for research in single and multi-robot RL for quadrotors that addresses both issues. QuadSwarm, with fast forward-dynamics propagation decoupled from rendering, is designed to be highly parallelizable such that throughput scales linearly with additional compute. It provides multiple components tailored toward multi-robot RL, including diverse training scenarios, and provides domain randomization to facilitate the development and sim2real transfer of multi-quadrotor control policies. Initial experiments suggest that QuadSwarm achieves over 48,500 simulation samples per second (SPS) on a single quadrotor and over 62,000 SPS on eight quadrotors on a 16-core CPU. The code can be found in https://github.com/Zhehui-Huang/quad-swarm-rl.

cs.RO↗
arXiv2023

Booster Free From Spin Resonance For Future 100~km-scale Circular e$^{+}$e$^{-}$ Colliders

Acceleration of polarized electron~(positron) beams in a booster synchrotron may suffer from depolarization due to crossings of many spin depolarization resonances, which could limit its applications. We have studied the spin depolarization resonance structure of a 100~km scale booster lattice of the Circular Electron Positron Collider~(CEPC). The lattice has 8 arc regions with hundreds of FODO cells, interleaved with straight sections, which leads to a high periodicity. Our analysis shows the contributions to the strength of intrinsic and imperfection spin resonances add up coherently near the super strong resonances beyond 120 GeV, but mostly cancel out and result in generally weak resonance strengths at lower beam energies. Detailed simulations confirm that beam polarization can be mostly maintained in the fast acceleration to 45.6 GeV and 80 GeV, but severe depolarization may occur at even higher energies. This study suggests the possibility of acceleration of polarized electron~(positron) beams to ultra-high beam energies without the help of Siberian snakes, and supports injecting highly polarized beams into the collider rings as an attractive solution for resonant depolarization measurements and longitudinal polarized colliding beam experiments for future 100~km scale circular e$^{+}$e$^{-}$ colliders.

physics.acc-ph↗
arXiv2023

Universal Robust Geometric Quantum Control via Geometric Trajectory Correction

Universal robust quantum control is essential for performing complex quantum algorithms and efficient quantum error correction protocols. Geometric phase, as a key element with intrinsic fault-tolerant feature, can be well integrated into quantum control processes to enhance control robustness. However, the current geometric quantum control is still controversial in robust universality, which leads to the unsatisfactory result that cannot sufficiently enhance the robustness of arbitrary type of geometric gate. In this study, we find that the finite choice on geometric evolution trajectory is one of the main roots that constrain the control robustness of previous geometric schemes, as it is unable to optionally avoid some trajectory segments that are seriously affected by systematic errors. In view of this, we here propose a new scheme for universal robust geometric control based on geometric trajectory correction, where enough available evolution parameters are introduced to ensure that the effective correction against systematic errors can be executed. From the results of our numerical simulation, arbitrary type of geometric gate implemented by using the corrected geometric trajectory has absolute robustness advantages over conventional quantum one. In addition, we also verify the feasibility of the high-fidelity physical implementation of our scheme in superconducting quantum circuit, and finally discuss in detail the potential researches based on our scheme. Therefore, our theoretical work is expected to offer an attractive avenue for realizing practical fault-tolerant quantum computation in existing experimental platforms.

quant-ph↗
arXiv2023

FX UMa: A New Heartbeat Binary System with Linear and Non-linear Tidal Oscillations and delta Sct Pulsations

We present a detailed analysis of an eclipsing double-lined binary FX UMa based on TESS photometry and newly acquired spectroscopic observations. The radial velocities and atmospheric parameters for each component star are obtained from the SONG high-resolution spectra. Combined with the radial-velocity measurements, our light-curve modeling yields absolute masses and radii of the two components. The Fourier amplitude spectrum of the residual light curve reveals a total of 103 frequencies with signal-to-noise ratio (S/N) > 4, including 12 independent frequencies, 17 multiples of the orbital frequency (Nforb), and 74 combination frequencies. Ten Nforb peaks with S/N > 10 have very high amplitudes and are likely due to tidally excited oscillations (TEOs). The remaining Nforb peaks (4 < S/N < 10) may be originated from the imperfect removal, or they are actually real TEOs. Four anharmonic frequencies can pair up and sum to give exact harmonics of the orbital frequency, suggesting the existence of non-linear tidal processes in the eccentric binary system FX UMa. Eight independent frequencies in the range of 20 to 32 day$^{-1}$ are typical low-order pressure modes of delta Scuti pulsators.

astro-ph.SR↗
arXiv2023

Strongly interacting Rydberg atoms in synthetic dimensions with a magnetic flux

Synthetic dimensions, wherein dynamics occurs in a set of internal states, have found great success in recent years in exploring topological effects in cold atoms and photonics. However, the phenomena thus far explored have largely been restricted to the non-interacting or weakly interacting regimes. Here, we extend the synthetic dimensions playbook to strongly interacting systems of Rydberg atoms prepared in optical tweezer arrays. We use precise control over driving microwave fields to introduce a tunable $U(1)$ flux in a four-site lattice of coupled Rydberg levels. We find highly coherent dynamics, in good agreement with theory. Single atoms show oscillatory dynamics controllable by the gauge field. Small arrays of interacting atoms exhibit behavior suggestive of the emergence of ergodic and arrested dynamics in the regimes of intermediate and strong interactions, respectively. These demonstrations pave the way for future explorations of strongly interacting dynamics and many-body phases in Rydberg synthetic lattices.

cond-mat.quant-gas↗
arXiv2023

StyleAvatar3D: Leveraging Image-Text Diffusion Models for High-Fidelity 3D Avatar Generation

The recent advancements in image-text diffusion models have stimulated research interest in large-scale 3D generative models. Nevertheless, the limited availability of diverse 3D resources presents significant challenges to learning. In this paper, we present a novel method for generating high-quality, stylized 3D avatars that utilizes pre-trained image-text diffusion models for data generation and a Generative Adversarial Network (GAN)-based 3D generation network for training. Our method leverages the comprehensive priors of appearance and geometry offered by image-text diffusion models to generate multi-view images of avatars in various styles. During data generation, we employ poses extracted from existing 3D models to guide the generation of multi-view images. To address the misalignment between poses and images in data, we investigate view-specific prompts and develop a coarse-to-fine discriminator for GAN training. We also delve into attribute-related prompts to increase the diversity of the generated avatars. Additionally, we develop a latent diffusion model within the style space of StyleGAN to enable the generation of avatars based on image inputs. Our approach demonstrates superior performance over current state-of-the-art methods in terms of visual quality and diversity of the produced avatars.

cs.CV↗
arXiv2023

Swin Transformer coupling CNNs Makes Strong Contextual Encoders for VHR Image Road Extraction

Accurately segmenting roads is challenging due to substantial intra-class variations, indistinct inter-class distinctions, and occlusions caused by shadows, trees, and buildings. To address these challenges, attention to important texture details and perception of global geometric contextual information are essential. Recent research has shown that CNN-Transformer hybrid structures outperform using CNN or Transformer alone. While CNN excels at extracting local detail features, the Transformer naturally perceives global contextual information. In this paper, we propose a dual-branch network block named ConSwin that combines ResNet and SwinTransformers for road extraction tasks. This ConSwin block harnesses the strengths of both approaches to better extract detailed and global features. Based on ConSwin, we construct an hourglass-shaped road extraction network and introduce two novel connection structures to better transmit texture and structural detail information to the decoder. Our proposed method outperforms state-of-the-art methods on both the Massachusetts and CHN6-CUG datasets in terms of overall accuracy, IOU, and F1 indicators. Additional experiments validate the effectiveness of our proposed module, while visualization results demonstrate its ability to obtain better road representations.

cs.CV↗
arXiv2023

Tucker Bilinear Attention Network for Multi-scale Remote Sensing Object Detection

Object detection on VHR remote sensing images plays a vital role in applications such as urban planning, land resource management, and rescue missions. The large-scale variation of the remote-sensing targets is one of the main challenges in VHR remote-sensing object detection. Existing methods improve the detection accuracy of high-resolution remote sensing objects by improving the structure of feature pyramids and adopting different attention modules. However, for small targets, there still be seriously missed detections due to the loss of key detail features. There is still room for improvement in the way of multiscale feature fusion and balance. To address this issue, this paper proposes two novel modules: Guided Attention and Tucker Bilinear Attention, which are applied to the stages of early fusion and late fusion respectively. The former can effectively retain clean key detail features, and the latter can better balance features through semantic-level correlation mining. Based on two modules, we build a new multi-scale remote sensing object detection framework. No bells and whistles. The proposed method largely improves the average precisions of small objects and achieves the highest mean average precisions compared with 9 state-of-the-art methods on DOTA, DIOR, and NWPU VHR-10.Code and models are available at https://github.com/Shinichict/GTNet.

cs.CV↗
arXiv2023

Spectroscopy of momentum state lattices

We explore a technique for probing energy spectra in synthetic lattices that is analogous to scanning tunneling microscopy. Using one-dimensional synthetic lattices of coupled atomic momentum states, we explore this spectroscopic technique and observe qualitative agreement between the measured and simulated energy spectra for small two- and three-site lattices as well as a uniform many-site lattice. Finally, through simulations, we show that this technique should allow for the exploration of the topological bands and the fractal energy spectrum of the Hofstadter model as realized in synthetic lattices.

cond-mat.quant-gas↗
arXiv2023

Executing your Commands via Motion Diffusion in Latent Space

We study a challenging task, conditional human motion generation, which produces plausible human motion sequences according to various conditional inputs, such as action classes or textual descriptors. Since human motions are highly diverse and have a property of quite different distribution from conditional modalities, such as textual descriptors in natural languages, it is hard to learn a probabilistic mapping from the desired conditional modality to the human motion sequences. Besides, the raw motion data from the motion capture system might be redundant in sequences and contain noises; directly modeling the joint distribution over the raw motion sequences and conditional modalities would need a heavy computational overhead and might result in artifacts introduced by the captured noises. To learn a better representation of the various human motion sequences, we first design a powerful Variational AutoEncoder (VAE) and arrive at a representative and low-dimensional latent code for a human motion sequence. Then, instead of using a diffusion model to establish the connections between the raw motion sequences and the conditional inputs, we perform a diffusion process on the motion latent space. Our proposed Motion Latent-based Diffusion model (MLD) could produce vivid motion sequences conforming to the given conditional inputs and substantially reduce the computational overhead in both the training and inference stages. Extensive experiments on various human motion generation tasks demonstrate that our MLD achieves significant improvements over the state-of-the-art methods among extensive human motion generation tasks, with two orders of magnitude faster than previous diffusion models on raw motion sequences.

cs.CV↗
arXiv2023

Boost Vision Transformer with GPU-Friendly Sparsity and Quantization

The transformer extends its success from the language to the vision domain. Because of the stacked self-attention and cross-attention blocks, the acceleration deployment of vision transformer on GPU hardware is challenging and also rarely studied. This paper thoroughly designs a compression scheme to maximally utilize the GPU-friendly 2:4 fine-grained structured sparsity and quantization. Specially, an original large model with dense weight parameters is first pruned into a sparse one by 2:4 structured pruning, which considers the GPU's acceleration of 2:4 structured sparse pattern with FP16 data type, then the floating-point sparse model is further quantized into a fixed-point one by sparse-distillation-aware quantization aware training, which considers GPU can provide an extra speedup of 2:4 sparse calculation with integer tensors. A mixed-strategy knowledge distillation is used during the pruning and quantization process. The proposed compression scheme is flexible to support supervised and unsupervised learning styles. Experiment results show GPUSQ-ViT scheme achieves state-of-the-art compression by reducing vision transformer models 6.4-12.7 times on model size and 30.3-62 times on FLOPs with negligible accuracy degradation on ImageNet classification, COCO detection and ADE20K segmentation benchmarking tasks. Moreover, GPUSQ-ViT can boost actual deployment performance by 1.39-1.79 times and 3.22-3.43 times of latency and throughput on A100 GPU, and 1.57-1.69 times and 2.11-2.51 times improvement of latency and throughput on AGX Orin.

cs.CV↗