arXiv subjects

Jie Song

Publications and source records attributed to Jie Song.

At least 217 records · Page 12Linked to original sources

arXiv2021

Human Performance Capture from Monocular Video in the Wild

Capturing the dynamically deforming 3D shape of clothed human is essential for numerous applications, including VR/AR, autonomous driving, and human-computer interaction. Existing methods either require a highly specialized capturing setup, such as expensive multi-view imaging systems, or they lack robustness to challenging body poses. In this work, we propose a method capable of capturing the dynamic 3D human shape from a monocular video featuring challenging body poses, without any additional input. We first build a 3D template human model of the subject based on a learned regression model. We then track this template model's deformation under challenging body articulations based on 2D image observations. Our method outperforms state-of-the-art methods on an in-the-wild human video dataset 3DPW. Moreover, we demonstrate its efficacy in robustness and generalizability on videos from iPER datasets.

cs.CV↗

arXiv2021

Systematic error tolerant multiqubit holonomic entangling gates

Quantum holonomic gates hold built-in resilience to local noises and provide a promising approach for implementing fault-tolerant quantum computation. We propose to realize high-fidelity holonomic $(N+1)$-qubit controlled gates using Rydberg atoms confined in optical arrays or superconducting circuits. We identify the scheme, deduce the effective multi-body Hamiltonian, and determine the working condition of the multiqubit gate. Uniquely, the multiqubit gate is immune to systematic errors, i.e., laser parameter fluctuations and motional dephasing, as the $N$ control atoms largely remain in the much stable qubit space during the operation. We show that $C_N$-NOT gates can reach same level of fidelity at a given gate time for $N\leq5$ under a suitable choice of parameters, and the gate tolerance against errors in systematic parameters can be further enhanced through optimal pulse engineering. In case of Rydberg atoms, the proposed protocol is intrinsically different from typical schemes based on Rydberg blockade or antiblockade. Our study paves a new route to build robust multiqubit gates with Rydberg atoms trapped in optical arrays or with superconducting circuits. It contributes to current efforts in developing scalable quantum computation with trapped atoms and fabricable superconducting devices.

quant-ph↗

arXiv2021

Unidirectional acoustic metamaterials based on nonadiabatic holonomic quantum transformations

Nonadiabatic holonomic quantum transformations (NHQTs) have attracted wide attention and have been applied in many aspects of quantum computation, whereas related research is usually limited to the field of quantum physics. Here we bring NHQTs into constructing a unidirectional acoustic metamaterial (UDAM) for shaping classical beams. The UDAM is made up of an array of three-waveguide couplers, where the propagation of acoustic waves mimics the evolution of NHQTs. The excellent agreement among analytical predictions, numerical simulations, and experimental measurements confirms the great applicability of NHQTs in acoustic metamaterial engineering. The present work extends research on NHQTs from quantum physics to the field of classical waves for designing metamaterials with simple structures and may pave a new way to design UDAMs that would be of potential applications in acoustic isolation, communication, and stealth.

physics.class-ph↗

arXiv2021

Render In-between: Motion Guided Video Synthesis for Action Interpolation

Upsampling videos of human activity is an interesting yet challenging task with many potential applications ranging from gaming to entertainment and sports broadcasting. The main difficulty in synthesizing video frames in this setting stems from the highly complex and non-linear nature of human motion and the complex appearance and texture of the body. We propose to address these issues in a motion-guided frame-upsampling framework that is capable of producing realistic human motion and appearance. A novel motion model is trained to inference the non-linear skeletal motion between frames by leveraging a large-scale motion-capture dataset (AMASS). The high-frame-rate pose predictions are then used by a neural rendering pipeline to produce the full-frame output, taking the pose and background consistency into consideration. Our pipeline only requires low-frame-rate videos and unpaired human motion data but does not require high-frame-rate videos for training. Furthermore, we contribute the first evaluation dataset that consists of high-quality and high-frame-rate videos of human activities for this task. Compared with state-of-the-art video interpolation techniques, our method produces in-between frames with better quality and accuracy, which is evident by state-of-the-art results on pixel-level, distributional metrics and comparative user evaluations. Our code and the collected dataset are available at https://git.io/Render-In-Between.

cs.CV↗

arXiv2021

Mosaicking to Distill: Knowledge Distillation from Out-of-Domain Data

Knowledge distillation~(KD) aims to craft a compact student model that imitates the behavior of a pre-trained teacher in a target domain. Prior KD approaches, despite their gratifying results, have largely relied on the premise that \emph{in-domain} data is available to carry out the knowledge transfer. Such an assumption, unfortunately, in many cases violates the practical setting, since the original training data or even the data domain is often unreachable due to privacy or copyright reasons. In this paper, we attempt to tackle an ambitious task, termed as \emph{out-of-domain} knowledge distillation~(OOD-KD), which allows us to conduct KD using only OOD data that can be readily obtained at a very low cost. Admittedly, OOD-KD is by nature a highly challenging task due to the agnostic domain gap. To this end, we introduce a handy yet surprisingly efficacious approach, dubbed as~\textit{MosaicKD}. The key insight behind MosaicKD lies in that, samples from various domains share common local patterns, even though their global semantic may vary significantly; these shared local patterns, in turn, can be re-assembled analogous to mosaic tiling, to approximate the in-domain data and to further alleviating the domain discrepancy. In MosaicKD, this is achieved through a four-player min-max game, in which a generator, a discriminator, a student network, are collectively trained in an adversarial manner, partially under the guidance of a pre-trained teacher. We validate MosaicKD over {classification and semantic segmentation tasks} across various benchmarks, and demonstrate that it yields results much superior to the state-of-the-art counterparts on OOD data. Our code is available at \url{https://github.com/zju-vipa/MosaicKD}.

cs.LG↗

arXiv2021

Revisiting Dimensionality Reduction Techniques for Visual Cluster Analysis: An Empirical Study

Dimensionality Reduction (DR) techniques can generate 2D projections and enable visual exploration of cluster structures of high-dimensional datasets. However, different DR techniques would yield various patterns, which significantly affect the performance of visual cluster analysis tasks. We present the results of a user study that investigates the influence of different DR techniques on visual cluster analysis. Our study focuses on the most concerned property types, namely the linearity and locality, and evaluates twelve representative DR techniques that cover the concerned properties. Four controlled experiments were conducted to evaluate how the DR techniques facilitate the tasks of 1) cluster identification, 2) membership identification, 3) distance comparison, and 4) density comparison, respectively. We also evaluated users' subjective preference of the DR techniques regarding the quality of projected clusters. The results show that: 1) Non-linear and Local techniques are preferred in cluster identification and membership identification; 2) Linear techniques perform better than non-linear techniques in density comparison; 3) UMAP (Uniform Manifold Approximation and Projection) and t-SNE (t-Distributed Stochastic Neighbor Embedding) perform the best in cluster identification and membership identification; 4) NMF (Nonnegative Matrix Factorization) has competitive performance in distance comparison; 5) t-SNLE (t-Distributed Stochastic Neighbor Linear Embedding) has competitive performance in density comparison.

cs.HC↗

arXiv2021

Shape-aware Multi-Person Pose Estimation from Multi-View Images

In this paper we contribute a simple yet effective approach for estimating 3D poses of multiple people from multi-view images. Our proposed coarse-to-fine pipeline first aggregates noisy 2D observations from multiple camera views into 3D space and then associates them into individual instances based on a confidence-aware majority voting technique. The final pose estimates are attained from a novel optimization scheme which links high-confidence multi-view 2D observations and 3D joint candidates. Moreover, a statistical parametric body model such as SMPL is leveraged as a regularizing prior for these 3D joint candidates. Specifically, both 3D poses and SMPL parameters are optimized jointly in an alternating fashion. Here the parametric models help in correcting implausible 3D pose estimates and filling in missing joint detections while updated 3D poses in turn guide obtaining better SMPL estimations. By linking 2D and 3D observations, our method is both accurate and generalizes to different data sources because it better decouples the final 3D pose from the inter-person constellation and is more robust to noisy 2D detections. We systematically evaluate our method on public datasets and achieve state-of-the-art performance. The code and video will be available on the project page: https://ait.ethz.ch/projects/2021/multi-human-pose/.

cs.CV↗

arXiv2021

DMN4: Few-shot Learning via Discriminative Mutual Nearest Neighbor Neural Network

Few-shot learning (FSL) aims to classify images under low-data regimes, where the conventional pooled global feature is likely to lose useful local characteristics. Recent work has achieved promising performances by using deep descriptors. They generally take all deep descriptors from neural networks into consideration while ignoring that some of them are useless in classification due to their limited receptive field, e.g., task-irrelevant descriptors could be misleading and multiple aggregative descriptors from background clutter could even overwhelm the object's presence. In this paper, we argue that a Mutual Nearest Neighbor (MNN) relation should be established to explicitly select the query descriptors that are most relevant to each task and discard less relevant ones from aggregative clutters in FSL. Specifically, we propose Discriminative Mutual Nearest Neighbor Neural Network (DMN4) for FSL. Extensive experiments demonstrate that our method outperforms the existing state-of-the-arts on both fine-grained and generalized datasets.

cs.CV↗

arXiv2021

Optimized nonadiabatic holonomic quantum computation based on Förster resonance in Rydberg atoms

In this paper, we propose a scheme for implementing the nonadiabatic holonomic quantum computation (NHQC+) of two Rydberg atoms by using invariant-based reverse engineering (IBRE). The scheme is based on Förster resonance induced by strong dipole-dipole interaction between two Rydberg atoms, which provides a selective coupling mechanism to simply the dynamics of system. Moreover, for improving the fidelity of the scheme, the optimal control method is introduced to enhance the gate robustness against systematic errors. Numerical simulations show the scheme is robust against the random noise in control fields, the deviation of dipole-dipole interaction, the Förster defect, and the spontaneous emission of atoms. Therefore, the scheme may provide some useful perspectives for the realization of quantum computation with Rydberg atoms.

quant-ph↗

arXiv2021

Unselective ground-state blockade of Rydberg atoms for implementing quantum gates

A dynamics regime of Rydberg atoms, unselective ground-state blockade (UGSB), is proposed in the context of Rydberg antiblockade (RAB), where the evolution of two atoms is suppressed when they populate in an identical ground state. UGSB is used to implement a SWAP gate in one step without individual addressing of atoms. Aiming at circumventing common issues in RAB-based gates including atomic decay, Doppler dephasing, and fluctuations in the interatomic coupling strength, we modify the RAB condition to achieve a dynamical SWAP gate whose robustness is much greater than that of the nonadiabatic holonomic one in the conventional RAB regime. In addition, on the basis of the proposed SWAP gates, we further investigate the implementation of a three-atom Fredkin gate by combining Rydberg blockade and RAB. The present work may facilitate to implement the RAB-based gates of strongly coupled atoms in experiment.

quant-ph↗

arXiv2021

Training Generative Adversarial Networks in One Stage

Generative Adversarial Networks (GANs) have demonstrated unprecedented success in various image generation tasks. The encouraging results, however, come at the price of a cumbersome training process, during which the generator and discriminator are alternately updated in two stages. In this paper, we investigate a general training scheme that enables training GANs efficiently in only one stage. Based on the adversarial losses of the generator and discriminator, we categorize GANs into two classes, Symmetric GANs and Asymmetric GANs, and introduce a novel gradient decomposition method to unify the two, allowing us to train both classes in one stage and hence alleviate the training effort. We also computationally analyze the efficiency of the proposed method, and empirically demonstrate that, the proposed method yields a solid $1.5\times$ acceleration across various datasets and network architectures. Furthermore, we show that the proposed method is readily applicable to other adversarial-training scenarios, such as data-free knowledge distillation. The code is available at https://github.com/zju-vipa/OSGAN.

cs.CV↗

arXiv2021

Contrastive Model Inversion for Data-Free Knowledge Distillation

Model inversion, whose goal is to recover training data from a pre-trained model, has been recently proved feasible. However, existing inversion methods usually suffer from the mode collapse problem, where the synthesized instances are highly similar to each other and thus show limited effectiveness for downstream tasks, such as knowledge distillation. In this paper, we propose Contrastive Model Inversion~(CMI), where the data diversity is explicitly modeled as an optimizable objective, to alleviate the mode collapse issue. Our main observation is that, under the constraint of the same amount of data, higher data diversity usually indicates stronger instance discrimination. To this end, we introduce in CMI a contrastive learning objective that encourages the synthesizing instances to be distinguishable from the already synthesized ones in previous batches. Experiments of pre-trained models on CIFAR-10, CIFAR-100, and Tiny-ImageNet demonstrate that CMI not only generates more visually plausible instances than the state of the arts, but also achieves significantly superior performance when the generated data are used for knowledge distillation. Code is available at \url{https://github.com/zju-vipa/DataFree}.

cs.AI↗

arXiv2021

KDExplainer: A Task-oriented Attention Model for Explaining Knowledge Distillation

Knowledge distillation (KD) has recently emerged as an efficacious scheme for learning compact deep neural networks (DNNs). Despite the promising results achieved, the rationale that interprets the behavior of KD has yet remained largely understudied. In this paper, we introduce a novel task-oriented attention model, termed as KDExplainer, to shed light on the working mechanism underlying the vanilla KD. At the heart of KDExplainer is a Hierarchical Mixture of Experts (HME), in which a multi-class classification is reformulated as a multi-task binary one. Through distilling knowledge from a free-form pre-trained DNN to KDExplainer, we observe that KD implicitly modulates the knowledge conflicts between different subtasks, and in reality has much more to offer than label smoothing. Based on such findings, we further introduce a portable tool, dubbed as virtual attention module (VAM), that can be seamlessly integrated with various DNNs to enhance their performance under KD. Experimental results demonstrate that with a negligible additional cost, student models equipped with VAM consistently outperform their non-VAM counterparts across different benchmarks. Furthermore, when combined with other KD methods, VAM remains competent in promoting results, even though it is only motivated by vanilla KD. The code is available at https://github.com/zju-vipa/KDExplainer.

cs.CV↗

arXiv2021

Auto-Validate: Unsupervised Data Validation Using Data-Domain Patterns Inferred from Data Lakes

Complex data pipelines are increasingly common in diverse applications such as BI reporting and ML modeling. These pipelines often recur regularly (e.g., daily or weekly), as BI reports need to be refreshed, and ML models need to be retrained. However, it is widely reported that in complex production pipelines, upstream data feeds can change in unexpected ways, causing downstream applications to break silently that are expensive to resolve. Data validation has thus become an important topic, as evidenced by notable recent efforts from Google and Amazon, where the objective is to catch data quality issues early as they arise in the pipelines. Our experience on production data suggests, however, that on string-valued data, these existing approaches yield high false-positive rates and frequently require human intervention. In this work, we develop a corpus-driven approach to auto-validate \emph{machine-generated data} by inferring suitable data-validation "patterns" that accurately describe the underlying data domain, which minimizes false positives while maximizing data quality issues caught. Evaluations using production data from real data lakes suggest that Auto-Validate is substantially more effective than existing methods. Part of this technology ships as an Auto-Tag feature in Microsoft Azure Purview.

cs.DB↗

arXiv2021

Data-Driven Short-Term Voltage Stability Assessment Based on Spatial-Temporal Graph Convolutional Network

Post-fault dynamics of short-term voltage stability (SVS) present spatial-temporal characteristics, but the existing data-driven methods for online SVS assessment fail to incorporate such characteristics into their models effectively. Confronted with this dilemma, this paper develops a novel spatial-temporal graph convolutional network (STGCN) to address this problem. The proposed STGCN utilizes graph convolution to integrate network topology information into the learning model to exploit spatial information. Then, it adopts one-dimensional convolution to exploit temporal information. In this way, it models the spatial-temporal characteristics of SVS with complete convolutional structures. After that, a node layer and a system layer are strategically designed in the STGCN for SVS assessment. The proposed STGCN incorporates the characteristics of SVS into the data-driven classification model. It can result in higher assessment accuracy, better robustness and adaptability than conventional methods. Besides, parameters in the system layer can provide valuable information about the influences of individual buses on SVS. Test results on the real-world Guangdong Power Grid in South China verify the effectiveness of the proposed network.

cs.LG↗

arXiv2021

Large-scale GHZ states through topologically protected zero-energy mode in a superconducting qutrit-resonator chain

We propose a superconducting qutrit-resonator chain model, and analytically work out forms of its topological edge states. The existence of the zero-energy mode enables to generate a state transfer between two ends of the chain, accompanied with state flips of all intermediate qutrits, based on which $N$-body Greenberger-Horne-Zeilinger (GHZ) states can be generated with great robustness against disorders of coupling strengths. Three schemes of generating large-scale GHZ states are designed, each of which possesses the robustness against loss of qutrits or of resonators, meeting a certain performance requirement of different experimental devices. With experimentally feasible qutrit-resonator coupling strengths and available coherence times of qutrits and resonators, it has a potential to generate large-scale GHZ states among dozens of qutrits with a high fidelity. Further, we show the experimental consideration of generating GHZ states based on the circuit QED system, and discuss the prospect of realizing fast GHZ states.

quant-ph↗

arXiv2021

Resilient quantum gates on periodically driven Rydberg atoms

Fault-tolerant implementation of quantum gates is one of preconditions for realizing quantum computation. The platform of Rydberg atoms is one of the most promising candidates for achieving quantum computation. We propose to implement a controlled-$Z$ gate on Rydberg atoms where an amplitude-modulated field is employed to induce Rydberg antiblockade. Gate robustness against the fluctuations in the Rydberg-Rydberg interaction can be largely enhanced by adjusting amplitude-modulated field. Furthermore, we introduce a Landau-Zener-Stückelberg transition on the target atom so as to improve the gate resilience to the deviation in the gate time and the drift in the pulse amplitude. With feasible experimental parameters, one can achieve the gate with low fidelity errors caused by atomic decay, interatomic dipole-dipole force, and Doppler effects. Finally, we generalize the gate scheme into multiqubit cases, where resilient multiqubit phase gates can be obtained in one step with an unchanged gate time as the number of qubits increases.

quant-ph↗

arXiv2020

Progressive Network Grafting for Few-Shot Knowledge Distillation

Knowledge distillation has demonstrated encouraging performances in deep model compression. Most existing approaches, however, require massive labeled data to accomplish the knowledge transfer, making the model compression a cumbersome and costly process. In this paper, we investigate the practical few-shot knowledge distillation scenario, where we assume only a few samples without human annotations are available for each category. To this end, we introduce a principled dual-stage distillation scheme tailored for few-shot data. In the first step, we graft the student blocks one by one onto the teacher, and learn the parameters of the grafted block intertwined with those of the other teacher blocks. In the second step, the trained student blocks are progressively connected and then together grafted onto the teacher network, allowing the learned student blocks to adapt themselves to each other and eventually replace the teacher network. Experiments demonstrate that our approach, with only a few unlabeled samples, achieves gratifying results on CIFAR10, CIFAR100, and ILSVRC-2012. On CIFAR10 and CIFAR100, our performances are even on par with those of knowledge distillation schemes that utilize the full datasets. The source code is available at https://github.com/zju-vipa/NetGraft.

cs.CV↗