arXiv subjects

Xiaodong Yang

Publications and source records attributed to Xiaodong Yang.

At least 73 records · Page 4Linked to original sources

arXiv2021

Learning Intrinsic Images for Clothing

Reconstruction of human clothing is an important task and often relies on intrinsic image decomposition. With a lack of domain-specific data and coarse evaluation metrics, existing models failed to produce satisfying results for graphics applications. In this paper, we focus on intrinsic image decomposition for clothing images and have comprehensive improvements. We collected CloIntrinsics, a clothing intrinsic image dataset, including a synthetic training set and a real-world testing set. A more interpretable edge-aware metric and an annotation scheme is designed for the testing set, which allows diagnostic evaluation for intrinsic models. Finally, we propose ClothInNet model with carefully designed loss terms and an adversarial module. It utilizes easy-to-acquire labels to learn from real-world shading, significantly improves performance with only minor additional annotation effort. We show that our proposed model significantly reduce texture-copying artifacts while retaining surprisingly tiny details, outperforming existing state-of-the-art methods.

cs.CV↗

arXiv2021

Exploring Simple 3D Multi-Object Tracking for Autonomous Driving

3D multi-object tracking in LiDAR point clouds is a key ingredient for self-driving vehicles. Existing methods are predominantly based on the tracking-by-detection pipeline and inevitably require a heuristic matching step for the detection association. In this paper, we present SimTrack to simplify the hand-crafted tracking paradigm by proposing an end-to-end trainable model for joint detection and tracking from raw point clouds. Our key design is to predict the first-appear location of each object in a given snippet to get the tracking identity and then update the location based on motion estimation. In the inference, the heuristic matching step can be completely waived by a simple read-off operation. SimTrack integrates the tracked object association, newborn object detection, and dead track killing in a single unified model. We conduct extensive evaluations on two large-scale datasets: nuScenes and Waymo Open Dataset. Experimental results reveal that our simple approach compares favorably with the state-of-the-art methods while ruling out the heuristic matching rules.

cs.CV↗

arXiv2021

Neural Network Repair with Reachability Analysis

Safety is a critical concern for the next generation of autonomy that is likely to rely heavily on deep neural networks for perception and control. Formally verifying the safety and robustness of well-trained DNNs and learning-enabled systems under attacks, model uncertainties, and sensing errors is essential for safe autonomy. This research proposes a framework to repair unsafe DNNs in safety-critical systems with reachability analysis. The repair process is inspired by adversarial training which has demonstrated high effectiveness in improving the safety and robustness of DNNs. Different from traditional adversarial training approaches where adversarial examples are utilized from random attacks and may not be representative of all unsafe behaviors, our repair process uses reachability analysis to compute the exact unsafe regions and identify sufficiently representative examples to enhance the efficacy and efficiency of the adversarial training. The performance of our framework is evaluated on two types of benchmarks without safe models as references. One is a DNN controller for aircraft collision avoidance with access to training data. The other is a rocket lander where our framework can be seamlessly integrated with the well-known deep deterministic policy gradient (DDPG) reinforcement learning algorithm. The experimental results show that our framework can successfully repair all instances on multiple safety specifications with negligible performance degradation. In addition, to increase the computational and memory efficiency of the reachability analysis algorithm, we propose a depth-first-search algorithm that combines an existing exact analysis method with an over-approximation approach based on a new set representation. Experimental results show that our method achieves a five-fold improvement in runtime and a two-fold improvement in memory usage compared to exact analysis.

cs.LG↗

arXiv2021

Reachability Analysis of Convolutional Neural Networks

Deep convolutional neural networks have been widely employed as an effective technique to handle complex and practical problems. However, one of the fundamental problems is the lack of formal methods to analyze their behavior. To address this challenge, we propose an approach to compute the exact reachable sets of a network given an input domain, where the reachable set is represented by the face lattice structure. Besides the computation of reachable sets, our approach is also capable of backtracking to the input domain given an output reachable set. Therefore, a full analysis of a network's behavior can be realized. In addition, an approach for fast analysis is also introduced, which conducts fast computation of reachable sets by considering selected sensitive neurons in each layer. The exact pixel-level reachability analysis method is evaluated on a CNN for the CIFAR10 dataset and compared to related works. The fast analysis method is evaluated over a CNN CIFAR10 dataset and VGG16 architecture for the ImageNet dataset.

cs.CV↗

arXiv2021

The 5th AI City Challenge

The AI City Challenge was created with two goals in mind: (1) pushing the boundaries of research and development in intelligent video analysis for smarter cities use cases, and (2) assessing tasks where the level of performance is enough to cause real-world adoption. Transportation is a segment ripe for such adoption. The fifth AI City Challenge attracted 305 participating teams across 38 countries, who leveraged city-scale real traffic data and high-quality synthetic data to compete in five challenge tracks. Track 1 addressed video-based automatic vehicle counting, where the evaluation being conducted on both algorithmic effectiveness and computational efficiency. Track 2 addressed city-scale vehicle re-identification with augmented synthetic data to substantially increase the training set for the task. Track 3 addressed city-scale multi-target multi-camera vehicle tracking. Track 4 addressed traffic anomaly detection. Track 5 was a new track addressing vehicle retrieval using natural language descriptions. The evaluation system shows a general leader board of all submitted results, and a public leader board of results limited to the contest participation rules, where teams are not allowed to use external data in their work. The public leader board shows results more close to real-world situations where annotated data is limited. Results show the promise of AI in Smarter Transportation. State-of-the-art performance for some tasks shows that these technologies are ready for adoption in real-world systems.

cs.CV↗

arXiv2021

Joint Discriminative and Generative Learning for Person Re-identification

Person re-identification (re-id) remains challenging due to significant intra-class variations across different cameras. Recently, there has been a growing interest in using generative models to augment training data and enhance the invariance to input changes. The generative pipelines in existing methods, however, stay relatively separate from the discriminative re-id learning stages. Accordingly, re-id models are often trained in a straightforward manner on the generated data. In this paper, we seek to improve learned re-id embeddings by better leveraging the generated data. To this end, we propose a joint learning framework that couples re-id learning and data generation end-to-end. Our model involves a generative module that separately encodes each person into an appearance code and a structure code, and a discriminative module that shares the appearance encoder with the generative module. By switching the appearance or structure codes, the generative module is able to generate high-quality cross-id composed images, which are online fed back to the appearance encoder and used to improve the discriminative module. The proposed joint learning framework renders significant improvement over the baseline without using generated data, leading to the state-of-the-art performance on several benchmark datasets.

cs.CV↗

arXiv2021

Self-Supervised Pillar Motion Learning for Autonomous Driving

Autonomous driving can benefit from motion behavior comprehension when interacting with diverse traffic participants in highly dynamic environments. Recently, there has been a growing interest in estimating class-agnostic motion directly from point clouds. Current motion estimation methods usually require vast amount of annotated training data from self-driving scenes. However, manually labeling point clouds is notoriously difficult, error-prone and time-consuming. In this paper, we seek to answer the research question of whether the abundant unlabeled data collections can be utilized for accurate and efficient motion learning. To this end, we propose a learning framework that leverages free supervisory signals from point clouds and paired camera images to estimate motion purely via self-supervision. Our model involves a point cloud based structural consistency augmented with probabilistic motion masking as well as a cross-sensor motion regularization to realize the desired self-supervision. Experiments reveal that our approach performs competitively to supervised methods, and achieves the state-of-the-art result when combining our self-supervised model with supervised fine-tuning.

cs.CV↗

arXiv2021

Experimental Adiabatic Quantum Metrology with the Heisenberg scaling

The critical quantum metrology, which exploits the quantum phase transition for high precision measurement, has gained increasing attention recently. The critical quantum metrology with the continuous quantum phase transition, however, is experimentally very challenging since the continuous quantum phase transition only exists at the thermal dynamical limit. Here, we propose an adiabatic scheme on a perturbed Ising spin model with the first order quantum phase transition. By employing the Landau-Zener anticrossing, we can not only encode the unknown parameter in the ground state but also tune the energy gap to control the evolution time of the adiabatic passage. We experimentally implement the adiabatic scheme on the nuclear magnetic resonance and show that the achieved precision attains the Heisenberg scaling. The advantages of the scheme-easy implementation, robust against the decay, tunable energy gap-are critical for practical applications of quantum metrology.

quant-ph↗

arXiv2021

Hybrid quantum-classical approach to enhanced quantum metrology

Quantum metrology plays a fundamental role in many scientific areas. However, the complexity of engineering entangled probes and the external noise raise technological barriers for realizing the expected precision of the to-be-estimated parameter with given resources. Here, we address this problem by introducing adjustable controls into the encoding process and then utilizing a hybrid quantum-classical approach to automatically optimize the controls online. Our scheme does not require any complex or intractable off-line design, and it can inherently correct certain unitary errors during the learning procedure. We also report the first experimental demonstration of this promising scheme for the task of finding optimal probes for frequency estimation on a nuclear magnetic resonance (NMR) processor. The proposed scheme paves the way to experimentally auto-search optimal protocol for improving the metrology precision.

quant-ph↗

arXiv2021

Robust Dynamical Decoupling for the Manipulation of a Spin Network via a Single Spin

High-fidelity control of quantum systems is crucial for quantum information processing, but is often limited by perturbations from the environment and imperfections in the applied control fields. Here, we investigate the combination of dynamical decoupling (DD) and robust optimal control (ROC) to address this problem. In this combination, ROC is employed to find robust shaped pulses, wherein the directional derivatives of the controlled dynamics with respect to control errors are reduced to a desired order. Then, we incorporate ROC pulses into DD sequences, achieving a remarkable improvement of robustness against multiple error channels. We demonstrate this method in the example of manipulating nuclear spin bath via an electron spin in the NV center system. Simulation results indicate that ROC based DD sequences outperform the state-of-the-art robust DD sequences. Our work has implications for robust quantum control on near-term noisy quantum devices.

quant-ph↗

arXiv2020

Network-Constrained Transactive Control for Multi- Microgrids-based Distribution Networks with SOPs

Different from most transactive control studies only focusing on economic aspect, this paper develops a novel network-constrained transactive control (NTC) framework that can address both economic and secure issues for a multi-microgrids-based distribution network considering uncertainties. In particular, we innovatively integrate a transactive energy market with the novel power-electronics device (i.e., soft open point) based AC power flow regulation technique to improve economic benefits for individual microgrids and meanwhile ensure the security of the entire distribution network. In this framework, a dynamic two-timescale NTC model consisting of slow-timescale pre-scheduling and real-time scheduling stages is formulated to work against multiple system uncertainties. Moreover, the original bilevel game problems are transformed into single-level mixed-integer second-order cone programming problems through KKT conditions, duality, linearization and relaxation techniques to avoid iterations of transitional methods, so as to improve computational efficiency. Finally, numerical simulations on a modified 33-bus test system with 3 MGs verify the effectiveness of the proposed framework.

math.OC↗

arXiv2020

Instance-aware, Context-focused, and Memory-efficient Weakly Supervised Object Detection

Weakly supervised learning has emerged as a compelling tool for object detection by reducing the need for strong supervision during training. However, major challenges remain: (1) differentiation of object instances can be ambiguous; (2) detectors tend to focus on discriminative parts rather than entire objects; (3) without ground truth, object proposals have to be redundant for high recalls, causing significant memory consumption. Addressing these challenges is difficult, as it often requires to eliminate uncertainties and trivial solutions. To target these issues we develop an instance-aware and context-focused unified framework. It employs an instance-aware self-training algorithm and a learnable Concrete DropBlock while devising a memory-efficient sequential batch back-propagation. Our proposed method achieves state-of-the-art results on COCO ($12.1\% ~AP$, $24.8\% ~AP_{50}$), VOC 2007 ($54.9\% ~AP$), and VOC 2012 ($52.1\% ~AP$), improving baselines by great margins. In addition, the proposed method is the first to benchmark ResNet based models and weakly supervised video object detection. Code, models, and more details will be made available at: https://github.com/NVlabs/wetectron.

cs.CV↗

arXiv2020

UFO$^2$: A Unified Framework towards Omni-supervised Object Detection

Existing work on object detection often relies on a single form of annotation: the model is trained using either accurate yet costly bounding boxes or cheaper but less expressive image-level tags. However, real-world annotations are often diverse in form, which challenges these existing works. In this paper, we present UFO$^2$, a unified object detection framework that can handle different forms of supervision simultaneously. Specifically, UFO$^2$ incorporates strong supervision (e.g., boxes), various forms of partial supervision (e.g., class tags, points, and scribbles), and unlabeled data. Through rigorous evaluations, we demonstrate that each form of label can be utilized to either train a model from scratch or to further improve a pre-trained model. We also use UFO$^2$ to investigate budget-aware omni-supervised learning, i.e., various annotation policies are studied under a fixed annotation budget: we show that competitive performance needs no strong labels for all data. Finally, we demonstrate the generalization of UFO$^2$, detecting more than 1,000 different objects without bounding box annotations.

cs.CV↗

arXiv2020

Contrastive Learning for Weakly Supervised Phrase Grounding

Phrase grounding, the problem of associating image regions to caption words, is a crucial component of vision-language tasks. We show that phrase grounding can be learned by optimizing word-region attention to maximize a lower bound on mutual information between images and caption words. Given pairs of images and captions, we maximize compatibility of the attention-weighted regions and the words in the corresponding caption, compared to non-corresponding pairs of images and captions. A key idea is to construct effective negative captions for learning through language model guided word substitutions. Training with our negatives yields a $\sim10\%$ absolute gain in accuracy over randomly-sampled negatives from the training data. Our weakly supervised phrase grounding model trained on COCO-Captions shows a healthy gain of $5.7\%$ to achieve $76.7\%$ accuracy on Flickr30K Entities benchmark.

cs.CV↗

arXiv2020

Simulating Content Consistent Vehicle Datasets with Attribute Descent

This paper uses a graphic engine to simulate a large amount of training data with free annotations. Between synthetic and real data, there is a two-level domain gap, i.e., content level and appearance level. While the latter has been widely studied, we focus on reducing the content gap in attributes like illumination and viewpoint. To reduce the problem complexity, we choose a smaller and more controllable application, vehicle re-identification (re-ID). We introduce a large-scale synthetic dataset VehicleX. Created in Unity, it contains 1,362 vehicles of various 3D models with fully editable attributes. We propose an attribute descent approach to let VehicleX approximate the attributes in real-world datasets. Specifically, we manipulate each attribute in VehicleX, aiming to minimize the discrepancy between VehicleX and real data in terms of the Fréchet Inception Distance (FID). This attribute descent algorithm allows content domain adaptation (DA) orthogonal to existing appearance DA methods. We mix the optimized VehicleX data with real-world vehicle re-ID datasets, and observe consistent improvement. With the augmented datasets, we report competitive accuracy. We make the dataset, engine and our codes available at https://github.com/yorkeyao/VehicleX.

cs.CV↗

arXiv2020

Probe optimization for quantum metrology via closed-loop learning control

Experimentally achieving the precision that standard quantum metrology schemes promise is always challenging. Recently, additional controls were applied to design feasible quantum metrology schemes. However, these approaches generally does not consider ease of implementation, raising technological barriers impeding its realization. In this paper, we circumvent this problem by applying closed-loop learning control to propose a practical controlled sequential scheme for quantum metrology. Purity loss of the probe state, which relates to quantum Fisher information, is measured efficiently as the fitness to guide the learning loop. We confirm its feasibility and certain superiorities over standard quantum metrology schemes by numerical analysis and proof-of-principle experiments in a nuclear magnetic resonance (NMR) system.

quant-ph↗

arXiv2020

Optimizing adiabatic quantum pathways via a learning algorithm

Designing proper time-dependent control fields for slowly varying the system to the ground state that encodes the problem solution is crucial for adiabatic quantum computation. However, inevitable perturbations in real applications demand us to accelerate the evolution so that the adiabatic errors can be prevented from accumulation. Here, by treating this trade-off task as a multiobjective optimization problem, we propose a gradient-free learning algorithm with pulse smoothing technique to search optimal adiabatic quantum pathways and apply it to the Landau-Zener Hamiltonian and Grover search Hamiltonian. Numerical comparisons with a linear schedule, local adiabatic theorem induced schedule, and gradient-based algorithm searched schedule reveal that the proposed method can achieve significant performance improvements in terms of the adiabatic time and the instantaneous ground-state population maintenance. The proposed method can be used to solve more complex and real adiabatic quantum computation problems.

quant-ph↗

arXiv2020

Joint Disentangling and Adaptation for Cross-Domain Person Re-Identification

Although a significant progress has been witnessed in supervised person re-identification (re-id), it remains challenging to generalize re-id models to new domains due to the huge domain gaps. Recently, there has been a growing interest in using unsupervised domain adaptation to address this scalability issue. Existing methods typically conduct adaptation on the representation space that contains both id-related and id-unrelated factors, thus inevitably undermining the adaptation efficacy of id-related features. In this paper, we seek to improve adaptation by purifying the representation space to be adapted. To this end, we propose a joint learning framework that disentangles id-related/unrelated features and enforces adaptation to work on the id-related feature space exclusively. Our model involves a disentangling module that encodes cross-domain images into a shared appearance space and two separate structure spaces, and an adaptation module that performs adversarial alignment and self-training on the shared appearance space. The two modules are co-designed to be mutually beneficial. Extensive experiments demonstrate that the proposed joint learning framework outperforms the state-of-the-art methods by clear margins.

cs.CV↗