arXiv subjects

Jian Sun

Publications and source records attributed to Jian Sun.

At least 289 records · Page 16Linked to original sources

arXiv2021

Data-Driven Resilient Predictive Control under Denial-of-Service

The study of resilient control of linear time-invariant (LTI) systems against denial-of-service (DoS) attacks is gaining popularity in emerging cyber-physical applications. In previous works, explicit system models are required to design a predictor-based resilient controller. These models can be either given a priori or obtained through a prior system identification step. Recent research efforts have focused on data-driven control based on pre-collected input-output trajectories (i.e., without explicit system models). In this paper, we take an initial step toward data-driven stabilization of stochastic LTI systems under DoS attacks, and develop a resilient model predictive control (MPC) scheme driven purely by data-dependent conditions. The proposed data-driven control method achieves the same level of resilience as the model-based control method. For example, local input-to-state stability (ISS) is achieved under mild assumptions on the noise and the DoS attacks. To recover global ISS, two modifications are further suggested at the price of reduced resilience against DoS attacks or increased computational complexity. Finally, a numerical example is given to validate the effectiveness of the proposed control method.

eess.SY↗

arXiv2021

Partial to Whole Knowledge Distillation: Progressive Distilling Decomposed Knowledge Boosts Student Better

Knowledge distillation field delicately designs various types of knowledge to shrink the performance gap between compact student and large-scale teacher. These existing distillation approaches simply focus on the improvement of \textit{knowledge quality}, but ignore the significant influence of \textit{knowledge quantity} on the distillation procedure. Opposed to the conventional distillation approaches, which extract knowledge from a fixed teacher computation graph, this paper explores a non-negligible research direction from a novel perspective of \textit{knowledge quantity} to further improve the efficacy of knowledge distillation. We introduce a new concept of knowledge decomposition, and further put forward the \textbf{P}artial to \textbf{W}hole \textbf{K}nowledge \textbf{D}istillation~(\textbf{PWKD}) paradigm. Specifically, we reconstruct teacher into weight-sharing sub-networks with same depth but increasing channel width, and train sub-networks jointly to obtain decomposed knowledge~(sub-networks with more channels represent more knowledge). Then, student extract partial to whole knowledge from the pre-trained teacher within multiple training stages where cyclic learning rate is leveraged to accelerate convergence. Generally, \textbf{PWKD} can be regarded as a plugin to be compatible with existing offline knowledge distillation approaches. To verify the effectiveness of \textbf{PWKD}, we conduct experiments on two benchmark datasets:~CIFAR-100 and ImageNet, and comprehensive evaluation results reveal that \textbf{PWKD} consistently improve existing knowledge distillation approaches without bells and whistles.

cs.CV↗

arXiv2021

DialogueCSE: Dialogue-based Contrastive Learning of Sentence Embeddings

Learning sentence embeddings from dialogues has drawn increasing attention due to its low annotation cost and high domain adaptability. Conventional approaches employ the siamese-network for this task, which obtains the sentence embeddings through modeling the context-response semantic relevance by applying a feed-forward network on top of the sentence encoders. However, as the semantic textual similarity is commonly measured through the element-wise distance metrics (e.g. cosine and L2 distance), such architecture yields a large gap between training and evaluating. In this paper, we propose DialogueCSE, a dialogue-based contrastive learning approach to tackle this issue. DialogueCSE first introduces a novel matching-guided embedding (MGE) mechanism, which generates a context-aware embedding for each candidate response embedding (i.e. the context-free embedding) according to the guidance of the multi-turn context-response matching matrices. Then it pairs each context-aware embedding with its corresponding context-free embedding and finally minimizes the contrastive loss across all pairs. We evaluate our model on three multi-turn dialogue datasets: the Microsoft Dialogue Corpus, the Jing Dong Dialogue Corpus, and the E-commerce Dialogue Corpus. Evaluation results show that our approach significantly outperforms the baselines across all three datasets in terms of MAP and Spearman's correlation measures, demonstrating its effectiveness. Further quantitative experiments show that our approach achieves better performance when leveraging more dialogue context and remains robust when less training data is provided.

cs.CL↗

arXiv2021

Joint Multi-Dimension Pruning via Numerical Gradient Update

We present joint multi-dimension pruning (abbreviated as JointPruning), an effective method of pruning a network on three crucial aspects: spatial, depth and channel simultaneously. To tackle these three naturally different dimensions, we proposed a general framework by defining pruning as seeking the best pruning vector (i.e., the numerical value of layer-wise channel number, spacial size, depth) and construct a unique mapping from the pruning vector to the pruned network structures. Then we optimize the pruning vector with gradient update and model joint pruning as a numerical gradient optimization process. To overcome the challenge that there is no explicit function between the loss and the pruning vectors, we proposed self-adapted stochastic gradient estimation to construct a gradient path through network loss to pruning vectors and enable efficient gradient update. We show that the joint strategy discovers a better status than previous studies that focused on individual dimensions solely, as our method is optimized collaboratively across the three dimensions in a single end-to-end training and it is more efficient than the previous exhaustive methods. Extensive experiments on large-scale ImageNet dataset across a variety of network architectures MobileNet V1&V2&V3 and ResNet demonstrate the effectiveness of our proposed method. For instance, we achieve significant margins of 2.5% and 2.6% improvement over the state-of-the-art approach on the already compact MobileNet V1&V2 under an extremely large compression ratio.

cs.CV↗

arXiv2021

Formation of copper boride on Cu(111)

Boron forms compounds with nearly all metals, with notable exception of copper and other group IB and IIB elements. Here, we report an unexpected discovery of ordered copper boride grown epitaxially on Cu(111) under ultrahigh vacuum. Scanning tunneling microscopy experiments combined with ab initio evolutionary structure prediction reveal a remarkably complex structure of 2D-Cu8B14. Strong intra-layer p-d hybridization and a large amount of charge transfer between Cu and B atoms are the key factors for the emergence of copper boride. This makes the discovered material unique and opens up the possibility of synthesizing ordered low-dimensional structures in similar immiscible systems.

cond-mat.mtrl-sci↗

arXiv2021

Learning Canonical View Representation for 3D Shape Recognition with Arbitrary Views

In this paper, we focus on recognizing 3D shapes from arbitrary views, i.e., arbitrary numbers and positions of viewpoints. It is a challenging and realistic setting for view-based 3D shape recognition. We propose a canonical view representation to tackle this challenge. We first transform the original features of arbitrary views to a fixed number of view features, dubbed canonical view representation, by aligning the arbitrary view features to a set of learnable reference view features using optimal transport. In this way, each 3D shape with arbitrary views is represented by a fixed number of canonical view features, which are further aggregated to generate a rich and robust 3D shape representation for shape recognition. We also propose a canonical view feature separation constraint to enforce that the view features in canonical view representation can be embedded into scattered points in a Euclidean space. Experiments on the ModelNet40, ScanObjectNN, and RGBD datasets show that our method achieves competitive results under the fixed viewpoint settings, and significantly outperforms the applicable methods under the arbitrary view setting.

cs.CV↗

arXiv2021

The observation of in-plane quantum Griffiths singularity in two-dimensional crystalline superconductors

Quantum Griffiths singularity (QGS) reveals the profound influence of quenched disorder on the quantum phase transitions, characterized by the divergence of the dynamical critical exponent at the boundary of the vortex glass-like phase, named as quantum Griffiths phase. However, in the absence of vortices, whether the QGS can exist under parallel magnetic field remains a puzzle. Here we study the magnetic field induced superconductor-metal transition in ultrathin crystalline PdTe2 films grown by molecular beam epitaxy. Remarkably, the QGS emerges under both perpendicular and parallel magnetic field in 4-monolayer PdTe2 films. The direct activated scaling analysis with a new irrelevant correction has been proposed, providing important evidence of QGS. With increasing film thickness to 6 monolayers, the QGS disappears under perpendicular field but persists under parallel field, and this discordance may originate from the differences in microscopic processes. Our work demonstrates the universality of parallel field induced QGS and can stimulate further investigations on novel quantum phase transitions under parallel magnetic field.

cond-mat.supr-con↗

arXiv2021

Dynamic ordering transitions in charged solid

The phenomenon of group motion is common in nature, ranging from the schools of fish, birds and insects, to avalanches, landslides and sand drift. If we treat objects as collectively moving particles, such phenomena can be studied from a physical point of view, and the research on many-body systems has proved that marvelous effects can arise from the simplest individuals. The motion of numerous individuals presents different dynamic phases related to the ordering of the system. However, it is usually difficult to study the dynamic ordering and their transitions through experiments. Electron bubble states formed in a two-dimensional electron gas, as a type of electron solids, can be driven by an external electric field and provide a platform to study the dynamic collective behaviors. Here, we demonstrate that noise spectrum is a powerful method to investigate the dynamics of bubble states. We observed not only the phenomena from dynamically ordered and disordered structures, but also unexpected alternations between them. Our results show that a dissipative system can convert between chaotic structures and ordered structures when tuning global parameters, which is concealed in conventional transport measurements of resistance or conductance. Moreover, charging the objects to study electrical noise spectrum in collective motions can be an additional approach to revealing dynamic ordering transitions.

cond-mat.mes-hall↗

arXiv2021

YOLOX: Exceeding YOLO Series in 2021

In this report, we present some experienced improvements to YOLO series, forming a new high-performance detector -- YOLOX. We switch the YOLO detector to an anchor-free manner and conduct other advanced detection techniques, i.e., a decoupled head and the leading label assignment strategy SimOTA to achieve state-of-the-art results across a large scale range of models: For YOLO-Nano with only 0.91M parameters and 1.08G FLOPs, we get 25.3% AP on COCO, surpassing NanoDet by 1.8% AP; for YOLOv3, one of the most widely used detectors in industry, we boost it to 47.3% AP on COCO, outperforming the current best practice by 3.0% AP; for YOLOX-L with roughly the same amount of parameters as YOLOv4-CSP, YOLOv5-L, we achieve 50.0% AP on COCO at a speed of 68.9 FPS on Tesla V100, exceeding YOLOv5-L by 1.8% AP. Further, we won the 1st Place on Streaming Perception Challenge (Workshop on Autonomous Driving at CVPR 2021) using a single YOLOX-L model. We hope this report can provide useful experience for developers and researchers in practical scenes, and we also provide deploy versions with ONNX, TensorRT, NCNN, and Openvino supported. Source code is at https://github.com/Megvii-BaseDetection/YOLOX.

cs.CV↗

arXiv2021

Learning from Manipulable Signals

We study a dynamic stopping game between a principal and an agent. The agent is privately informed about his type. The principal learns about the agent's type from a noisy performance measure, which can be manipulated by the agent via a costly and hidden action. We fully characterize the unique Markov equilibrium of this game. We find that terminations/market crashes are often preceded by a spike in (expected) performance. Our model also predicts that, due to endogenous signal manipulation, too much transparency can inhibit learning. As the players get arbitrarily patient, the principal elicits no useful information from the observed signal.

econ.TH↗

arXiv2021

Partially Diffusive Helium-Silica Compound in the Deep Interiors of Giant Planets

Helium is the second most abundant element in the universe, and together with silica, they are major components of giant planets. Exploring the reactivity and state of helium and silica under high pressure is of fundamental importance for developing and understanding of the evolution and internal structure of giant planets. Here, using first-principles calculations and crystal structure predictions, we identify four stable phases of a helium-silica compound with seven/eight-coordinated silicon atoms at pressure range of 600-4000 GPa, corresponding to the interior condition of the outer planets in the solar system. The density of HeSiO2 agrees with current structure models of the planets. This helium-silica compound exhibits a superionic-like helium diffusive state at the high pressure and high temperature conditions along the isentropes of Saturn, a metallic fluid state in Jupiter, and a solid state in the deep interiors of Uranus and Neptune. The reaction of helium and silica may lead to the erosion of the rocky core of giant planets and form a diluted core region. These results highlight the reactivity of helium under high pressure to form new compounds, and also provides evidence to help build more sophisticated interior models of giant planets.

cond-mat.mtrl-sci↗

arXiv2021

Superionic silica-water and silica-hydrogen compounds under high pressure

Silica, water and hydrogen are known to be the major components of celestial bodies, and have significant influence on the formation and evolution of giant planets, such as Uranus and Neptune. Thus, it is of fundamental importance to investigate their states and possible reactions under the planetary conditions. Here, using advanced crystal structure searches and first-principles calculations in the Si-O-H system, we find that a silica-water compound (SiO2)2(H2O) and a silica-hydrogen compound SiO2H2 can exist under high pressures above 450 and 650 GPa, respectively. Further simulations reveal that, at high pressure and high temperature conditions corresponding to the interiors of Uranus and Neptune, these compounds exhibit superionic behavior, in which protons diffuse freely like liquid while the silicon and oxygen framework is fixed as solid. Therefore, these superionic silica-water and silica-hydrogen compounds could be regarded as important components of the deep mantle or core of giants, which also provides an alternative origin for their anomalous magnetic fields. These unexpected physical and chemical properties of the most common natural materials at high pressure offer key clues to understand some abstruse issues including demixing and erosion of the core in giant planets, and shed light on building reliable models for solar giants and exoplanets.

cond-mat.mtrl-sci↗

arXiv2021

Workshop on Autonomous Driving at CVPR 2021: Technical Report for Streaming Perception Challenge

In this report, we introduce our real-time 2D object detection system for the realistic autonomous driving scenario. Our detector is built on a newly designed YOLO model, called YOLOX. On the Argoverse-HD dataset, our system achieves 41.0 streaming AP, which surpassed second place by 7.8/6.1 on detection-only track/fully track, respectively. Moreover, equipped with TensorRT, our model achieves the 30FPS inference speed with a high-resolution input size (e.g., 1440-2304). Code and models will be available at https://github.com/Megvii-BaseDetection/YOLOX

cs.CV↗

arXiv2021

A Unified Hyper-GAN Model for Unpaired Multi-contrast MR Image Translation

Cross-contrast image translation is an important task for completing missing contrasts in clinical diagnosis. However, most existing methods learn separate translator for each pair of contrasts, which is inefficient due to many possible contrast pairs in real scenarios. In this work, we propose a unified Hyper-GAN model for effectively and efficiently translating between different contrast pairs. Hyper-GAN consists of a pair of hyper-encoder and hyper-decoder to first map from the source contrast to a common feature space, and then further map to the target contrast image. To facilitate the translation between different contrast pairs, contrast-modulators are designed to tune the hyper-encoder and hyper-decoder adaptive to different contrasts. We also design a common space loss to enforce that multi-contrast images of a subject share a common feature space, implicitly modeling the shared underlying anatomical structures. Experiments on two datasets of IXI and BraTS 2019 show that our Hyper-GAN achieves state-of-the-art results in both accuracy and efficiency, e.g., improving more than 1.47 and 1.09 dB in PSNR on two datasets with less than half the amount of parameters.

eess.IV↗

arXiv2021

Distributed stochastic gradient tracking algorithm with variance reduction for non-convex optimization

This paper proposes a distributed stochastic algorithm with variance reduction for general smooth non-convex finite-sum optimization, which has wide applications in signal processing and machine learning communities. In distributed setting, large number of samples are allocated to multiple agents in the network. Each agent computes local stochastic gradient and communicates with its neighbors to seek for the global optimum. In this paper, we develop a modified variance reduction technique to deal with the variance introduced by stochastic gradients. Combining gradient tracking and variance reduction techniques, this paper proposes a distributed stochastic algorithm, GT-VR, to solve large-scale non-convex finite-sum optimization over multi-agent networks. A complete and rigorous proof shows that the GT-VR algorithm converges to first-order stationary points with $O(\frac{1}{k})$ convergence rate. In addition, we provide the complexity analysis of the proposed algorithm. Compared with some existing first-order methods, the proposed algorithm has a lower $\mathcal{O}(PMε^{-1})$ gradient complexity under some mild condition. By comparing state-of-the-art algorithms and GT-VR in experimental simulations, we verify the efficiency of the proposed algorithm.

math.OC↗

arXiv2021

PointINS: Point-based Instance Segmentation

In this paper, we explore the mask representation in instance segmentation with Point-of-Interest (PoI) features. Differentiating multiple potential instances within a single PoI feature is challenging because learning a high-dimensional mask feature for each instance using vanilla convolution demands a heavy computing burden. To address this challenge, we propose an instance-aware convolution. It decomposes this mask representation learning task into two tractable modules as instance-aware weights and instance-agnostic features. The former is to parametrize convolution for producing mask features corresponding to different instances, improving mask learning efficiency by avoiding employing several independent convolutions. Meanwhile, the latter serves as mask templates in a single point. Together, instance-aware mask features are computed by convolving the template with dynamic weights, used for the mask prediction. Along with instance-aware convolution, we propose PointINS, a simple and practical instance segmentation approach, building upon dense one-stage detectors. Through extensive experiments, we evaluated the effectiveness of our framework built upon RetinaNet and FCOS. PointINS in ResNet101 backbone achieves a 38.3 mask mean average precision (mAP) on COCO dataset, outperforming existing point-based methods by a large margin. It gives a comparable performance to the region-based Mask R-CNN with faster inference.

cs.CV↗

arXiv2021

NTIRE 2021 Challenge on Burst Super-Resolution: Methods and Results

This paper reviews the NTIRE2021 challenge on burst super-resolution. Given a RAW noisy burst as input, the task in the challenge was to generate a clean RGB image with 4 times higher resolution. The challenge contained two tracks; Track 1 evaluating on synthetically generated data, and Track 2 using real-world bursts from mobile camera. In the final testing phase, 6 teams submitted results using a diverse set of solutions. The top-performing methods set a new state-of-the-art for the burst super-resolution task.

cs.CV↗

arXiv2021

Relational Learning with Gated and Attentive Neighbor Aggregator for Few-Shot Knowledge Graph Completion

Aiming at expanding few-shot relations' coverage in knowledge graphs (KGs), few-shot knowledge graph completion (FKGC) has recently gained more research interests. Some existing models employ a few-shot relation's multi-hop neighbor information to enhance its semantic representation. However, noise neighbor information might be amplified when the neighborhood is excessively sparse and no neighbor is available to represent the few-shot relation. Moreover, modeling and inferring complex relations of one-to-many (1-N), many-to-one (N-1), and many-to-many (N-N) by previous knowledge graph completion approaches requires high model complexity and a large amount of training instances. Thus, inferring complex relations in the few-shot scenario is difficult for FKGC models due to limited training instances. In this paper, we propose a few-shot relational learning with global-local framework to address the above issues. At the global stage, a novel gated and attentive neighbor aggregator is built for accurately integrating the semantics of a few-shot relation's neighborhood, which helps filtering the noise neighbors even if a KG contains extremely sparse neighborhoods. For the local stage, a meta-learning based TransH (MTransH) method is designed to model complex relations and train our model in a few-shot learning fashion. Extensive experiments show that our model outperforms the state-of-the-art FKGC approaches on the frequently-used benchmark datasets NELL-One and Wiki-One. Compared with the strong baseline model MetaR, our model achieves 5-shot FKGC performance improvements of 8.0% on NELL-One and 2.8% on Wiki-One by the metric Hits@10.

cs.AI↗