arXiv subjects

Jian Sun

Publications and source records attributed to Jian Sun.

At least 271 records · Page 15Linked to original sources

arXiv2022

ThunderNet: Towards Real-time Generic Object Detection

Real-time generic object detection on mobile platforms is a crucial but challenging computer vision task. However, previous CNN-based detectors suffer from enormous computational cost, which hinders them from real-time inference in computation-constrained scenarios. In this paper, we investigate the effectiveness of two-stage detectors in real-time generic detection and propose a lightweight two-stage detector named ThunderNet. In the backbone part, we analyze the drawbacks in previous lightweight backbones and present a lightweight backbone designed for object detection. In the detection part, we exploit an extremely efficient RPN and detection head design. To generate more discriminative feature representation, we design two efficient architecture blocks, Context Enhancement Module and Spatial Attention Module. At last, we investigate the balance between the input resolution, the backbone, and the detection head. Compared with lightweight one-stage detectors, ThunderNet achieves superior performance with only 40% of the computational cost on PASCAL VOC and COCO benchmarks. Without bells and whistles, our model runs at 24.1 fps on an ARM-based device. To the best of our knowledge, this is the first real-time detector reported on ARM platforms. Our code and models are available at \url{https://github.com/qinzheng93/ThunderNet}.

cs.CV↗

arXiv2022

Boosting Black-Box Adversarial Attacks with Meta Learning

Deep neural networks (DNNs) have achieved remarkable success in diverse fields. However, it has been demonstrated that DNNs are very vulnerable to adversarial examples even in black-box settings. A large number of black-box attack methods have been proposed to in the literature. However, those methods usually suffer from low success rates and large query counts, which cannot fully satisfy practical purposes. In this paper, we propose a hybrid attack method which trains meta adversarial perturbations (MAPs) on surrogate models and performs black-box attacks by estimating gradients of the models. Our method uses the meta adversarial perturbation as an initialization and subsequently trains any black-box attack method for several epochs. Furthermore, the MAPs enjoy favorable transferability and universality, in the sense that they can be employed to boost performance of other black-box adversarial attack methods. Extensive experiments demonstrate that our method can not only improve the attack success rates, but also reduces the number of queries compared to other methods.

cs.LG↗

arXiv2022

FS6D: Few-Shot 6D Pose Estimation of Novel Objects

6D object pose estimation networks are limited in their capability to scale to large numbers of object instances due to the close-set assumption and their reliance on high-fidelity object CAD models. In this work, we study a new open set problem; the few-shot 6D object poses estimation: estimating the 6D pose of an unknown object by a few support views without extra training. To tackle the problem, we point out the importance of fully exploring the appearance and geometric relationship between the given support views and query scene patches and propose a dense prototypes matching framework by extracting and matching dense RGBD prototypes with transformers. Moreover, we show that the priors from diverse appearances and shapes are crucial to the generalization capability under the problem setting and thus propose a large-scale RGBD photorealistic dataset (ShapeNet6D) for network pre-training. A simple and effective online texture blending approach is also introduced to eliminate the domain gap from the synthesis dataset, which enriches appearance diversity at a low cost. Finally, we discuss possible solutions to this problem and establish benchmarks on popular datasets to facilitate future research. The project page is at \url{https://fs6d.github.io/}.

cs.CV↗

arXiv2022

Ill-posed Surface Emissivity Retrieval from Multi-Geometry Hyperspectral Images using a Hybrid Deep Neural Network

Atmospheric correction is a fundamental task in remote sensing because observations are taken either of the atmosphere or looking through the atmosphere. Atmospheric correction errors can significantly alter the spectral signature of the observations, and lead to invalid classifications or target detection. This is even more crucial when working with hyperspectral data, where a precise measurement of spectral properties is required. State-of-the-art physics-based atmospheric correction approaches require extensive prior knowledge about sensor characteristics, collection geometry, and environmental characteristics of the scene being collected. These approaches are computationally expensive, prone to inaccuracy due to lack of sufficient environmental and collection information, and often impossible for real-time applications. In this paper, a geometry-dependent hybrid neural network is proposed for automatic atmospheric correction using multi-scan hyperspectral data collected from different geometries. The proposed network can characterize the atmosphere without any additional meteorological data. A grid-search method is also proposed to solve the temperature emissivity separation problem. Results show that the proposed network has the capacity to accurately characterize the atmosphere and estimate target emissivity spectra with a Mean Absolute Error (MAE) under 0.02 for 29 different materials. This solution can lead to accurate atmospheric correction to improve target detection for real time applications.

eess.IV↗

arXiv2022

Tree Energy Loss: Towards Sparsely Annotated Semantic Segmentation

Sparsely annotated semantic segmentation (SASS) aims to train a segmentation network with coarse-grained (i.e., point-, scribble-, and block-wise) supervisions, where only a small proportion of pixels are labeled in each image. In this paper, we propose a novel tree energy loss for SASS by providing semantic guidance for unlabeled pixels. The tree energy loss represents images as minimum spanning trees to model both low-level and high-level pair-wise affinities. By sequentially applying these affinities to the network prediction, soft pseudo labels for unlabeled pixels are generated in a coarse-to-fine manner, achieving dynamic online self-training. The tree energy loss is effective and easy to be incorporated into existing frameworks by combining it with a traditional segmentation loss. Compared with previous SASS methods, our method requires no multistage training strategies, alternating optimization procedures, additional supervised data, or time-consuming post-processing while outperforming them in all SASS settings. Code is available at https://github.com/megvii-research/TreeEnergyLoss.

cs.CV↗

arXiv2022

A Slot Is Not Built in One Utterance: Spoken Language Dialogs with Sub-Slots

A slot value might be provided segment by segment over multiple-turn interactions in a dialog, especially for some important information such as phone numbers and names. It is a common phenomenon in daily life, but little attention has been paid to it in previous work. To fill the gap, this paper defines a new task named Sub-Slot based Task-Oriented Dialog (SSTOD) and builds a Chinese dialog dataset SSD for boosting research on SSTOD. The dataset includes a total of 40K dialogs and 500K utterances from four different domains: Chinese names, phone numbers, ID numbers and license plate numbers. The data is well annotated with sub-slot values, slot values, dialog states and actions. We find some new linguistic phenomena and interactive manners in SSTOD which raise critical challenges of building dialog agents for the task. We test three state-of-the-art dialog models on SSTOD and find they cannot handle the task well on any of the four domains. We also investigate an improved model by involving slot knowledge in a plug-in manner. More work should be done to meet the new challenges raised from SSTOD which widely exists in real-life applications. The dataset and code are publicly available via https://github.com/shunjiu/SSTOD.

cs.CL↗

arXiv2022

S$^2$SQL: Injecting Syntax to Question-Schema Interaction Graph Encoder for Text-to-SQL Parsers

The task of converting a natural language question into an executable SQL query, known as text-to-SQL, is an important branch of semantic parsing. The state-of-the-art graph-based encoder has been successfully used in this task but does not model the question syntax well. In this paper, we propose S$^2$SQL, injecting Syntax to question-Schema graph encoder for Text-to-SQL parsers, which effectively leverages the syntactic dependency information of questions in text-to-SQL to improve the performance. We also employ the decoupling constraint to induce diverse relational edge embedding, which further improves the network's performance. Experiments on the Spider and robustness setting Spider-Syn demonstrate that the proposed approach outperforms all existing methods when pre-training models are used, resulting in a performance ranks first on the Spider leaderboard.

cs.CL↗

arXiv2022

Data-driven Control of Dynamic Event-triggered Systems with Delays

This paper studies data-driven control of unknown sampled-data systems with communication delays under an event-triggering transmission mechanism. Data-based representations for time-invariant linear systems with known or unknown system input matrices are first developed, along with a novel class of dynamic triggering schemes for sampled-data systems with time delays. A model-based stability condition for the resulting event-triggered time-delay system is established using a looped-functional approach. Combining this model-based condition with the data-driven representations, data-based stability conditions are derived. Building on the data-based conditions, methods for co-designing the controller gain and the event-triggering matrix are subsequently provided for both cases with or without using the input matrix. Finally, numerical examples are presented to corroborate the usefulness of additional prior knowledge of the input matrix in reducing the conservatism of stability conditions, as well as the merits of the proposed data-driven event-triggered control schemes relative to existing results.

eess.SY↗

arXiv2022

Linking-Enhanced Pre-Training for Table Semantic Parsing

Recently pre-training models have significantly improved the performance of various NLP tasks by leveraging large-scale text corpora to improve the contextual representation ability of the neural network. The large pre-training language model has also been applied in the area of table semantic parsing. However, existing pre-training approaches have not carefully explored explicit interaction relationships between a question and the corresponding database schema, which is a key ingredient for uncovering their semantic and structural correspondence. Furthermore, the question-aware representation learning in the schema grounding context has received less attention in pre-training objective.To alleviate these issues, this paper designs two novel pre-training objectives to impose the desired inductive bias into the learned representations for table pre-training. We further propose a schema-aware curriculum learning approach to mitigate the impact of noise and learn effectively from the pre-training data in an easy-to-hard manner. We evaluate our pre-trained framework by fine-tuning it on two benchmarks, Spider and SQUALL. The results demonstrate the effectiveness of our pre-training objective and curriculum compared to a variety of baselines.

cs.CL↗

arXiv2022

Critical topology and pressure-induced superconductivity in the van der Waals compound AuTe2Br

The study on quantum spin Hall effect and topological insulators formed the prologue to the surge of research activities in topological materials in the past decade. Compared to intricately engineered quantum wells, three-dimensional weak topological insulators provide a natural route to the quantum spin Hall effect, due to the adiabatic connection between them and a stack of quantum spin Hall insulators, and the convenience in exfoliation of samples associated with their van der Waals-type structure. Despite these advantages, both theoretical prediction and experimental identification of weak topological insulators remain scarce. Here, based on first-principles calculations, we show that AuTe2Br locates at the boundary between a strong and a weak topological insulating state. More interestingly, the critical topology of AuTe2Br persists up to an applied pressure of ~ 15.4 GPa before a structural phase transition accompanied by a change of electronic topology and the onset of superconductivity. Our results establish AuTe2Br as a new candidate for weak topological insulators with the potential to realize various other topological phases of matter.

cond-mat.mtrl-sci↗

arXiv2022

Instance-Conditional Knowledge Distillation for Object Detection

Knowledge distillation has shown great success in classification, however, it is still challenging for detection. In a typical image for detection, representations from different locations may have different contributions to detection targets, making the distillation hard to balance. In this paper, we propose a conditional distillation framework to distill the desired knowledge, namely knowledge that is beneficial in terms of both classification and localization for every instance. The framework introduces a learnable conditional decoding module, which retrieves information given each target instance as query. Specifically, we encode the condition information as query and use the teacher's representations as key. The attention between query and key is used to measure the contribution of different features, guided by a localization-recognition-sensitive auxiliary task. Extensive experiments demonstrate the efficacy of our method: we observe impressive improvements under various settings. Notably, we boost RetinaNet with ResNet-50 backbone from 37.4 to 40.7 mAP (+3.3) under 1x schedule, that even surpasses the teacher (40.4 mAP) with ResNet-101 backbone under 3x schedule. Code has been released on https://github.com/megvii-research/ICD.

cs.CV↗

arXiv2022

Anchor DETR: Query Design for Transformer-Based Object Detection

In this paper, we propose a novel query design for the transformer-based object detection. In previous transformer-based detectors, the object queries are a set of learned embeddings. However, each learned embedding does not have an explicit physical meaning and we cannot explain where it will focus on. It is difficult to optimize as the prediction slot of each object query does not have a specific mode. In other words, each object query will not focus on a specific region. To solved these problems, in our query design, object queries are based on anchor points, which are widely used in CNN-based detectors. So each object query focuses on the objects near the anchor point. Moreover, our query design can predict multiple objects at one position to solve the difficulty: "one region, multiple objects". In addition, we design an attention variant, which can reduce the memory cost while achieving similar or better performance than the standard attention in DETR. Thanks to the query design and the attention variant, the proposed detector that we called Anchor DETR, can achieve better performance and run faster than the DETR with 10$\times$ fewer training epochs. For example, it achieves 44.2 AP with 19 FPS on the MSCOCO dataset when using the ResNet50-DC5 feature for training 50 epochs. Extensive experiments on the MSCOCO benchmark prove the effectiveness of the proposed methods. Code is available at \url{https://github.com/megvii-research/AnchorDETR}.

cs.CV↗

arXiv2021

Reentrance of metal-insulator transition and magnetic competitions on a triangular lattice with second nearest-neighbor hopping

The $120^{\circ}$ antiferromagnetism (AFM) is widely believed as the magnetic ground state of the triangular systems because of the geometrical frustration. The emergence of novel magnetism, such as the row-wise AFM in Mn/Cu(111) and Sn/Si(111), reveals the importance of the longer-range hopping on magnetic competitions in realistic material systems. By utilizing advanced many-body techniques, we systematically studied the isotropic triangular Hubbard model with second nearest-neighbor hopping $t^{\prime}$, including both the single- and the two-particle responses. We found that both electronic and magnetic phase transitions show a clear dependence on $t^{\prime}/t$. Consequently, we observed a remarkable reentrance of the metal-insulator transition and a crossover between the $120^{\circ}$- and the row-wise AFM. The Fermi surface (FS) shows two distinct structures with the nesting vectors consistent with the magnetic correlations. When $t^{\prime}$ evolves from 0 to 1, the correlated Fermi surface demonstrates a Lifschitz transition between the two nesting structures, and exotic phases like the featureless insulating state can be realized. Our work sheds light on the engineering of electronic and magnetic correlations of correlated triangular surfaces via longer-range hopping. The rich phase diagram and the high degree of tunability make the triangular lattice with longer-range hopping a more realistic platform to study the emergent magnetic competitions.

cond-mat.str-el↗

arXiv2021

Improving Text-to-SQL with Schema Dependency Learning

Text-to-SQL aims to map natural language questions to SQL queries. The sketch-based method combined with execution-guided (EG) decoding strategy has shown a strong performance on the WikiSQL benchmark. However, execution-guided decoding relies on database execution, which significantly slows down the inference process and is hence unsatisfactory for many real-world applications. In this paper, we present the Schema Dependency guided multi-task Text-to-SQL model (SDSQL) to guide the network to effectively capture the interactions between questions and schemas. The proposed model outperforms all existing methods in both the settings with or without EG. We show the schema dependency learning partially cover the benefit from EG and alleviates the need for it. SDSQL without EG significantly reduces time consumption during inference, sacrificing only a small amount of performance and provides more flexibility for downstream applications.

cs.CL↗

arXiv2021

Evolution between two orbital-selective Mott phases driven by interorbital hopping

The effect of interorbital hopping on the orbital selective Mottness in a two-band correlation system is investigated by using the dynamical mean-field theory with the Lanczos method as impurity solver. We construct the phase diagram of the two-orbital Hubbard model with interorbital hopping ($t_{12})$, where the orbital selective Mott phases (OSMP) show different evolution trends. We find that the negative interorbital hopping ($t_{12}<0$) can enhance the OSMP regime upon tuning the effective bandwidth ratio. On the contrary, for the cases with positive interorbital hopping ($t_{12}>0$), the OSMP region becomes narrow with the increase of orbital hybridization until it disappears. It is also shown that a new OSMP emerges for a large enough positive interorbital hopping, owing to the role exchange of wide and narrow effective orbitals caused by the large $t_{12}$. Our results are also applicable to the hole-overdoped Ba$_2$CuO$_{4-δ}$ superconductor, which is an orbital-selective Mott compound at half-filling.

cond-mat.str-el↗

arXiv2021

Learning to Select the Next Reasonable Mention for Entity Linking

Entity linking aims to establish a link between entity mentions in a document and the corresponding entities in knowledge graphs (KGs). Previous work has shown the effectiveness of global coherence for entity linking. However, most of the existing global linking methods based on sequential decisions focus on how to utilize previously linked entities to enhance the later decisions. In those methods, the order of mention is fixed, making the model unable to adjust the subsequent linking targets according to the previously linked results, which will cause the previous information to be unreasonably utilized. To address the problem, we propose a novel model, called DyMen, to dynamically adjust the subsequent linking target based on the previously linked entities via reinforcement learning, enabling the model to select a link target that can fully use previously linked information. We sample mention by sliding window to reduce the action sampling space of reinforcement learning and maintain the semantic coherence of mention. Experiments conducted on several benchmark datasets have shown the effectiveness of the proposed model.

cs.CL↗

arXiv2021

Path-Enhanced Multi-Relational Question Answering with Knowledge Graph Embeddings

The multi-relational Knowledge Base Question Answering (KBQA) system performs multi-hop reasoning over the knowledge graph (KG) to achieve the answer. Recent approaches attempt to introduce the knowledge graph embedding (KGE) technique to handle the KG incompleteness but only consider the triple facts and neglect the significant semantic correlation between paths and multi-relational questions. In this paper, we propose a Path and Knowledge Embedding-Enhanced multi-relational Question Answering model (PKEEQA), which leverages multi-hop paths between entities in the KG to evaluate the ambipolar correlation between a path embedding and a multi-relational question embedding via a customizable path representation mechanism, benefiting for achieving more accurate answers from the perspective of both the triple facts and the extra paths. Experimental results illustrate that PKEEQA improves KBQA models' performance for multi-relational question answering with explainability to some extent derived from paths.

cs.CL↗

arXiv2021

A Survey on Dialog Management: Recent Advances and Challenges

Dialog management (DM) is a crucial component in a task-oriented dialog system. Given the dialog history, DM predicts the dialog state and decides the next action that the dialog agent should take. Recently, dialog policy learning has been widely formulated as a Reinforcement Learning (RL) problem, and more works focus on the applicability of DM. In this paper, we survey recent advances and challenges within three critical topics for DM: (1) improving model scalability to facilitate dialog system modeling in new scenarios, (2) dealing with the data scarcity problem for dialog policy learning, and (3) enhancing the training efficiency to achieve better task-completion performance . We believe that this survey can shed a light on future research in dialog management.

cs.CL↗