arXiv subjects

Jun Xiao

Publications and source records attributed to Jun Xiao.

At least 235 records · Page 13Linked to original sources

arXiv2021

Progressive and Selective Fusion Network for High Dynamic Range Imaging

This paper considers the problem of generating an HDR image of a scene from its LDR images. Recent studies employ deep learning and solve the problem in an end-to-end fashion, leading to significant performance improvements. However, it is still hard to generate a good quality image from LDR images of a dynamic scene captured by a hand-held camera, e.g., occlusion due to the large motion of foreground objects, causing ghosting artifacts. The key to success relies on how well we can fuse the input images in their feature space, where we wish to remove the factors leading to low-quality image generation while performing the fundamental computations for HDR image generation, e.g., selecting the best-exposed image/region. We propose a novel method that can better fuse the features based on two ideas. One is multi-step feature fusion; our network gradually fuses the features in a stack of blocks having the same structure. The other is the design of the component block that effectively performs two operations essential to the problem, i.e., comparing and selecting appropriate images/regions. Experimental results show that the proposed method outperforms the previous state-of-the-art methods on the standard benchmark tests.

cs.CV↗

arXiv2021

Video Relation Detection via Tracklet based Visual Transformer

Video Visual Relation Detection (VidVRD), has received significant attention of our community over recent years. In this paper, we apply the state-of-the-art video object tracklet detection pipeline MEGA and deepSORT to generate tracklet proposals. Then we perform VidVRD in a tracklet-based manner without any pre-cutting operations. Specifically, we design a tracklet-based visual Transformer. It contains a temporal-aware decoder which performs feature interactions between the tracklets and learnable predicate query embeddings, and finally predicts the relations. Experimental results strongly demonstrate the superiority of our method, which outperforms other methods by a large margin on the Video Relation Understanding (VRU) Grand Challenge in ACM Multimedia 2021. Codes are released at https://github.com/Dawn-LX/VidVRD-tracklets.

cs.CV↗

arXiv2021

Improving Weakly-supervised Object Localization via Causal Intervention

The recent emerged weakly supervised object localization (WSOL) methods can learn to localize an object in the image only using image-level labels. Previous works endeavor to perceive the interval objects from the small and sparse discriminative attention map, yet ignoring the co-occurrence confounder (e.g., bird and sky), which makes the model inspection (e.g., CAM) hard to distinguish between the object and context. In this paper, we make an early attempt to tackle this challenge via causal intervention (CI). Our proposed method, dubbed CI-CAM, explores the causalities among images, contexts, and categories to eliminate the biased co-occurrence in the class activation maps thus improving the accuracy of object localization. Extensive experiments on several benchmarks demonstrate the effectiveness of CI-CAM in learning the clear object boundaries from confounding contexts. Particularly, in CUB-200-2011 which severely suffers from the co-occurrence confounder, CI-CAM significantly outperforms the traditional CAM-based baseline (58.39% vs 52.4% in top-1 localization accuracy). While in more general scenarios such as ImageNet, CI-CAM can also perform on par with the state of the arts.

cs.CV↗

arXiv2021

INQ, a modern GPU-accelerated computational framework for (time-dependent) density functional theory

We present INQ, a new implementation of density functional theory (DFT) and time-dependent DFT (TDDFT) written from scratch to work on graphical processing units (GPUs). Besides GPU support, INQ makes use of modern code design features and takes advantage of newly available hardware. By designing the code around algorithms, rather than against specific implementations and numerical libraries, we aim to provide a concise and modular code. The result is a fairly complete DFT/TDDFT implementation in roughly 12,000 lines of open-source C++ code representing a modular platform for community-driven application development on emerging high-performance computing architectures for the simulation of materials.

cond-mat.mtrl-sci↗

arXiv2021

Deep Learning for Weakly-Supervised Object Detection and Object Localization: A Survey

Weakly-Supervised Object Detection (WSOD) and Localization (WSOL), i.e., detecting multiple and single instances with bounding boxes in an image using image-level labels, are long-standing and challenging tasks in the CV community. With the success of deep neural networks in object detection, both WSOD and WSOL have received unprecedented attention. Hundreds of WSOD and WSOL methods and numerous techniques have been proposed in the deep learning era. To this end, in this paper, we consider WSOL is a sub-task of WSOD and provide a comprehensive survey of the recent achievements of WSOD. Specifically, we firstly describe the formulation and setting of the WSOD, including the background, challenges, basic framework. Meanwhile, we summarize and analyze all advanced techniques and training tricks for improving detection performance. Then, we introduce the widely-used datasets and evaluation metrics of WSOD. Lastly, we discuss the future directions of WSOD. We believe that these summaries can help pave a way for future research on WSOD and WSOL.

cs.CV↗

arXiv2021

Efficient Ring-topology Decentralized Federated Learning with Deep Generative Models for Industrial Artificial Intelligent

By leveraging deep learning based technologies, the data-driven based approaches have reached great success with the rapid increase of data generated of Industrial Indernet of Things(IIot). However, security and privacy concerns are obstacles for data providers in many sensitive data-driven industrial scenarios, such as healthcare and auto-driving. Many Federated Learning(FL) approaches have been proposed with DNNs for IIoT applications, these works still suffer from low usability of data due to data incompleteness, low quality, insufficient quantity, sensitivity, etc. Therefore, we propose a ring-topogy based decentralized federated learning(RDFL) scheme for Deep Generative Models(DGMs), where DGMs is a promising solution for solving the aforementioned data usability issues. Compare with existing IIoT FL works, our RDFL schemes provides communication efficiency and maintain training performance to boost DGMs in target IIoT tasks. A novel ring FL topology as well as a map-reduce based synchronizing method are designed in the proposed RDFL to improve decentralized FL performance and bandwidth utilization. In addition, InterPlanetary File System(IPFS) is introduced to further improve communication efficiency and FL security. Extensive experiments have been taken to demonstate the superiority of RDFL with either independent and identically distributed(IID) datasets or non-independent and identically distributed(Non-IID) datasets.

cs.LG↗

arXiv2021

GFL: A Decentralized Federated Learning Framework Based On Blockchain

Federated learning(FL) is a rapidly growing field and many centralized and decentralized FL frameworks have been proposed. However, it is of great challenge for current FL frameworks to improve communication performance and maintain the security and robustness under malicious node attacks. In this paper, we propose Galaxy Federated Learning Framework(GFL), a decentralized FL framework based on blockchain. GFL introduces the consistent hashing algorithm to improve communication performance and proposes a novel ring decentralized FL algorithm(RDFL) to improve decentralized FL performance and bandwidth utilization. In addition, GFL introduces InterPlanetary File System(IPFS) and blockchain to further improve communication efficiency and FL security. Our experiments show that GFL improves communication performance and decentralized FL performance under the data poisoning of malicious nodes and non-independent and identically distributed(Non-IID) datasets.

cs.LG↗

arXiv2021

Human-like Controllable Image Captioning with Verb-specific Semantic Roles

Controllable Image Captioning (CIC) -- generating image descriptions following designated control signals -- has received unprecedented attention over the last few years. To emulate the human ability in controlling caption generation, current CIC studies focus exclusively on control signals concerning objective properties, such as contents of interest or descriptive patterns. However, we argue that almost all existing objective control signals have overlooked two indispensable characteristics of an ideal control signal: 1) Event-compatible: all visual contents referred to in a single sentence should be compatible with the described activity. 2) Sample-suitable: the control signals should be suitable for a specific image sample. To this end, we propose a new control signal for CIC: Verb-specific Semantic Roles (VSR). VSR consists of a verb and some semantic roles, which represents a targeted activity and the roles of entities involved in this activity. Given a designated VSR, we first train a grounded semantic role labeling (GSRL) model to identify and ground all entities for each role. Then, we propose a semantic structure planner (SSP) to learn human-like descriptive semantic structures. Lastly, we use a role-shift captioning model to generate the captions. Extensive experiments and ablations demonstrate that our framework can achieve better controllability than several strong baselines on two challenging CIC benchmarks. Besides, we can generate multi-level diverse captions easily. The code is available at: https://github.com/mad-red/VSR-guided-CIC.

cs.CV↗

arXiv2021

Benchmarking calculations with spectroscopic accuracy of level energies and wavelengths in W LVII - W LXII tungsten ions

Atomic properties of $n=3$ states of the W$^{56+}$ $-$ W$^{61+}$ ions are systematically investigated through two state-of-the-art methods, namely, the second-order many-body perturbation theory, and the multi-configuration Dirac-Hartree-Fock method combined with the relativistic configuration interaction approach. The contributions of valence-valence and core-valence electron correlations, the Breit interaction, the higher-order retardation correction beyond the Breit interaction through the transverse photon interaction, and the quantum electrodynamical corrections to the excitation energies are studied in detail. The excitation energies and wavelengths obtained with the two methods agree with each other within \approx 0.01 %. The present results achieve spectroscopic accuracy and provide a benchmark test for various applications and other theoretical calculations of W$^{56+}$ $-$ W$^{61+}$ ions. They will assist spectroscopists in their assignment and direct identification of observed lines in complex spectra.

physics.atom-ph↗

arXiv2021

Ref-NMS: Breaking Proposal Bottlenecks in Two-Stage Referring Expression Grounding

The prevailing framework for solving referring expression grounding is based on a two-stage process: 1) detecting proposals with an object detector and 2) grounding the referent to one of the proposals. Existing two-stage solutions mostly focus on the grounding step, which aims to align the expressions with the proposals. In this paper, we argue that these methods overlook an obvious mismatch between the roles of proposals in the two stages: they generate proposals solely based on the detection confidence (i.e., expression-agnostic), hoping that the proposals contain all right instances in the expression (i.e., expression-aware). Due to this mismatch, current two-stage methods suffer from a severe performance drop between detected and ground-truth proposals. To this end, we propose Ref-NMS, which is the first method to yield expression-aware proposals at the first stage. Ref-NMS regards all nouns in the expression as critical objects, and introduces a lightweight module to predict a score for aligning each box with a critical object. These scores can guide the NMS operation to filter out the boxes irrelevant to the expression, increasing the recall of critical objects, resulting in a significantly improved grounding performance. Since Ref- NMS is agnostic to the grounding step, it can be easily integrated into any state-of-the-art two-stage method. Extensive ablation studies on several backbones, benchmarks, and tasks consistently demonstrate the superiority of Ref-NMS. Codes are available at: https://github.com/ChopinSharp/ref-nms.

cs.CV↗

arXiv2021

Kinetic Energy Distribution of Fragments for Thermal Neutron-Induced $^{235}$U and $^{239}$Pu Fission Reactions

Focused on the generation and evolution of vast complementary pairs of the primary fission fragments at scission moment, Dinuclear and Statistical Model (DSM) is proposed. (1) It is assumed that the fissile nucleus elongates along a symmetric coaxis until it breaks into two primary fission fragments. (2) Every complementary pair of the primary fission fragments is approximatively described as two ellipsoids with large deformation at scission moment. (3) The kinetic energy in every complementary pair of the primary fragments is mainly provided by Coulomb repulsion, which is explicitly expressed through strict six-dimensional integrals. (4) Only three phenomenological coefficients are obtained to globally describe the quadrupole deformation parameters of arbitrary primary fragments both for $^{235}$U($n_{th}, f$) and $^{239}$Pu($n_{th}, f$) reactions, on the basis of the common characteristics of the measured data, such as mass and charge distributions, kinetic energy distributions. In the framework of DSM, the explicit average total kinetic energy distribution $\overline{TKE}(A)$ and the average kinetic energy distribution $\overline{KE}(A)$ are consistently represented. The theoretical results in this paper agree well with the experimental data. Furthermore, this model is expected as the reliable approach to generally evaluate the corresponding observebles for thermal neutron-induced fission of actinides.

nucl-th↗

arXiv2021

Probing Multiple Electric Dipole Forbidden Optical Transitions in Highly Charged Nickel Ions

Highly charged ions (HCIs) are promising candidates for the next generation of atomic clocks, owing to their tightly bound electron cloud, which significantly suppresses the common environmental disturbances to the quantum oscillator. Here we propose and pursue an experimental strategy that, while focusing on various HCIs of a single atomic element, keeps the number of candidate clock transitions as large as possible. Following this strategy, we identify four adjacent charge states of nickel HCIs that offer as many as six optical transitions. Experimentally, we demonstrated the essential capability of producing these ions in the low-energy compact Shanghai-Wuhan Electron Beam Ion Trap. We measured the wavelengths of four magnetic-dipole ($M$1) and one electric-quadrupole ($E$2) clock transitions with an accuracy of several ppm with a novel calibration method; two of these lines were observed and characterized for the first time in controlled laboratory settings. Compared to the earlier determinations, our measurements improved wavelength accuracy by an order of magnitude. Such measurements are crucial for constraining the range of laser wavelengths for finding the "needle in a haystack" narrow lines. In addition, we calculated frequencies and quality factors, evaluated sensitivity of these six transitions to the hypothetical variation of the electromagnetic fine structure constant $α$ needed for fundamental physics applications. We argue that all the six transitions in nickel HCIs offer intrinsic immunity to all common perturbations of quantum oscillators, and one of them has the projected fractional frequency uncertainty down to the remarkable level of 10$^{-19}$.

physics.atom-ph↗

arXiv2020

ROBY: Evaluating the Robustness of a Deep Model by its Decision Boundaries

With the successful application of deep learning models in many real-world tasks, the model robustness becomes more and more critical. Often, we evaluate the robustness of the deep models by attacking them with purposely generated adversarial samples, which is computationally costly and dependent on the specific attackers and the model types. This work proposes a generic evaluation metric ROBY, a novel attack-independent robustness measure based on the model's decision boundaries. Independent of adversarial samples, ROBY uses the inter-class and intra-class statistic features to capture the features of the model's decision boundaries. We experimented on ten state-of-the-art deep models and showed that ROBY matches the robustness gold standard of attack success rate (ASR) by a strong first-order generic attacker. with only 1% of time cost. To the best of our knowledge, ROBY is the first lightweight attack-independent robustness evaluation metric that can be applied to a wide range of deep models. The code of ROBY is open sourced at https://github.com/baaaad/ROBY-Evaluating-the-Robustness-of-a-Deep-Model-by-its-Decision-Boundaries.

cs.LG↗

arXiv2020

Federated Unsupervised Representation Learning

To leverage enormous unlabeled data on distributed edge devices, we formulate a new problem in federated learning called Federated Unsupervised Representation Learning (FURL) to learn a common representation model without supervision while preserving data privacy. FURL poses two new challenges: (1) data distribution shift (Non-IID distribution) among clients would make local models focus on different categories, leading to the inconsistency of representation spaces. (2) without the unified information among clients in FURL, the representations across clients would be misaligned. To address these challenges, we propose Federated Constrastive Averaging with dictionary and alignment (FedCA) algorithm. FedCA is composed of two key modules: (1) dictionary module to aggregate the representations of samples from each client and share with all clients for consistency of representation space and (2) alignment module to align the representation of each client on a base model trained on a public data. We adopt the contrastive loss for local model training. Through extensive experiments with three evaluation protocols in IID and Non-IID settings, we demonstrate that FedCA outperforms all baselines with significant margins.

cs.LG↗

arXiv2020

Topic Adaptation and Prototype Encoding for Few-Shot Visual Storytelling

Visual Storytelling~(VIST) is a task to tell a narrative story about a certain topic according to the given photo stream. The existing studies focus on designing complex models, which rely on a huge amount of human-annotated data. However, the annotation of VIST is extremely costly and many topics cannot be covered in the training dataset due to the long-tail topic distribution. In this paper, we focus on enhancing the generalization ability of the VIST model by considering the few-shot setting. Inspired by the way humans tell a story, we propose a topic adaptive storyteller to model the ability of inter-topic generalization. In practice, we apply the gradient-based meta-learning algorithm on multi-modal seq2seq models to endow the model the ability to adapt quickly from topic to topic. Besides, We further propose a prototype encoding structure to model the ability of intra-topic derivation. Specifically, we encode and restore the few training story text to serve as a reference to guide the generation at inference time. Experimental results show that topic adaptation and prototype encoding structure mutually bring benefit to the few-shot model on BLEU and METEOR metric. The further case study shows that the stories generated after few-shot adaptation are more relative and expressive.

cs.CL↗

arXiv2020

Berry curvature memory through electrically driven stacking transitions

In two-dimensional layered quantum materials, the stacking order of the layers determines both the crystalline symmetry and electronic properties such as the Berry curvature, topology and electron correlation. Electrical stimuli can influence quasiparticle interactions and the free-energy landscape, making it possible to dynamically modify the stacking order and reveal hidden structures that host different quantum properties. Here we demonstrate electrically driven stacking transitions that can be applied to design nonvolatile memory based on Berry curvature in few-layer WTe$_2$. The interplay of out-of-plane electric fields and electrostatic doping controls in-plane interlayer sliding and creates multiple polar and centrosymmetric stacking orders. In situ nonlinear Hall transport reveals such stacking rearrangements result in a layer-parity-selective Berry curvature memory in momentum space, where the sign reversal of the Berry curvature and its dipole only occurs in odd-layer crystals. Our findings open an avenue towards exploring coupling between topology, electron correlations, and ferroelectricity in hidden stacking orders and demonstrate a new low-energy-cost, electrically controlled topological memory in the atomically thin limit.

cond-mat.mtrl-sci↗

arXiv2020

Hierarchical Fashion Graph Network for Personalized Outfit Recommendation

Fashion outfit recommendation has attracted increasing attentions from online shopping services and fashion communities.Distinct from other scenarios (e.g., social networking or content sharing) which recommend a single item (e.g., a friend or picture) to a user, outfit recommendation predicts user preference on a set of well-matched fashion items.Hence, performing high-quality personalized outfit recommendation should satisfy two requirements -- 1) the nice compatibility of fashion items and 2) the consistence with user preference. However, present works focus mainly on one of the requirements and only consider either user-outfit or outfit-item relationships, thereby easily leading to suboptimal representations and limiting the performance. In this work, we unify two tasks, fashion compatibility modeling and personalized outfit recommendation. Towards this end, we develop a new framework, Hierarchical Fashion Graph Network(HFGN), to model relationships among users, items, and outfits simultaneously. In particular, we construct a hierarchical structure upon user-outfit interactions and outfit-item mappings. We then get inspirations from recent graph neural networks, and employ the embedding propagation on such hierarchical graph, so as to aggregate item information into an outfit representation, and then refine a user's representation via his/her historical outfits. Furthermore, we jointly train these two tasks to optimize these representations. To demonstrate the effectiveness of HFGN, we conduct extensive experiments on a benchmark dataset, and HFGN achieves significant improvements over the state-of-the-art compatibility matching models like NGNN and outfit recommenders like FHN.

cs.IR↗

arXiv2020

Strain-Induced Room-Temperature Ferroelectricity in SrTiO$_3$ Membranes

Advances in complex oxide heteroepitaxy have highlighted the enormous potential of utilizing strain engineering via lattice mismatch to control ferroelectricity in thin-film heterostructures. This approach, however, lacks the ability to produce large and continuously variable strain states, thus limiting the potential for designing and tuning the desired properties of ferroelectric films. Here, we observe and explore dynamic strain-induced ferroelectricity in SrTiO$_3$ by laminating freestanding oxide films onto a stretchable polymer substrate. Using a combination of scanning probe microscopy, optical second harmonic generation measurements, and atomistic modeling, we demonstrate robust room-temperature ferroelectricity in SrTiO$_3$ with 2.0% uniaxial tensile strain, corroborated by the notable features of 180° ferroelectric domains and an extrapolated transition temperature of 400 K. Our work reveals the enormous potential of employing oxide membranes to create and enhance ferroelectricity in environmentally benign lead-free oxides, which hold great promise for applications ranging from non-volatile memories and microwave electronics.

cond-mat.mtrl-sci↗