arXiv subjects

Yu Li

Publications and source records attributed to Yu Li.

At least 451 records · Page 25Linked to original sources

arXiv2022

Interpretable RNA Foundation Model from Unannotated Data for Highly Accurate RNA Structure and Function Predictions

Non-coding RNA structure and function are essential to understanding various biological processes, such as cell signaling, gene expression, and post-transcriptional regulations. These are all among the core problems in the RNA field. With the rapid growth of sequencing technology, we have accumulated a massive amount of unannotated RNA sequences. On the other hand, expensive experimental observatory results in only limited numbers of annotated data and 3D structures. Hence, it is still challenging to design computational methods for predicting their structures and functions. The lack of annotated data and systematic study causes inferior performance. To resolve the issue, we propose a novel RNA foundation model (RNA-FM) to take advantage of all the 23 million non-coding RNA sequences through self-supervised learning. Within this approach, we discover that the pre-trained RNA-FM could infer sequential and evolutionary information of non-coding RNAs without using any labels. Furthermore, we demonstrate RNA-FM's effectiveness by applying it to the downstream secondary/3D structure prediction, SARS-CoV-2 genome structure and evolution prediction, protein-RNA binding preference modeling, and gene expression regulation modeling. The comprehensive experiments show that the proposed method improves the RNA structural and functional modelling results significantly and consistently. Despite only being trained with unlabelled data, RNA-FM can serve as the foundational model for the field.

q-bio.QM↗

arXiv2022

Deep Learning-enabled Spatial Phase Unwrapping for 3D Measurement

In terms of 3D imaging speed and system cost, the single-camera system projecting single-frequency patterns is the ideal option among all proposed Fringe Projection Profilometry (FPP) systems. This system necessitates a robust spatial phase unwrapping (SPU) algorithm. However, robust SPU remains a challenge in complex scenes. Quality-guided SPU algorithms need more efficient ways to identify the unreliable points in phase maps before unwrapping. End-to-end deep learning SPU methods face generality and interpretability problems. This paper proposes a hybrid method combining deep learning and traditional path-following for robust SPU in FPP. This hybrid SPU scheme demonstrates better robustness than traditional quality-guided SPU methods, better interpretability than end-to-end deep learning scheme, and generality on unseen data. Experiments on the real dataset of multiple illumination conditions and multiple FPP systems differing in image resolution, the number of fringes, fringe direction, and optics wavelength verify the effectiveness of the proposed method.

eess.IV↗

arXiv2022

Knowledge-Grounded Dialogue Generation with a Unified Knowledge Representation

Knowledge-grounded dialogue systems are challenging to build due to the lack of training data and heterogeneous knowledge sources. Existing systems perform poorly on unseen topics due to limited topics covered in the training data. In addition, heterogeneous knowledge sources make it challenging for systems to generalize to other tasks because knowledge sources in different knowledge representations require different knowledge encoders. To address these challenges, we present PLUG, a language model that homogenizes different knowledge sources to a unified knowledge representation for knowledge-grounded dialogue generation tasks. PLUG is pre-trained on a dialogue generation task conditioned on a unified essential knowledge representation. It can generalize to different downstream knowledge-grounded dialogue generation tasks with a few training examples. The empirical evaluation on two benchmarks shows that our model generalizes well across different knowledge-grounded tasks. It can achieve comparable performance with state-of-the-art methods under a fully-supervised setting and significantly outperforms other methods in zero-shot and few-shot settings.

cs.CL↗

arXiv2022

Using Chatbots to Teach Languages

This paper reports on progress towards building an online language learning tool to provide learners with conversational experience by using dialog systems as conversation practice partners. Our system can adapt to users' language proficiency on the fly. We also provide automatic grammar error feedback to help users learn from their mistakes. According to our first adopters, our system is entertaining and useful. Furthermore, we will provide the learning technology community a large-scale conversation dataset on language learning and grammar correction. Our next step is to make our system more adaptive to user profile information by using reinforcement learning algorithms.

cs.CL↗

arXiv2022

Neutron Scattering Study of Fluctuating and Static Spin Correlations in the Anisotropic Spin Glass Fe$_2$TiO$_5$

The anisotropic spin glass transition, in which spin freezing is observed only along the c-axis in pseudobrookite Fe$_2$TiO$_5$, has long been perplexing because the Fe$^{3+}$ moments (d$^5$) are expected to be isotropic. Recently, neutron diffraction demonstrated that surfboard-shaped antiferromagnetic nanoregions coalesce above the glass transition temperature, T$_g$ $\approx$ 55 K, and a model was proposed in which the freezing of the fluctuations of the surfboards' magnetization leads to the anisotropic spin glass state. Given this new model, we have carried out high resolution inelastic neutron scattering measurements of the spin-spin correlations to understand the temperature dependence of the intra-surfboard spin dynamics on neutron (picosecond) time-scales. Here, we report on the temperature-dependence of the spin fluctuations measured from single crystal Fe$_2$TiO$_5$. Strong quasi-elastic magnetic scattering, arising from intra-surfboard correlations, is observed well above T$_g$. The spin fluctuations possess a steep energy-wave vector relation and are indicative of strong exchange interactions, consistent with the large Curie-Weiss temperature. As the temperature approaches T$_g$ from above, a shift in spectral weight from inelastic to elastic scattering is observed. At various temperatures between 4 K and 300 K, a characteristic relaxation rate of the fluctuations is determined. Despite the freezing of the majority of the spin correlations, an inelastic contribution remains even at base temperature, signifying the presence of fluctuating intra-surfboard spin correlations to at least T/T$_g$ $\approx$ 0.1 consistent with a description of Fe$_2$TiO$_5$ as a hybrid between conventional and geometrically frustrated spin glasses.

cond-mat.dis-nn↗

arXiv2022

Unseasonal super ionospheric plasma bubble and scintillations seeded by the 2022 Tonga Volcano Eruption related perturbations

The Hunga-Tonga volcano eruption at 04:14:45 UT on 15 January 2022 produced various waves propagating globally, disturbing the background atmosphere and ionosphere. Coinciding with the arrival of perturbation waves, several equatorial plasma bubbles (EPBs) were consecutively generated at post-sunset hours over the East/Southeast Asian region, with the largest extension to middle latitudes. These EPBs caused intense L-band amplitude scintillations at middle-to-low latitudes, with signal fading depths up to ~16 dB. Considering the very rare occurrence of EPBs during this season in East/Southeast Asian sector and the significantly modulated background ionosphere, we believe that the perturbation waves launched by the volcano eruption triggered the generation of unseasonal super EPBs. The ionospheric perturbations linked with the 2022 Tonga volcano eruption propagated coincidently through the East/Southeast Asia longitude sector near sunset, modulated the equatorial F region bottomside plasma density and acted as the seeding source for the generation of unseasonal super bubbles. Our results implicate that volcano eruption could indirectly affect the satellite communication links in the region more than ten thousand kilometers away.

physics.space-ph↗

arXiv2022

"Think Before You Speak": Improving Multi-Action Dialog Policy by Planning Single-Action Dialogs

Multi-action dialog policy (MADP), which generates multiple atomic dialog actions per turn, has been widely applied in task-oriented dialog systems to provide expressive and efficient system responses. Existing MADP models usually imitate action combinations from the labeled multi-action dialog samples. Due to data limitations, they generalize poorly toward unseen dialog flows. While interactive learning and reinforcement learning algorithms can be applied to incorporate external data sources of real users and user simulators, they take significant manual effort to build and suffer from instability. To address these issues, we propose Planning Enhanced Dialog Policy (PEDP), a novel multi-task learning framework that learns single-action dialog dynamics to enhance multi-action prediction. Our PEDP method employs model-based planning for conceiving what to express before deciding the current response through simulating single-action dialogs. Experimental results on the MultiWOZ dataset demonstrate that our fully supervised learning-based method achieves a solid task success rate of 90.6%, improving 3% compared to the state-of-the-art methods.

cs.CL↗

arXiv2022

Parameter-robust Braess-Sarazin-type smoothers for linear elasticity problems

In this work, we propose three Braess-Sarazin-type multigrid relaxation schemes for solving linear elasticity problems, where the marker and cell scheme, a finite difference method, is used for the discretization. The three relaxation schemes are Jacobi-Braess-Sarazin, Mass-Braess-Sarazin, and Vanka-Braess-Sarazin. A local Fourier analysis (LFA) for the block-structured relaxation schemes is presented to study multigrid convergence behavior. From LFA, we derive optimal LFA smoothing factor for each case. We obtain highly efficient smoothing factors, which are independent of Lamé constants. Vanka-Braess-Sarazin relaxation scheme leads to the most efficient one. In each relaxation, a Schur complement system needs to be solved. Due to the fact that direct solve is often expensive, an inexact version is developed, where we simply use at most three weighted Jacobi iterations on the Schur complement system. Finally, two-grid and V-cycle multigrid performances are presented to validate our theoretical results. Our numerical results show that inexact versions can achieve the same performance as that of exact versions and our methods are robust to the Lamé constants.

math.NA↗

arXiv2022

Temporally Efficient Vision Transformer for Video Instance Segmentation

Recently vision transformer has achieved tremendous success on image-level visual recognition tasks. To effectively and efficiently model the crucial temporal information within a video clip, we propose a Temporally Efficient Vision Transformer (TeViT) for video instance segmentation (VIS). Different from previous transformer-based VIS methods, TeViT is nearly convolution-free, which contains a transformer backbone and a query-based video instance segmentation head. In the backbone stage, we propose a nearly parameter-free messenger shift mechanism for early temporal context fusion. In the head stages, we propose a parameter-shared spatiotemporal query interaction mechanism to build the one-to-one correspondence between video instances and queries. Thus, TeViT fully utilizes both framelevel and instance-level temporal context information and obtains strong temporal modeling capacity with negligible extra computational cost. On three widely adopted VIS benchmarks, i.e., YouTube-VIS-2019, YouTube-VIS-2021, and OVIS, TeViT obtains state-of-the-art results and maintains high inference speed, e.g., 46.6 AP with 68.9 FPS on YouTube-VIS-2019. Code is available at https://github.com/hustvl/TeViT.

cs.CV↗

arXiv2022

A physics and data co-driven surrogate modeling approach for temperature field prediction on irregular geometric domain

In the whole aircraft structural optimization loop, thermal analysis plays a very important role. But it faces a severe computational burden when directly applying traditional numerical analysis tools, especially when each optimization involves repetitive parameter modification and thermal analysis followed. Recently, with the fast development of deep learning, several Convolutional Neural Network (CNN) surrogate models have been introduced to overcome this obstacle. However, for temperature field prediction on irregular geometric domains (TFP-IGD), CNN can hardly be competent since most of them stem from processing for regular images. To alleviate this difficulty, we propose a novel physics and data co-driven surrogate modeling method. First, after adapting the Bezier curve in geometric parameterization, a body-fitted coordinate mapping is introduced to generate coordinate transforms between the irregular physical plane and regular computational plane. Second, a physics-driven CNN surrogate with partial differential equation (PDE) residuals as a loss function is utilized for fast meshing (meshing surrogate); then, we present a data-driven surrogate model based on the multi-level reduced-order method, aiming to learn solutions of temperature field in the above regular computational plane (thermal surrogate). Finally, combining the grid position information provided by the meshing surrogate with the scalar temperature field information provided by the thermal surrogate (combined model), we reach an end-to-end surrogate model from geometric parameters to temperature field prediction on an irregular geometric domain. Numerical results demonstrate that our method can significantly improve accuracy prediction on a smaller dataset while reducing the training time when compared with other CNN methods.

cs.LG↗

arXiv2022

Enhanced two-component superconductivity in CoSi2/TiSi2 heterojunctions

We report enhanced two-component superconductivity in (CoSi2/Si)/TiSi2 superconductor/normal-metal (S/N) heterojunctions. An enhanced superconducting transition temperature about twice that of CoSi2 and an upper critical field about 20 times bigger than that of epitaxial CoSi2/Si films were found. The tunneling spectra of three-terminal S/N junctions show pronounced zero-bias conductance peaks (ZBCPs) that signify penetration of odd-frequency, spin-triplet and even-parity Cooper pairs in TiSi2 from triplet dominant pairing in CoSi2/Si driven by symmetry reduction at the CoSi2/Si interface. Both the enhancement of the superconducting transition temperature and the ZBCPs are found to be more pronounced if TiSi2 is made more diffusive.

cond-mat.supr-con↗

arXiv2022

A Virtual Reality-based Training and Assessment System for Bridge Inspectors with an Assistant Drone

Over 600,000 bridges in the U.S. must be inspected every two years to identify flaws, defects, or potential problems that may need follow-up maintenance. Bridge inspection has adopted unmanned aerial vehicles (or drones) for improving safety, efficiency, and cost-effectiveness. Although drones can operate in an autonomous mode, keeping inspectors in the loop is critical for complex tasks in bridge inspection. Therefore, inspectors need to develop the skill and confidence to operate drones in their jobs. This paper presents the design and development of a virtual reality-based training and assessment system for inspectors assisted by a drone in bridge inspection. The system is composed of four integrated modules: a simulated bridge inspection developed in Unity, an interface that allows a trainee to operate the drone in simulation using a remote controller, data monitoring and analysis to provide real-time, in-task feedback to trainees to assist their learning, and a post-study assessment supporting personalized training. The paper also conducts a proof-of-concept pilot study to illustrate the functionality of this system. The study demonstrated that TASBID, as a tool for the early-stage training, can objectively identify the training needs of individuals in detail and, further, help them develop the skill and confidence in collaborating with a drone in bridge inspection. The system has built a modeling and analysis platform for exploring advanced solutions to the human-drone cooperative inspection of civil infrastructure.

cs.RO↗

arXiv2022

MixDefense: A Defense-in-Depth Framework for Adversarial Example Detection Based on Statistical and Semantic Analysis

Machine learning with deep neural networks (DNNs) has become one of the foundation techniques in many safety-critical systems, such as autonomous vehicles and medical diagnosis systems. DNN-based systems, however, are known to be vulnerable to adversarial examples (AEs) that are maliciously perturbed variants of legitimate inputs. While there has been a vast body of research to defend against AE attacks in the literature, the performances of existing defense techniques are still far from satisfactory, especially for adaptive attacks, wherein attackers are knowledgeable about the defense mechanisms and craft AEs accordingly. In this work, we propose a multilayer defense-in-depth framework for AE detection, namely MixDefense. For the first layer, we focus on those AEs with large perturbations. We propose to leverage the `noise' features extracted from the inputs to discover the statistical difference between natural images and tampered ones for AE detection. For AEs with small perturbations, the inference result of such inputs would largely deviate from their semantic information. Consequently, we propose a novel learning-based solution to model such contradictions for AE detection. Both layers are resilient to adaptive attacks because there do not exist gradient propagation paths for AE generation. Experimental results with various AE attack methods on image classification datasets show that the proposed MixDefense solution outperforms the existing AE detection techniques by a considerable margin.

cs.CR↗

arXiv2022

Hot-Refresh Model Upgrades with Regression-Alleviating Compatible Training in Image Retrieval

The task of hot-refresh model upgrades of image retrieval systems plays an essential role in the industry but has never been investigated in academia before. Conventional cold-refresh model upgrades can only deploy new models after the gallery is overall backfilled, taking weeks or even months for massive data. In contrast, hot-refresh model upgrades deploy the new model immediately and then gradually improve the retrieval accuracy by backfilling the gallery on-the-fly. Compatible training has made it possible, however, the problem of model regression with negative flips poses a great challenge to the stable improvement of user experience. We argue that it is mainly due to the fact that new-to-old positive query-gallery pairs may show less similarity than new-to-new negative pairs. To solve the problem, we introduce a Regression-Alleviating Compatible Training (RACT) method to properly constrain the feature compatibility while reducing negative flips. The core is to encourage the new-to-old positive pairs to be more similar than both the new-to-old negative pairs and the new-to-new negative pairs. An efficient uncertainty-based backfilling strategy is further introduced to fasten accuracy improvements. Extensive experiments on large-scale retrieval benchmarks (e.g., Google Landmark) demonstrate that our RACT effectively alleviates the model regression for one more step towards seamless model upgrades. The code will be available at https://github.com/binjiezhang/RACT_ICLR2022.

cs.CV↗

arXiv2022

What You See is Not What the Network Infers: Detecting Adversarial Examples Based on Semantic Contradiction

Adversarial examples (AEs) pose severe threats to the applications of deep neural networks (DNNs) to safety-critical domains, e.g., autonomous driving. While there has been a vast body of AE defense solutions, to the best of our knowledge, they all suffer from some weaknesses, e.g., defending against only a subset of AEs or causing a relatively high accuracy loss for legitimate inputs. Moreover, most existing solutions cannot defend against adaptive attacks, wherein attackers are knowledgeable about the defense mechanisms and craft AEs accordingly. In this paper, we propose a novel AE detection framework based on the very nature of AEs, i.e., their semantic information is inconsistent with the discriminative features extracted by the target DNN model. To be specific, the proposed solution, namely ContraNet, models such contradiction by first taking both the input and the inference result to a generator to obtain a synthetic output and then comparing it against the original input. For legitimate inputs that are correctly inferred, the synthetic output tries to reconstruct the input. On the contrary, for AEs, instead of reconstructing the input, the synthetic output would be created to conform to the wrong label whenever possible. Consequently, by measuring the distance between the input and the synthetic output with metric learning, we can differentiate AEs from legitimate inputs. We perform comprehensive evaluations under various AE attack scenarios, and experimental results show that ContraNet outperforms existing solutions by a large margin, especially under adaptive attacks. Moreover, our analysis shows that successful AEs that can bypass ContraNet tend to have much-weakened adversarial semantics. We have also shown that ContraNet can be easily combined with adversarial training techniques to achieve further improved AE defense capabilities.

cs.CR↗

arXiv2022

Semantic Guided Single Image Reflection Removal

Reflection is common in images capturing scenes behind a glass window, which is not only a disturbance visually but also influence the performance of other computer vision algorithms. Single image reflection removal is an ill-posed problem because the color at each pixel needs to be separated into two values, i.e., the desired clear background and the reflection. To solve it, existing methods propose priors such as smoothness, color consistency. However, the low-level priors are not reliable in complex scenes, for instance, when capturing a real outdoor scene through a window, both the foreground and background contain both smooth and sharp area and a variety of color. In this paper, inspired by the fact that human can separate the two layers easily by recognizing the objects, we use the object semantic as guidance to force the same semantic object belong to the same layer. Extensive experiments on different datasets show that adding the semantic information offers a significant improvement to reflection separation. We also demonstrate the applications of the proposed method to other computer vision tasks.

cs.CV↗

arXiv2022

Active Learning for Open-set Annotation

Existing active learning studies typically work in the closed-set setting by assuming that all data examples to be labeled are drawn from known classes. However, in real annotation tasks, the unlabeled data usually contains a large amount of examples from unknown classes, resulting in the failure of most active learning methods. To tackle this open-set annotation (OSA) problem, we propose a new active learning framework called LfOSA, which boosts the classification performance with an effective sampling strategy to precisely detect examples from known classes for annotation. The LfOSA framework introduces an auxiliary network to model the per-example max activation value (MAV) distribution with a Gaussian Mixture Model, which can dynamically select the examples with highest probability from known classes in the unlabeled set. Moreover, by reducing the temperature $T$ of the loss function, the detection model will be further optimized by exploiting both known and unknown supervision. The experimental results show that the proposed method can significantly improve the selection quality of known classes, and achieve higher classification accuracy with lower annotation cost than state-of-the-art active learning methods. To the best of our knowledge, this is the first work of active learning for open-set annotation.

cs.LG↗

arXiv2022

Towards explainable artificial intelligence (XAI) for early anticipation of traffic accidents

Traffic accident anticipation is a vital function of Automated Driving Systems (ADSs) for providing a safety-guaranteed driving experience. An accident anticipation model aims to predict accidents promptly and accurately before they occur. Existing Artificial Intelligence (AI) models of accident anticipation lack a human-interpretable explanation of their decision-making. Although these models perform well, they remain a black-box to the ADS users, thus difficult to get their trust. To this end, this paper presents a Gated Recurrent Unit (GRU) network that learns spatio-temporal relational features for the early anticipation of traffic accidents from dashcam video data. A post-hoc attention mechanism named Grad-CAM is integrated into the network to generate saliency maps as the visual explanation of the accident anticipation decision. An eye tracker captures human eye fixation points for generating human attention maps. The explainability of network-generated saliency maps is evaluated in comparison to human attention maps. Qualitative and quantitative results on a public crash dataset confirm that the proposed explainable network can anticipate an accident on average 4.57 seconds before it occurs, with 94.02% average precision. In further, various post-hoc attention-based XAI methods are evaluated and compared. It confirms that the Grad-CAM chosen by this study can generate high-quality, human-interpretable saliency maps (with 1.23 Normalized Scanpath Saliency) for explaining the crash anticipation decision. Importantly, results confirm that the proposed AI model, with a human-inspired design, can outperform humans in the accident anticipation.

cs.CV↗