Searcharxiv⌕ Search

arXiv subjects

Chen Liu

Publications and source records attributed to Chen Liu.

At least 181 records · Page 10Linked to original sources
arXiv2023

Towards Control-Centric Representations in Reinforcement Learning from Images

Image-based Reinforcement Learning is a practical yet challenging task. A major hurdle lies in extracting control-centric representations while disregarding irrelevant information. While approaches that follow the bisimulation principle exhibit the potential in learning state representations to address this issue, they still grapple with the limited expressive capacity of latent dynamics and the inadaptability to sparse reward environments. To address these limitations, we introduce ReBis, which aims to capture control-centric information by integrating reward-free control information alongside reward-specific knowledge. ReBis utilizes a transformer architecture to implicitly model the dynamics and incorporates block-wise masking to eliminate spatiotemporal redundancy. Moreover, ReBis combines bisimulation-based loss with asymmetric reconstruction loss to prevent feature collapse in environments with sparse rewards. Empirical studies on two large benchmarks, including Atari games and DeepMind Control Suit, demonstrate that ReBis has superior performance compared to existing methods, proving its effectiveness.

cs.LG↗
arXiv2023

Deep Learning Enhanced Realized GARCH

We propose a new approach to volatility modeling by combining deep learning (LSTM) and realized volatility measures. This LSTM-enhanced realized GARCH framework incorporates and distills modeling advances from financial econometrics, high frequency trading data and deep learning. Bayesian inference via the Sequential Monte Carlo method is employed for statistical inference and forecasting. The new framework can jointly model the returns and realized volatility measures, has an excellent in-sample fit and superior predictive performance compared to several benchmark models, while being able to adapt well to the stylized facts in volatility. The performance of the new framework is tested using a wide range of metrics, from marginal likelihood, volatility forecasting, to tail risk forecasting and option pricing. We report on a comprehensive empirical study using 31 widely traded stock indices over a time period that includes COVID-19 pandemic.

econ.EM↗
arXiv2023

Structure preserving schemes for Fokker-Planck equations of irreversible processes

In this paper, we introduce second order and fourth order space discretization via finite difference implementation of the finite element method for solving Fokker-Planck equations associated with irreversible processes. The proposed schemes are first order in time and second order and fourth order in space. Under mild mesh conditions and time step constraints for smooth solutions, the schemes are proved to be monotone, thus are positivity-preserving and energy dissipative. In particular, our scheme is suitable for capturing steady state solutions in large final time simulations.

math.NA↗
arXiv2023

Application of Deep Learning Methods Combined with Physical Background in Wide Field of View Imaging Atmospheric Cherenkov Telescopes

The HADAR experiment, which will be constructed in Tibet, China, combines the wide-angle advantages of traditional EAS array detectors with the high sensitivity advantages of focused Cherenkov detectors. Its physics objective is to observe transient sources such as gamma-ray bursts and counterparts of gravitational waves. The aim of this study is to utilize the latest AI technology to enhance the sensitivity of the HADAR experiment. We have built training datasets and models with distinctive creativity by incorporating relevant physical theories for various applications. They are able to determine the kind, energy, and direction of incident particles after careful design. We have obtained a background identification accuracy of 98.6%, a relative energy reconstruction error of 10.0%, and an angular resolution of 0.22-degrees in a test dataset at 10 TeV. These findings demonstrate the enormous potential for enhancing the precision and dependability of detector data analysis in astrophysical research. Thanks to deep learning techniques, the HADAR experiment's observational sensitivity to the Crab Nebula has surpassed that of MAGIC and H.E.S.S. at energies below 0.5 TeV and remains competitive with conventional narrow-field Cherenkov telescopes at higher energies. Additionally, our experiment offers a fresh approach to dealing with strongly connected scattered data.

astro-ph.IM↗
arXiv2023

Divide and Ensemble: Progressively Learning for the Unknown

In the wheat nutrient deficiencies classification challenge, we present the DividE and EnseMble (DEEM) method for progressive test data predictions. We find that (1) test images are provided in the challenge; (2) samples are equipped with their collection dates; (3) the samples of different dates show notable discrepancies. Based on the findings, we partition the dataset into discrete groups by the dates and train models on each divided group. We then adopt the pseudo-labeling approach to label the test data and incorporate those with high confidence into the training set. In pseudo-labeling, we leverage models ensemble with different architectures to enhance the reliability of predictions. The pseudo-labeling and ensembled model training are iteratively conducted until all test samples are labeled. Finally, the separated models for each group are unified to obtain the model for the whole dataset. Our method achieves an average of 93.6\% Top-1 test accuracy~(94.0\% on WW2020 and 93.2\% on WR2021) and wins the 1$st$ place in the Deep Nutrient Deficiency Challenge~\footnote{https://cvppa2023.github.io/challenges/}.

cs.AI↗
arXiv2023

Enhancing the Authenticity of Rendered Portraits with Identity-Consistent Transfer Learning

Despite rapid advances in computer graphics, creating high-quality photo-realistic virtual portraits is prohibitively expensive. Furthermore, the well-know ''uncanny valley'' effect in rendered portraits has a significant impact on the user experience, especially when the depiction closely resembles a human likeness, where any minor artifacts can evoke feelings of eeriness and repulsiveness. In this paper, we present a novel photo-realistic portrait generation framework that can effectively mitigate the ''uncanny valley'' effect and improve the overall authenticity of rendered portraits. Our key idea is to employ transfer learning to learn an identity-consistent mapping from the latent space of rendered portraits to that of real portraits. During the inference stage, the input portrait of an avatar can be directly transferred to a realistic portrait by changing its appearance style while maintaining the facial identity. To this end, we collect a new dataset, Daz-Rendered-Faces-HQ (DRFHQ), that is specifically designed for rendering-style portraits. We leverage this dataset to fine-tune the StyleGAN2 generator, using our carefully crafted framework, which helps to preserve the geometric and color features relevant to facial identity. We evaluate our framework using portraits with diverse gender, age, and race variations. Qualitative and quantitative evaluations and ablation studies show the advantages of our method compared to state-of-the-art approaches.

cs.CV↗
arXiv2023

Topological edge and corner states in Bi fractals on InSb

Topological materials hosting metallic edges characterized by integer quantized conductivity in an insulating bulk have revolutionized our understanding of transport in matter. The topological protection of these edge states is based on symmetries and dimensionality. However, only integer-dimensional models have been classified, and the interplay of topology and fractals, which may have a non-integer dimension, remained largely unexplored. Quantum fractals have recently been engineered in metamaterials, but up to present no topological states were unveiled in fractals realized in real materials. Here, we show theoretically and experimentally that topological edge and corner modes arise in fractals formed upon depositing thin layers of bismuth on an indium antimonide substrate. Scanning tunneling microscopy reveals the appearance of (nearly) zero-energy modes at the corners of Sierpiński triangles, as well as the formation of outer and inner edge modes at higher energies. Unexpectedly, a robust and sharp depleted mode appears at the outer and inner edges of the samples at negative bias voltages. The experimental findings are corroborated by theoretical calculations in the framework of a continuum muffin-tin and a lattice tight-binding model. The stability of the topological features to the introduction of a Rashba spin-orbit coupling and disorder is discussed. This work opens the perspective to novel electronics in real materials at non-integer dimensions with robust and protected topological states.

cond-mat.mes-hall↗
arXiv2023

A positivity-preserving implicit-explicit scheme with high order polynomial basis for compressible Navier-Stokes equations

In this paper, we are interested in constructing a scheme solving compressible Navier--Stokes equations, with desired properties including high order spatial accuracy, conservation, and positivity-preserving of density and internal energy under a standard hyperbolic type CFL constraint on the time step size, e.g., $Δt=\mathcal O(Δx)$. Strang splitting is used to approximate convection and diffusion operators separately. For the convection part, i.e., the compressible Euler equation, the high order accurate postivity-preserving Runge--Kutta discontinuous Galerkin method can be used. For the diffusion part, the equation of internal energy instead of the total energy is considered, and a first order semi-implicit time discretization is used for the ease of achieving positivity. A suitable interior penalty discontinuous Galerkin method for the stress tensor can ensure the conservation of momentum and total energy for any high order polynomial basis. In particular, positivity can be proven with $Δt=\mathcal{O}(Δx)$ if the Laplacian operator of internal energy is approximated by the $\mathbb{Q}^k$ spectral element method with $k=1,2,3$. So the full scheme with $\mathbb{Q}^k$ ($k=1,2,3$) basis is conservative and positivity-preserving with $Δt=\mathcal{O}(Δx)$, which is robust for demanding problems such as solutions with low density and low pressure induced by high-speed shock diffraction. Even though the full scheme is only first order accurate in time, numerical tests indicate that higher order polynomial basis produces much better numerical solutions, e.g., better resolution for capturing the roll-ups during shock reflection.

math.NA↗
arXiv2023

Lung Nodule Segmentation and Uncertain Region Prediction with an Uncertainty-Aware Attention Mechanism

Radiologists possess diverse training and clinical experiences, leading to variations in the segmentation annotations of lung nodules and resulting in segmentation uncertainty.Conventional methods typically select a single annotation as the learning target or attempt to learn a latent space comprising multiple annotations. However, these approaches fail to leverage the valuable information inherent in the consensus and disagreements among the multiple annotations. In this paper, we propose an Uncertainty-Aware Attention Mechanism (UAAM) that utilizes consensus and disagreements among multiple annotations to facilitate better segmentation. To this end, we introduce the Multi-Confidence Mask (MCM), which combines a Low-Confidence (LC) Mask and a High-Confidence (HC) Mask.The LC mask indicates regions with low segmentation confidence, where radiologists may have different segmentation choices. Following UAAM, we further design an Uncertainty-Guide Multi-Confidence Segmentation Network (UGMCS-Net), which contains three modules: a Feature Extracting Module that captures a general feature of a lung nodule, an Uncertainty-Aware Module that produces three features for the the annotations' union, intersection, and annotation set, and an Intersection-Union Constraining Module that uses distances between the three features to balance the predictions of final segmentation and MCM. To comprehensively demonstrate the performance of our method, we propose a Complex Nodule Validation on LIDC-IDRI, which tests UGMCS-Net's segmentation performance on lung nodules that are difficult to segment using common methods. Experimental results demonstrate that our method can significantly improve the segmentation performance on nodules that are difficult to segment using conventional methods.

eess.IV↗
arXiv2023

Strong magnon-magnon coupling in an ultralow damping all-magnetic-insulator heterostructure

Magnetic insulators such as yttrium iron garnets (YIGs) are of paramount importance for spin-wave or magnonic devices as their ultralow damping enables ultralow power dissipation that is free of Joule heating, exotic magnon quantum state, and coherent coupling to other wave excitations. Magnetic insulator heterostructures bestow superior structural and magnetic properties and house immense design space thanks to the strong and engineerable exchange interaction between individual layers. To fully unleash their potential, realizing low damping and strong exchange coupling simultaneously is critical, which often requires high quality interface. Here, we show that such a demand is realized in an all-insulator thulium iron garnet (TmIG)/YIG bilayer system. The ultralow dissipation rates in both YIG and TmIG, along with their significant spin-spin interaction at the interface, enable strong and coherent magnon-magnon coupling with a benchmarking cooperativity value larger than the conventional ferromagnetic metal-based heterostructures. The coupling strength can be tuned by varying the magnetic insulator layer thickness and magnon modes, which is consistent with analytical calculations and micromagnetic simulations. Our results demonstrate TmIG/YIG as a novel platform for investigating hybrid magnonic phenomena and open opportunities in magnon devices comprising all-insulator heterostructures.

cond-mat.mtrl-sci↗
arXiv2023

Optimal Sample Selection Through Uncertainty Estimation and Its Application in Deep Learning

Modern deep learning heavily relies on large labeled datasets, which often comse with high costs in terms of both manual labeling and computational resources. To mitigate these challenges, researchers have explored the use of informative subset selection techniques, including coreset selection and active learning. Specifically, coreset selection involves sampling data with both input ($\bx$) and output ($\by$), active learning focuses solely on the input data ($\bx$). In this study, we present a theoretically optimal solution for addressing both coreset selection and active learning within the context of linear softmax regression. Our proposed method, COPS (unCertainty based OPtimal Sub-sampling), is designed to minimize the expected loss of a model trained on subsampled data. Unlike existing approaches that rely on explicit calculations of the inverse covariance matrix, which are not easily applicable to deep learning scenarios, COPS leverages the model's logits to estimate the sampling ratio. This sampling ratio is closely associated with model uncertainty and can be effectively applied to deep learning tasks. Furthermore, we address the challenge of model sensitivity to misspecification by incorporating a down-weighting approach for low-density samples, drawing inspiration from previous works. To assess the effectiveness of our proposed method, we conducted extensive empirical experiments using deep neural networks on benchmark datasets. The results consistently showcase the superior performance of COPS compared to baseline methods, reaffirming its efficacy.

stat.ML↗
arXiv2023

When 3D Bounding-Box Meets SAM: Point Cloud Instance Segmentation with Weak-and-Noisy Supervision

Learning from bounding-boxes annotations has shown great potential in weakly-supervised 3D point cloud instance segmentation. However, we observed that existing methods would suffer severe performance degradation with perturbed bounding box annotations. To tackle this issue, we propose a complementary image prompt-induced weakly-supervised point cloud instance segmentation (CIP-WPIS) method. CIP-WPIS leverages pretrained knowledge embedded in the 2D foundation model SAM and 3D geometric prior to achieve accurate point-wise instance labels from the bounding box annotations. Specifically, CP-WPIS first selects image views in which 3D candidate points of an instance are fully visible. Then, we generate complementary background and foreground prompts from projections to obtain SAM 2D instance mask predictions. According to these, we assign the confidence values to points indicating the likelihood of points belonging to the instance. Furthermore, we utilize 3D geometric homogeneity provided by superpoints to decide the final instance label assignments. In this fashion, we achieve high-quality 3D point-wise instance labels. Extensive experiments on both Scannet-v2 and S3DIS benchmarks demonstrate that our method is robust against noisy 3D bounding-box annotations and achieves state-of-the-art performance.

cs.CV↗
arXiv2023

Asymmetric Co-Training with Explainable Cell Graph Ensembling for Histopathological Image Classification

Convolutional neural networks excel in histopathological image classification, yet their pixel-level focus hampers explainability. Conversely, emerging graph convolutional networks spotlight cell-level features and medical implications. However, limited by their shallowness and suboptimal use of high-dimensional pixel data, GCNs underperform in multi-class histopathological image classification. To make full use of pixel-level and cell-level features dynamically, we propose an asymmetric co-training framework combining a deep graph convolutional network and a convolutional neural network for multi-class histopathological image classification. To improve the explainability of the entire framework by embedding morphological and topological distribution of cells, we build a 14-layer deep graph convolutional network to handle cell graph data. For the further utilization and dynamic interactions between pixel-level and cell-level information, we also design a co-training strategy to integrate the two asymmetric branches. Notably, we collect a private clinically acquired dataset termed LUAD7C, including seven subtypes of lung adenocarcinoma, which is rare and more challenging. We evaluated our approach on the private LUAD7C and public colorectal cancer datasets, showcasing its superior performance, explainability, and generalizability in multi-class histopathological image classification.

cs.CV↗
arXiv2023

Ionically-Driven Synthesis and Exchange Bias in Mn$_{4}$N/MnN$_{x}$ Heterostructures

Ferrimagnets have received renewed attention as a promising platform for spintronic applications. Of particular interest is the Mn4N from the $ε$-phase of the manganese nitride as an emergent rare-earth-free spintronic material due to its perpendicular magnetic anisotropy, small saturation magnetization, high thermal stability, and large domain wall velocity. We have achieved high-quality (001)-ordered Mn$_{4}$N thin film by sputtering Mn onto $η$-phase Mn$_{3}$N$_{2}$ seed layers on Si substrates. As the deposited Mn thickness varies, nitrogen ion migration across the Mn$_{3}$N$_{2}$/Mn layers leads to a continuous evolution of the layers to Mn$_{3}$N$_{2}$/Mn$_{2}$N/Mn$_{4}$N, Mn$_{2}$N/Mn$_{4}$N, and eventually Mn$_{4}$N alone. The ferrimagnetic Mn$_{4}$N indeed exhibits perpendicular magnetic anisotropy, and forms via a nucleation-and-growth mechanism. The nitrogen ion migration is also manifested in a significant exchange bias, up to 0.3 T at 5 K, due to the interactions between ferrimagnetic Mn$_{4}$N and antiferromagnetic Mn$_{3}$N$_{2}$ and Mn$_{2}$N. These results demonstrate a promising all-nitride magneto-ionic platform with remarkable tunability for device applications.

cond-mat.mtrl-sci↗
arXiv2023

BAVS: Bootstrapping Audio-Visual Segmentation by Integrating Foundation Knowledge

Given an audio-visual pair, audio-visual segmentation (AVS) aims to locate sounding sources by predicting pixel-wise maps. Previous methods assume that each sound component in an audio signal always has a visual counterpart in the image. However, this assumption overlooks that off-screen sounds and background noise often contaminate the audio recordings in real-world scenarios. They impose significant challenges on building a consistent semantic mapping between audio and visual signals for AVS models and thus impede precise sound localization. In this work, we propose a two-stage bootstrapping audio-visual segmentation framework by incorporating multi-modal foundation knowledge. In a nutshell, our BAVS is designed to eliminate the interference of background noise or off-screen sounds in segmentation by establishing the audio-visual correspondences in an explicit manner. In the first stage, we employ a segmentation model to localize potential sounding objects from visual data without being affected by contaminated audio signals. Meanwhile, we also utilize a foundation audio classification model to discern audio semantics. Considering the audio tags provided by the audio foundation model are noisy, associating object masks with audio tags is not trivial. Thus, in the second stage, we develop an audio-visual semantic integration strategy (AVIS) to localize the authentic-sounding objects. Here, we construct an audio-visual tree based on the hierarchical correspondence between sounds and object categories. We then examine the label concurrency between the localized objects and classified audio tags by tracing the audio-visual tree. With AVIS, we can effectively segment real-sounding objects. Extensive experiments demonstrate the superiority of our method on AVS datasets, particularly in scenarios involving background noise. Our project website is https://yenanliu.github.io/AVSS.github.io/.

cs.CV↗
arXiv2023

Audio-Visual Segmentation by Exploring Cross-Modal Mutual Semantics

The audio-visual segmentation (AVS) task aims to segment sounding objects from a given video. Existing works mainly focus on fusing audio and visual features of a given video to achieve sounding object masks. However, we observed that prior arts are prone to segment a certain salient object in a video regardless of the audio information. This is because sounding objects are often the most salient ones in the AVS dataset. Thus, current AVS methods might fail to localize genuine sounding objects due to the dataset bias. In this work, we present an audio-visual instance-aware segmentation approach to overcome the dataset bias. In a nutshell, our method first localizes potential sounding objects in a video by an object segmentation network, and then associates the sounding object candidates with the given audio. We notice that an object could be a sounding object in one video but a silent one in another video. This would bring ambiguity in training our object segmentation network as only sounding objects have corresponding segmentation masks. We thus propose a silent object-aware segmentation objective to alleviate the ambiguity. Moreover, since the category information of audio is unknown, especially for multiple sounding sources, we propose to explore the audio-visual semantic correlation and then associate audio with potential objects. Specifically, we attend predicted audio category scores to potential instance masks and these scores will highlight corresponding sounding instances while suppressing inaudible ones. When we enforce the attended instance masks to resemble the ground-truth mask, we are able to establish audio-visual semantics correlation. Experimental results on the AVS benchmarks demonstrate that our method can effectively segment sounding objects without being biased to salient objects.

cs.SD↗
arXiv2023

Delving Deeper into Cross-lingual Visual Question Answering

Visual question answering (VQA) is one of the crucial vision-and-language tasks. Yet, existing VQA research has mostly focused on the English language, due to a lack of suitable evaluation resources. Previous work on cross-lingual VQA has reported poor zero-shot transfer performance of current multilingual multimodal Transformers with large gaps to monolingual performance, without any deeper analysis. In this work, we delve deeper into the different aspects of cross-lingual VQA, aiming to understand the impact of 1) modeling methods and choices, including architecture, inductive bias, fine-tuning; 2) learning biases: including question types and modality biases in cross-lingual setups. The key results of our analysis are: 1) We show that simple modifications to the standard training setup can substantially reduce the transfer gap to monolingual English performance, yielding +10 accuracy points over existing methods. 2) We analyze cross-lingual VQA across different question types of varying complexity for different multilingual multimodal Transformers, and identify question types that are the most difficult to improve on. 3) We provide an analysis of modality biases present in training data and models, revealing why zero-shot performance gaps remain for certain question types and languages.

cs.CL↗
arXiv2023

Frequency Throttling Side-Channel Attack

Modern processors dynamically control their operating frequency to optimize resource utilization, maximize energy savings, and conform to system-defined constraints. If, during the execution of a software workload, the running average of any electrical or thermal parameter exceeds its corresponding predefined threshold value, the power management architecture will reactively adjust CPU frequency to ensure safe operating conditions. In this paper, we demonstrate how such power management-based frequency throttling activity forms a source of timing side-channel information leakage, which can be exploited by an attacker to infer secret data even from a constant-cycle victim workload. The proposed frequency throttling side-channel attack can be launched by both kernel-space and user-space attackers, thus compromising security guarantees provided by isolation boundaries. We validate our attack methodology across different systems and threat models by performing experiments on a constant-cycle implementation of AES algorithm based on AES-NI instructions. The results of our experimental evaluations demonstrate that the attacker can successfully recover all bytes of an AES key by measuring encryption execution times. Finally, we discuss different options to mitigate the threat posed by frequency throttling side-channel attacks, as well as their advantages and disadvantages.

cs.CR↗