arXiv subjects

Hong Liu

Publications and source records attributed to Hong Liu.

At least 325 records · Page 18Linked to original sources

arXiv2022

Simultaneous Alignment and Surface Regression Using Hybrid 2D-3D Networks for 3D Coherent Layer Segmentation of Retina OCT Images

Automated surface segmentation of retinal layer is important and challenging in analyzing optical coherence tomography (OCT). Recently, many deep learning based methods have been developed for this task and yield remarkable performance. However, due to large spatial gap and potential mismatch between the B-scans of OCT data, all of them are based on 2D segmentation of individual B-scans, which may loss the continuity information across the B-scans. In addition, 3D surface of the retina layers can provide more diagnostic information, which is crucial in quantitative image analysis. In this study, a novel framework based on hybrid 2D-3D convolutional neural networks (CNNs) is proposed to obtain continuous 3D retinal layer surfaces from OCT. The 2D features of individual B-scans are extracted by an encoder consisting of 2D convolutions. These 2D features are then used to produce the alignment displacement field and layer segmentation by two 3D decoders, which are coupled via a spatial transformer module. The entire framework is trained end-to-end. To the best of our knowledge, this is the first study that attempts 3D retinal layer segmentation in volumetric OCT images based on CNNs. Experiments on a publicly available dataset show that our framework achieves superior results to state-of-the-art 2D methods in terms of both layer segmentation accuracy and cross-B-scan 3D continuity, thus offering more clinical values than previous works.

eess.IV↗

arXiv2022

Learning Deep Direct-Path Relative Transfer Function for Binaural Sound Source Localization

Direct-path relative transfer function (DP-RTF) refers to the ratio between the direct-path acoustic transfer functions of two microphone channels. Though DP-RTF fully encodes the sound spatial cues and serves as a reliable localization feature, it is often erroneously estimated in the presence of noise and reverberation. This paper proposes to learn DP-RTF with deep neural networks for robust binaural sound source localization. A DP-RTF learning network is designed to regress the binaural sensor signals to a real-valued representation of DP-RTF. It consists of a branched convolutional neural network module to separately extract the inter-channel magnitude and phase patterns, and a convolutional recurrent neural network module for joint feature learning. To better explore the speech spectra to aid the DP-RTF estimation, a monaural speech enhancement network is used to recover the direct-path spectrograms from the noisy ones. The enhanced spectrograms are stacked onto the noisy spectrograms to act as the input of the DP-RTF learning network. We train one unique DP-RTF learning network using many different binaural arrays to enable the generalization of DP-RTF learning across arrays. This way avoids time-consuming training data collection and network retraining for a new array, which is very useful in practical application. Experimental results on both simulated and real-world data show the effectiveness of the proposed method for direction of arrival (DOA) estimation in the noisy and reverberant environment, and a good generalization ability to unseen binaural arrays.

cs.SD↗

arXiv2022

SRP-DNN: Learning Direct-Path Phase Difference for Multiple Moving Sound Source Localization

Multiple moving sound source localization in real-world scenarios remains a challenging issue due to interaction between sources, time-varying trajectories, distorted spatial cues, etc. In this work, we propose to use deep learning techniques to learn competing and time-varying direct-path phase differences for localizing multiple moving sound sources. A causal convolutional recurrent neural network is designed to extract the direct-path phase difference sequence from signals of each microphone pair. To avoid the assignment ambiguity and the problem of uncertain output-dimension encountered when simultaneously predicting multiple targets, the learning target is designed in a weighted sum format, which encodes source activity in the weight and direct-path phase differences in the summed value. The learned direct-path phase differences for all microphone pairs can be directly used to construct the spatial spectrum according to the formulation of steered response power (SRP). This deep neural network (DNN) based SRP method is referred to as SRP-DNN. The locations of sources are estimated by iteratively detecting and removing the dominant source from the spatial spectrum, in which way the interaction between sources is reduced. Experimental results on both simulated and real-world data show the superiority of the proposed method in the presence of noise and reverberation.

cs.SD↗

arXiv2022

Virtual Adversarial Training for Semi-supervised Breast Mass Classification

This study aims to develop a novel computer-aided diagnosis (CAD) scheme for mammographic breast mass classification using semi-supervised learning. Although supervised deep learning has achieved huge success across various medical image analysis tasks, its success relies on large amounts of high-quality annotations, which can be challenging to acquire in practice. To overcome this limitation, we propose employing a semi-supervised method, i.e., virtual adversarial training (VAT), to leverage and learn useful information underlying in unlabeled data for better classification of breast masses. Accordingly, our VAT-based models have two types of losses, namely supervised and virtual adversarial losses. The former loss acts as in supervised classification, while the latter loss aims at enhancing model robustness against virtual adversarial perturbation, thus improving model generalizability. To evaluate the performance of our VAT-based CAD scheme, we retrospectively assembled a total of 1024 breast mass images, with equal number of benign and malignant masses. A large CNN and a small CNN were used in this investigation, and both were trained with and without the adversarial loss. When the labeled ratios were 40% and 80%, VAT-based CNNs delivered the highest classification accuracy of 0.740 and 0.760, respectively. The experimental results suggest that the VAT-based CAD scheme can effectively utilize meaningful knowledge from unlabeled data to better classify mammographic breast mass images.

cs.CV↗

arXiv2022

High-order Photonic Cavity Modes Enabled 3D Structural Color

It remains a challenge to directly print three-dimensional arbitrary shapes that exhibit structural colors at the micrometer scale. Woodpile photonic crystals (WPCs) fabricated via two-photon lithography (TPL) are promising as building blocks to produce 3D geometries that generate structural colors due to their ability to exhibit either omnidirectional or anisotropic photonic stopbands. However, existing approaches have focused on achieving structural colors when illuminating WPCs from the top, which necessitates print resolutions beyond the limit of commercial TPL and/or post-processing techniques. Here, we devised a new strategy to support high-order photonic cavity modes upon side-illumination on WPCs that surprisingly generate large reflectance peaks in the visible spectrum. Based on that, we demonstrate one-step printing of 3D photonic structural colors without requiring post-processing or subwavelength features. Vivid colors with reflectance peaks exhibiting a full width at half maximum of ~25 nm, a maximum reflectance of 50%, gamut of ~85% of sRGB, and large viewing angles, were achieved. In addition, we also demonstrated voxel-level manipulation and control of colors in arbitrary-shaped 3D objects constituted with WPCs as unit cells, which has great potential for applications in dynamic color displays, colorimetric sensing, anti-counterfeiting, and light-matter interaction platforms.

physics.optics↗

arXiv2022

Recent Advances in Tunable Metasurfaces: Materials, Design and Applications

Metasurfaces, a two-dimensional (2D) form of metamaterials constituted by planar meta-atoms, exhibit exotic abilities to freely tailor electromagnetic (EM) waves. Over the past decade, tunable metasurfaces have come to the frontier in the field of nanophotonics, with tremendous effort focused on developing and integrating various active materials into metasurfaces. As a result, tunable/reconfigurable metasurfaces with multi-functionalities triggered by various external stimuli have been successfully demonstrated, openings a new avenue to dynamically manipulate and control EM waves for photonic applications in demand. In this review, we first brief the progress of tunable metasurfaces development in the last decade and highlight representative works from the perspectives of active materials development, design methodologies and application-driven exploration. Then, we elaborate on the active tuning mechanisms and relevant active materials. Next, we discuss recent achievements in theory as well as machine learning (ML) assisted design methodologies to sustain the development of this field. After that, we summarize and describe typical application areas of the tunable metasurfaces. We conclude this review by analyzing existing challenges and presenting our perspectives on future directions and opportunities in this vibrant and fast-developing field.

physics.optics↗

arXiv2022

Exploiting Temporal Contexts with Strided Transformer for 3D Human Pose Estimation

Despite the great progress in 3D human pose estimation from videos, it is still an open problem to take full advantage of a redundant 2D pose sequence to learn representative representations for generating one 3D pose. To this end, we propose an improved Transformer-based architecture, called Strided Transformer, which simply and effectively lifts a long sequence of 2D joint locations to a single 3D pose. Specifically, a Vanilla Transformer Encoder (VTE) is adopted to model long-range dependencies of 2D pose sequences. To reduce the redundancy of the sequence, fully-connected layers in the feed-forward network of VTE are replaced with strided convolutions to progressively shrink the sequence length and aggregate information from local contexts. The modified VTE is termed as Strided Transformer Encoder (STE), which is built upon the outputs of VTE. STE not only effectively aggregates long-range information to a single-vector representation in a hierarchical global and local fashion, but also significantly reduces the computation cost. Furthermore, a full-to-single supervision scheme is designed at both full sequence and single target frame scales applied to the outputs of VTE and STE, respectively. This scheme imposes extra temporal smoothness constraints in conjunction with the single target frame supervision and hence helps produce smoother and more accurate 3D poses. The proposed Strided Transformer is evaluated on two challenging benchmark datasets, Human3.6M and HumanEva-I, and achieves state-of-the-art results with fewer parameters. Code and models are available at \url{https://github.com/Vegetebird/StridedTransformer-Pose3D}.

cs.CV↗

arXiv2022

Exponential decay of intersection volume with applications on list-decodability and Gilbert-Varshamov type bound

We give some natural sufficient conditions for balls in a metric space to have small intersection. Roughly speaking, this happens when the metric space is (i) expanding and (ii) well-spread, and (iii) a certain random variable on the boundary of a ball has a small tail. As applications, we show that the volume of intersection of balls in Hamming, Johnson spaces and symmetric groups decay exponentially as their centers drift apart. To verify condition (iii), we prove some large deviation inequalities `on a slice' for functions with Lipschitz conditions. We then use these estimates on intersection volumes to $\bullet$ obtain a sharp lower bound on list-decodability of random $q$-ary codes, confirming a conjecture of Li and Wootters; and $\bullet$ improve the classical bound of Levenshtein from 1971 on constant weight codes by a factor linear in dimension, resolving a problem raised by Jiang and Vardy. Our probabilistic point of view also offers a unified framework to obtain improvements on other Gilbert--Varshamov type bounds, giving conceptually simple and calculation-free proofs for $q$-ary codes, permutation codes, and spherical codes. Another consequence is a counting result on the number of codes, showing ampleness of large codes.

math.CO↗

arXiv2021

Multi-Modal Perception Attention Network with Self-Supervised Learning for Audio-Visual Speaker Tracking

Multi-modal fusion is proven to be an effective method to improve the accuracy and robustness of speaker tracking, especially in complex scenarios. However, how to combine the heterogeneous information and exploit the complementarity of multi-modal signals remains a challenging issue. In this paper, we propose a novel Multi-modal Perception Tracker (MPT) for speaker tracking using both audio and visual modalities. Specifically, a novel acoustic map based on spatial-temporal Global Coherence Field (stGCF) is first constructed for heterogeneous signal fusion, which employs a camera model to map audio cues to the localization space consistent with the visual cues. Then a multi-modal perception attention network is introduced to derive the perception weights that measure the reliability and effectiveness of intermittent audio and video streams disturbed by noise. Moreover, a unique cross-modal self-supervised learning method is presented to model the confidence of audio and visual observations by leveraging the complementarity and consistency between different modalities. Experimental results show that the proposed MPT achieves 98.6% and 78.3% tracking accuracy on the standard and occluded datasets, respectively, which demonstrates its robustness under adverse conditions and outperforms the current state-of-the-art methods.

cs.CV↗

arXiv2021

Pose-guided Feature Disentangling for Occluded Person Re-identification Based on Transformer

Occluded person re-identification is a challenging task as human body parts could be occluded by some obstacles (e.g. trees, cars, and pedestrians) in certain scenes. Some existing pose-guided methods solve this problem by aligning body parts according to graph matching, but these graph-based methods are not intuitive and complicated. Therefore, we propose a transformer-based Pose-guided Feature Disentangling (PFD) method by utilizing pose information to clearly disentangle semantic components (e.g. human body or joint parts) and selectively match non-occluded parts correspondingly. First, Vision Transformer (ViT) is used to extract the patch features with its strong capability. Second, to preliminarily disentangle the pose information from patch information, the matching and distributing mechanism is leveraged in Pose-guided Feature Aggregation (PFA) module. Third, a set of learnable semantic views are introduced in transformer decoder to implicitly enhance the disentangled body part features. However, those semantic views are not guaranteed to be related to the body without additional supervision. Therefore, Pose-View Matching (PVM) module is proposed to explicitly match visible body parts and automatically separate occlusion features. Fourth, to better prevent the interference of occlusions, we design a Pose-guided Push Loss to emphasize the features of visible body parts. Extensive experiments over five challenging datasets for two tasks (occluded and holistic Re-ID) demonstrate that our proposed PFD is superior promising, which performs favorably against state-of-the-art methods. Code is available at https://github.com/WangTaoAs/PFD_Net

cs.CV↗

arXiv2021

Contrastive Learning from Extremely Augmented Skeleton Sequences for Self-supervised Action Recognition

In recent years, self-supervised representation learning for skeleton-based action recognition has been developed with the advance of contrastive learning methods. The existing contrastive learning methods use normal augmentations to construct similar positive samples, which limits the ability to explore novel movement patterns. In this paper, to make better use of the movement patterns introduced by extreme augmentations, a Contrastive Learning framework utilizing Abundant Information Mining for self-supervised action Representation (AimCLR) is proposed. First, the extreme augmentations and the Energy-based Attention-guided Drop Module (EADM) are proposed to obtain diverse positive samples, which bring novel movement patterns to improve the universality of the learned representations. Second, since directly using extreme augmentations may not be able to boost the performance due to the drastic changes in original identity, the Dual Distributional Divergence Minimization Loss (D$^3$M Loss) is proposed to minimize the distribution divergence in a more gentle way. Third, the Nearest Neighbors Mining (NNM) is proposed to further expand positive samples to make the abundant information mining process more reasonable. Exhaustive experiments on NTU RGB+D 60, PKU-MMD, NTU RGB+D 120 datasets have verified that our AimCLR can significantly perform favorably against state-of-the-art methods under a variety of evaluation protocols with observed higher quality action representations. Our code is available at https://github.com/Levigty/AimCLR.

cs.CV↗

arXiv2021

Mixed-state entanglement and information recovery in thermalized states and evaporating black holes

We study the universal behavior of quantum information-theoretic quantities in thermalized isolated quantum many-body systems and evaporating black holes. In particular, we study a genuine mixed-state entanglement measure called the logarithmic negativity, other correlation measures including the Renyi negativities and the mutual information, and a signature of multipartite entanglement called the reflected entropy. We also probe the feasibility of recovering quantum information from subsystems of a thermalized quantum many-body system or from the radiation of an evaporating black hole, using quantities such as relative entropy and Petz map fidelity. A recently developed technique called the equilibrium approximation allows us to probe these quantities at finite temperature. We find striking qualitative differences from the infinite temperature case, which has been the topic of previous studies using Haar-random states. In particular, we find regimes where the logarithmic negativity is extensive but the mutual information is sub-extensive, indicating a large amount of undistillable, bound entanglement in thermalized states. For evaporating black holes at finite temperature, both the logarithmic negativity and the Petz map fidelity reveal an important new time scale $t_b$, which is earlier than the Page time $t_p$ by a finite fraction of the total evaporation time. We find that $t_b$, as opposed to $t_p$, is the time scale at which quantum entanglement between different parts of the radiation becomes extensive, and the fidelity of information recovery for a large diary thrown into the black hole starts to grow.

hep-th↗

arXiv2021

Interest-oriented Universal User Representation via Contrastive Learning

User representation is essential for providing high-quality commercial services in industry. Universal user representation has received many interests recently, with which we can be free from the cumbersome work of training a specific model for each downstream application. In this paper, we attempt to improve universal user representation from two points of views. First, a contrastive self-supervised learning paradigm is presented to guide the representation model training. It provides a unified framework that allows for long-term or short-term interest representation learning in a data-driven manner. Moreover, a novel multi-interest extraction module is presented. The module introduces an interest dictionary to capture principal interests of the given user, and then generate his/her interest-oriented representations via behavior aggregation. Experimental results demonstrate the effectiveness and applicability of the learned user representations.

cs.LG↗

arXiv2021

Sound Event Detection Transformer: An Event-based End-to-End Model for Sound Event Detection

Sound event detection (SED) has gained increasing attention with its wide application in surveillance, video indexing, etc. Existing models in SED mainly generate frame-level prediction, converting it into a sequence multi-label classification problem. A critical issue with the frame-based model is that it pursues the best frame-level prediction rather than the best event-level prediction. Besides, it needs post-processing and cannot be trained in an end-to-end way. This paper firstly presents the one-dimensional Detection Transformer (1D-DETR), inspired by Detection Transformer for image object detection. Furthermore, given the characteristics of SED, the audio query branch and a one-to-many matching strategy for fine-tuning the model are added to 1D-DETR to form Sound Event Detection Transformer (SEDT). To our knowledge, SEDT is the first event-based and end-to-end SED model. Experiments are conducted on the URBAN-SED dataset and the DCASE2019 Task4 dataset, and both show that SEDT can achieve competitive performance.

cs.SD↗

arXiv2021

Improving Camouflaged Object Detection with the Uncertainty of Pseudo-edge Labels

This paper focuses on camouflaged object detection (COD), which is a task to detect objects hidden in the background. Most of the current COD models aim to highlight the target object directly while outputting ambiguous camouflaged boundaries. On the other hand, the performance of the models considering edge information is not yet satisfactory. To this end, we propose a new framework that makes full use of multiple visual cues, i.e., saliency as well as edges, to refine the predicted camouflaged map. This framework consists of three key components, i.e., a pseudo-edge generator, a pseudo-map generator, and an uncertainty-aware refinement module. In particular, the pseudo-edge generator estimates the boundary that outputs the pseudo-edge label, and the conventional COD method serves as the pseudo-map generator that outputs the pseudo-map label. Then, we propose an uncertainty-based module to reduce the uncertainty and noise of such two pseudo labels, which takes both pseudo labels as input and outputs an edge-accurate camouflaged map. Experiments on various COD datasets demonstrate the effectiveness of our method with superior performance to the existing state-of-the-art methods.

cs.CV↗

arXiv2021

Cycle Self-Training for Domain Adaptation

Mainstream approaches for unsupervised domain adaptation (UDA) learn domain-invariant representations to narrow the domain shift. Recently, self-training has been gaining momentum in UDA, which exploits unlabeled target data by training with target pseudo-labels. However, as corroborated in this work, under distributional shift in UDA, the pseudo-labels can be unreliable in terms of their large discrepancy from target ground truth. Thereby, we propose Cycle Self-Training (CST), a principled self-training algorithm that explicitly enforces pseudo-labels to generalize across domains. CST cycles between a forward step and a reverse step until convergence. In the forward step, CST generates target pseudo-labels with a source-trained classifier. In the reverse step, CST trains a target classifier using target pseudo-labels, and then updates the shared representations to make the target classifier perform well on the source data. We introduce the Tsallis entropy as a confidence-friendly regularization to improve the quality of target pseudo-labels. We analyze CST theoretically under realistic assumptions, and provide hard cases where CST recovers target ground truth, while both invariant feature learning and vanilla self-training fail. Empirical results indicate that CST significantly improves over the state-of-the-arts on visual recognition and sentiment analysis benchmarks.

cs.LG↗

arXiv2021

On mixing enhancement by secondary baroclinic vorticity in shock-bubble interaction

To investigate the intrinsic mechanism for mixing enhancement by variable density behaviour, a canonical variable density (VD) mixing extracted from a supersonic streamwise vortex protocol, shock bubble interaction (SBI), is numerically studied and compared with a counterpart of passive scalar (PS) mixing. It is meaningful to observe that the maximum concentration decays much faster in VD SBI than in PS SBI regardless of the shock Mach number (Ma = 1.22 ~ 4). The quasi-Lamb-Oseen type velocity distribution in the PS SBI is found by analyzing the azimuthal velocity that stretches the bubble. Meanwhile, for the VD SBI, an additional stretching enhanced by the secondary baroclinic vorticity (SBV) production contributes to the faster-mixing decay. The underlying mechanism of the SBV-enhanced stretching is further revealed through the density and velocity difference between the shocked light bubble and the heavy ambient air. By combining the SBV-accelerated stretching model and the initial shock compression, a novel mixing time estimation for VD SBI is theoretically proposed by solving the advection-diffusion equation under a deformation field of an axisymmetric vortex with the additional SBV induced azimuthal velocity. Based on the mixing time model, a mixing enhancement number defined by the ratio of VD and PS mixing time further reveals the contribution from the variable-density effect, which implies a better control of density distribution for mixing enhancement in a supersonic streamwise vortex.

physics.flu-dyn↗

arXiv2021

Hybrid dark excitons in monolayer $\hbox{MoS}_2$

Transition metal dichalcogenides have a rich exciton landscape consisting of a variety of bright and dark excitonic states. We consider the lowest-energy dark states in $\hbox{MoS}_2$, which can be referred to as hybrid excitons, as they are formed by a Dirac electron and a Schrödinger hole. The chiral nature of the Dirac electron introduces asymmetry to the excited exciton state spectrum and entangles the relative motion of the electron and hole with the center-of-mass motion. We demonstrate that the entanglement generates an additional contribution to the Berry curvature of hybrid excitons. The additional contribution is substrate-dependent and accounts for almost one quarter of the Berry curvature in suspended samples. The nontrivial geometry of hybrid excitons is manifested in the optical anomalous valley Hall effect, which can be observed via recently developed pump-probe photoemission spectroscopy. We argue that the Hall angle of hybrid excitons is approximately one half of that for bright excitons. Moreover, the anticipated long lifetime of hybrid excitons favors an extended propagation distance and allows the spatial separation of hybrid excitons with different valley indices.

cond-mat.mes-hall↗