Searcharxiv⌕ Search

arXiv subjects

Liang Liu

Publications and source records attributed to Liang Liu.

At least 163 records · Page 9Linked to original sources
arXiv2023

Rethinking Mobile Block for Efficient Attention-based Models

This paper focuses on developing modern, efficient, lightweight models for dense predictions while trading off parameters, FLOPs, and performance. Inverted Residual Block (IRB) serves as the infrastructure for lightweight CNNs, but no counterpart has been recognized by attention-based studies. This work rethinks lightweight infrastructure from efficient IRB and effective components of Transformer from a unified perspective, extending CNN-based IRB to attention-based models and abstracting a one-residual Meta Mobile Block (MMB) for lightweight model design. Following simple but effective design criterion, we deduce a modern Inverted Residual Mobile Block (iRMB) and build a ResNet-like Efficient MOdel (EMO) with only iRMB for down-stream tasks. Extensive experiments on ImageNet-1K, COCO2017, and ADE20K benchmarks demonstrate the superiority of our EMO over state-of-the-art methods, e.g., EMO-1M/2M/5M achieve 71.5, 75.1, and 78.4 Top-1 that surpass equal-order CNN-/Attention-based models, while trading-off the parameter, efficiency, and accuracy well: running 2.8-4.0x faster than EdgeNeXt on iPhone14.

cs.CV↗
arXiv2023

Metallic states in Pb$_{10}$(PO$_4$)$_6$O induced by the Cu/O-insertions and carrier doping

The reports of room-temperature superconductivity in Cu-doped Pb-apatite draw intense interests and debates. Herein, based on the density functional theory, we show that the Cu-insertion in Pb-apatite is thermodynamically stable and further carrier doping can convert the system CuPb$_{10}$(PO$_4$)$_6$O into metal. The electric conduction is mainly along the one-dimensional (1D) Cu-O chains in the c-axis (out-of-plane). The calculated conductance along the c-axis is larger than the in-plane ones by 2 magnitude orders, indicating the 1D conduction behavior. Moreover, the 1D Cu-O chain is an anti-ferromagnetic Mott insulator at zero-doping point due to the super-exchange. Further electron/hole-doping will erase the anti-ferromagnetism. Therefore, the Cu-inserted system CuPb$_{10}$(PO$_4$)$_6$O show transport and magnetic features similar to the cuprate superconductors. On the other hand, the O-insertion can also induce the metallic states, in which the conductance along the out-of-plane direction is higher than the in-plane direction by 6-folds. Our results display the metallization of Pb$_{10}$(PO$_4$)$_6$O via Cu/O-insertions, and suggesting the electric conductions along the c-axis might dominate the transport behaviors.

cond-mat.supr-con↗
arXiv2023

Fundamental CRB-Rate Tradeoff in Multi-Antenna ISAC Systems with Information Multicasting and Multi-Target Sensing

This paper investigates the performance tradeoff for a multi-antenna integrated sensing and communication (ISAC) system with simultaneous information multicasting and multi-target sensing, in which a multi-antenna base station (BS) sends the common information messages to a set of single-antenna communication users (CUs) and estimates the parameters of multiple sensing targets based on the echo signals concurrently. We consider two target sensing scenarios without and with prior target knowledge at the BS, in which the BS is interested in estimating the complete multi-target response matrix and the target reflection coefficients/angles, respectively. First, we consider the capacity-achieving transmission and characterize the fundamental tradeoff between the achievable rate and the multi-target estimation Cramér-Rao bound (CRB) accordingly.

cs.IT↗
arXiv2023

Generation and modulation of multiple 2D bulk photovoltaic effects in space-time reversal asymmetric 2H-FeCl2

The two-dimensional (2D) bulk photovoltaic effect (BPVE) is a cornerstone for future highly efficient 2D solar cells and optoelectronics. The ferromagnetic semiconductor 2H-FeCl2 is shown to realize a new type of BPVE in which spatial inversion (P), time reversal (T), and space-time reversal (PT) symmetries are broken (PT-broken). Using density functional theory and perturbation theory, we show that 2H-FeCl2 exhibits giant photocurrents, photo-spin-currents, and photo-orbital-currents under illumination by linearly polarized light. The injection-like and shift-like photocurrents coexist and propagate in different directions. The material also demonstrates substantial photoconductance, photo-spin-conductance, and photo-orbital-conductance, with magnitudes up to 4650 (nm{\cdot}μA/V2), 4620 (nm{\cdot}μA/V2 {\hbar}/2e), and 6450 (nm{\cdot}μA/V2 {\hbar}/e), respectively. Furthermore, the injection-currents, shift-spin-currents, and shift-orbital-currents can be readily switched via rotating the magnetizations of 2H-FeCl2. These results demonstrate the superior performance and intriguing control of a new type of BPVE in 2H-FeCl2.

cond-mat.mtrl-sci↗
arXiv2023

A Mask Free Neural Network for Monaural Speech Enhancement

In speech enhancement, the lack of clear structural characteristics in the target speech phase requires the use of conservative and cumbersome network frameworks. It seems difficult to achieve competitive performance using direct methods and simple network architectures. However, we propose the MFNet, a direct and simple network that can not only map speech but also map reverse noise. This network is constructed by stacking global local former blocks (GLFBs), which combine the advantages of Mobileblock for global processing and Metaformer architecture for local interaction. Our experimental results demonstrate that our network using mapping method outperforms masking methods, and direct mapping of reverse noise is the optimal solution in strong noise environments. In a horizontal comparison on the 2020 Deep Noise Suppression (DNS) challenge test set without reverberation, to the best of our knowledge, MFNet is the current state-of-the-art (SOTA) mapping model.

cs.SD↗
arXiv2023

Dual Path Transformer with Partition Attention

This paper introduces a novel attention mechanism, called dual attention, which is both efficient and effective. The dual attention mechanism consists of two parallel components: local attention generated by Convolutional Neural Networks (CNNs) and long-range attention generated by Vision Transformers (ViTs). To address the high computational complexity and memory footprint of vanilla Multi-Head Self-Attention (MHSA), we introduce a novel Multi-Head Partition-wise Attention (MHPA) mechanism. The partition-wise attention approach models both intra-partition and inter-partition attention simultaneously. Building on the dual attention block and partition-wise attention mechanism, we present a hierarchical vision backbone called DualFormer. We evaluate the effectiveness of our model on several computer vision tasks, including image classification on ImageNet, object detection on COCO, and semantic segmentation on Cityscapes. Specifically, the proposed DualFormer-XS achieves 81.5\% top-1 accuracy on ImageNet, outperforming the recent state-of-the-art MPViT-XS by 0.6\% top-1 accuracy with much higher throughput.

cs.CV↗
arXiv2023

A Heterogeneous 6G Networked Sensing Architecture with Active and Passive Anchors

In the future 6G integrated sensing and communication (ISAC) cellular systems, networked sensing is a promising technique that can leverage the cooperation among the base stations (BSs) to perform high-resolution localization. However, a dense deployment of BSs to fully reap the networked sensing gain is not a cost-efficient solution in practice. Motivated by the advance in the intelligent reflecting surface (IRS) technology for 6G communication, this paper examines the feasibility of deploying the low-cost IRSs to enhance the anchor density for networked sensing. Specifically, we propose a novel heterogeneous networked sensing architecture, which consists of both the active anchors, i.e., the BSs, and the passive anchors, i.e., the IRSs. Under this framework, the BSs emit the orthogonal frequency division multiplexing (OFDM) communication signals in the downlink for localizing the targets based on their echoes reflected via/not via the IRSs. However, there are two challenges for using passive anchors in localization. First, it is impossible to utilize the round-trip signal between a passive IRS and a passive target for estimating their distance. Second, before localizing a target, we do not know which IRS is closest to it and serves as its anchor. In this paper, we show that the distance between a target and its associated IRS can be indirectly estimated based on the length of the BS-target-BS path and the BS-target-IRS-BS path. Moreover, we propose an efficient data association method to match each target to its associated IRS. Numerical results are given to validate the feasibility and effectiveness of our proposed heterogeneous networked sensing architecture with both active and passive anchors.

eess.SP↗
arXiv2023

Device Activity Detection in mMTC with Low-Resolution ADC: A New Protocol

This paper investigates the effect of low-resolution analog-to-digital converters (ADCs) on device activity detection in massive machine-type communications (mMTC). The low-resolution ADCs induce two challenges on the device activity detection compared with the traditional setup with the assumption of infinite ADC resolution. First, the codebook design for signal quantization by the low-resolution ADC is particularly important since a good design of the codebook can lead to small quantization error on the received signal, which in turn has significant influence on the activity detector performance. To this end, prior information about the received signal power is needed, which depends on the number of active devices $K$. This is sharply different from the activity detection problem in traditional setups, in which the knowledge of $K$ is not required by the BS as a prerequisite. Second, the covariance-based approach achieves good activity detection performance in traditional setups while it is not clear if it can still achieve good performance in this paper. To solve the above challenges, we propose a communication protocol that consists of an estimator for $K$ and a detector for active device identities: 1) For the estimator, the technical difficulty is that the design of the ADC quantizer and the estimation of $K$ are closely intertwined and doing one needs the information/execution from the other. We propose a progressive estimator which iteratively performs the estimation of $K$ and the design of the ADC quantizer; 2) For the activity detector, we propose a custom-designed stochastic gradient descent algorithm to estimate the active device identities. Numerical results demonstrate the effectiveness of the communication protocol.

cs.IT↗
arXiv2023

MixTeacher: Mining Promising Labels with Mixed Scale Teacher for Semi-Supervised Object Detection

Scale variation across object instances remains a key challenge in object detection task. Despite the remarkable progress made by modern detection models, this challenge is particularly evident in the semi-supervised case. While existing semi-supervised object detection methods rely on strict conditions to filter high-quality pseudo labels from network predictions, we observe that objects with extreme scale tend to have low confidence, resulting in a lack of positive supervision for these objects. In this paper, we propose a novel framework that addresses the scale variation problem by introducing a mixed scale teacher to improve pseudo label generation and scale-invariant learning. Additionally, we propose mining pseudo labels using score promotion of predictions across scales, which benefits from better predictions from mixed scale features. Our extensive experiments on MS COCO and PASCAL VOC benchmarks under various semi-supervised settings demonstrate that our method achieves new state-of-the-art performance. The code and models are available at \url{https://github.com/lliuz/MixTeacher}.

cs.CV↗
arXiv2023

Calibrated Teacher for Sparsely Annotated Object Detection

Fully supervised object detection requires training images in which all instances are annotated. This is actually impractical due to the high labor and time costs and the unavoidable missing annotations. As a result, the incomplete annotation in each image could provide misleading supervision and harm the training. Recent works on sparsely annotated object detection alleviate this problem by generating pseudo labels for the missing annotations. Such a mechanism is sensitive to the threshold of the pseudo label score. However, the effective threshold is different in different training stages and among different object detectors. Therefore, the current methods with fixed thresholds have sub-optimal performance, and are difficult to be applied to other detectors. In order to resolve this obstacle, we propose a Calibrated Teacher, of which the confidence estimation of the prediction is well calibrated to match its real precision. In this way, different detectors in different training stages would share a similar distribution of the output confidence, so that multiple detectors could share the same fixed threshold and achieve better performance. Furthermore, we present a simple but effective Focal IoU Weight (FIoU) for the classification loss. FIoU aims at reducing the loss weight of false negative samples caused by the missing annotation, and thus works as the complement of the teacher-student paradigm. Extensive experiments show that our methods set new state-of-the-art under all different sparse settings in COCO. Code will be available at https://github.com/Whileherham/CalibratedTeacher.

cs.CV↗
arXiv2023

Spacecraft Anomaly Detection with Attention Temporal Convolution Network

Spacecraft faces various situations when carrying out exploration missions in complex space, thus monitoring the anomaly status of spacecraft is crucial to the development of \textcolor{blue}{the} aerospace industry. The time series telemetry data generated by on-orbit spacecraft \textcolor{blue}{contains} important information about the status of spacecraft. However, traditional domain knowledge-based spacecraft anomaly detection methods are not effective due to high dimensionality and complex correlation among variables. In this work, we propose an anomaly detection framework for spacecraft multivariate time-series data based on temporal convolution networks (TCNs). First, we employ dynamic graph attention to model the complex correlation among variables and time series. Second, temporal convolution networks with parallel processing ability are used to extract multidimensional \textcolor{blue}{features} for \textcolor{blue}{the} downstream prediction task. Finally, many potential anomalies are detected by the best threshold. Experiments on real NASA SMAP/MSL spacecraft datasets show the superiority of our proposed model with respect to state-of-the-art methods.

cs.LG↗
arXiv2023

Detecting Abrupt Changes in Channel Covariance Matrix for MIMO Communication

The acquisition of the channel covariance matrix is of paramount importance to many strategies in multiple-input-multiple-output (MIMO) communications, such as the minimum mean-square error (MMSE) channel estimation. Therefore, plenty of efficient channel covariance matrix estimation schemes have been proposed in the literature. However, an abrupt change in the channel covariance matrix may happen occasionally in practice due to the change in the scattering environment and the user location. Our paper aims to adopt the classic change detection theory to detect the change in the channel covariance matrix as accurately and quickly as possible such that the new covariance matrix can be re-estimated in time. Specifically, this paper first considers the technique of on-line change detection (also known as quickest/sequential change detection), where we need to detect whether a change in the channel covariance matrix occurs at each channel coherence time interval. Next, because the complexity of detecting the change in a high-dimension covariance matrix at each coherence time interval is too high, we devise a low-complexity off-line strategy in massive MIMO systems, where change detection is merely performed at the last channel coherence time interval of a given time period. Numerical results show that our proposed on-line and off-line schemes can detect the channel covariance change with a small delay and a low false alarm rate. Therefore, our paper theoretically and numerically verifies the feasibility of detecting the channel covariance change accurately and quickly in practice.

eess.SP↗
arXiv2023

Iterative Few-shot Semantic Segmentation from Image Label Text

Few-shot semantic segmentation aims to learn to segment unseen class objects with the guidance of only a few support images. Most previous methods rely on the pixel-level label of support images. In this paper, we focus on a more challenging setting, in which only the image-level labels are available. We propose a general framework to firstly generate coarse masks with the help of the powerful vision-language model CLIP, and then iteratively and mutually refine the mask predictions of support and query images. Extensive experiments on PASCAL-5i and COCO-20i datasets demonstrate that our method not only outperforms the state-of-the-art weakly supervised approaches by a significant margin, but also achieves comparable or better results to recent supervised methods. Moreover, our method owns an excellent generalization ability for the images in the wild and uncommon classes. Code will be available at https://github.com/Whileherham/IMR-HSNet.

cs.CV↗
arXiv2023

High-Precision Machine-Learning Based Indoor Localization with Massive MIMO System

High-precision cellular-based localization is one of the key technologies for next-generation communication systems. In this paper, we investigate the potential of applying machine learning (ML) to a massive multiple-input multiple-output (MIMO) system to enhance localization accuracy. We analyze a new ML-based localization pipeline that has two parallel fully connected neural networks (FCNN). The first FCNN takes the instantaneous spatial covariance matrix to capture angular information, while the second FCNN takes the channel impulse responses to capture delay information. We fuse the estimated coordinates of these two FCNNs for further accuracy improvement. To test the localization algorithm, we performed an indoor measurement campaign with a massive MIMO testbed at 3.7GHz. In the measured scenario, the proposed pipeline can achieve centimeter-level accuracy by combining delay and angular information.

eess.SP↗
arXiv2023

Spin- and orbital-angular-momentum nonlinear optical selectivity of single-mode nanolasers

Selective control of light is essential for optical science and technology with numerous applications. Nanophotonic waveguides and integrated couplers have been developed to achieve selective coupling and spatial control of an optical beam according to its multiple degrees of freedom. However, previous coupling devices remain passive with an inherently linear response to the power of incident light limiting their maximal optical selectivity. Here, we demonstrate nonlinear optical selectivity through selective excitation of individual single-mode nanolasers based on the spin and orbital angular momentum of light. Our designed nanolaser circuits consist of plasmonic metasurfaces and individual perovskite nanowires, enabling subwavelength focusing of angular-momentum-distinctive plasmonic fields and further selective excitation of single transverse laser modes in nanowires. The optically selected nanolaser with nonlinear increase of light emission greatly enhances the baseline optical selectivity offered by the metasurface from about 0.4 up to near unity. Our demonstrated nonlinear optical selectivity may find important applications in all-optical logic gates and nanowire networks, ultrafast optical switches, nanophotonic detectors, and on-chip optical and quantum information processing.

physics.optics↗
arXiv2023

Learning from Noisy Labels with Decoupled Meta Label Purifier

Training deep neural networks(DNN) with noisy labels is challenging since DNN can easily memorize inaccurate labels, leading to poor generalization ability. Recently, the meta-learning based label correction strategy is widely adopted to tackle this problem via identifying and correcting potential noisy labels with the help of a small set of clean validation data. Although training with purified labels can effectively improve performance, solving the meta-learning problem inevitably involves a nested loop of bi-level optimization between model weights and hyper-parameters (i.e., label distribution). As compromise, previous methods resort to a coupled learning process with alternating update. In this paper, we empirically find such simultaneous optimization over both model weights and label distribution can not achieve an optimal routine, consequently limiting the representation ability of backbone and accuracy of corrected labels. From this observation, a novel multi-stage label purifier named DMLP is proposed. DMLP decouples the label correction process into label-free representation learning and a simple meta label purifier. In this way, DMLP can focus on extracting discriminative feature and label correction in two distinctive stages. DMLP is a plug-and-play label purifier, the purified labels can be directly reused in naive end-to-end network retraining or other robust learning methods, where state-of-the-art results are obtained on several synthetic and real-world noisy datasets, especially under high noise levels.

cs.CV↗
arXiv2023

Joint Data Association, NLOS Mitigation, and Clutter Suppression for Networked Device-Free Sensing in 6G Cellular Network

Recently, there is a growing interest in achieving integrated sensing and communication (ISAC) in the sixth-generation (6G) cellular network. Inspired by this trend and the success of cooperative communication in cloud radio access network, this paper considers a networked device-free sensing architecture based on base station (BS) cooperation to transform the cellular network into a huge sensor that can provide ubiquitous and high-performance sensing services. Under this framework, the BSs first transmit the downlink communication signals to the mobile users and then estimate the range information of the targets based on their echoes. Next, a central processor collects the range information from all the BSs via the fronthaul links and localizes each target based on its distances to various BSs. To enable the above strategy in the 6G network, we will perform joint data association, non-line-of-sight (NLOS) mitigation, and clutter suppression, such that the central processor is able to find out the useful range estimations extracted from the line-of-sight (LOS) paths and match them to the right targets for localization. Numerical results show that our interested networked device-free sensing scheme for the 6G network can localize the targets with high accuracy in the challenging multi-path propagation environment.

eess.SP↗
arXiv2023

Learning with Noisy labels via Self-supervised Adversarial Noisy Masking

Collecting large-scale datasets is crucial for training deep models, annotating the data, however, inevitably yields noisy labels, which poses challenges to deep learning algorithms. Previous efforts tend to mitigate this problem via identifying and removing noisy samples or correcting their labels according to the statistical properties (e.g., loss values) among training samples. In this paper, we aim to tackle this problem from a new perspective, delving into the deep feature maps, we empirically find that models trained with clean and mislabeled samples manifest distinguishable activation feature distributions. From this observation, a novel robust training approach termed adversarial noisy masking is proposed. The idea is to regularize deep features with a label quality guided masking scheme, which adaptively modulates the input data and label simultaneously, preventing the model to overfit noisy samples. Further, an auxiliary task is designed to reconstruct input data, it naturally provides noise-free self-supervised signals to reinforce the generalization ability of deep models. The proposed method is simple and flexible, it is tested on both synthetic and real-world noisy datasets, where significant improvements are achieved over previous state-of-the-art methods.

cs.CV↗