Searcharxiv⌕ Search

arXiv subjects

Jiawei Liu

Publications and source records attributed to Jiawei Liu.

At least 199 records · Page 11Linked to original sources

Abnormal Event Detection via Hypergraph Contrastive Learning

Abnormal event detection, which refers to mining unusual interactions among involved entities, plays an important role in many real applications. Previous works mostly over-simplify this task as detecting abnormal pair-wise interactions. However, real-world events may contain multi-typed attributed entities and complex interactions among them, which forms an Attributed Heterogeneous Information Network (AHIN). With the boom of social networks, abnormal event detection in AHIN has become an important, but seldom explored task. In this paper, we firstly study the unsupervised abnormal event detection problem in AHIN. The events are considered as star-schema instances of AHIN and are further modeled by hypergraphs. A novel hypergraph contrastive learning method, named AEHCL, is proposed to fully capture abnormal event patterns. AEHCL designs the intra-event and inter-event contrastive modules to exploit self-supervised AHIN information. The intra-event contrastive module captures the pair-wise and multivariate interaction anomalies within an event, and the inter-event module captures the contextual anomalies among events. These two modules collaboratively boost the performance of each other and improve the detection results. During the testing phase, a contrastive learning-based abnormal event score function is further proposed to measure the abnormality degree of events. Extensive experiments on three datasets in different scenarios demonstrate the effectiveness of AEHCL, and the results improve state-of-the-art baselines up to 12.0% in Average Precision (AP) and 4.6% in Area Under Curve (AUC) respectively.

cs.LG↗

Sounding Video Generator: A Unified Framework for Text-guided Sounding Video Generation

As a combination of visual and audio signals, video is inherently multi-modal. However, existing video generation methods are primarily intended for the synthesis of visual frames, whereas audio signals in realistic videos are disregarded. In this work, we concentrate on a rarely investigated problem of text guided sounding video generation and propose the Sounding Video Generator (SVG), a unified framework for generating realistic videos along with audio signals. Specifically, we present the SVG-VQGAN to transform visual frames and audio melspectrograms into discrete tokens. SVG-VQGAN applies a novel hybrid contrastive learning method to model inter-modal and intra-modal consistency and improve the quantized representations. A cross-modal attention module is employed to extract associated features of visual frames and audio signals for contrastive learning. Then, a Transformer-based decoder is used to model associations between texts, visual frames, and audio signals at token level for auto-regressive sounding video generation. AudioSetCap, a human annotated text-video-audio paired dataset, is produced for training SVG. Experimental results demonstrate the superiority of our method when compared with existing textto-video generation methods as well as audio generation methods on Kinetics and VAS datasets.

cs.CV↗

PD-Quant: Post-Training Quantization based on Prediction Difference Metric

Post-training quantization (PTQ) is a neural network compression technique that converts a full-precision model into a quantized model using lower-precision data types. Although it can help reduce the size and computational cost of deep neural networks, it can also introduce quantization noise and reduce prediction accuracy, especially in extremely low-bit settings. How to determine the appropriate quantization parameters (e.g., scaling factors and rounding of weights) is the main problem facing now. Existing methods attempt to determine these parameters by minimize the distance between features before and after quantization, but such an approach only considers local information and may not result in the most optimal quantization parameters. We analyze this issue and ropose PD-Quant, a method that addresses this limitation by considering global information. It determines the quantization parameters by using the information of differences between network prediction before and after quantization. In addition, PD-Quant can alleviate the overfitting problem in PTQ caused by the small number of calibration sets by adjusting the distribution of activations. Experiments show that PD-Quant leads to better quantization parameters and improves the prediction accuracy of quantized models, especially in low-bit settings. For example, PD-Quant pushes the accuracy of ResNet-18 up to 53.14% and RegNetX-600MF up to 40.67% in weight 2-bit activation 2-bit. The code is released at https://github.com/hustvl/PD-Quant.

cs.CV↗

Benchmarking the Reliability of Post-training Quantization: a Particular Focus on Worst-case Performance

Post-training quantization (PTQ) is a popular method for compressing deep neural networks (DNNs) without modifying their original architecture or training procedures. Despite its effectiveness and convenience, the reliability of PTQ methods in the presence of some extrem cases such as distribution shift and data noise remains largely unexplored. This paper first investigates this problem on various commonly-used PTQ methods. We aim to answer several research questions related to the influence of calibration set distribution variations, calibration paradigm selection, and data augmentation or sampling strategies on PTQ reliability. A systematic evaluation process is conducted across a wide range of tasks and commonly-used PTQ paradigms. The results show that most existing PTQ methods are not reliable enough in term of the worst-case group performance, highlighting the need for more robust methods. Our findings provide insights for developing PTQ methods that can effectively handle distribution shift scenarios and enable the deployment of quantized DNNs in real-world applications.

cs.LG↗

When Optical Microscopy Meets All-Optical Analog Computing: A Brief Review

As a revolutionary observation tool in life science, biomedical, and material science, optical microscopy allows imaging of samples with high spatial resolution and a wide field of view. However, conventional microscopy methods are limited to single imaging and cannot accomplish real-time image processing. The edge detection, image enhancement and phase visualization schemes have attracted great interest with the rapid development of optical analog computing. The two main physical mechanisms that enable optical analog computing originate from two geometric phases: the spin-redirection Rytov-Vlasimirskii-Berry (RVB) phase and the Pancharatnam-Berry (PB) phase. Here, we review the basic principles and recent research progress of the RVB phase and PB phase based optical differentiators. Then we focus on the innovative and emerging applications of optical analog computing in microscopic imaging. Optical analog computing is accelerating the transformation of information processing from classical imaging to quantum techniques. Its intersection with optical microscopy opens opportunities for the development of versatile and compact optical microscopy systems.

physics.optics↗

Local Label Point Correction for Edge Detection of Overlapping Cervical Cells

Accurate labeling is essential for supervised deep learning methods. However, it is almost impossible to accurately and manually annotate thousands of images, which results in many labeling errors for most datasets. We proposes a local label point correction (LLPC) method to improve annotation quality for edge detection and image segmentation tasks. Our algorithm contains three steps: gradient-guided point correction, point interpolation and local point smoothing. We correct the labels of object contours by moving the annotated points to the pixel gradient peaks. This can improve the edge localization accuracy, but it also causes unsmooth contours due to the interference of image noise. Therefore, we design a point smoothing method based on local linear fitting to smooth the corrected edge. To verify the effectiveness of our LLPC, we construct a largest overlapping cervical cell edge detection dataset (CCEDD) with higher precision label corrected by our label correction method. Our LLPC only needs to set three parameters, but yields 30-40$\%$ average precision improvement on multiple networks. The qualitative and quantitative experimental results show that our LLPC can improve the quality of manual labels and the accuracy of overlapping cell edge detection. We hope that our study will give a strong boost to the development of the label correction for edge detection and image segmentation. We will release the dataset and code at https://github.com/nachifur/LLPC.

cs.CV↗

AI vs. Human -- Differentiation Analysis of Scientific Content Generation

Recent neural language models have taken a significant step forward in producing remarkably controllable, fluent, and grammatical text. Although studies have found that AI-generated text is not distinguishable from human-written text for crowd-sourcing workers, there still exist errors in AI-generated text which are even subtler and harder to spot. We primarily focus on the scenario in which scientific AI writing assistant is deeply involved. First, we construct a feature description framework to distinguish between AI-generated text and human-written text from syntax, semantics, and pragmatics based on the human evaluation. Then we utilize the features, i.e., writing style, coherence, consistency, and argument logistics, from the proposed framework to analyze two types of content. Finally, we adopt several publicly available methods to investigate the gap of between AI-generated scientific text and human-written scientific text by AI-generated scientific text detection models. The results suggest that while AI has the potential to generate scientific content that is as accurate as human-written content, there is still a gap in terms of depth and overall quality. The AI-generated scientific content is more likely to contain errors in factual issues. We find that there exists a "writing style" gap between AI-generated scientific text and human-written scientific text. Based on the analysis result, we summarize a series of model-agnostic and distribution-agnostic features for detection tasks in other domains. Findings in this paper contribute to guiding the optimization of AI models to produce high-quality content and addressing related ethical and security concerns.

cs.CL↗

Generalized Few-Shot 3D Object Detection of LiDAR Point Cloud for Autonomous Driving

Recent years have witnessed huge successes in 3D object detection to recognize common objects for autonomous driving (e.g., vehicles and pedestrians). However, most methods rely heavily on a large amount of well-labeled training data. This limits their capability of detecting rare fine-grained objects (e.g., police cars and ambulances), which is important for special cases, such as emergency rescue, and so on. To achieve simultaneous detection for both common and rare objects, we propose a novel task, called generalized few-shot 3D object detection, where we have a large amount of training data for common (base) objects, but only a few data for rare (novel) classes. Specifically, we analyze in-depth differences between images and point clouds, and then present a practical principle for the few-shot setting in the 3D LiDAR dataset. To solve this task, we propose a simple and effective detection framework, including (1) an incremental fine-tuning method to extend existing 3D detection models to recognize both common and rare objects, and (2) a sample adaptive balance loss to alleviate the issue of long-tailed data distribution in autonomous driving scenarios. On the nuScenes dataset, we conduct sufficient experiments to demonstrate that our approach can successfully detect the rare (novel) classes that contain only a few training data, while also maintaining the detection accuracy of common objects.

cs.CV↗

Low-loss interconnects for modular superconducting quantum processors

Scaling is now a key challenge in superconducting quantum computing. One solution is to build modular systems in which smaller-scale quantum modules are individually constructed and calibrated, and then assembled into a larger architecture. This, however, requires the development of suitable interconnects. Here, we report low-loss interconnects based on pure aluminium coaxial cables and on-chip impedance transformers featuring quality factors up to $8.1 \times 10^5$, which is comparable to the performance of our transmon qubits fabricated on single-crystal sapphire substrate. We use these interconnects to link five quantum modules with inter-module quantum state transfer and Bell state fidelities up to 99\%. To benchmark the overall performance of the processor, we create maximally-entangled, multi-qubit Greenberger-Horne-Zeilinger (GHZ) states. The generated inter-module four-qubit GHZ state exhibits 92.0\% fidelity. We also entangle up to 12 qubits in a GHZ state with $55.8 \pm 1.8\%$ fidelity, which is above the genuine multipartite entanglement threshold of 1/2. These results represent a viable modular approach for large-scale superconducting quantum processors.

quant-ph↗

NNSmith: Generating Diverse and Valid Test Cases for Deep Learning Compilers

Deep-learning (DL) compilers such as TVM and TensorRT are increasingly being used to optimize deep neural network (DNN) models to meet performance, resource utilization and other requirements. Bugs in these compilers can result in models whose semantics differ from the original ones, producing incorrect results that corrupt the correctness of downstream applications. However, finding bugs in these compilers is challenging due to their complexity. In this work, we propose a new fuzz testing approach for finding bugs in deep-learning compilers. Our core approach consists of (i) generating diverse yet valid DNN test models that can exercise a large part of the compiler's transformation logic using light-weight operator specifications; (ii) performing gradient-based search to find model inputs that avoid any floating-point exceptional values during model execution, reducing the chance of missed bugs or false alarms; and (iii) using differential testing to identify bugs. We implemented this approach in NNSmith which has found 72 new bugs for TVM, TensorRT, ONNXRuntime, and PyTorch to date. Of these 58 have been confirmed and 51 have been fixed by their respective project maintainers.

cs.LG↗

Fast and Flexible Human Pose Estimation with HyperPose

Estimating human pose is an important yet challenging task in multimedia applications. Existing pose estimation libraries target reproducing standard pose estimation algorithms. When it comes to customising these algorithms for real-world applications, none of the existing libraries can offer both the flexibility of developing custom pose estimation algorithms and the high-performance of executing these algorithms on commodity devices. In this paper, we introduce Hyperpose, a novel flexible and high-performance pose estimation library. Hyperpose provides expressive Python APIs that enable developers to easily customise pose estimation algorithms for their applications. It further provides a model inference engine highly optimised for real-time pose estimation. This engine can dynamically dispatch carefully designed pose estimation tasks to CPUs and GPUs, thus automatically achieving high utilisation of hardware resources irrespective of deployment environments. Extensive evaluation results show that Hyperpose can achieve up to 3.1x~7.3x higher pose estimation throughput compared to state-of-the-art pose estimation libraries without compromising estimation accuracy. By 2021, Hyperpose has received over 1000 stars on GitHub and attracted users from both industry and academy.

cs.CV↗

Task-load-Aware Game-Theoretic Framework for Wireless Federated Learning

Federated learning (FL) has been proposed as a popular learning framework to protect the users' data privacy but it has difficulties in motivating the users to participate in task training. This paper proposes a Bertrand-game-based framework for FL in wireless networks, where the model server as a resource buyer can issue an FL task, whereas the employed user equipment (UEs) as the resource sellers can help train the model by using their local data. Specially, the influence of time-varying \textit{task load} and \textit{channel quality} on UE's motivation to participate in FL is considered. Firstly, we adopt the finite-state discrete-time Markov chain (FSDT-MC) method to predict the \textit{existing task load} and \textit{channel gain} of a UE during a FL task. Depending on the performance metrics set by the model server and the estimated overall energy cost for engaging in the FL task, each UE seeks the best price to maximize its own profit in the game. To this end, the Nash equilibrium (NE) of the game is obtained in closed form, and a distributed iterative algorithm is also developed to find the NE. Simulation result verifies the effectiveness of the proposed approach.

cs.GT↗

Improve Ranking Correlation of Super-net through Training Scheme from One-shot NAS to Few-shot NAS

The algorithms of one-shot neural architecture search(NAS) have been widely used to reduce computation consumption. However, because of the interference among the subnets in which weights are shared, the subnets inherited from these super-net trained by those algorithms have poor consistency in precision ranking. To address this problem, we propose a step-by-step training super-net scheme from one-shot NAS to few-shot NAS. In the training scheme, we firstly train super-net in a one-shot way, and then we disentangle the weights of super-net by splitting them into multi-subnets and training them gradually. Finally, our method ranks 4th place in the CVPR2022 3rd Lightweight NAS Challenge Track1. Our code is available at https://github.com/liujiawei2333/CVPR2022-NAS-competition-Track-1-4th-solution.

cs.CV↗

Label Noise-Resistant Mean Teaching for Weakly Supervised Fake News Detection

Fake news spreads at an unprecedented speed, reaches global audiences and poses huge risks to users and communities. Most existing fake news detection algorithms focus on building supervised training models on a large amount of manually labeled data, which is expensive to acquire or often unavailable. In this work, we propose a novel label noise-resistant mean teaching approach (LNMT) for weakly supervised fake news detection. LNMT leverages unlabeled news and feedback comments of users to enlarge the amount of training data and facilitates model training by generating refined labels as weak supervision. Specifically, LNMT automatically assigns initial weak labels to unlabeled samples based on semantic correlation and emotional association between news content and the comments. Moreover, in order to suppress the noises in weak labels, LNMT establishes a mean teacher framework equipped with label propagation and label reliability estimation. The framework measures a weak label similarity matrix between the teacher and student networks, and propagates different valuable weak label information to refine the weak labels. Meanwhile, it exploits the consistency between the output class likelihood vectors of the two networks to evaluate the reliability of the weak labels and incorporates the reliability into model optimization to alleviate the negative effect of noisy weak labels. Extensive experiments show the superior performance of LNMT.

cs.CL↗

Heteromoiré Engineering on Magnetic Bloch Transport in Twisted Graphene Superlattices

Localized electrons subject to applied magnetic fields can restart to propagate freely through the lattice in delocalized magnetic Bloch states (MBSs) when the lattice periodicity is commensurate with the magnetic length. Twisted graphene superlattices with moiré wavelength tunability enable experimental access to the unique delocalization in a controllable fashion. Here we report the observation and characterization of high-temperature Brown-Zak (BZ) oscillations which come in two types, 1/B and B periodicity, originating from the generation of integer and fractional MBSs, in the twisted bilayer and trilayer graphene superlattices, respectively. Coexisting periodic-in-1/B oscillations assigned to different moiré wavelengths, are dramatically observed in small-angle twisted bilayer graphene, which may arise from angle-disorder-induced in-plane heteromoiré superlattices. Moreover, the vertical stacking of heteromoiré supercells in double-twisted trilayer graphene results in a mega-sized superlattice. The exotic superlattice contributes to the periodic-in-B oscillation and dominates the magnetic Bloch transport.

cond-mat.mes-hall↗

Realizing Ultra-Fast and Energy-Efficient Baseband Processing Using Analogue Resistive Switching Memory

To support emerging applications ranging from holographic communications to extended reality, next-generation mobile wireless communication systems require ultra-fast and energy-efficient (UFEE) baseband processors. Traditional complementary metal-oxide-semiconductor (CMOS)-based baseband processors face two challenges in transistor scaling and the von Neumann bottleneck. To address these challenges, in-memory computing-based baseband processors using resistive random-access memory (RRAM) present an attractive solution. In this paper, we propose and demonstrate RRAM-based in-memory baseband processing for the widely adopted multiple-input-multiple-output orthogonal frequency division multiplexing (MIMO-OFDM) air interface. Its key feature is to execute the key operations, including discrete Fourier transform (DFT) and MIMO detection using linear minimum mean square error (L-MMSE) and zero forcing (ZF), in one-step. In addition, RRAM-based channel estimation as well as mapper/demapper modules are proposed. By prototyping and simulations, we demonstrate that the RRAM-based full-fledged communication system can significantly outperform its CMOS-based counterpart in terms of speed and energy efficiency by $10^3$ and $10^6$ times, respectively. The results pave a potential pathway for RRAM-based in-memory computing to be implemented in the era of the sixth generation (6G) mobile communications.

eess.SP↗

Scaling Limits of Controlled Branching Processes

In this paper, a special sequence of controlled branching processes is considered. We provide a simple set of sufficient conditions for the weak convergence of such processes to a weak solution to a kind of continuous branching processes with dependent immigration.

math.PR↗

Multi-grating design for integrated single-atom trapping, manipulation, and readout

An on-chip multi-grating device is proposed to interface single-atoms and integrated photonic circuits, by guiding and focusing lasers to the area with ~10um above the chip for trapping, state manipulation, and readout of single Rubidium atoms. For the optical dipole trap, two 850 nm laser beams are diffracted and overlapped to form a lattice of single-atom dipole trap, with the diameter of optical dipole trap around 2.7um. Similar gratings are designed for guiding 780 nm probe laser to excite and also collect the fluorescence of 87Rb atoms. Such a device provides a compact solution for future applications of single atoms, including the single photon source, single-atom quantum register, and sensor.

physics.optics↗