arXiv subjects

Zhizhong Zhang

Publications and source records attributed to Zhizhong Zhang.

67 records · Page 4Linked to original sources

arXiv2022

Image Understands Point Cloud: Weakly Supervised 3D Semantic Segmentation via Association Learning

Weakly supervised point cloud semantic segmentation methods that require 1\% or fewer labels, hoping to realize almost the same performance as fully supervised approaches, which recently, have attracted extensive research attention. A typical solution in this framework is to use self-training or pseudo labeling to mine the supervision from the point cloud itself, but ignore the critical information from images. In fact, cameras widely exist in LiDAR scenarios and this complementary information seems to be greatly important for 3D applications. In this paper, we propose a novel cross-modality weakly supervised method for 3D segmentation, incorporating complementary information from unlabeled images. Basically, we design a dual-branch network equipped with an active labeling strategy, to maximize the power of tiny parts of labels and directly realize 2D-to-3D knowledge transfer. Afterwards, we establish a cross-modal self-training framework in an Expectation-Maximum (EM) perspective, which iterates between pseudo labels estimation and parameters updating. In the M-Step, we propose a cross-modal association learning to mine complementary supervision from images by reinforcing the cycle-consistency between 3D points and 2D superpixels. In the E-step, a pseudo label self-rectification mechanism is derived to filter noise labels thus providing more accurate labels for the networks to get fully trained. The extensive experimental results demonstrate that our method even outperforms the state-of-the-art fully supervised competitors with less than 1\% actively selected annotations.

cs.CV↗

arXiv2022

Attentive pooling for Group Activity Recognition

In group activity recognition, hierarchical framework is widely adopted to represent the relationships between individuals and their corresponding group, and has achieved promising performance. However, the existing methods simply employed max/average pooling in this framework, which ignored the distinct contributions of different individuals to the group activity recognition. In this paper, we propose a new contextual pooling scheme, named attentive pooling, which enables the weighted information transition from individual actions to group activity. By utilizing the attention mechanism, the attentive pooling is intrinsically interpretable and able to embed member context into the existing hierarchical model. In order to verify the effectiveness of the proposed scheme, two specific attentive pooling methods, i.e., global attentive pooling (GAP) and hierarchical attentive pooling (HAP) are designed. GAP rewards the individuals that are significant to group activity, while HAP further considers the hierarchical division by introducing subgroup structure. The experimental results on the benchmark dataset demonstrate that our proposal is significantly superior beyond the baseline and is comparable to the state-of-the-art methods.

cs.CV↗

arXiv2022

Boosting Night-time Scene Parsing with Learnable Frequency

Night-Time Scene Parsing (NTSP) is essential to many vision applications, especially for autonomous driving. Most of the existing methods are proposed for day-time scene parsing. They rely on modeling pixel intensity-based spatial contextual cues under even illumination. Hence, these methods do not perform well in night-time scenes as such spatial contextual cues are buried in the over-/under-exposed regions in night-time scenes. In this paper, we first conduct an image frequency-based statistical experiment to interpret the day-time and night-time scene discrepancies. We find that image frequency distributions differ significantly between day-time and night-time scenes, and understanding such frequency distributions is critical to NTSP problem. Based on this, we propose to exploit the image frequency distributions for night-time scene parsing. First, we propose a Learnable Frequency Encoder (LFE) to model the relationship between different frequency coefficients to measure all frequency components dynamically. Second, we propose a Spatial Frequency Fusion module (SFF) that fuses both spatial and frequency information to guide the extraction of spatial context features. Extensive experiments show that our method performs favorably against the state-of-the-art methods on the NightCity, NightCity+ and BDD100K-night datasets. In addition, we demonstrate that our method can be applied to existing day-time scene parsing methods and boost their performance on night-time scenes.

cs.CV↗

arXiv2022

Variational Distillation for Multi-View Learning

Information Bottleneck (IB) based multi-view learning provides an information theoretic principle for seeking shared information contained in heterogeneous data descriptions. However, its great success is generally attributed to estimate the multivariate mutual information which is intractable when the network becomes complicated. Moreover, the representation learning tradeoff, {\it i.e.}, prediction-compression and sufficiency-consistency tradeoff, makes the IB hard to satisfy both requirements simultaneously. In this paper, we design several variational information bottlenecks to exploit two key characteristics ({\it i.e.}, sufficiency and consistency) for multi-view representation learning. Specifically, we propose a Multi-View Variational Distillation (MV$^2$D) strategy to provide a scalable, flexible and analytical solution to fitting MI by giving arbitrary input of viewpoints but without explicitly estimating it. Under rigorously theoretical guarantee, our approach enables IB to grasp the intrinsic correlation between observations and semantic labels, producing predictive and compact representations naturally. Also, our information-theoretic constraint can effectively neutralize the sensitivity to heterogeneous data by eliminating both task-irrelevant and view-specific information, preventing both tradeoffs in multiple view cases. To verify our theoretically grounded strategies, we apply our approaches to various benchmarks under three different applications. Extensive experiments to quantitatively and qualitatively demonstrate the effectiveness of our approach against state-of-the-art methods.

cs.CV↗

arXiv2022

NTIRE 2022 Challenge on Efficient Super-Resolution: Methods and Results

This paper reviews the NTIRE 2022 challenge on efficient single image super-resolution with focus on the proposed solutions and results. The task of the challenge was to super-resolve an input image with a magnification factor of $\times$4 based on pairs of low and corresponding high resolution images. The aim was to design a network for single image super-resolution that achieved improvement of efficiency measured according to several metrics including runtime, parameters, FLOPs, activations, and memory consumption while at least maintaining the PSNR of 29.00dB on DIV2K validation set. IMDN is set as the baseline for efficiency measurement. The challenge had 3 tracks including the main track (runtime), sub-track one (model complexity), and sub-track two (overall performance). In the main track, the practical runtime performance of the submissions was evaluated. The rank of the teams were determined directly by the absolute value of the average runtime on the validation set and test set. In sub-track one, the number of parameters and FLOPs were considered. And the individual rankings of the two metrics were summed up to determine a final ranking in this track. In sub-track two, all of the five metrics mentioned in the description of the challenge including runtime, parameter count, FLOPs, activations, and memory consumption were considered. Similar to sub-track one, the rankings of five metrics were summed up to determine a final ranking. The challenge had 303 registered participants, and 43 teams made valid submissions. They gauge the state-of-the-art in efficient single image super-resolution.

cs.CV↗

arXiv2021

Multi-scale 2D Representation Learning for weakly-supervised moment retrieval

Video moment retrieval aims to search the moment most relevant to a given language query. However, most existing methods in this community often require temporal boundary annotations which are expensive and time-consuming to label. Hence weakly supervised methods have been put forward recently by only using coarse video-level label. Despite effectiveness, these methods usually process moment candidates independently, while ignoring a critical issue that the natural temporal dependencies between candidates in different temporal scales. To cope with this issue, we propose a Multi-scale 2D Representation Learning method for weakly supervised video moment retrieval. Specifically, we first construct a two-dimensional map for each temporal scale to capture the temporal dependencies between candidates. Two dimensions in this map indicate the start and end time points of these candidates. Then, we select top-K candidates from each scale-varied map with a learnable convolutional neural network. With a newly designed Moments Evaluation Module, we obtain the alignment scores of the selected candidates. At last, the similarity between captions and language query is served as supervision for further training the candidates' selector. Experiments on two benchmark datasets Charades-STA and ActivityNet Captions demonstrate that our approach achieves superior performance to state-of-the-art results.

cs.CV↗

arXiv2021

Towards Compact Single Image Super-Resolution via Contrastive Self-distillation

Convolutional neural networks (CNNs) are highly successful for super-resolution (SR) but often require sophisticated architectures with heavy memory cost and computational overhead, significantly restricts their practical deployments on resource-limited devices. In this paper, we proposed a novel contrastive self-distillation (CSD) framework to simultaneously compress and accelerate various off-the-shelf SR models. In particular, a channel-splitting super-resolution network can first be constructed from a target teacher network as a compact student network. Then, we propose a novel contrastive loss to improve the quality of SR images and PSNR/SSIM via explicit knowledge transfer. Extensive experiments demonstrate that the proposed CSD scheme effectively compresses and accelerates several standard SR models such as EDSR, RCAN and CARN. Code is available at https://github.com/Booooooooooo/CSD.

cs.CV↗

arXiv2021

Contrastive Learning for Compact Single Image Dehazing

Single image dehazing is a challenging ill-posed problem due to the severe information degeneration. However, existing deep learning based dehazing methods only adopt clear images as positive samples to guide the training of dehazing network while negative information is unexploited. Moreover, most of them focus on strengthening the dehazing network with an increase of depth and width, leading to a significant requirement of computation and memory. In this paper, we propose a novel contrastive regularization (CR) built upon contrastive learning to exploit both the information of hazy images and clear images as negative and positive samples, respectively. CR ensures that the restored image is pulled to closer to the clear image and pushed to far away from the hazy image in the representation space. Furthermore, considering trade-off between performance and memory storage, we develop a compact dehazing network based on autoencoder-like (AE) framework. It involves an adaptive mixup operation and a dynamic feature enhancement module, which can benefit from preserving information flow adaptively and expanding the receptive field to improve the network's transformation capability, respectively. We term our dehazing network with autoencoder and contrastive regularization as AECR-Net. The extensive experiments on synthetic and real-world datasets demonstrate that our AECR-Net surpass the state-of-the-art approaches. The code is released in https://github.com/GlassyWu/AECR-Net.

cs.CV↗

arXiv2021

Field-free spin-orbit torque-induced switching of perpendicular magnetization in a ferrimagnetic layer with vertical composition gradient

Current-induced spin-orbit torques (SOTs) are of interest for fast and energy-efficient manipulation of magnetic order in spintronic devices. To be deterministic, however, switching of perpendicularly magnetized materials by SOT requires a mechanism for in-plane symmetry breaking. Existing methods to do so involve the application of an in-plane bias magnetic field, or incorporation of in-plane structural asymmetry in the device, both of which can be difficult to implement in practical applications. Here, we reported bias-field-free SOT switching in a single perpendicular CoTb layer with an engineered vertical composition gradient. The vertical structural inversion asymmetry induces strong intrinsic SOTs and a gradient-driven Dzyaloshinskii-Moriya interaction (g-DMI), which breaks the in-plane symmetry during the switching process. Micromagnetic simulations are in agreement with experimental results, and elucidate the role of g-DMI in the deterministic switching. This bias-field-free switching scheme for perpendicular ferrimagnets with g-DMI provides a strategy for efficient and compact SOT device design.

cond-mat.mes-hall↗

arXiv2018

Domain Wall Motion Driven by Laplace Pressure in CoFeB-MgO Nanodots with Perpendicular Anisotropy

We have studied the magnetization reversal of CoFeB-MgO nanodots with perpendicular anisotropy for size ranging from w=400 nm to 1 μm. Contrary to previous experiments, the switching field distribution is shifted toward lower magnetic fields as the size of the elements is reduced with a mean switching field varying as 1/w. We show that this mechanism can be explained by the nucleation of a pinned magnetic domain wall (DW) at the edges of the nanodots where damages are introduced by the patterning process. As the surface tension (Laplace pressure) applied on the DW increases when reducing the size of the nanodots, we demonstrate that the depinning field to reverse the entire elements varies as 1/w. These results suggest that the presence of DWs has to be considered in the switching process of nanoscale elements and open a path toward scalable spintronic devices.

physics.app-ph↗

arXiv2018

Ultra-Fast Ferrimagnetic All Spin Logic Device

All spin logic device (ASLD) blazes an alternative path for realizing ultra-low power computing in the Post-Moore era. However, initial device structure relying on ferromagnetic input/output and spin transfer torque (STT) driven magnetization switching degrades its performance and even hinders its realization. In this paper, we propose an ASLD based on rare-earth (RE)-transition-metal (TM) ferromagnetic alloy that can achieve an ultra-high frequency up to terahertz. The spin orbit torque (SOT) induced fast precession near the spin angular momentum compensated point is investigated through the macrospin model. Combining the non-local spin current diffusing from the input to the output, a deterministic picosecond switching can be realized without any external magnetic field. Our results show that ASLD has the potential to exceed the performance of mainstream computing.

physics.app-ph↗

arXiv2017

Effective Image Retrieval via Multilinear Multi-index Fusion

Multi-index fusion has demonstrated impressive performances in retrieval task by integrating different visual representations in a unified framework. However, previous works mainly consider propagating similarities via neighbor structure, ignoring the high order information among different visual representations. In this paper, we propose a new multi-index fusion scheme for image retrieval. By formulating this procedure as a multilinear based optimization problem, the complementary information hidden in different indexes can be explored more thoroughly. Specially, we first build our multiple indexes from various visual representations. Then a so-called index-specific functional matrix, which aims to propagate similarities, is introduced for updating the original index. The functional matrices are then optimized in a unified tensor space to achieve a refinement, such that the relevant images can be pushed more closer. The optimization problem can be efficiently solved by the augmented Lagrangian method with theoretical convergence guarantee. Unlike the traditional multi-index fusion scheme, our approach embeds the multi-index subspace structure into the new indexes with sparse constraint, thus it has little additional memory consumption in online query stage. Experimental evaluation on three benchmark datasets reveals that the proposed approach achieves the state-of-the-art performance, i.e., N-score 3.94 on UKBench, mAP 94.1\% on Holiday and 62.39\% on Market-1501.

cs.CV↗

arXiv2016

Perspectives of Racetrack Memory for Large-Capacity On-Chip Memory: From Device to System

Current-induced domain wall motion (CIDWM) is regarded as a promising way towards achieving emerging high-density, high-speed and low-power non-volatile devices. Racetrack memory is an attractive spintronic memory based on this phenomenon, which can store and transfer a series of data along a magnetic nanowire. However, storage capacity issue is always one of the most serious bottlenecks hindering its application for practical systems. This paper focuses on the potential of racetrack memory towards large capacity. The investigations covering from device level to system level have been carried out. Various alternative mechanisms to improve the capacity of racetrack memory have been proposed and elucidated, e.g. magnetic field assistance, chiral DW motion and voltage-controlled flexible DW pinning. All of them can increase nanowire length, allowing enhanced feasibility of large-capacity racetrack memory. By using SPICE compatible racetrack memory electrical model and commercial CMOS 28 nm design kit, mixed simulations are performed to validate their functionalities and analyze their performance. System level evaluations demonstrate the impact of capacity improvement on overall system. Compared with traditional SRAM based cache, racetrack memory based cache shows its advantages in terms of execution time and energy consumption.

cond-mat.mtrl-sci↗