arXiv subjects

Han Zhang

Publications and source records attributed to Han Zhang.

At least 379 records · Page 21Linked to original sources

arXiv2022

Learning to Prompt for Continual Learning

The mainstream paradigm behind continual learning has been to adapt the model parameters to non-stationary data distributions, where catastrophic forgetting is the central challenge. Typical methods rely on a rehearsal buffer or known task identity at test time to retrieve learned knowledge and address forgetting, while this work presents a new paradigm for continual learning that aims to train a more succinct memory system without accessing task identity at test time. Our method learns to dynamically prompt (L2P) a pre-trained model to learn tasks sequentially under different task transitions. In our proposed framework, prompts are small learnable parameters, which are maintained in a memory space. The objective is to optimize prompts to instruct the model prediction and explicitly manage task-invariant and task-specific knowledge while maintaining model plasticity. We conduct comprehensive experiments under popular image classification benchmarks with different challenging continual learning settings, where L2P consistently outperforms prior state-of-the-art methods. Surprisingly, L2P achieves competitive results against rehearsal-based methods even without a rehearsal buffer and is directly applicable to challenging task-agnostic continual learning. Source code is available at https://github.com/google-research/l2p.

cs.LG↗

arXiv2022

PNM: Pixel Null Model for General Image Segmentation

A major challenge in image segmentation is classifying object boundaries. Recent efforts propose to refine the segmentation result with boundary masks. However, models are still prone to misclassifying boundary pixels even when they correctly capture the object contours. In such cases, even a perfect boundary map is unhelpful for segmentation refinement. In this paper, we argue that assigning proper prior weights to error-prone pixels such as object boundaries can significantly improve the segmentation quality. Specifically, we present the \textit{pixel null model} (PNM), a prior model that weights each pixel according to its probability of being correctly classified by a random segmenter. Empirical analysis shows that PNM captures the misclassification distribution of different state-of-the-art (SOTA) segmenters. Extensive experiments on semantic, instance, and panoptic segmentation tasks over three datasets (Cityscapes, ADE20K, MS COCO) confirm that PNM consistently improves the segmentation quality of most SOTA methods (including the vision transformers) and outperforms boundary-based methods by a large margin. We also observe that the widely-used mean IoU (mIoU) metric is insensitive to boundaries of different sharpness. As a byproduct, we propose a new metric, \textit{PNM IoU}, which perceives the boundary sharpness and better reflects the model segmentation performance in error-prone regions.

cs.CV↗

arXiv2022

Givens Coordinate Descent Methods for Rotation Matrix Learning in Trainable Embedding Indexes

Product quantization (PQ) coupled with a space rotation, is widely used in modern approximate nearest neighbor (ANN) search systems to significantly compress the disk storage for embeddings and speed up the inner product computation. Existing rotation learning methods, however, minimize quantization distortion for fixed embeddings, which are not applicable to an end-to-end training scenario where embeddings are updated constantly. In this paper, based on geometric intuitions from Lie group theory, in particular the special orthogonal group $SO(n)$, we propose a family of block Givens coordinate descent algorithms to learn rotation matrix that are provably convergent on any convex objectives. Compared to the state-of-the-art SVD method, the Givens algorithms are much more parallelizable, reducing runtime by orders of magnitude on modern GPUs, and converge more stably according to experimental studies. They further improve upon vanilla product quantization significantly in an end-to-end training scenario.

cs.IR↗

arXiv2022

Spherical accretion of collisional gas in modified gravity I: self-similar solutions and a new cosmological hydrodynamical code

The spherical collapse scenario has great importance in cosmology since it captures several crucial aspects of structure formation. The presence of self-similar solutions in the Einstein-de Sitter (EdS) model greatly simplifies its analysis, making it a powerful tool to gain valuable insights into the real and more complicated physical processes involved in galaxy formation. While there has been a large body of research to incorporate various additional physical processes into spherical collapse, the effect of modified gravity (MG) models, which are popular alternatives to the $ΛCDM$ paradigm to explain the cosmic acceleration, is still not well understood in this scenario. In this paper, we study the spherical accretion of collisional gas in a particular MG model, which is a rare case that also admits self-similar solutions. The model displays interesting behaviours caused by the enhanced gravity and a screening mechanism. Despite the strong effects of MG, we find that its self-similar solution agrees well with that of the EdS model. These results are used to assess a new cosmological hydrodynamical code for spherical collapse simulations introduced here, which is based on the hyperbolic partial differential equation engine ExaHyPE 2. Its good agreement with the theoretical predictions confirms the reliability of this code in modelling astrophysical processes in spherical collapse. We will use this code to study the evolution of gas in more realistic MG models in future work.

astro-ph.CO↗

arXiv2022

Topology-Preserving Segmentation Network: A Deep Learning Segmentation Framework for Connected Component

Medical image segmentation, which aims to automatically extract anatomical or pathological structures, plays a key role in computer-aided diagnosis and disease analysis. Despite the problem has been widely studied, existing methods are prone to topological errors. In medical imaging, the topology of the structure, such as the kidney or lung, is usually known. Preserving the topology of the structure in the segmentation process is of utmost importance for accurate image analysis. In this work, a novel learning-based segmentation model is proposed. A {\it topology-preserving segmentation network (TPSN)} is trained to give an accurate segmentation result of an input image that preserves the prescribed topology. TPSN is a deformation-based model that yields a deformation map through a UNet, which takes the medical image and a template mask as inputs. The main idea is to deform a template mask describing the prescribed topology by a diffeomorphism to segment the object in the image. The topology of the shape in the template mask is well preserved under the diffeomorphic map. The diffeomorphic property of the map is controlled by introducing a regularization term related to the Jacobian in the loss function. As such, a topology-preserving segmentation result can be guaranteed. Furthermore, a multi-scale TPSN is developed in this paper that incorporates multi-level information of images to produce more precise segmentation results. To evaluate our method, we applied the 2D TPSN on Ham10000 and 3D TPSN on KiTS21. Experimental results illustrate our method outperforms the baseline UNet segmentation model with/without connected-component analysis (CCA) by both the dice score and IoU score. Besides, results show that our method can produce reliable results even in challenging cases, where pixel-wise segmentation models by UNet and CCA fail to obtain accurate results.

eess.IV↗

arXiv2022

StyleBERT: Chinese pretraining by font style information

With the success of down streaming task using English pre-trained language model, the pre-trained Chinese language model is also necessary to get a better performance of Chinese NLP task. Unlike the English language, Chinese has its special characters such as glyph information. So in this article, we propose the Chinese pre-trained language model StyleBERT which incorporate the following embedding information to enhance the savvy of language model, such as word, pinyin, five stroke and chaizi. The experiments show that the model achieves well performances on a wide range of Chinese NLP tasks.

cs.CL↗

arXiv2022

MaskGIT: Masked Generative Image Transformer

Generative transformers have experienced rapid popularity growth in the computer vision community in synthesizing high-fidelity and high-resolution images. The best generative transformer models so far, however, still treat an image naively as a sequence of tokens, and decode an image sequentially following the raster scan ordering (i.e. line-by-line). We find this strategy neither optimal nor efficient. This paper proposes a novel image synthesis paradigm using a bidirectional transformer decoder, which we term MaskGIT. During training, MaskGIT learns to predict randomly masked tokens by attending to tokens in all directions. At inference time, the model begins with generating all tokens of an image simultaneously, and then refines the image iteratively conditioned on the previous generation. Our experiments demonstrate that MaskGIT significantly outperforms the state-of-the-art transformer model on the ImageNet dataset, and accelerates autoregressive decoding by up to 64x. Besides, we illustrate that MaskGIT can be easily extended to various image editing tasks, such as inpainting, extrapolation, and image manipulation.

cs.CV↗

arXiv2022

Best of both worlds: Synergistically derived material properties via additive manufacturing of nanocomposites

With an exponential rise in the popularity and availability of additive manufacturing (AM), a large focus has been directed toward research in this topic's movement, while trying to distinguish themselves from similar works by simply adding nanomaterials to their process. Though nanomaterials can add impressive properties to nanocomposites (NCs), there are expansive amounts of opportunities that are left unexplored by simply combining AM with NCs without discovering synergistic effects and novel emerging material properties that are not possible by each of these alone. Cooperative, evolving properties of NCs in AM can be investigated at the processing, morphological, and architectural levels. Each of these categories are studied as a function of the amplifying relationship between nanomaterials and AM, with each showing the systematically selected material and method to advance the material performance, explore emergent properties, as well as improve the AM process itself. Innovative, advanced materials are key to faster development cycles in disruptive technologies for bioengineering, defense, and transportation sectors. This is only possible by focusing on synergism and amplification within additive manufacturing of nanocomposites.

cond-mat.mtrl-sci↗

arXiv2022

Team Learning-Based Resource Allocation for Open Radio Access Network (O-RAN)

Recently, the concept of open radio access network (O-RAN) has been proposed, which aims to adopt intelligence and openness in the next generation radio access networks (RAN). It provides standardized interfaces and the ability to host network applications from third-party vendors by x-applications (xAPPs), which enables higher flexibility for network management. However, this may lead to conflicts in network function implementations, especially when these functions are implemented by different vendors. In this paper, we aim to mitigate the conflicts between xAPPs for near-real-time (near-RT) radio intelligent controller (RIC) of O-RAN. In particular, we propose a team learning algorithm to enhance the performance of the network by increasing cooperation between xAPPs. We compare the team learning approach with independent deep Q-learning where network functions individually optimize resources. Our simulations show that team learning has better network performance under various user mobility and traffic loads. With 6 Mbps traffic load and 20 m/s user movement speed, team learning achieves 8% higher throughput and 64.8% lower PDR.

cs.NI↗

arXiv2021

ERNIE-ViLG: Unified Generative Pre-training for Bidirectional Vision-Language Generation

Conventional methods for the image-text generation tasks mainly tackle the naturally bidirectional generation tasks separately, focusing on designing task-specific frameworks to improve the quality and fidelity of the generated samples. Recently, Vision-Language Pre-training models have greatly improved the performance of the image-to-text generation tasks, but large-scale pre-training models for text-to-image synthesis task are still under-developed. In this paper, we propose ERNIE-ViLG, a unified generative pre-training framework for bidirectional image-text generation with transformer model. Based on the image quantization models, we formulate both image generation and text generation as autoregressive generative tasks conditioned on the text/image input. The bidirectional image-text generative modeling eases the semantic alignments across vision and language. For the text-to-image generation process, we further propose an end-to-end training method to jointly learn the visual sequence generator and the image reconstructor. To explore the landscape of large-scale pre-training for bidirectional text-image generation, we train a 10-billion parameter ERNIE-ViLG model on a large-scale dataset of 145 million (Chinese) image-text pairs which achieves state-of-the-art performance for both text-to-image and image-to-text tasks, obtaining an FID of 7.9 on MS-COCO for text-to-image synthesis and best results on COCO-CN and AIC-ICC for image captioning.

cs.CV↗

arXiv2021

Nested Hierarchical Transformer: Towards Accurate, Data-Efficient and Interpretable Visual Understanding

Hierarchical structures are popular in recent vision transformers, however, they require sophisticated designs and massive datasets to work well. In this paper, we explore the idea of nesting basic local transformers on non-overlapping image blocks and aggregating them in a hierarchical way. We find that the block aggregation function plays a critical role in enabling cross-block non-local information communication. This observation leads us to design a simplified architecture that requires minor code changes upon the original vision transformer. The benefits of the proposed judiciously-selected design are threefold: (1) NesT converges faster and requires much less training data to achieve good generalization on both ImageNet and small datasets like CIFAR; (2) when extending our key ideas to image generation, NesT leads to a strong decoder that is 8$\times$ faster than previous transformer-based generators; and (3) we show that decoupling the feature learning and abstraction processes via this nested hierarchy in our design enables constructing a novel method (named GradCAT) for visually interpreting the learned model. Source code is available https://github.com/google-research/nested-transformer.

cs.CV↗

arXiv2021

Improved Transformer for High-Resolution GANs

Attention-based models, exemplified by the Transformer, can effectively model long range dependency, but suffer from the quadratic complexity of self-attention operation, making them difficult to be adopted for high-resolution image generation based on Generative Adversarial Networks (GANs). In this paper, we introduce two key ingredients to Transformer to address this challenge. First, in low-resolution stages of the generative process, standard global self-attention is replaced with the proposed multi-axis blocked self-attention which allows efficient mixing of local and global attention. Second, in high-resolution stages, we drop self-attention while only keeping multi-layer perceptrons reminiscent of the implicit neural function. To further improve the performance, we introduce an additional self-modulation component based on cross-attention. The resulting model, denoted as HiT, has a nearly linear computational complexity with respect to the image size and thus directly scales to synthesizing high definition images. We show in the experiments that the proposed HiT achieves state-of-the-art FID scores of 30.83 and 2.95 on unconditional ImageNet $128 \times 128$ and FFHQ $256 \times 256$, respectively, with a reasonable throughput. We believe the proposed HiT is an important milestone for generators in GANs which are completely free of convolutions. Our code is made publicly available at https://github.com/google-research/hit-gan

cs.CV↗

arXiv2021

Temporal Transformer Networks with Self-Supervision for Action Recognition

In recent years, 2D Convolutional Networks-based video action recognition has encouragingly gained wide popularity; However, constrained by the lack of long-range non-linear temporal relation modeling and reverse motion information modeling, the performance of existing models is, therefore, undercut seriously. To address this urgent problem, we introduce a startling Temporal Transformer Network with Self-supervision (TTSN). Our high-performance TTSN mainly consists of a temporal transformer module and a temporal sequence self-supervision module. Concisely speaking, we utilize the efficient temporal transformer module to model the non-linear temporal dependencies among non-local frames, which significantly enhances complex motion feature representations. The temporal sequence self-supervision module we employ unprecedentedly adopts the streamlined strategy of "random batch random channel" to reverse the sequence of video frames, allowing robust extractions of motion information representation from inversed temporal dimensions and improving the generalization capability of the model. Extensive experiments on three widely used datasets (HMDB51, UCF101, and Something-something V1) have conclusively demonstrated that our proposed TTSN is promising as it successfully achieves state-of-the-art performance for action recognition.

cs.CV↗

arXiv2021

Degradation Mechanism of Perovskite under High Charge Carrier Density Condition

Extensive studies have focused on degradation of perovskite at low charge carrier density (<10^16 cm^-3), but few have surveyed the degradation mechanism at high charge carrier density (~10^18 cm^-3). Here, we investigate the degradation mechanisms of perovskite under high charge carrier conditions. Unlike the observations in previous works, we find that MAPbI3 degradation starts at surface defects and progressing from the surface defects towards neighboring regions under high charge carrier density condition. By using PbI2 passivation, the defect-initiated degradation is significantly suppressed and the nanoplatelet degrades in a layer-by-layer way, enabling the MAPbI3 laser sustain for 4500 s (2.7*10^7 pulses), which is almost 3 times longer than that of the nanoplatelet laser without passivation. Meanwhile, the PbI2 passivated MAPbI3 nanoplatelet laser with the nanoplatelet cavity displaying a maximum quality factor up to ~7800, the highest reported for all MAPbI3 nanoplatelet cavities. Furthermore, a high stability MAPbI3 nanoplatelet laser that can last for 8500 s (5.1*10^7 pulses) is demonstrated based on a dual passivation strategy, by retarding the defect-initiated degradation and surface-initiated degradation, simultaneously. This work provides in-depth insights for understanding the degradation of perovskite at high charge carrier density.

physics.optics↗

arXiv2021

GloDyNE: Global Topology Preserving Dynamic Network Embedding

Learning low-dimensional topological representation of a network in dynamic environments is attracting much attention due to the time-evolving nature of many real-world networks. The main and common objective of Dynamic Network Embedding (DNE) is to efficiently update node embeddings while preserving network topology at each time step. The idea of most existing DNE methods is to capture the topological changes at or around the most affected nodes (instead of all nodes) and accordingly update node embeddings. Unfortunately, this kind of approximation, although can improve efficiency, cannot effectively preserve the global topology of a dynamic network at each time step, due to not considering the inactive sub-networks that receive accumulated topological changes propagated via the high-order proximity. To tackle this challenge, we propose a novel node selecting strategy to diversely select the representative nodes over a network, which is coordinated with a new incremental learning paradigm of Skip-Gram based embedding approach. The extensive experiments show GloDyNE, with a small fraction of nodes being selected, can already achieve the superior or comparable performance w.r.t. the state-of-the-art DNE methods in three typical downstream tasks. Particularly, GloDyNE significantly outperforms other methods in the graph reconstruction task, which demonstrates its ability of global topology preservation. The source code is available at https://github.com/houchengbin/GloDyNE

cs.SI↗

arXiv2021

Nilpotent Category of Abelian Category and Self-Adjoint Functors

Let $\mathcal{C}$ be an additive category. The nilpotent category $\mathrm{Nil} (\mathcal{C})$ of $\mathcal{C}$, consists of objects pairs $(X, x)$ with $X\in\mathcal{C}, x\in\mathrm{End}_{\mathcal{C}}(X)$ such that $x^n=0$ for some positive integer $n$, and a morphism $f:(X, x)\rightarrow (Y,y)$ is $f\in \mathrm{Hom}_{\mathcal{C}}(X, Y)$ satisfying $fx=yf$. A general theory of $\mathrm{Nil}(\mathcal{C})$ is established and it is abelian in the case that $\mathcal{C}$ is abelian. Two abelian categories are equivalent if and only if their nilpotent categories are equivalent, which generalizes a Song, Wu, and Zhang's result. As an application, it is proved all self-adjoint functors are naturally isomorphic to $\mathrm{Hom}$ and $\mathrm{Tensor}$ functors over the category $\mathrm{Nil}$ of finite-dimensional vector spaces. Both $\mathrm{Hom}$ and $\mathrm{Tensor}$ can be naturally generalized to $\mathrm{HOM}$ and $\mathrm{Tensor}$ functor over $\mathrm{Nil}(\mathcal{V})$. They are still self-adjoint, but intrinsically different.

math.CT↗

arXiv2021

threaTrace: Detecting and Tracing Host-based Threats in Node Level Through Provenance Graph Learning

Host-based threats such as Program Attack, Malware Implantation, and Advanced Persistent Threats (APT), are commonly adopted by modern attackers. Recent studies propose leveraging the rich contextual information in data provenance to detect threats in a host. Data provenance is a directed acyclic graph constructed from system audit data. Nodes in a provenance graph represent system entities (e.g., $processes$ and $files$) and edges represent system calls in the direction of information flow. However, previous studies, which extract features of the whole provenance graph, are not sensitive to the small number of threat-related entities and thus result in low performance when hunting stealthy threats. We present threaTrace, an anomaly-based detector that detects host-based threats at system entity level without prior knowledge of attack patterns. We tailor GraphSAGE, an inductive graph neural network, to learn every benign entity's role in a provenance graph. threaTrace is a real-time system, which is scalable of monitoring a long-term running host and capable of detecting host-based intrusion in their early phase. We evaluate threaTrace on three public datasets. The results show that threaTrace outperforms three state-of-the-art host intrusion detection systems.

cs.CR↗

arXiv2021

Locally unipotent invariant measures and limit distribution of a sequence of polynomial trajectories on homogeneous spaces

Let $G$ be a Lie group and $Γ$ be a lattice in $G$. We introduce the notion of locally unipotent invariant measures on $G/Γ$. We then prove that under some conditions, the limit measure supported on the image of polynomial trajectories on $G/Γ$ is locally unipotent invariant, thus give a partial answer to an equidistribution problem for higher dimensional polynomial trajectories on homogeneous spaces, which was raised by Shah in \cite{shah1994limit}. The proof relies on Ratner's measure classification theorem, linearization technique for polynomial trajectories near singular sets and a twisting technique of Shah.

math.DS↗