arXiv subjects

Ping Li

Publications and source records attributed to Ping Li.

At least 163 records · Page 9Linked to original sources

arXiv2023

Fully Transformer-Equipped Architecture for End-to-End Referring Video Object Segmentation

Referring Video Object Segmentation (RVOS) requires segmenting the object in video referred by a natural language query. Existing methods mainly rely on sophisticated pipelines to tackle such cross-modal task, and do not explicitly model the object-level spatial context which plays an important role in locating the referred object. Therefore, we propose an end-to-end RVOS framework completely built upon transformers, termed \textit{Fully Transformer-Equipped Architecture} (FTEA), which treats the RVOS task as a mask sequence learning problem and regards all the objects in video as candidate objects. Given a video clip with a text query, the visual-textual features are yielded by encoder, while the corresponding pixel-level and word-level features are aligned in terms of semantic similarity. To capture the object-level spatial context, we have developed the Stacked Transformer, which individually characterizes the visual appearance of each candidate object, whose feature map is decoded to the binary mask sequence in order directly. Finally, the model finds the best matching between mask sequence and text query. In addition, to diversify the generated masks for candidate objects, we impose a diversity loss on the model for capturing more accurate mask of the referred object. Empirical studies have shown the superiority of the proposed method on three benchmarks, e.g., FETA achieves 45.1% and 38.7% in terms of mAP on A2D Sentences (3782 videos) and J-HMDB Sentences (928 videos), respectively; it achieves 56.6% in terms of $\mathcal{J\&F}$ on Ref-YouTube-VOS (3975 videos and 7451 objects). Particularly, compared to the best candidate method, it has a gain of 2.1% and 3.2% in terms of P$@$0.5 on the former two, respectively, while it has a gain of 2.9% in terms of $\mathcal{J}$ on the latter one.

cs.CV↗

arXiv2023

Efficient Long-Short Temporal Attention Network for Unsupervised Video Object Segmentation

Unsupervised Video Object Segmentation (VOS) aims at identifying the contours of primary foreground objects in videos without any prior knowledge. However, previous methods do not fully use spatial-temporal context and fail to tackle this challenging task in real-time. This motivates us to develop an efficient Long-Short Temporal Attention network (termed LSTA) for unsupervised VOS task from a holistic view. Specifically, LSTA consists of two dominant modules, i.e., Long Temporal Memory and Short Temporal Attention. The former captures the long-term global pixel relations of the past frames and the current frame, which models constantly present objects by encoding appearance pattern. Meanwhile, the latter reveals the short-term local pixel relations of one nearby frame and the current frame, which models moving objects by encoding motion pattern. To speedup the inference, the efficient projection and the locality-based sliding window are adopted to achieve nearly linear time complexity for the two light modules, respectively. Extensive empirical studies on several benchmarks have demonstrated promising performances of the proposed method with high efficiency.

cs.CV↗

arXiv2023

Towards High-Quality Specular Highlight Removal by Leveraging Large-Scale Synthetic Data

This paper aims to remove specular highlights from a single object-level image. Although previous methods have made some progresses, their performance remains somewhat limited, particularly for real images with complex specular highlights. To this end, we propose a three-stage network to address them. Specifically, given an input image, we first decompose it into the albedo, shading, and specular residue components to estimate a coarse specular-free image. Then, we further refine the coarse result to alleviate its visual artifacts such as color distortion. Finally, we adjust the tone of the refined result to match that of the input as closely as possible. In addition, to facilitate network training and quantitative evaluation, we present a large-scale synthetic dataset of object-level images, covering diverse objects and illumination conditions. Extensive experiments illustrate that our network is able to generalize well to unseen real object-level images, and even produce good results for scene-level images with multiple background objects and complex lighting.

cs.CV↗

arXiv2023

Transition metal single-atom anchored on MoSi2N4 monolayer as highly efficient electrocatalyst for hydrogen evolution reaction

Single-atom catalysts are considered as a promising method for efficient energy conversion, owing to their advantages of high atom utilization and low catalyst cost. However, finding a stable two-dimensional structure and high hydrogen evolution reaction (HER) performance is a current research hotspot. Herein, based on the first-principles calculations, we identify the HER properties of six catalysts (TM@MoSi2N4, TM = Sc, Ti, V, Fe, Co, and Ni) comprising transition metal atoms anchored on MoSi2N4 monolayer. The results show that the spin-polarized states appear around the Fermi level after anchoring TM atoms. Therefore, the energy level of the first available unoccupied state for accommodating hydrogen drops, regulating the bonding strength of hydrogen. Thus, the single transition metal atom activates the active site of the MoSi2N4 inert base plane, becoming a quite suitable site for the HER. Based on ΔGH*, the exchange current density and volcano diagram of the corresponding catalytic system were also calculated. Among them, V@MoSi2N4 (ΔGH* = -0.07 eV) and Ni@MoSi2N4 (ΔGH* = 0.06 eV) systems show efficient the HER property. Our study confirms that the transition metal atom anchoring is an effective means to improve the performance of electrocatalysis, and TM@MoSi2N4 has practical application potential as a high efficiency HER electrocatalyst.

cond-mat.mtrl-sci↗

arXiv2023

Empirical Risk Minimization for Losses without Variance

This paper considers an empirical risk minimization problem under heavy-tailed settings, where data does not have finite variance, but only has $p$-th moment with $p \in (1,2)$. Instead of using estimation procedure based on truncated observed data, we choose the optimizer by minimizing the risk value. Those risk values can be robustly estimated via using the remarkable Catoni's method (Catoni, 2012). Thanks to the structure of Catoni-type influence functions, we are able to establish excess risk upper bounds via using generalized generic chaining methods. Moreover, we take computational issues into consideration. We especially theoretically investigate two types of optimization methods, robust gradient descent algorithm and empirical risk-based methods. With an extensive numerical study, we find that the optimizer based on empirical risks via Catoni-style estimation indeed shows better performance than other baselines. It indicates that estimation directly based on truncated data may lead to unsatisfactory results.

stat.ML↗

arXiv2023

Progress on two-dimensional ferrovalley materials

The electron's charge and spin degrees of freedom are at the core of modern electronic devices. With the in-depth investigation of two-dimensional materials, another degree of freedom, valley, has also attracted tremendous research interest. The intrinsic spontaneous valley polarization in two-dimensional magnetic systems, ferrovalley material, provides convenience for detecting and modulating the valley. In this review, we first introduce the development of valleytronics. Then, the valley polarization forms by the p, d, and f-orbit that are discussed. Following, we discuss the investigation progress of modulating the valley polarization of two-dimensional ferrovalley materials by multiple physical fields, such as electric, stacking mode, strain, and interface. Finally, we look forward to the future developments of valleytronics.

cond-mat.mtrl-sci↗

arXiv2023

Nanopore-patterned CuSe drives the realization of PbSe-CuSe lateral heterostructure

Monolayer PbSe has been predicted to be a two-dimensional (2D) topological crystalline insulator (TCI) with crystalline symmetry-protected Dirac-cone-like edge states. Recently, few-layered epitaxial PbSe has been grown on the SrTiO3 substrate successfully, but the corresponding signature of the TCI was only observed for films not thinner than seven monolayers, largely due to interfacial strain. Here, we demonstrate a two-step method based on molecular beam epitaxy for the growth of the PbSe-CuSe lateral heterostructure on the Cu(111) substrate, in which we observe a nanopore patterned CuSe layer that acts as the template for lateral epitaxial growth of PbSe. This further results in a monolayer PbSe-CuSe lateral heterostructure with an atomically sharp interface. Scanning tunneling microscopy and spectroscopy measurements reveal a four-fold symmetric square lattice of such monolayer PbSe with a quasi-particle band gap of 1.8 eV, a value highly comparable with the theoretical value of freestanding PbSe. The weak monolayer-substrate interaction is further supported by both density functional theory (DFT) and projected crystal orbital Hamilton population, with the former predicting the monolayer's anti-bond state to reside below the Fermi level. Our work demonstrates a practical strategy to fabricate a high-quality in-plane heterostructure, involving a monolayer TCI, which is viable for further exploration of the topology-derived quantum physics and phenomena in the monolayer limit.

cond-mat.mtrl-sci↗

arXiv2023

Two-dimensional germanium islands with Dirac signature on Ag2Ge surface alloy

Two-dimensional (2D) Dirac materials have attracted intense research efforts due to their promise for applications ranging from field-effect transistors and low-power electronics to fault-tolerant quantum computation. One key challenge is to fabricate 2D Dirac materials hosting Dirac electrons. Here, monolayer germanene is successfully fabricated on a Ag2Ge surface alloy. Scanning tunneling spectroscopy measurements revealed a linear energy dispersion relation. The latter was supported by density functional theory calculations. These results demonstrate that monolayer germanene can be realistically fabricated on a Ag2Ge surface alloy. The finding opens the door to exploration and study of 2D Dirac material physics and device applications.

cond-mat.mtrl-sci↗

arXiv2023

Native Pb vacancy defects induced p-type characteristic in epitaxial monolayer PbSe

PbSe, a predicted two-dimensional (2D) topological crystalline insulator (TCI) in the monolayer limit, possess excellent thermoelectric and infrared optical properties. Native defects in PbSe take a crucial role for the applications. However, little attention has been paid to the defect induced doping characteristics. Here, we provide an experimental and theoretical investigation of defects induced p-type characteristic on epitaxial monolayer PbSe on Au(111). Scanning tunneling microscopy (STM) measurements demonstrate an epitaxial PbSe monolayer with a fourfold symmetric lattice. Combined scanning tunneling spectroscopy (STS) and density functional theory (DFT) calculations reveal a quasi-particle bandgap of 0.8eV of PbSe. STM results unveil that there are two types of defects on the surface, one is related the vacancies of Pb atoms and the other is the replacement of the absent Se atoms by Pb. Corresponding theoretical optimization confirms the structures of the defects. More importantly, both STS measurements and DFT calculations give evidence that the Pb vacancies move the Fermi energy inside the valence band and produce extra holes, leading to p-type characteristics of PbSe. Our work provides effective information for the future research of device performance based on PbSe films.

cond-mat.mtrl-sci↗

arXiv2023

Rethinking Client Drift in Federated Learning: A Logit Perspective

Federated Learning (FL) enables multiple clients to collaboratively learn in a distributed way, allowing for privacy protection. However, the real-world non-IID data will lead to client drift which degrades the performance of FL. Interestingly, we find that the difference in logits between the local and global models increases as the model is continuously updated, thus seriously deteriorating FL performance. This is mainly due to catastrophic forgetting caused by data heterogeneity between clients. To alleviate this problem, we propose a new algorithm, named FedCSD, a Class prototype Similarity Distillation in a federated framework to align the local and global models. FedCSD does not simply transfer global knowledge to local clients, as an undertrained global model cannot provide reliable knowledge, i.e., class similarity information, and its wrong soft labels will mislead the optimization of local models. Concretely, FedCSD introduces a class prototype similarity distillation to align the local logits with the refined global logits that are weighted by the similarity between local logits and the global prototype. To enhance the quality of global logits, FedCSD adopts an adaptive mask to filter out the terrible soft labels of the global models, thereby preventing them to mislead local optimization. Extensive experiments demonstrate the superiority of our method over the state-of-the-art federated learning approaches in various heterogeneous settings. The source code will be released.

cs.LG↗

arXiv2023

Copula for Instance-wise Feature Selection and Ranking

Instance-wise feature selection and ranking methods can achieve a good selection of task-friendly features for each sample in the context of neural networks. However, existing approaches that assume feature subsets to be independent are imperfect when considering the dependency between features. To address this limitation, we propose to incorporate the Gaussian copula, a powerful mathematical technique for capturing correlations between variables, into the current feature selection framework with no additional changes needed. Experimental results on both synthetic and real datasets, in terms of performance comparison and interpretability, demonstrate that our method is capable of capturing meaningful correlations.

cs.LG↗

arXiv2023

Decay estimates for fourth-order Schrödinger operators in dimension two

In this paper we study the decay estimates of the fourth order Schrödinger operator $H=Δ^{2}+V(x)$ on $\mathbb{R}^2$ with a bounded decaying potential $V(x)$. We first deduce the asymptotic expansions of resolvent of $H$ near the zero threshold in the presence of resonances or eigenvalue, and then use them to establish the $L^1-L^\infty$ decay estimates of $e^{-itH}$generated by the fourth order Schrödinger operator $H$. Our methods used in the decay estimates depend on Littlewood-Paley decomposition and oscillatory integral theory. Moreover, we classify these zero resonances as the distributional solutions of $Hϕ=0$ in suitable weighted spaces. Due to the degeneracy of $Δ^{2}$ at zero threshold and the lower even dimension (i.e. $n=2$), we remark that the asymptotic expansions of resolvent $R_V(λ^4)$ and the classifications of resonances are more involved than Schrödinger operator $-Δ+V$ in dimension two.

math.AP↗

arXiv2023

Improving the Transferability of Adversarial Attacks on Face Recognition with Beneficial Perturbation Feature Augmentation

Face recognition (FR) models can be easily fooled by adversarial examples, which are crafted by adding imperceptible perturbations on benign face images. The existence of adversarial face examples poses a great threat to the security of society. In order to build a more sustainable digital nation, in this paper, we improve the transferability of adversarial face examples to expose more blind spots of existing FR models. Though generating hard samples has shown its effectiveness in improving the generalization of models in training tasks, the effectiveness of utilizing this idea to improve the transferability of adversarial face examples remains unexplored. To this end, based on the property of hard samples and the symmetry between training tasks and adversarial attack tasks, we propose the concept of hard models, which have similar effects as hard samples for adversarial attack tasks. Utilizing the concept of hard models, we propose a novel attack method called Beneficial Perturbation Feature Augmentation Attack (BPFA), which reduces the overfitting of adversarial examples to surrogate FR models by constantly generating new hard models to craft the adversarial examples. Specifically, in the backpropagation, BPFA records the gradients on pre-selected feature maps and uses the gradient on the input image to craft the adversarial example. In the next forward propagation, BPFA leverages the recorded gradients to add beneficial perturbations on their corresponding feature maps to increase the loss. Extensive experiments demonstrate that BPFA can significantly boost the transferability of adversarial attacks on FR.

cs.CV↗

arXiv2023

CVSformer: Cross-View Synthesis Transformer for Semantic Scene Completion

Semantic scene completion (SSC) requires an accurate understanding of the geometric and semantic relationships between the objects in the 3D scene for reasoning the occluded objects. The popular SSC methods voxelize the 3D objects, allowing the deep 3D convolutional network (3D CNN) to learn the object relationships from the complex scenes. However, the current networks lack the controllable kernels to model the object relationship across multiple views, where appropriate views provide the relevant information for suggesting the existence of the occluded objects. In this paper, we propose Cross-View Synthesis Transformer (CVSformer), which consists of Multi-View Feature Synthesis and Cross-View Transformer for learning cross-view object relationships. In the multi-view feature synthesis, we use a set of 3D convolutional kernels rotated differently to compute the multi-view features for each voxel. In the cross-view transformer, we employ the cross-view fusion to comprehensively learn the cross-view relationships, which form useful information for enhancing the features of individual views. We use the enhanced features to predict the geometric occupancies and semantic labels of all voxels. We evaluate CVSformer on public datasets, where CVSformer yields state-of-the-art results.

cs.CV↗

arXiv2023

Blockwise Feature Interaction in Recommendation Systems

Feature interactions can play a crucial role in recommendation systems as they capture complex relationships between user preferences and item characteristics. Existing methods such as Deep & Cross Network (DCNv2) may suffer from high computational requirements due to their cross-layer operations. In this paper, we propose a novel approach called blockwise feature interaction (BFI) to help alleviate this issue. By partitioning the feature interaction process into smaller blocks, we can significantly reduce both the memory footprint and the computational burden. Four variants (denoted by P, Q, T, S, respectively) of BFI have been developed and empirically compared. Our experimental results demonstrate that the proposed algorithms achieves close accuracy compared to the standard DCNv2, while greatly reducing the computational overhead and the number of parameters. This paper contributes to the development of efficient recommendation systems by providing a practical solution for improving feature interaction efficiency.

cs.IR↗

arXiv2023

Pb-Hash: Partitioned b-bit Hashing

Many hashing algorithms including minwise hashing (MinHash), one permutation hashing (OPH), and consistent weighted sampling (CWS) generate integers of $B$ bits. With $k$ hashes for each data vector, the storage would be $B\times k$ bits; and when used for large-scale learning, the model size would be $2^B\times k$, which can be expensive. A standard strategy is to use only the lowest $b$ bits out of the $B$ bits and somewhat increase $k$, the number of hashes. In this study, we propose to re-use the hashes by partitioning the $B$ bits into $m$ chunks, e.g., $b\times m =B$. Correspondingly, the model size becomes $m\times 2^b \times k$, which can be substantially smaller than the original $2^B\times k$. Our theoretical analysis reveals that by partitioning the hash values into $m$ chunks, the accuracy would drop. In other words, using $m$ chunks of $B/m$ bits would not be as accurate as directly using $B$ bits. This is due to the correlation from re-using the same hash. On the other hand, our analysis also shows that the accuracy would not drop much for (e.g.,) $m=2\sim 4$. In some regions, Pb-Hash still works well even for $m$ much larger than 4. We expect Pb-Hash would be a good addition to the family of hashing methods/applications and benefit industrial practitioners. We verify the effectiveness of Pb-Hash in machine learning tasks, for linear SVM models as well as deep learning models. Since the hashed data are essentially categorical (ID) features, we follow the standard practice of using embedding tables for each hash. With Pb-Hash, we need to design an effective strategy to combine $m$ embeddings. Our study provides an empirical evaluation on four pooling schemes: concatenation, max pooling, mean pooling, and product pooling. There is no definite answer which pooling would be always better and we leave that for future study.

cs.LG↗

arXiv2023

An Energy-Based Prior for Generative Saliency

We propose a novel generative saliency prediction framework that adopts an informative energy-based model as a prior distribution. The energy-based prior model is defined on the latent space of a saliency generator network that generates the saliency map based on a continuous latent variables and an observed image. Both the parameters of saliency generator and the energy-based prior are jointly trained via Markov chain Monte Carlo-based maximum likelihood estimation, in which the sampling from the intractable posterior and prior distributions of the latent variables are performed by Langevin dynamics. With the generative saliency model, we can obtain a pixel-wise uncertainty map from an image, indicating model confidence in the saliency prediction. Different from existing generative models, which define the prior distribution of the latent variables as a simple isotropic Gaussian distribution, our model uses an energy-based informative prior which can be more expressive in capturing the latent space of the data. With the informative energy-based prior, we extend the Gaussian distribution assumption of generative models to achieve a more representative distribution of the latent space, leading to more reliable uncertainty estimation. We apply the proposed frameworks to both RGB and RGB-D salient object detection tasks with both transformer and convolutional neural network backbones. We further propose an adversarial learning algorithm and a variational inference algorithm as alternatives to train the proposed generative framework. Experimental results show that our generative saliency model with an energy-based prior can achieve not only accurate saliency predictions but also reliable uncertainty maps that are consistent with human perception. Results and code are available at \url{https://github.com/JingZhang617/EBMGSOD}.

cs.CV↗

arXiv2023

A Semi-Autoregressive Graph Generative Model for Dependency Graph Parsing

Recent years have witnessed the impressive progress in Neural Dependency Parsing. According to the different factorization approaches to the graph joint probabilities, existing parsers can be roughly divided into autoregressive and non-autoregressive patterns. The former means that the graph should be factorized into multiple sequentially dependent components, then it can be built up component by component. And the latter assumes these components to be independent so that they can be outputted in a one-shot manner. However, when treating the directed edge as an explicit dependency relationship, we discover that there is a mixture of independent and interdependent components in the dependency graph, signifying that both aforementioned models fail to precisely capture the explicit dependencies among nodes and edges. Based on this property, we design a Semi-Autoregressive Dependency Parser to generate dependency graphs via adding node groups and edge groups autoregressively while pouring out all group elements in parallel. The model gains a trade-off between non-autoregression and autoregression, which respectively suffer from the lack of target inter-dependencies and the uncertainty of graph generation orders. The experiments show the proposed parser outperforms strong baselines on Enhanced Universal Dependencies of multiple languages, especially achieving $4\%$ average promotion at graph-level accuracy. Also, the performances of model variations show the importance of specific parts.

cs.CL↗