arXiv subjects

Rui Feng

Publications and source records attributed to Rui Feng.

90 records · Page 5Linked to original sources

arXiv2022

Pyramid Region-based Slot Attention Network for Temporal Action Proposal Generation

It has been found that temporal action proposal generation, which aims to discover the temporal action instances within the range of the start and end frames in the untrimmed videos, can largely benefit from proper temporal and semantic context exploitation. The latest efforts were dedicated to considering the temporal context and similarity-based semantic contexts through self-attention modules. However, they still suffer from cluttered background information and limited contextual feature learning. In this paper, we propose a novel Pyramid Region-based Slot Attention (PRSlot) module to address these issues. Instead of using the similarity computation, our PRSlot module directly learns the local relations in an encoder-decoder manner and generates the representation of a local region enhanced based on the attention over input features called \textit{slot}. Specifically, upon the input snippet-level features, PRSlot module takes the target snippet as \textit{query}, its surrounding region as \textit{key} and then generates slot representations for each \textit{query-key} slot by aggregating the local snippet context with a parallel pyramid strategy. Based on PRSlot modules, we present a novel Pyramid Region-based Slot Attention Network termed PRSA-Net to learn a unified visual representation with rich temporal and semantic context for better proposal generation. Extensive experiments are conducted on two widely adopted THUMOS14 and ActivityNet-1.3 benchmarks. Our PRSA-Net outperforms other state-of-the-art methods. In particular, we improve the AR@100 from the previous best 50.67% to 56.12% for proposal generation and raise the mAP under 0.5 tIoU from 51.9\% to 58.7\% for action detection on THUMOS14. \textit{Code is available at} \url{https://github.com/handhand123/PRSA-Net}

cs.CV↗

arXiv2022

Reconfigurable intelligent surfaces: Channel characterization and modeling

Reconfigurable intelligent surfaces (RISs) are two dimensional (2D) metasurfaces which can intelligently manipulate electromagnetic waves by low-cost near passive reflecting elements. RIS is viewed as a potential key technology for the sixth generation (6G) wireless communication systems mainly due to its advantages in tuning wireless signals, thus smartly controlling propagation environments. In this paper, we aim at addressing channel characterization and modeling issues of RIS-assisted wireless communication systems. At first, the concept, principle, and potential applications of RIS are given. An overview of RIS based channel measurements and experiments is presented by classifying frequency bands, scenarios, system configurations, RIS constructions, experiment purposes, and channel observations. Then, RIS based channel characteristics are studied, including reflection and transmission, Doppler effect and multipath fading mitigation, channel reciprocity, channel hardening, rank improvement, far field and near field, etc. RIS based channel modeling works are investigated, including largescale path loss models and small-scale multipath fading models. At last, future research directions related to RIS-assisted channels are also discussed.

eess.SP↗

arXiv2022

CREAM: Weakly Supervised Object Localization via Class RE-Activation Mapping

Weakly Supervised Object Localization (WSOL) aims to localize objects with image-level supervision. Existing works mainly rely on Class Activation Mapping (CAM) derived from a classification model. However, CAM-based methods usually focus on the most discriminative parts of an object (i.e., incomplete localization problem). In this paper, we empirically prove that this problem is associated with the mixup of the activation values between less discriminative foreground regions and the background. To address it, we propose Class RE-Activation Mapping (CREAM), a novel clustering-based approach to boost the activation values of the integral object regions. To this end, we introduce class-specific foreground and background context embeddings as cluster centroids. A CAM-guided momentum preservation strategy is developed to learn the context embeddings during training. At the inference stage, the re-activation mapping is formulated as a parameter estimation problem under Gaussian Mixture Model, which can be solved by deriving an unsupervised Expectation-Maximization based soft-clustering algorithm. By simply integrating CREAM into various WSOL approaches, our method significantly improves their performance. CREAM achieves the state-of-the-art performance on CUB, ILSVRC and OpenImages benchmark datasets. Code will be available at https://github.com/Jazzcharles/CREAM.

cs.CV↗

arXiv2022

Self-Supervised Video Representation Learning with Motion-Contrastive Perception

Visual-only self-supervised learning has achieved significant improvement in video representation learning. Existing related methods encourage models to learn video representations by utilizing contrastive learning or designing specific pretext tasks. However, some models are likely to focus on the background, which is unimportant for learning video representations. To alleviate this problem, we propose a new view called long-range residual frame to obtain more motion-specific information. Based on this, we propose the Motion-Contrastive Perception Network (MCPNet), which consists of two branches, namely, Motion Information Perception (MIP) and Contrastive Instance Perception (CIP), to learn generic video representations by focusing on the changing areas in videos. Specifically, the MIP branch aims to learn fine-grained motion features, and the CIP branch performs contrastive learning to learn overall semantics information for each instance. Experiments on two benchmark datasets UCF-101 and HMDB-51 show that our method outperforms current state-of-the-art visual-only self-supervised approaches.

cs.CV↗

arXiv2022

CERES: Pretraining of Graph-Conditioned Transformer for Semi-Structured Session Data

User sessions empower many search and recommendation tasks on a daily basis. Such session data are semi-structured, which encode heterogeneous relations between queries and products, and each item is described by the unstructured text. Despite recent advances in self-supervised learning for text or graphs, there lack of self-supervised learning models that can effectively capture both intra-item semantics and inter-item interactions for semi-structured sessions. To fill this gap, we propose CERES, a graph-based transformer model for semi-structured session data. CERES learns representations that capture both inter- and intra-item semantics with (1) a graph-conditioned masked language pretraining task that jointly learns from item text and item-item relations; and (2) a graph-conditioned transformer architecture that propagates inter-item contexts to item-level representations. We pretrained CERES using ~468 million Amazon sessions and find that CERES outperforms strong pretraining baselines by up to 9% in three session search and entity linking tasks.

cs.IR↗

arXiv2021

Simple and Robust Loss Design for Multi-Label Learning with Missing Labels

Multi-label learning in the presence of missing labels (MLML) is a challenging problem. Existing methods mainly focus on the design of network structures or training schemes, which increase the complexity of implementation. This work seeks to fulfill the potential of loss function in MLML without increasing the procedure and complexity. Toward this end, we propose two simple yet effective methods via robust loss design based on an observation that a model can identify missing labels during training with a high precision. The first is a novel robust loss for negatives, namely the Hill loss, which re-weights negatives in the shape of a hill to alleviate the effect of false negatives. The second is a self-paced loss correction (SPLC) method, which uses a loss derived from the maximum likelihood criterion under an approximate distribution of missing labels. Comprehensive experiments on a vast range of multi-label image classification datasets demonstrate that our methods can remarkably boost the performance of MLML and achieve new state-of-the-art loss functions in MLML.

cs.LG↗

arXiv2021

A Modular Design of Continuously Tunable Full Color Plasmonic Pixels with Broken Rotational Symmetry

Color tuning is a fascinating and indispensable property in applications such as advanced display, active camouflaging and information encryption. Thus far, a variety of reconfigurable approaches have been implemented to achieve color change. However, it is still a challenge to enable a continuous color tuning over the entire hue range in a simple, stable and rapid manner without changes in configuration and material properties. Here, we demonstrate an all-optical continuously tunable plasmonic pixel scheme via a modular design approach to realize polarization-controlled full color tuning by breaking the intrinsic symmetry of the unit cell layout. The polarization-controlled full color tunable plasmonic pixels consist of three different types of color modules oriented at an angle of 60° with respect to each other, corresponding to three subtractive primary colors. Without changing the structural properties or surrounding environment, the structural colors can be continuously and precisely tuned across all hues by illuminating linearly polarized light with different polarization directions. Meanwhile, the plasmonic pixels can be flexibly customized for various color tuning processes, such as different initial output colors and color tuning sequences, through the appropriate choice of component modules and the elaborate design of module layouts. Furthermore, we extend the color tuning to achromatic colors, white or black, with the utilization of a single module or the introduction of a black module. The proposed polarization-controlled full color tunable plasmonic pixels hold considerable potential to function as next-generation color pixels integrated with liquid-crystal polarizers.

physics.optics↗

arXiv2021

MPN: Multimodal Parallel Network for Audio-Visual Event Localization

Audio-visual event localization aims to localize an event that is both audible and visible in the wild, which is a widespread audio-visual scene analysis task for unconstrained videos. To address this task, we propose a Multimodal Parallel Network (MPN), which can perceive global semantics and unmixed local information parallelly. Specifically, our MPN framework consists of a classification subnetwork to predict event categories and a localization subnetwork to predict event boundaries. The classification subnetwork is constructed by the Multimodal Co-attention Module (MCM) and obtains global contexts. The localization subnetwork consists of Multimodal Bottleneck Attention Module (MBAM), which is designed to extract fine-grained segment-level contents. Extensive experiments demonstrate that our framework achieves the state-of-the-art performance both in fully supervised and weakly supervised settings on the Audio-Visual Event (AVE) dataset.

cs.CV↗

arXiv2020

Probing and Fine-tuning Reading Comprehension Models for Few-shot Event Extraction

We study the problem of event extraction from text data, which requires both detecting target event types and their arguments. Typically, both the event detection and argument detection subtasks are formulated as supervised sequence labeling problems. We argue that the event extraction models so trained are inherently label-hungry, and can generalize poorly across domains and text genres.We propose a reading comprehension framework for event extraction.Specifically, we formulate event detection as a textual entailment prediction problem, and argument detection as a question answer-ing problem. By constructing proper query templates, our approach can effectively distill rich knowledge about tasks and label semantics from pretrained reading comprehension models. Moreover, our model can be fine-tuned with a small amount of data to boost its performance. Our experiment results show that our method performs strongly for zero-shot and few-shot event extraction, and it achieves state-of-the-art performance on the ACE 2005 benchmark when trained with full supervision.

cs.CL↗

arXiv2020

Transformer-Based Neural Text Generation with Syntactic Guidance

We study the problem of using (partial) constituency parse trees as syntactic guidance for controlled text generation. Existing approaches to this problem use recurrent structures, which not only suffer from the long-term dependency problem but also falls short in modeling the tree structure of the syntactic guidance. We propose to leverage the parallelism of Transformer to better incorporate parse trees. Our method first expands a partial template constituency parse tree to a full-fledged parse tree tailored for the input source text, and then uses the expanded tree to guide text generation. The effectiveness of our model in this process hinges upon two new attention mechanisms: 1) a path attention mechanism that forces one node to attend to only other nodes located in its path in the syntax tree to better incorporate syntax guidance; 2) a multi-encoder attention mechanism that allows the decoder to dynamically attend to information from multiple encoders. Our experiments in the controlled paraphrasing task show that our method outperforms SOTA models both semantically and syntactically, improving the best baseline's BLEU score from 11.83 to 26.27.

cs.CL↗

arXiv2020

Edge Dislocations Can Control Yield Strength in Refractory Body-Centered-Cubic High Entropy Alloys

Energy efficiency is motivating the search for new high-temperature metals. Some new body-centered-cubic random multicomponent "high entropy alloys (HEAs)" based on refractory elements (Cr-Mo-Nb-Ta-V-W-Hf-Ti-Zr) possess exceptional strengths at high temperatures but the physical origins of this outstanding behavior are not known. Here we show, using integrated neutron-diffraction (ND), high-resolution transmission electron microscopy (HRTEM), and theory, that the high strength and strength retention of a NbTaVTi alloy and a new high-strength/low-density CrMoNbV alloy are attributable to edge dislocations. This is surprising because plastic-flow in BCC elemental metals and dilute alloys is universally accepted to be controlled by screw dislocations. We use the insight and theory to perform a computationally-guided search over $10^7$ BCC HEAs and identify over $10^6$ possible ultra-strong high-temperature alloy compositions for future exploration.

cond-mat.mtrl-sci↗

arXiv2020

Look, Listen, and Attend: Co-Attention Network for Self-Supervised Audio-Visual Representation Learning

When watching videos, the occurrence of a visual event is often accompanied by an audio event, e.g., the voice of lip motion, the music of playing instruments. There is an underlying correlation between audio and visual events, which can be utilized as free supervised information to train a neural network by solving the pretext task of audio-visual synchronization. In this paper, we propose a novel self-supervised framework with co-attention mechanism to learn generic cross-modal representations from unlabelled videos in the wild, and further benefit downstream tasks. Specifically, we explore three different co-attention modules to focus on discriminative visual regions correlated to the sounds and introduce the interactions between them. Experiments show that our model achieves state-of-the-art performance on the pretext task while having fewer parameters compared with existing methods. To further evaluate the generalizability and transferability of our approach, we apply the pre-trained model on two downstream tasks, i.e., sound source localization and action recognition. Extensive experiments demonstrate that our model provides competitive results with other self-supervised methods, and also indicate that our approach can tackle the challenging scenes which contain multiple sound sources.

cs.MM↗

arXiv2020

Learning Error-Driven Curriculum for Crowd Counting

Density regression has been widely employed in crowd counting. However, the frequency imbalance of pixel values in the density map is still an obstacle to improve the performance. In this paper, we propose a novel learning strategy for learning error-driven curriculum, which uses an additional network to supervise the training of the main network. A tutoring network called TutorNet is proposed to repetitively indicate the critical errors of the main network. TutorNet generates pixel-level weights to formulate the curriculum for the main network during training, so that the main network will assign a higher weight to those hard examples than easy examples. Furthermore, we scale the density map by a factor to enlarge the distance among inter-examples, which is well known to improve the performance. Extensive experiments on two challenging benchmark datasets show that our method has achieved state-of-the-art performance.

cs.CV↗

arXiv2019

Learning Fair Representations via an Adversarial Framework

Fairness has become a central issue for our research community as classification algorithms are adopted in societally critical domains such as recidivism prediction and loan approval. In this work, we consider the potential bias based on protected attributes (e.g., race and gender), and tackle this problem by learning latent representations of individuals that are statistically indistinguishable between protected groups while sufficiently preserving other information for classification. To do that, we develop a minimax adversarial framework with a generator to capture the data distribution and generate latent representations, and a critic to ensure that the distributions across different protected groups are similar. Our framework provides a theoretical guarantee with respect to statistical parity and individual fairness. Empirical results on four real-world datasets also show that the learned representation can effectively be used for classification tasks such as credit risk prediction while obstructing information related to protected groups, especially when removing protected attributes is not sufficient for fair classification.

cs.LG↗

arXiv2017

Representation Learning for Scale-free Networks

Network embedding aims to learn the low-dimensional representations of vertexes in a network, while structure and inherent properties of the network is preserved. Existing network embedding works primarily focus on preserving the microscopic structure, such as the first- and second-order proximity of vertexes, while the macroscopic scale-free property is largely ignored. Scale-free property depicts the fact that vertex degrees follow a heavy-tailed distribution (i.e., only a few vertexes have high degrees) and is a critical property of real-world networks, such as social networks. In this paper, we study the problem of learning representations for scale-free networks. We first theoretically analyze the difficulty of embedding and reconstructing a scale-free network in the Euclidean space, by converting our problem to the sphere packing problem. Then, we propose the "degree penalty" principle for designing scale-free property preserving network embedding algorithm: punishing the proximity between high-degree vertexes. We introduce two implementations of our principle by utilizing the spectral techniques and a skip-gram model respectively. Extensive experiments on six datasets show that our algorithms are able to not only reconstruct heavy-tailed distributed degree distribution, but also outperform state-of-the-art embedding models in various network mining tasks, such as vertex classification and link prediction.

cs.SI↗

arXiv2017

First-principles prediction of high-entropy-alloy stability

High entropy alloys (HEAs) are multicomponent compounds whose high configurational entropy allows them to solidify into a single phase, with a simple crystal lattice structure. Some HEA's exhibit desirable properties, such as high specific strength, ductility, and corrosion resistance, while challenging the scientist to make confident predictions in the face of multiple competing phases. We demonstrate phase stability in the multicomponent alloy system of Cr-Mo-Nb-V, for which some of its binary subsystems are subject to phase separation and complex intermetallic-phase formation. Our first-principles calculation of free energy predicts that the configurational entropy stabilizes a single body-centered cubic (BCC) phase from T = 1,700K up to melting, while precipitation of a complex intermetallic is favored at lower temperatures. We form the compound experimentally and confirm that it forms as a single BCC phase from the melt, but that it transforms reversibly at lower temperatures.

cond-mat.mtrl-sci↗

arXiv2014

Regularization Methods for High-Dimensional Instrumental Variables Regression With an Application to Genetical Genomics

In genetical genomics studies, it is important to jointly analyze gene expression data and genetic variants in exploring their associations with complex traits, where the dimensionality of gene expressions and genetic variants can both be much larger than the sample size. Motivated by such modern applications, we consider the problem of variable selection and estimation in high-dimensional sparse instrumental variables models. To overcome the difficulty of high dimensionality and unknown optimal instruments, we propose a two-stage regularization framework for identifying and estimating important covariate effects while selecting and estimating optimal instruments. The methodology extends the classical two-stage least squares estimator to high dimensions by exploiting sparsity using sparsity-inducing penalty functions in both stages. The resulting procedure is efficiently implemented by coordinate descent optimization. For the representative $L_1$ regularization and a class of concave regularization methods, we establish estimation, prediction, and model selection properties of the two-stage regularized estimators in the high-dimensional setting where the dimensionality of covariates and instruments are both allowed to grow exponentially with the sample size. The practical performance of the proposed method is evaluated by simulation studies and its usefulness is illustrated by an analysis of mouse obesity data. Supplementary materials for this article are available online.

stat.ME↗

arXiv2004

Ultrathin epitaxial graphite: 2D electron gas properties and a route toward graphene-based nanoelectronics

We have produced ultrathin epitaxial graphite films which show remarkable 2D electron gas (2DEG) behavior. The films, composed of typically 3 graphene sheets, were grown by thermal decomposition on the (0001) surface of 6H-SiC, and characterized by surface-science techniques. The low-temperature conductance spans a range of localization regimes according to the structural state (square resistance 1.5 kOhm to 225 kOhm at 4 K, with positive magnetoconductance). Low resistance samples show characteristics of weak-localization in two dimensions, from which we estimate elastic and inelastic mean free paths. At low field, the Hall resistance is linear up to 4.5 T, which is well-explained by n-type carriers of density 10^{12} cm^{-2} per graphene sheet. The most highly-ordered sample exhibits Shubnikov - de Haas oscillations which correspond to nonlinearities observed in the Hall resistance, indicating a potential new quantum Hall system. We show that the high-mobility films can be patterned via conventional lithographic techniques, and we demonstrate modulation of the film conductance using a top-gate electrode. These key elements suggest electronic device applications based on nano-patterned epitaxial graphene (NPEG), with the potential for large-scale integration.

cond-mat.mtrl-sci↗