arXiv subjects

Kai Wang

Publications and source records attributed to Kai Wang.

At least 595 records · Page 33Linked to original sources

arXiv2022

Prompt Vision Transformer for Domain Generalization

Though vision transformers (ViTs) have exhibited impressive ability for representation learning, we empirically find that they cannot generalize well to unseen domains with previous domain generalization algorithms. In this paper, we propose a novel approach DoPrompt based on prompt learning to embed the knowledge of source domains in domain prompts for target domain prediction. Specifically, domain prompts are prepended before ViT input tokens from the corresponding source domain. Each domain prompt learns domain-specific knowledge efficiently since it is optimized only for one domain. Meanwhile, we train a prompt adapter to produce a suitable prompt for each input image based on the learned source domain prompts. At test time, the adapted prompt generated by the prompt adapter can exploit the similarity between the feature of the out-of-domain image and source domains to properly integrate the source domain knowledge. Extensive experiments are conducted on four benchmark datasets. Our approach achieves 1.4% improvements in the averaged accuracy, which is 3.5 times the improvement of the state-of-the-art algorithm with a ViT backbone.

cs.CV↗

arXiv2022

QuickSkill: Novice Skill Estimation in Online Multiplayer Games

Matchmaking systems are vital for creating fair matches in online multiplayer games, which directly affects players' satisfactions and game experience. Most of the matchmaking systems largely rely on precise estimation of players' game skills to construct equitable games. However, the skill rating of a novice is usually inaccurate, as current matchmaking rating algorithms require considerable amount of games for learning the true skill of a new player. Using these unreliable skill scores at early stages for matchmaking usually leads to disparities in terms of team performance, which causes negative game experience. This is known as the ''cold-start'' problem for matchmaking rating algorithms. To overcome this conundrum, this paper proposes QuickSKill, a deep learning based novice skill estimation framework to quickly probe abilities of new players in online multiplayer games. QuickSKill extracts sequential performance features from initial few games of a player to predict his/her future skill rating with a dedicated neural network, thus delivering accurate skill estimation at the player's early game stage. By employing QuickSKill for matchmaking, game fairness can be dramatically improved in the initial cold-start period. We conduct experiments in a popular mobile multiplayer game in both offline and online scenarios. Results obtained with two real-world anonymized gaming datasets demonstrate that proposed QuickSKill delivers precise estimation of game skills for novices, leading to significantly lower team skill disparities and better player game experience. To the best of our knowledge, proposed QuickSKill is the first framework that tackles the cold-start problem for traditional skill rating algorithms.

cs.LG↗

arXiv2022

Skeleton-based Action Recognition via Temporal-Channel Aggregation

Skeleton-based action recognition methods are limited by the semantic extraction of spatio-temporal skeletal maps. However, current methods have difficulty in effectively combining features from both temporal and spatial graph dimensions and tend to be thick on one side and thin on the other. In this paper, we propose a Temporal-Channel Aggregation Graph Convolutional Networks (TCA-GCN) to learn spatial and temporal topologies dynamically and efficiently aggregate topological features in different temporal and channel dimensions for skeleton-based action recognition. We use the Temporal Aggregation module to learn temporal dimensional features and the Channel Aggregation module to efficiently combine spatial dynamic channel-wise topological features with temporal dynamic topological features. In addition, we extract multi-scale skeletal features on temporal modeling and fuse them with an attention mechanism. Extensive experiments show that our model results outperform state-of-the-art methods on the NTU RGB+D, NTU RGB+D 120, and NW-UCLA datasets.

cs.CV↗

arXiv2022

Cohomology theory of Rota-Baxter pre-Lie algebras of arbitrary weights

This paper is devoted to studying deformation, cohomology theory of Rota-Baxter pre-Lie algebras of arbitrary weights. First we give the notion of a new representation of a Rota-Baxter pre-Lie algebra of arbitrary weight and define the cohomology theory of a Rota-Baxter pre-Lie algebra of arbitrary weight. Then we study formal deformations by lower degree cohomology groups. Finally, we classify abelian extensions of Rota-Baxter pre-Lie algebras of arbitrary weight using the second cohomology group, and classify skeletal Rota-Baxter pre-Lie 2-algebra of arbitrary weight using the third cohomology group as applications.

math.RA↗

arXiv2022

Investigation of variable temperature Mössbauer spectrum of YFe$_{0.5}$Cr$_{0.5}$O$_3$ perovskite

In this paper, we reported the preparation of YFe$_{0.5}$Cr$_{0.5}$O$_3$ by the sol-gel method and studied its structure and Mössbauer spectrum at variable temperatures. X-ray diffraction(XRD) analysis exhibits that the sample has the orthorhombic structure with the Pnma space group, and the energy dispersive spectroscopy (EDS) analysis shows that the sample has Fe/Cr = 1:1, indicating that the sample is Fe half-doped YCrO$_3$. The hyperfine parameters of the Mössbauer spectrum at room temperature confirm that the characteristics of 57Fe in the sample were trivalent hexacoordinated high-spin(s=5/2), and the coexistence of doublet and the sextets at 250K indicate that the sample has superparamagnetic relaxation. The Mössbauer spectrum records the magnetic phase transition in the temperature range of 250K-300K.

cond-mat.mtrl-sci↗

arXiv2022

The Shape Part Slot Machine: Contact-based Reasoning for Generating 3D Shapes from Parts

We present the Shape Part Slot Machine, a new method for assembling novel 3D shapes from existing parts by performing contact-based reasoning. Our method represents each shape as a graph of ``slots,'' where each slot is a region of contact between two shape parts. Based on this representation, we design a graph-neural-network-based model for generating new slot graphs and retrieving compatible parts, as well as a gradient-descent-based optimization scheme for assembling the retrieved parts into a complete shape that respects the generated slot graph. This approach does not require any semantic part labels; interestingly, it also does not require complete part geometries -- reasoning about the slots proves sufficient to generate novel, high-quality 3D shapes. We demonstrate that our method generates shapes that outperform existing modeling-by-assembly approaches regarding quality, diversity, and structural complexity.

cs.GR↗

arXiv2022

Reliable Label Correction is a Good Booster When Learning with Extremely Noisy Labels

Learning with noisy labels has aroused much research interest since data annotations, especially for large-scale datasets, may be inevitably imperfect. Recent approaches resort to a semi-supervised learning problem by dividing training samples into clean and noisy sets. This paradigm, however, is prone to significant degeneration under heavy label noise, as the number of clean samples is too small for conventional methods to behave well. In this paper, we introduce a novel framework, termed as LC-Booster, to explicitly tackle learning under extreme noise. The core idea of LC-Booster is to incorporate label correction into the sample selection, so that more purified samples, through the reliable label correction, can be utilized for training, thereby alleviating the confirmation bias. Experiments show that LC-Booster advances state-of-the-art results on several noisy-label benchmarks, including CIFAR-10, CIFAR-100, Clothing1M and WebVision. Remarkably, under the extreme 90\% noise ratio, LC-Booster achieves 92.9\% and 48.4\% accuracy on CIFAR-10 and CIFAR-100, surpassing state-of-the-art methods by a large margin.

cs.CV↗

arXiv2022

Learning MDPs from Features: Predict-Then-Optimize for Sequential Decision Problems by Reinforcement Learning

In the predict-then-optimize framework, the objective is to train a predictive model, mapping from environment features to parameters of an optimization problem, which maximizes decision quality when the optimization is subsequently solved. Recent work on decision-focused learning shows that embedding the optimization problem in the training pipeline can improve decision quality and help generalize better to unseen tasks compared to relying on an intermediate loss function for evaluating prediction quality. We study the predict-then-optimize framework in the context of sequential decision problems (formulated as MDPs) that are solved via reinforcement learning. In particular, we are given environment features and a set of trajectories from training MDPs, which we use to train a predictive model that generalizes to unseen test MDPs without trajectories. Two significant computational challenges arise in applying decision-focused learning to MDPs: (i) large state and action spaces make it infeasible for existing techniques to differentiate through MDP problems, and (ii) the high-dimensional policy space, as parameterized by a neural network, makes differentiating through a policy expensive. We resolve the first challenge by sampling provably unbiased derivatives to approximate and differentiate through optimality conditions, and the second challenge by using a low-rank approximation to the high-dimensional sample-based derivatives. We implement both Bellman--based and policy gradient--based decision-focused learning on three different MDP problems with missing parameters, and show that decision-focused learning performs better in generalization to unseen tasks.

cs.LG↗

arXiv2022

Deformations and Cohomology Theory of Rota-Baxter Systems

Inspired by the work of Wang and Zhou [4] for Rota-Baxter algebras, we develop a cohomology theory of Rota-Baxter systems and justify it by interpreting the lower degree cohomology groups as formal deformations and as abelian extensions of Rota-Baxter systems. A further study on an $L_\infty$-algebra structure associated to this cohomology theory will be given in a subsequent paper.

math.RA↗

arXiv2022

Detection of diffuse gamma-ray emission towards a massive star forming region hosting Wolf-Rayet stars

Isotopic and elemental abundances seen in Galactic cosmic rays imply that $\sim20\%$ of the cosmic-ray (CR) nuclei are probably synthesized by massive Wolf-Rayet (WR) stars. Massive star clusters hosting WR and OB-type stars have been proposed as potential Galactic cosmic-ray accelerators for decades, in particular via diffusive shock acceleration at wind termination shocks. Here we report the analysis of {\em Fermi} Large Area Telescope's data towards the direction of Masgomas-6a, a young massive star cluster candidate hosting two WR stars. We detect an extended $γ$-ray source with $\rm{TS}=183$ in the vicinity of Masgomas-6a, spatially coincident with two unassociated {\em Fermi} 4FGL sources. We also present the CO observational results of molecular clouds in this region, using the data from the Milky Way Imaging Scroll Painting project. The $γ$-ray emission intensity correlates well with the distribution of molecular gas at the distance of Masgomas-6a, indicating that these gamma rays may be produced by CRs accelerated by massive stars in Masgomas-6a. At the distance of $3.9{\rm \ kpc}$ of Masgomas-6a, the luminosity of the extended source is $(1.81\pm0.02)\times 10^{35}{\rm \ erg \ s^{-1}}$. With a kinetic luminosity of $\sim 10^{37}{\rm erg \ s^{-1}}$ in the stellar winds, the WR stars are capable of powering the $γ$-ray emission via neutral pion decay resulted from cosmic ray $pp$ interactions. The size of the GeV source and the energetic requirement suggests a CR diffusion coefficient smaller than that in the Galactic interstellar medium, indicating strong suppression of CR diffusion in the molecular cloud.

astro-ph.HE↗

arXiv2022

Monitoring AGNs with H$β$ Asymmetry. III. Long-term Reverberation Mapping Results of 15 Palomar-Green Quasars

In this third paper of the series reporting on the reverberation mapping (RM) campaign of active galactic nuclei with asymmetric H$β$ emission-line profiles, we present results for 15 Palomar-Green (PG) quasars using spectra obtained between the end of 2016 to May 2021. This campaign combines long time spans with relatively high cadence. For 8 objects, both the time lags obtained from the entire light curves and the measurements from individual observing seasons are provided. Reverberation mapping of 9 of our targets has been attempted for the first time, while the results for 6 others can be compared with previous campaigns. We measure the H$β$ time lags over periods of years and estimate their black hole masses. The long duration of the campaign enables us to investigate their broad line region (BLR) geometry and kinematics for different years by using velocity-resolved lags, which demonstrate signatures of diverse BLR geometry and kinematics. The BLR geometry and kinematics of individual objects are discussed. In this sample, the BLR kinematics of Keplerian/virialized motion and inflow is more common than outflow.

astro-ph.GA↗

arXiv2022

Background-penalty-free waveguide enhancement of CARS signal in air-filled anti-resonance hollow-core fiber

We study coherent anti-Stokes Raman spectroscopy in air-filled anti-resonance hollow-core photonic crystal fiber, otherwise known as 'revolver' fiber. We compare the vibrational coherent anti-Stokes Raman signal of N$_2$, at 2331 cm$^{-1}$, generated in ambient air (no fiber present), with the one generated in a 2.96 cm of a revolver fiber. We show a 170 times enhancement for the signal produced in the fiber, due to an increased interaction path. Remarkably, the N$_2$ signal obtained in the revolver fiber shows near-zero non-resonant background, due to near-zero overlap between the laser field and the fiber cladding. Through our study, we find that the revolver fiber properties make it an ideal candidate for the coherent Raman spectroscopy signal enhancement.

physics.optics↗

arXiv2022

Could TDE outflows produce the PeV neutrino events?

A tidal disruption event (TDE), AT2019dsg, was observed to be associated with a PeV neutrino event, IceCube-191001A, lagging the optical outburst by a half year. It is known that TDEs may generate ultrafast outflows. If the TDE occurs in a cloudy environment, the outflow-cloud interactions may form shock waves which generate accelerated protons and hence delayed neutrinos from hadronic interactions in clouds. Here we investigate the neutrino production in AT2019dsg by examining the TDE outflow-cloud interaction model. We find that, for an outflow with a velocity of 0.07c and a kinetic luminosity of $10^{45}\rm erg\ s^{-1}$, protons may be accelerated up to $\sim$ 60 PeV by the bow shocks, and generate PeV neutrinos by interactions with clouds. The predicted neutrino number in this model depends on the uncertainties of model parameters and in order to match the observations, some challenging values of parameters have been involved. The PeV neutrino event number can be $\sim 4\times10^{-3}$ for a hard proton index $Γ=1.5$.

astro-ph.HE↗

arXiv2022

Multi-label classification for biomedical literature: an overview of the BioCreative VII LitCovid Track for COVID-19 literature topic annotations

The COVID-19 pandemic has been severely impacting global society since December 2019. Massive research has been undertaken to understand the characteristics of the virus and design vaccines and drugs. The related findings have been reported in biomedical literature at a rate of about 10,000 articles on COVID-19 per month. Such rapid growth significantly challenges manual curation and interpretation. For instance, LitCovid is a literature database of COVID-19-related articles in PubMed, which has accumulated more than 200,000 articles with millions of accesses each month by users worldwide. One primary curation task is to assign up to eight topics (e.g., Diagnosis and Treatment) to the articles in LitCovid. Despite the continuing advances in biomedical text mining methods, few have been dedicated to topic annotations in COVID-19 literature. To close the gap, we organized the BioCreative LitCovid track to call for a community effort to tackle automated topic annotation for COVID-19 literature. The BioCreative LitCovid dataset, consisting of over 30,000 articles with manually reviewed topics, was created for training and testing. It is one of the largest multilabel classification datasets in biomedical scientific literature. 19 teams worldwide participated and made 80 submissions in total. Most teams used hybrid systems based on transformers. The highest performing submissions achieved 0.8875, 0.9181, and 0.9394 for macro F1-score, micro F1-score, and instance-based F1-score, respectively. The level of participation and results demonstrate a successful track and help close the gap between dataset curation and method development. The dataset is publicly available via https://ftp.ncbi.nlm.nih.gov/pub/lu/LitCovid/biocreative/ for benchmarking and further development.

cs.DL↗

arXiv2022

Progressive Multi-scale Consistent Network for Multi-class Fundus Lesion Segmentation

Effectively integrating multi-scale information is of considerable significance for the challenging multi-class segmentation of fundus lesions because different lesions vary significantly in scales and shapes. Several methods have been proposed to successfully handle the multi-scale object segmentation. However, two issues are not considered in previous studies. The first is the lack of interaction between adjacent feature levels, and this will lead to the deviation of high-level features from low-level features and the loss of detailed cues. The second is the conflict between the low-level and high-level features, this occurs because they learn different scales of features, thereby confusing the model and decreasing the accuracy of the final prediction. In this paper, we propose a progressive multi-scale consistent network (PMCNet) that integrates the proposed progressive feature fusion (PFF) block and dynamic attention block (DAB) to address the aforementioned issues. Specifically, PFF block progressively integrates multi-scale features from adjacent encoding layers, facilitating feature learning of each layer by aggregating fine-grained details and high-level semantics. As features at different scales should be consistent, DAB is designed to dynamically learn the attentive cues from the fused features at different scales, thus aiming to smooth the essential conflicts existing in multi-scale features. The two proposed PFF and DAB blocks can be integrated with the off-the-shelf backbone networks to address the two issues of multi-scale and feature inconsistency in the multi-class segmentation of fundus lesions, which will produce better feature representation in the feature space. Experimental results on three public datasets indicate that the proposed method is more effective than recent state-of-the-art methods.

eess.IV↗

arXiv2022

Divide to Adapt: Mitigating Confirmation Bias for Domain Adaptation of Black-Box Predictors

Domain Adaptation of Black-box Predictors (DABP) aims to learn a model on an unlabeled target domain supervised by a black-box predictor trained on a source domain. It does not require access to both the source-domain data and the predictor parameters, thus addressing the data privacy and portability issues of standard domain adaptation. Existing DABP approaches mostly rely on model distillation from the black-box predictor, \emph{i.e.}, training the model with its noisy target-domain predictions, which however inevitably introduces the confirmation bias accumulated from the prediction noises. To mitigate such bias, we propose a new method, named BETA, to incorporate knowledge distillation and noisy label learning into one coherent framework. This is enabled by a new divide-to-adapt strategy. BETA divides the target domain into an easy-to-adapt subdomain with less noise and a hard-to-adapt subdomain. Then it deploys mutually-teaching twin networks to filter the predictor errors for each other and improve them progressively, from the easy to hard subdomains. As such, BETA effectively purifies the noisy labels and reduces error accumulation. We theoretically show that the target error of BETA is minimized by decreasing the noise ratio of the subdomains. Extensive experiments demonstrate BETA outperforms existing methods on all DABP benchmarks, and is even comparable with the standard domain adaptation methods that use the source-domain data.

cs.LG↗

arXiv2022

Swift and Sure: Hardness-aware Contrastive Learning for Low-dimensional Knowledge Graph Embeddings

Knowledge graph embedding (KGE) has shown great potential in automatic knowledge graph (KG) completion and knowledge-driven tasks. However, recent KGE models suffer from high training cost and large storage space, thus limiting their practicality in real-world applications. To address this challenge, based on the latest findings in the field of Contrastive Learning, we propose a novel KGE training framework called Hardness-aware Low-dimensional Embedding (HaLE). Instead of the traditional Negative Sampling, we design a new loss function based on query sampling that can balance two important training targets, Alignment and Uniformity. Furthermore, we analyze the hardness-aware ability of recent low-dimensional hyperbolic models and propose a lightweight hardness-aware activation mechanism. The experimental results show that in the limited training time, HaLE can effectively improve the performance and training speed of KGE models on five commonly-used datasets. After training just a few minutes, the HaLE-trained models are competitive compared to the state-of-the-art models in both low- and high-dimensional conditions.

cs.LG↗

arXiv2022

FaceMAE: Privacy-Preserving Face Recognition via Masked Autoencoders

Face recognition, as one of the most successful applications in artificial intelligence, has been widely used in security, administration, advertising, and healthcare. However, the privacy issues of public face datasets have attracted increasing attention in recent years. Previous works simply mask most areas of faces or synthesize samples using generative models to construct privacy-preserving face datasets, which overlooks the trade-off between privacy protection and data utility. In this paper, we propose a novel framework FaceMAE, where the face privacy and recognition performance are considered simultaneously. Firstly, randomly masked face images are used to train the reconstruction module in FaceMAE. We tailor the instance relation matching (IRM) module to minimize the distribution gap between real faces and FaceMAE reconstructed ones. During the deployment phase, we use trained FaceMAE to reconstruct images from masked faces of unseen identities without extra training. The risk of privacy leakage is measured based on face retrieval between reconstructed and original datasets. Experiments prove that the identities of reconstructed images are difficult to be retrieved. We also perform sufficient privacy-preserving face recognition on several public face datasets (i.e. CASIA-WebFace and WebFace260M). Compared to previous state of the arts, FaceMAE consistently \textbf{reduces at least 50\% error rate} on LFW, CFP-FP and AgeDB.

cs.CV↗