arXiv subjects

Pei Wang

Publications and source records attributed to Pei Wang.

At least 181 records · Page 10Linked to original sources

arXiv2022

Generalized Intent Discovery: Learning from Open World Dialogue System

Traditional intent classification models are based on a pre-defined intent set and only recognize limited in-domain (IND) intent classes. But users may input out-of-domain (OOD) queries in a practical dialogue system. Such OOD queries can provide directions for future improvement. In this paper, we define a new task, Generalized Intent Discovery (GID), which aims to extend an IND intent classifier to an open-world intent set including IND and OOD intents. We hope to simultaneously classify a set of labeled IND intent classes while discovering and recognizing new unlabeled OOD types incrementally. We construct three public datasets for different application scenarios and propose two kinds of frameworks, pipeline-based and end-to-end for future work. Further, we conduct exhaustive experiments and qualitative analysis to comprehend key challenges and provide new guidance for future GID research.

cs.CL↗

arXiv2022

A Support Vector Machine Based Cure Rate Model For Interval Censored Data

The mixture cure rate model is the most commonly used cure rate model in the literature. In the context of mixture cure rate model, the standard approach to model the effect of covariates on the cured or uncured probability is to use a logistic function. This readily implies that the boundary classifying the cured and uncured subjects is linear. In this paper, we propose a new mixture cure rate model based on interval censored data that uses the support vector machine (SVM) to model the effect of covariates on the uncured or the cured probability (i.e., on the incidence part of the model). Our proposed model inherits the features of the SVM and provides flexibility to capture classification boundaries that are non-linear and more complex. Furthermore, the new model can be used to model the effect of covariates on the incidence part when the dimension of covariates is high. The latency part is modeled by a proportional hazards structure. We develop an estimation procedure based on the expectation maximization (EM) algorithm to estimate the cured/uncured probability and the latency model parameters. Our simulation study results show that the proposed model performs better in capturing complex classification boundaries when compared to the existing logistic regression based mixture cure rate model. We also show that our model's ability to capture complex classification boundaries improve the estimation results corresponding to the latency parameters. For illustrative purpose, we present our analysis by applying the proposed methodology to an interval censored data on smoking cessation.

stat.ME↗

arXiv2022

Radio detection of an elusive millisecond pulsar in the Globular Cluster NGC 6397

We report the discovery of a new 5.78 ms-period millisecond pulsar (MSP), PSR J1740-5340B (NGC 6397B), in an eclipsing binary system discovered with the Parkes radio telescope (now also known as Murriyang), Australia, and confirmed with the MeerKAT radio telescope in South Africa. The measured orbital period, 1.97 days, is the longest among all eclipsing binaries in globular clusters (GCs) and consistent with that of the coincident X-ray source U18, previously suggested to be a 'hidden MSP'. Our XMM-Newton observations during NGC 6397B's radio quiescent epochs detected no X-ray flares. NGC 6397B is either a transitional MSP or an eclipsing binary in its initial stage of mass transfer after the companion star left the main sequence. The discovery of NGC 6397B potentially reveals a subgroup of extremely faint and heavily obscured binary pulsars, thus providing a plausible explanation to the apparent dearth of binary neutron stars in core-collapsed GCs as well as a critical constraint on the evolution of GCs.

astro-ph.HE↗

arXiv2022

Discrete Probabilistic Inverse Optimal Transport

Optimal transport (OT) formalizes the problem of finding an optimal coupling between probability measures given a cost matrix. The inverse problem of inferring the cost given a coupling is Inverse Optimal Transport (IOT). IOT is less well understood than OT. We formalize and systematically analyze the properties of IOT using tools from the study of entropy-regularized OT. Theoretical contributions include characterization of the manifold of cross-ratio equivalent costs, the implications of model priors, and derivation of an MCMC sampler. Empirical contributions include visualizations of cross-ratio equivalent effect on basic examples and simulations validating theoretical results.

stat.ML↗

arXiv2022

Lightweight Cross-Lingual Sentence Representation Learning

Large-scale models for learning fixed-dimensional cross-lingual sentence representations like LASER (Artetxe and Schwenk, 2019b) lead to significant improvement in performance on downstream tasks. However, further increases and modifications based on such large-scale models are usually impractical due to memory limitations. In this work, we introduce a lightweight dual-transformer architecture with just 2 layers for generating memory-efficient cross-lingual sentence representations. We explore different training tasks and observe that current cross-lingual training tasks leave a lot to be desired for this shallow architecture. To ameliorate this, we propose a novel cross-lingual language model, which combines the existing single-word masked language model with the newly proposed cross-lingual token-level reconstruction task. We further augment the training task by the introduction of two computationally-lite sentence-level contrastive learning tasks to enhance the alignment of cross-lingual sentence representation space, which compensates for the learning bottleneck of the lightweight transformer for generative tasks. Our comparisons with competing models on cross-lingual sentence retrieval and multilingual document classification confirm the effectiveness of the newly proposed training tasks for a shallow model.

cs.CL↗

arXiv2022

Evolution of beliefs in social networks

Evolution of beliefs of a society are a product of interactions between people (horizontal transmission) in the society over generations (vertical transmission). Researchers have studied both horizontal and vertical transmission separately. Extending prior work, we propose a new theoretical framework which allows application of tools from Markov chain theory to the analysis of belief evolution via horizontal and vertical transmission. We analyze three cases: static network, randomly changing network, and homophily-based dynamic network. Whereas the former two assume network structure is independent of beliefs, the latter assumes that people tend to communicate with those who have similar beliefs. We prove under general conditions that both static and randomly changing networks converge to a single set of beliefs among all individuals along with the rate of convergence. We prove that homophily-based network structures do not in general converge to a single set of beliefs shared by all and prove lower bounds on the number of different limiting beliefs as a function of initial beliefs. We conclude by discussing implications for prior theories and directions for future work.

cs.LG↗

arXiv2022

Simulating high-time resolution radio-telescope observations

We describe a new software package for simulating channelised, high-time resolution data streams from radio telescopes. The software simulates data from the telescope and observing system taking into account the observation strategy, receiver system and digitisation. The signatures of pulsars, fast radio bursts and flare stars are modelled, including frequency-dependent effects such as scattering and scintillation. We also simulate more generic signals using spline curves and images. Models of radio frequency interference include signals from satellites, terrestrial transmitters and impulsive, broadband signals. The simulated signals can also be injected into real data sets. Uses of this software include the production of machine learning training data sets, development and testing of new algorithms to search for anomalous patterns and to characterise processing pipelines.

astro-ph.IM↗

arXiv2022

High-accuracy three-dimensional surface detection in smoothed particle hydrodynamics for free-surface flows

In this study, we investigate high-accuracy three-dimensional surface detection in smoothed particle hydrodynamics for free-surface flows. A new geometrical method is first developed to enhance the accuracy of free-surface particle detection in complex flows. This method detects free-surface particles via continuous global scanning inside the sphere of a particle through a cone region whose vertex corresponds to the particle position. The particle is identified as a free-surface particle if there exists a cone region with no neighboring particles. Next, an efficient semi-geometrical method is proposed based on the geometrical method to reduce the computational cost. It consists of finding particles near the free surface via position divergence and then detecting these particles using the geometrical method to identify free-surface particles. The accuracy and robustness of the proposed method are demonstrated by performing tests on several model problems.

physics.flu-dyn↗

arXiv2022

Learning Distinctive Margin toward Active Domain Adaptation

Despite plenty of efforts focusing on improving the domain adaptation ability (DA) under unsupervised or few-shot semi-supervised settings, recently the solution of active learning started to attract more attention due to its suitability in transferring model in a more practical way with limited annotation resource on target data. Nevertheless, most active learning methods are not inherently designed to handle domain gap between data distribution, on the other hand, some active domain adaptation methods (ADA) usually requires complicated query functions, which is vulnerable to overfitting. In this work, we propose a concise but effective ADA method called Select-by-Distinctive-Margin (SDM), which consists of a maximum margin loss and a margin sampling algorithm for data selection. We provide theoretical analysis to show that SDM works like a Support Vector Machine, storing hard examples around decision boundaries and exploiting them to find informative and transferable data. In addition, we propose two variants of our method, one is designed to adaptively adjust the gradient from margin loss, the other boosts the selectivity of margin sampling by taking the gradient direction into account. We benchmark SDM with standard active learning setting, demonstrating our algorithm achieves competitive results with good data scalability. Code is available at https://github.com/TencentYoutuResearch/ActiveLearning-SDM

cs.LG↗

arXiv2022

Omni-DETR: Omni-Supervised Object Detection with Transformers

We consider the problem of omni-supervised object detection, which can use unlabeled, fully labeled and weakly labeled annotations, such as image tags, counts, points, etc., for object detection. This is enabled by a unified architecture, Omni-DETR, based on the recent progress on student-teacher framework and end-to-end transformer based object detection. Under this unified architecture, different types of weak labels can be leveraged to generate accurate pseudo labels, by a bipartite matching based filtering mechanism, for the model to learn. In the experiments, Omni-DETR has achieved state-of-the-art results on multiple datasets and settings. And we have found that weak annotations can help to improve detection performance and a mixture of them can achieve a better trade-off between annotation cost and accuracy than the standard complete annotation. These findings could encourage larger object detection datasets with mixture annotations. The code is available at https://github.com/amazon-research/omni-detr.

cs.CV↗

arXiv2022

Neurosymbolic hybrid approach to driver collision warning

There are two main algorithmic approaches to autonomous driving systems: (1) An end-to-end system in which a single deep neural network learns to map sensory input directly into appropriate warning and driving responses. (2) A mediated hybrid recognition system in which a system is created by combining independent modules that detect each semantic feature. While some researchers believe that deep learning can solve any problem, others believe that a more engineered and symbolic approach is needed to cope with complex environments with less data. Deep learning alone has achieved state-of-the-art results in many areas, from complex gameplay to predicting protein structures. In particular, in image classification and recognition, deep learning models have achieved accuracies as high as humans. But sometimes it can be very difficult to debug if the deep learning model doesn't work. Deep learning models can be vulnerable and are very sensitive to changes in data distribution. Generalization can be problematic. It's usually hard to prove why it works or doesn't. Deep learning models can also be vulnerable to adversarial attacks. Here, we combine deep learning-based object recognition and tracking with an adaptive neurosymbolic network agent, called the Non-Axiomatic Reasoning System (NARS), that can adapt to its environment by building concepts based on perceptual sequences. We achieved an improved intersection-over-union (IOU) object recognition performance of 0.65 in the adaptive retraining model compared to IOU 0.31 in the COCO data pre-trained model. We improved the object detection limits using RADAR sensors in a simulated environment, and demonstrated the weaving car detection capability by combining deep learning-based object detection and tracking with a neurosymbolic model.

cs.CV↗

arXiv2022

Frequency-dependent polarization of repeating fast radio bursts-implications for their origin

The polarization of fast radio bursts (FRBs), bright astronomical transients, contains crucial information about their environments. We report polarization measurements of five repeating FRBs, the abundant signals of which enable wide-band observations with two telescopes. A clear trend of lower polarization at lower frequencies was found, which can be well characterized by a single parameter rotation-measure-scatter (σRM) and modeled by multi-path scatter. Sources with higher σRM have higher RM magnitude and scattering timescales. The two sources with the most substantial σRM, FRB 20121102A and FRB 20190520B, are associated with a compact persistent radio source. These properties indicate a complex environment near the repeating FRBs, such as a supernova remnant or a pulsar wind nebula, consistent with their arising from young populations.

astro-ph.HE↗

arXiv2021

Neurosymbolic Systems of Perception & Cognition: The Role of Attention

A cognitive architecture aimed at cumulative learning must provide the necessary information and control structures to allow agents to learn incrementally and autonomously from their experience. This involves managing an agent's goals as well as continuously relating sensory information to these in its perception-cognition information stack. The more varied the environment of a learning agent is, the more general and flexible must be these mechanisms to handle a wider variety of relevant patterns, tasks, and goal structures. While many researchers agree that information at different levels of abstraction likely differs in its makeup and structure and processing mechanisms, agreement on the particulars of such differences is not generally shared in the research community. A binary processing architecture (often referred to as System-1 and System-2) has been proposed as a model of cognitive processing for low- and high-level information, respectively. We posit that cognition is not binary in this way and that knowledge at any level of abstraction involves what we refer to as neurosymbolic information, meaning that data at both high and low levels must contain both symbolic and subsymbolic information. Further, we argue that the main differentiating factor between the processing of high and low levels of data abstraction can be largely attributed to the nature of the involved attention mechanisms. We describe the key arguments behind this view and review relevant evidence from the literature.

cs.AI↗

arXiv2021

Dynamic Parameterized Network for CTR Prediction

Learning to capture feature relations effectively and efficiently is essential in click-through rate (CTR) prediction of modern recommendation systems. Most existing CTR prediction methods model such relations either through tedious manually-designed low-order interactions or through inflexible and inefficient high-order interactions, which both require extra DNN modules for implicit interaction modeling. In this paper, we proposed a novel plug-in operation, Dynamic Parameterized Operation (DPO), to learn both explicit and implicit interaction instance-wisely. We showed that the introduction of DPO into DNN modules and Attention modules can respectively benefit two main tasks in CTR prediction, enhancing the adaptiveness of feature-based modeling and improving user behavior modeling with the instance-wise locality. Our Dynamic Parameterized Networks significantly outperforms state-of-the-art methods in the offline experiments on the public dataset and real-world production dataset, together with an online A/B test. Furthermore, the proposed Dynamic Parameterized Networks has been deployed in the ranking system of one of the world's largest e-commerce companies, serving the main traffic of hundreds of millions of active users.

cs.IR↗

arXiv2021

DeepAg: Deep Learning Approach for Measuring the Effects of Outlier Events on Agricultural Production and Policy

Quantitative metrics that measure the global economy's equilibrium have strong and interdependent relationships with the agricultural supply chain and international trade flows. Sudden shocks in these processes caused by outlier events such as trade wars, pandemics, or weather can have complex effects on the global economy. In this paper, we propose a novel framework, namely: DeepAg, that employs econometrics and measures the effects of outlier events detection using Deep Learning (DL) to determine relationships between commonplace financial indices (such as the DowJones), and the production values of agricultural commodities (such as Cheese and Milk). We employed a DL technique called Long Short-Term Memory (LSTM) networks successfully to predict commodity production with high accuracy and also present five popular models (regression and boosting) as baselines to measure the effects of outlier events. The results indicate that DeepAg with outliers' considerations (using Isolation Forests) outperforms baseline models, as well as the same model without outliers detection. Outlier events make a considerable impact when predicting commodity production with respect to financial indices. Moreover, we present the implications of DeepAg on public policy, provide insights for policymakers and farmers, and for operational decisions in the agricultural ecosystem. Data are collected, models developed, and the results are recorded and presented.

cs.LG↗

arXiv2021

A GPU based single-pulse search pipeline (GSP) with database and its application to the commensal radio astronomy FAST survey (CRAFTS)

We developed a GPU based single-pulse search pipeline (GSP) with candidate-archiving database. Largely based upon the infrastructure of Open source pulsar search and analysis toolkit (PRESTO), GSP implements GPU acceleration of the de-dispersion and integrates a candidate-archiving database. We applied GSP to the data streams from the commensal radio astronomy FAST survey (CRAFTS), which resulted in a quasi-real-time processing. The integrated candidate database facilitates synergistic usage of multiple machine-learning tools and thus improves efficient identification of radio pulsars such as rotating radio transients (RRATs) and Fast Radio Bursts (FRBs). We first tested GSP on pilot CRAFTS observations with the FAST Ultra-Wide Band (UWB) receiver. GSP detected all pulsars known from the the Parkes multibeam pulsar survey in the respective sky area covered by the FAST-UWB. GSP also discovered 13 new pulsars. We measured the computational efficiency of GSP to be ~120 times faster than the original PRESTO and ~60 times faster than a MPI-parallelized version of PRESTO.

astro-ph.IM↗

arXiv2021

An Eclipsing Black Widow Pulsar in NGC 6712

We report the discovery of the first radio pulsar associated with NGC 6712, an eclipsing black widow (BW) pulsar, J1853$-$0842A, found by high-sensitivity searches using the Five-hundred-meter Aperture Spherical radio Telescope. This 2.15 ms pulsar is in a 3.56 hr compact circular orbit with a very low mass companion likely of mass 0.018 to 0.036 $M_{\rm \odot}$ and exhibits eclipsing of the pulsar signal. Though the distance to PSR J1853$-$0842A predicted from its dispersion measure ($155.125 \pm 0.004$ cm$^{-3}$ pc) and Galactic free electron density models are about 30\% smaller than that of NGC 6712 obtained from interstellar reddening measurements, this is likely due to limited knowledge about the spiral arms and Scutum stellar cloud in this direction. Follow-up timing observations spanning 445 days allow us to localize the pulsar's position to be 0.14 core radii from the center of NGC 6712 and measure a negative spin-down rate for this pulsar of $-2.39(2)\times10^{-21}\rm s s^{-1}$. The latter cannot be explained without the acceleration of the GC and decisively supports the association between PSR J1853--0842A and NGC 6712. Considering the maximum GC acceleration, Galactic acceleration, and Shklovskii effect, we place an upper limit on the intrinsic spin-down rate to be $1.11\times10^{-20}\rm~s~s^{-1}$. From an analysis of the eclipsing observations, we estimate the electron density of the eclipse region to be about $1.88\times10^6\rm cm^{-3}$. We also place an upper limit of the accretion rate from the companion is about $3.05\times10^{-13}~M_{\rm \odot}\rm~yr^{-1}$ which is comparable with some other BWs.

astro-ph.HE↗

arXiv2021

Fast tree skeleton extraction using voxel thinning based on tree point cloud

Tree skeleton plays an important role in tree structure analysis, forest inventory and ecosystem monitoring. However, it is a challenge to extract a skeleton from a tree point cloud with complex branches. In this paper, an automatic and fast tree skeleton extraction method (FTSEM) based on voxel thinning is proposed. In this method, a wood-leaf classification algorithm was introduced to filter leaf points for the reduction of the leaf interference on tree skeleton generation, tree voxel thinning was adopted to extract raw tree skeleton quickly, and a breakpoint connection algorithm was used to improve the skeleton connectivity and completeness. Experiments were carried out in Haidian Park, Beijing, in which 24 trees were scanned and processed to obtain tree skeletons. The graph search algorithm (GSA) is used to extract tree skeletons based on the same datasets. Compared with GSA method, the FTSEM method obtained more complete tree skeletons. And the time cost of the FTSEM method is evaluated using the runtime and time per million points (TPMP). The runtime of FTSEM is from 1.0 s to 13.0 s, and the runtime of GSA is from 6.4 s to 309.3 s. The average value of TPMP is 1.8 s for FTSEM, and 22.3 s for GSA respectively. The experimental results demonstrate that the proposed method is feasible, robust, and fast with a good potential on tree skeleton extraction.

cs.CV↗