arXiv subjects

Ziwei Wu

Publications and source records attributed to Ziwei Wu.

27 records · Page 2Linked to original sources

arXiv2023

Fengshenbang 1.0: Being the Foundation of Chinese Cognitive Intelligence

Nowadays, foundation models become one of fundamental infrastructures in artificial intelligence, paving ways to the general intelligence. However, the reality presents two urgent challenges: existing foundation models are dominated by the English-language community; users are often given limited resources and thus cannot always use foundation models. To support the development of the Chinese-language community, we introduce an open-source project, called Fengshenbang, which leads by the research center for Cognitive Computing and Natural Language (CCNL). Our project has comprehensive capabilities, including large pre-trained models, user-friendly APIs, benchmarks, datasets, and others. We wrap all these in three sub-projects: the Fengshenbang Model, the Fengshen Framework, and the Fengshen Benchmark. An open-source roadmap, Fengshenbang, aims to re-evaluate the open-source community of Chinese pre-trained large-scale models, prompting the development of the entire Chinese large-scale model community. We also want to build a user-centered open-source ecosystem to allow individuals to access the desired models to match their computing resources. Furthermore, we invite companies, colleges, and research institutions to collaborate with us to build the large-scale open-source model-based ecosystem. We hope that this project will be the foundation of Chinese cognitive intelligence.

cs.CL↗

arXiv2023

Pulsar Scintillation Studies with LOFAR: II. Dual-frequency scattering study of PSR J0826+2637 with LOFAR and NenuFAR

Interstellar scattering (ISS) of radio pulsar emission can be used as a probe of the ionised interstellar medium (IISM) and causes corruptions in pulsar timing experiments. Two types of ISS phenomena (intensity scintillation and pulse broadening) are caused by electron density fluctuations on small scales (< 0.01 AU). Theory predicts that these are related, and both have been widely employed to study the properties of the IISM. Larger scales ($\sim$1-100\,AU) cause measurable changes in dispersion and these can be correlated with ISS observations to estimate the fluctuation spectrum over a very wide scale range. IISM measurements can often be modeled by a homogeneous power-law spatial spectrum of electron density with the Kolmogorov ($-11/3$) spectral exponent. Here we aim to test the validity of using the Kolmogorov exponent with PSR~J0826+2637. We do so using observations of intensity scintillation, pulse broadening and dispersion variations across a wide fractional bandwidth (20 -- 180\,MHz). We present that the frequency dependence of the intensity scintillation in the high frequency band matches the expectations of a Kolmogorov spectral exponent but the pulse broadening in the low frequency band does not change as rapidly as predicted with this assumption. We show that this behavior is due to an inhomogeneity in the scattering region, specifically that the scattering is dominated by a region of transverse size $\sim$40\,AU. The power spectrum of the electron density, however, maintains the Kolmogorov spectral exponent from spatial scales of 5$\times10^{-6}$\,AU to $\sim$100\,AU.

astro-ph.HE↗

arXiv2022

Deep Active Learning by Leveraging Training Dynamics

Active learning theories and methods have been extensively studied in classical statistical learning settings. However, deep active learning, i.e., active learning with deep learning models, is usually based on empirical criteria without solid theoretical justification, thus suffering from heavy doubts when some of those fail to provide benefits in real applications. In this paper, by exploring the connection between the generalization performance and the training dynamics, we propose a theory-driven deep active learning method (dynamicAL) which selects samples to maximize training dynamics. In particular, we prove that the convergence speed of training and the generalization performance are positively correlated under the ultra-wide condition and show that maximizing the training dynamics leads to better generalization performance. Furthermore, to scale up to large deep neural networks and data sets, we introduce two relaxations for the subset selection problem and reduce the time complexity from polynomial to constant. Empirical results show that dynamicAL not only outperforms the other baselines consistently but also scales well on large deep learning models. We hope our work would inspire more attempts on bridging the theoretical findings of deep networks and practical impacts of deep active learning in real applications.

cs.LG↗

arXiv2022

Zero-Shot Learners for Natural Language Understanding via a Unified Multiple Choice Perspective

We propose a new paradigm for zero-shot learners that is format agnostic, i.e., it is compatible with any format and applicable to a list of language tasks, such as text classification, commonsense reasoning, coreference resolution, and sentiment analysis. Zero-shot learning aims to train a model on a given task such that it can address new learning tasks without any additional training. Our approach converts zero-shot learning into multiple-choice tasks, avoiding problems in commonly used large-scale generative models such as FLAN. It not only adds generalization ability to models but also significantly reduces the number of parameters. Our method shares the merits of efficient training and deployment. Our approach shows state-of-the-art performance on several benchmarks and produces satisfactory results on tasks such as natural language inference and text classification. Our model achieves this success with only 235M parameters, which is substantially smaller than state-of-the-art models with billions of parameters. The code and pre-trained models are available at https://github.com/IDEA-CCNL/Fengshenbang-LM .

cs.CL↗

arXiv2022

Fairness-aware Model-agnostic Positive and Unlabeled Learning

With the increasing application of machine learning in high-stake decision-making problems, potential algorithmic bias towards people from certain social groups poses negative impacts on individuals and our society at large. In the real-world scenario, many such problems involve positive and unlabeled data such as medical diagnosis, criminal risk assessment and recommender systems. For instance, in medical diagnosis, only the diagnosed diseases will be recorded (positive) while others will not (unlabeled). Despite the large amount of existing work on fairness-aware machine learning in the (semi-)supervised and unsupervised settings, the fairness issue is largely under-explored in the aforementioned Positive and Unlabeled Learning (PUL) context, where it is usually more severe. In this paper, to alleviate this tension, we propose a fairness-aware PUL method named FairPUL. In particular, for binary classification over individuals from two populations, we aim to achieve similar true positive rates and false positive rates in both populations as our fairness metric. Based on the analysis of the optimal fair classifier for PUL, we design a model-agnostic post-processing framework, leveraging both the positive examples and unlabeled ones. Our framework is proven to be statistically consistent in terms of both the classification error and the fairness metric. Experiments on the synthetic and real-world data sets demonstrate that our framework outperforms state-of-the-art in both PUL and fair classification.

cs.LG↗

arXiv2022

Long-term scintillation studies of EPTA pulsars. I. Observations and basic results

Interstellar scintillation analysis of pulsars allows us to probe the small-scale distribution and inhomogeneities of the ionized interstellar medium. Our priority is to present the data set and the basic measurements of scintillation parameters of pulsars employing long-term scintillation observations carried out from 2011 January to 2020 August by the European Pulsar Timing Array radio telescopes in the 21-cm and 11-cm bands. Additionally, we aim to identify future possible lines of study using this long-term scintillation dataset. We present the long-term time series of $ν_{\rm d}$ and $τ_{\rm d}$ for 13 pulsars. Sanity-checks and comparisons indicate that the scintillation parameters of our work and previously published works are mostly consistent. For two pulsars, PSRs~J1857+0943 and J1939+2134, we were able to obtain measurements of the $ν_{\rm d}$ at both bands, which allows us to derive the time series of frequency scaling indices with a mean and a standard deviation of 2.82$\pm$1.95 and 3.18$\pm$0.60, respectively. We found some interesting features which will be studied in more detail in subsequent papers in this series: (i) in the time series of PSR~J1939+2134, where the scintillation bandwidth sharply increases or decreases associated with a sharp change of dispersion measure; (ii) PSR~J0613$-$0200 and PSR~J0636+5126 show a strong annual variation in the time series of the $τ_{\rm d}$; (iii) PSR~J1939+2134 shows a weak anti-correlation between scintillation timescale and dispersion in WSRT data.

astro-ph.HE↗

arXiv2022

Pulsar scintillation studies with LOFAR. I. The census

Context. Interstellar scintillation (ISS) of pulsar emission can be used both as a probe of the ionised interstellar medium (IISM) and cause corruptions in pulsar timing experiments. Of particular interest are so-called scintillation arcs which can be used to measure time-variable interstellar scattering delays directly, potentially allowing high-precision improvements to timing precision. Aims. The primary aim of this study is to carry out the first sizeable and self-consistent census of diffractive pulsar scintillation and scintillation-arc detectability at low frequencies, as a primer for larger-scale IISM studies and pulsar-timing related propagation studies with the LOw-Frequency ARray (LOFAR) High Band Antennae (HBA). Results. In this initial set of 31 sources, 15 allow full determination of the scintillation properties; nine of these show detectable scintillation arcs at 120-180 MHz. Eight of the observed sources show unresolved scintillation; and the final eight don't display diffractive scintillation. Some correlation between scintillation detectability and pulsar brightness and dispersion measure is apparent, although no clear cut-off values can be determined. Our measurements across a large fractional bandwidth allow a meaningful test of the frequency scaling of scintillation parameters, uncorrupted by influences from refractive scintillation variations. Conclusions. Our results indicate the powerful advantage and great potential of ISS studies at low frequencies and the complex dependence of scintillation detectability on parameters like pulsar brightness and interstellar dispersion. This work provides the first installment of a larger-scale census and longer-term monitoring of interstellar scintillation effects at low frequencies.

astro-ph.HE↗

arXiv2021

Model-Agnostic Counterfactual Reasoning for Eliminating Popularity Bias in Recommender System

The general aim of the recommender system is to provide personalized suggestions to users, which is opposed to suggesting popular items. However, the normal training paradigm, i.e., fitting a recommender model to recover the user behavior data with pointwise or pairwise loss, makes the model biased towards popular items. This results in the terrible Matthew effect, making popular items be more frequently recommended and become even more popular. Existing work addresses this issue with Inverse Propensity Weighting (IPW), which decreases the impact of popular items on the training and increases the impact of long-tail items. Although theoretically sound, IPW methods are highly sensitive to the weighting strategy, which is notoriously difficult to tune. In this work, we explore the popularity bias issue from a novel and fundamental perspective -- cause-effect. We identify that popularity bias lies in the direct effect from the item node to the ranking score, such that an item's intrinsic property is the cause of mistakenly assigning it a higher ranking score. To eliminate popularity bias, it is essential to answer the counterfactual question that what the ranking score would be if the model only uses item property. To this end, we formulate a causal graph to describe the important cause-effect relations in the recommendation process. During training, we perform multi-task learning to achieve the contribution of each cause; during testing, we perform counterfactual inference to remove the effect of item popularity. Remarkably, our solution amends the learning process of recommendation which is agnostic to a wide range of models -- it can be easily implemented in existing methods. We demonstrate it on Matrix Factorization (MF) and LightGCN [20]. Experiments on five real-world datasets demonstrate the effectiveness of our method.

cs.IR↗

arXiv2018

The X-ray bursts within the 2010 outburst of the accreting millisecond X-ray pulsar SAX J1748.9-2021

With the observations from \textit{Rossi X-ray Timing Explorer}, we search and study the X-ray bursts of accreting millisecond X-ray pulsar SAX~J1748.9-2021 during its 2010 outburst. We find 13 X-ray bursts, including 12 standard type-\uppercase\expandafter{\romannumeral1} X-ray bursts and an irregular X-ray burst which lacks cooling tail. During the outburst, the persistent emission occurred at $\sim$(1-5)$\%\rm {\dot{M}_{Edd}}$. We use a combination model of a blackbody (BB), a powerlaw, and a line component to fit the persistent emission spectra. Another BB is added into the combination model to account for the emission of the X-ray bursts due to the thermonuclear burning on the surface of the neutron star. Finally, we modify the combination model with a multiplicative factor $f_{\rm a}$, plus a BB to fit the spectra during the X-ray bursts. It is found that the $f_{\rm a}$ is inversely correlated with the burst flux in some cases. Our analysis suggests that the ignition depth of the irregular X-ray burst is obviously smaller than those of the type-\uppercase\expandafter{\romannumeral1} X-ray bursts. We argue that the detected type-\uppercase\expandafter{\romannumeral1} X-ray bursts originate from helium-rich or pure-helium environment, while the irregular X-ray burst originates from the thermonuclear flash in a shallow ocean.

astro-ph.HE↗