arXiv subjects

Lu Liu

Publications and source records attributed to Lu Liu.

At least 181 records · Page 10Linked to original sources

arXiv2021

A Method for Pulsar Searching: Combining a Two-dimensional Autocorrelation Profile Map and a Deep Convolutional Neural Network

In pulsar astronomy, detecting effective pulsar signals among numerous pulsar candidates is an important research topic. Starting from space X-ray pulsar signals, the two-dimensional autocorrelation profile map (2D-APM) feature modelling method, which utilizes epoch folding of the autocorrelation function of X-ray signals and expands the time-domain information of the periodic axis, is proposed. A uniform setting criterion regarding the time resolution of the periodic axis addresses pulsar signals without any prior information. Compared with the traditional profile, the model has a strong anti-noise ability, a greater abundance of information and consistent characteristics. The new feature is simulated with double Gaussian components, and the characteristic distribution of the model is revealed to be closely related to the distance between the double peaks of the profile. Next, a deep convolutional neural network (DCNN) is built, named Inception-ResNet. According to the order of the peak separation and number of arriving photons, 30 data sets based on the Poisson process are simulated to construct the training set, and the observation data of PSRs B0531+21, B0540-69 and B1509-58 from the Rossi X-ray Timing Explorer (RXTE) are selected to generate the test set. The number of training sets and the number of test sets are 30,000 and 5,400, respectively. After achieving convergence stability, more than 99$\%$ of the pulsar signals are recognized, and more than 99$\%$ of the interference is successfully rejected, which verifies the high degree of agreement between the network and the feature model and the high potential of the proposed method in searching for pulsars.

astro-ph.IM↗

arXiv2021

Understanding the onset of hot streaks across artistic, cultural, and scientific careers

Hot streaks dominate the main impact of creative careers. Despite their ubiquitous nature across a wide range of creative domains, it remains unclear if there is any regularity underlying the beginning of hot streaks. Here, we develop computational methods using deep learning and network science and apply them to novel, large-scale datasets tracing the career outputs of artists, film directors, and scientists, allowing us to build high-dimensional representations of the artworks, films, and scientific publications they produce. By examining individuals' career trajectories within the underlying creative space, we find that across all three domains, individuals tend to explore diverse styles or topics before their hot streak, but become notably more focused in what they work on after the hot streak begins. Crucially, we find that hot streaks are associated with neither exploration nor exploitation behavior in isolation, but a particular sequence of exploration followed by exploitation, where the transition from exploration to exploitation closely traces the onset of a hot streak. Overall, these results unveil among the first identifiable regularity underlying the onset of hot streaks, which appears universal across diverse creative domains, suggesting that a sequential view of creative strategies that balances experimentation and implementation may be particularly powerful for producing long-lasting contributions, which may have broad implications for identifying and nurturing creative talents.

physics.soc-ph↗

arXiv2021

From Technology to Society: An Overview of Blockchain-based DAO

Decentralized Autonomous Organization (DAO) is believed to play a significant role in our future society governed in a decentralized way. In this article, we first explain the definitions and preliminaries of DAO. Then, we conduct a literature review of the existing studies of DAO published in the recent few years. Through the literature review, we find out that a comprehensive survey towards the state-of-the-art studies of DAO is still missing. To fill this gap, we perform such an overview by identifying and classifying the most valuable proposals and perspectives closely related to the combination of DAO and blockchain technologies. We anticipate that this survey can help researchers, engineers, and educators acknowledge the cutting-edge development of blockchain-related DAO technologies.

cs.CR↗

arXiv2021

Isometric Propagation Network for Generalized Zero-shot Learning

Zero-shot learning (ZSL) aims to classify images of an unseen class only based on a few attributes describing that class but no access to any training sample. A popular strategy is to learn a mapping between the semantic space of class attributes and the visual space of images based on the seen classes and their data. Thus, an unseen class image can be ideally mapped to its corresponding class attributes. The key challenge is how to align the representations in the two spaces. For most ZSL settings, the attributes for each seen/unseen class are only represented by a vector while the seen-class data provide much more information. Thus, the imbalanced supervision from the semantic and the visual space can make the learned mapping easily overfitting to the seen classes. To resolve this problem, we propose Isometric Propagation Network (IPN), which learns to strengthen the relation between classes within each space and align the class dependency in the two spaces. Specifically, IPN learns to propagate the class representations on an auto-generated graph within each space. In contrast to only aligning the resulted static representation, we regularize the two dynamic propagation procedures to be isometric in terms of the two graphs' edge weights per step by minimizing a consistency loss between them. IPN achieves state-of-the-art performance on three popular ZSL benchmarks. To evaluate the generalization capability of IPN, we further build two larger benchmarks with more diverse unseen classes and demonstrate the advantages of IPN on them.

cs.CV↗

arXiv2021

NATS-Bench: Benchmarking NAS Algorithms for Architecture Topology and Size

Neural architecture search (NAS) has attracted a lot of attention and has been illustrated to bring tangible benefits in a large number of applications in the past few years. Architecture topology and architecture size have been regarded as two of the most important aspects for the performance of deep learning models and the community has spawned lots of searching algorithms for both aspects of the neural architectures. However, the performance gain from these searching algorithms is achieved under different search spaces and training setups. This makes the overall performance of the algorithms to some extent incomparable and the improvement from a sub-module of the searching model unclear. In this paper, we propose NATS-Bench, a unified benchmark on searching for both topology and size, for (almost) any up-to-date NAS algorithm. NATS-Bench includes the search space of 15,625 neural cell candidates for architecture topology and 32,768 for architecture size on three datasets. We analyze the validity of our benchmark in terms of various criteria and performance comparison of all candidates in the search space. We also show the versatility of NATS-Bench by benchmarking 13 recent state-of-the-art NAS algorithms on it. All logs and diagnostic information trained using the same setup for each candidate are provided. This facilitates a much larger community of researchers to focus on developing better NAS algorithms in a more comparable and computationally cost friendly environment. All codes are publicly available at: https://xuanyidong.com/assets/projects/NATS-Bench.

cs.LG↗

arXiv2021

Episodic memory governs choices: An RNN-based reinforcement learning model for decision-making task

Typical methods to study cognitive function are to record the electrical activities of animal neurons during the training of animals performing behavioral tasks. A key problem is that they fail to record all the relevant neurons in the animal brain. To alleviate this problem, we develop an RNN-based Actor-Critic framework, which is trained through reinforcement learning (RL) to solve two tasks analogous to the monkeys' decision-making tasks. The trained model is capable of reproducing some features of neural activities recorded from animal brain, or some behavior properties exhibited in animal experiments, suggesting that it can serve as a computational platform to explore other cognitive functions. Furthermore, we conduct behavioral experiments on our framework, trying to explore an open question in neuroscience: which episodic memory in the hippocampus should be selected to ultimately govern future decisions. We find that the retrieval of salient events sampled from episodic memories can effectively shorten deliberation time than common events in the decision-making process. The results indicate that salient events stored in the hippocampus could be prioritized to propagate reward information, and thus allow decision-makers to learn a strategy faster.

q-bio.NC↗

arXiv2021

Extreme Suppression of Antiferromagnetic Order and Critical Scaling in a Two-Dimensional Random Quantum Magnet

Sr$_2$CuTeO$_6$ is a square-lattice Néel antiferromagnet with superexchange between first-neighbor $S=1/2$ Cu spins mediated by plaquette centered Te ions. Substituting Te by W, the affected impurity plaquettes have predominantly second-neighbor interactions, thus causing local magnetic frustration. Here we report a study of Sr$_2$CuTe$_{1-x}$W$_x$O$_6$ using neutron diffraction and $μ$SR techniques, showing that the Néel order vanishes already at $x = 0.025 \pm 0.005$. We explain this extreme order suppression using a two-dimensional Heisenberg spin model, demonstrating that a W-type impurity induces a deformation of the order parameter that decays with distance as $1/r^2$ at temperature $T=0$. The associated logarithmic singularity leads to loss of order for any $x>0$. Order for small $x>0$ and $T>0$ is induced by weak interplane couplings. In the nonmagnetic phase of Sr$_2$CuTe$_{1-x}$W$_x$O$_6$, the $μ$SR relaxation rate exhibits quantum critical scaling with a large dynamic exponent, $z \approx 3$, consistent with a random-singlet state.

cond-mat.str-el↗

arXiv2020

The combinatorial equivalence of a computability theoretic question

We show that a question of Miller and Solomon -- that whether there exists a coloring $c:d^{<ω}\rightarrow k$ that does not admit a $c$-computable variable word infinite solution, is equivalent to a natural, nontrivial combinatorial question. The combinatorial question asked whether there is an infinite sequence of integers such that each of its initial segment satisfies a Ramsian type property. This is the first computability theoretic question known to be equivalent to a natural, nontrivial question that does not concern complexity notions. It turns out that the negation of the combinatorial question is a generalization of Hales-Jewett theorem. We solve some special cases of the combinatorial question and obtain a generalization of Hales-Jewett theorem on some particular parameters.

math.LO↗

arXiv2020

Two-dimensional ferromagnetic semiconductor VBr3 with tunable anisotropy

Two-dimensional (2D) ferromagnets (FMs) have attracted widespread attention due to their prospects in spintronic applications. Here we explore the electronic structure and magnetic properties of the bulk and monolayer of VBr$_{3}$ in the honeycomb lattice, using first-principles calculations, crystal field level analyses, and Monte Carlo simulations. Our results show that VBr$_{3}$ bulk has the $e'_{g}$$^2$ ($S$=1) ground state and possesses a small orbital moment and weak in-plane magnetic anisotropy. Those results well explain the recent experiments. More interestingly, we find that a tensile strain on the semiconducting VBr$_{3}$ monolayer tunes the ground state into $a_{1g}$$^1$$e'_{g}$$^1$ and thus produces a large orbital moment and a strong out-of-plane anisotropy. Then, the significantly enhanced FM superexchange and single ion anisotropy (SIA) would raise $T_{\rm C}$ from 20 K for the bare VBr$_{3}$ monolayer to 100-115 K under a 2.5$\%$-5$\%$ strain. Therefore, VBr$_{3}$ would be a promising 2D FM semiconductor with a tunable anisotropy.

cond-mat.mtrl-sci↗

arXiv2020

Attribute Propagation Network for Graph Zero-shot Learning

The goal of zero-shot learning (ZSL) is to train a model to classify samples of classes that were not seen during training. To address this challenging task, most ZSL methods relate unseen test classes to seen(training) classes via a pre-defined set of attributes that can describe all classes in the same semantic space, so the knowledge learned on the training classes can be adapted to unseen classes. In this paper, we aim to optimize the attribute space for ZSL by training a propagation mechanism to refine the semantic attributes of each class based on its neighbors and related classes on a graph of classes. We show that the propagated attributes can produce classifiers for zero-shot classes with significantly improved performance in different ZSL settings. The graph of classes is usually free or very cheap to acquire such as WordNet or ImageNet classes. When the graph is not provided, given pre-defined semantic embeddings of the classes, we can learn a mechanism to generate the graph in an end-to-end manner along with the propagation mechanism. However, this graph-aided technique has not been well-explored in the literature. In this paper, we introduce the attribute propagation network (APNet), which is composed of 1) a graph propagation model generating attribute vector for each class and 2) a parameterized nearest neighbor (NN) classifier categorizing an image to the class with the nearest attribute vector to the image's embedding. For better generalization over unseen classes, different from previous methods, we adopt a meta-learning strategy to train the propagation mechanism and the similarity metric for the NN classifier on multiple sub-graphs, each associated with a classification task over a subset of training classes. In experiments with two zero-shot learning settings and five benchmark datasets, APNet achieves either compelling performance or new state-of-the-art results.

cs.CV↗

arXiv2020

A Universal Representation Transformer Layer for Few-Shot Image Classification

Few-shot classification aims to recognize unseen classes when presented with only a small number of samples. We consider the problem of multi-domain few-shot image classification, where unseen classes and examples come from diverse data sources. This problem has seen growing interest and has inspired the development of benchmarks such as Meta-Dataset. A key challenge in this multi-domain setting is to effectively integrate the feature representations from the diverse set of training domains. Here, we propose a Universal Representation Transformer (URT) layer, that meta-learns to leverage universal features for few-shot classification by dynamically re-weighting and composing the most appropriate domain-specific representations. In experiments, we show that URT sets a new state-of-the-art result on Meta-Dataset. Specifically, it achieves top-performance on the highest number of data sources compared to competing methods. We analyze variants of URT and present a visualization of the attention score heatmaps that sheds light on how the model performs cross-domain generalization. Our code is available at https://github.com/liulu112601/URT.

cs.LG↗

arXiv2020

Masked Pre-trained Encoder base on Joint CTC-Transformer

This study (The work was accomplished during the internship in Tencent AI lab) addresses semi-supervised acoustic modeling, i.e. attaining high-level representations from unsupervised audio data and fine-tuning the parameters of pre-trained model with supervised data. The proposed approach adopts a two-stage training framework, consisting of masked pre-trained encoder (MPE) and Joint CTC-Transformer (JCT). In the MPE framework, part of input frames are masked and reconstructed after the encoder with massive unsupervised data. In JCT framework, compared with original Transformer, acoustic features are applied as input instead of plain text. CTC loss performs as the prediction target on top of the encoder, and decoder blocks remain unchanged. This paper presents a comparison between two-stage training method and the fully supervised JCT. In addition, this paper investigates the our approach's robustness against different volumns of training data. Experiments on the two-stage training method deliver much better performance than fully supervised model. The word error rate (WER) with two-stage training which only exploits 30\% of WSJ labeled data achieves 17\% reduction than which trained by 50\% of WSJ in a fully supervised way. Moreover, increasing unlabeled data for MPE from WSJ (81h) to Librispeech (960h) attains about 22\% WER reduction.

eess.AS↗

arXiv2020

Interpretable Time-series Classification on Few-shot Samples

Recent few-shot learning works focus on training a model with prior meta-knowledge to fast adapt to new tasks with unseen classes and samples. However, conventional time-series classification algorithms fail to tackle the few-shot scenario. Existing few-shot learning methods are proposed to tackle image or text data, and most of them are neural-based models that lack interpretability. This paper proposes an interpretable neural-based framework, namely \textit{Dual Prototypical Shapelet Networks (DPSN)} for few-shot time-series classification, which not only trains a neural network-based model but also interprets the model from dual granularity: 1) global overview using representative time series samples, and 2) local highlights using discriminative shapelets. In particular, the generated dual prototypical shapelets consist of representative samples that can mostly demonstrate the overall shapes of all samples in the class and discriminative partial-length shapelets that can be used to distinguish different classes. We have derived 18 few-shot TSC datasets from public benchmark datasets and evaluated the proposed method by comparing with baselines. The DPSN framework outperforms state-of-the-art time-series classification methods, especially when training with limited amounts of data. Several case studies have been given to demonstrate the interpret ability of our model.

cs.LG↗

arXiv2020

Many-Class Few-Shot Learning on Multi-Granularity Class Hierarchy

We study many-class few-shot (MCFS) problem in both supervised learning and meta-learning settings. Compared to the well-studied many-class many-shot and few-class few-shot problems, the MCFS problem commonly occurs in practical applications but has been rarely studied in previous literature. It brings new challenges of distinguishing between many classes given only a few training samples per class. In this paper, we leverage the class hierarchy as a prior knowledge to train a coarse-to-fine classifier that can produce accurate predictions for MCFS problem in both settings. The propose model, "memory-augmented hierarchical-classification network (MahiNet)", performs coarse-to-fine classification where each coarse class can cover multiple fine classes. Since it is challenging to directly distinguish a variety of fine classes given few-shot data per class, MahiNet starts from learning a classifier over coarse-classes with more training data whose labels are much cheaper to obtain. The coarse classifier reduces the searching range over the fine classes and thus alleviates the challenges from "many classes". On architecture, MahiNet firstly deploys a convolutional neural network (CNN) to extract features. It then integrates a memory-augmented attention module and a multi-layer perceptron (MLP) together to produce the probabilities over coarse and fine classes. While the MLP extends the linear classifier, the attention module extends the KNN classifier, both together targeting the "few-shot" problem. We design several training strategies of MahiNet for supervised learning and meta-learning. In addition, we propose two novel benchmark datasets "mcfsImageNet" and "mcfsOmniglot" specially designed for MCFS problem. In experiments, we show that MahiNet outperforms several state-of-the-art models on MCFS problems in both supervised learning and meta-learning.

cs.LG↗

arXiv2020

Quantum-critical scaling properties of the two-dimensional random-singlet state

We use QMC simulations to study effects of disorder on the $S=1/2$ Heisenberg model with exchange constant $J$ on the square lattice supplemented by multispin interactions $Q$. It was found recently [L. Lu et al., Phys. Rev. X 8, 041040 (2018)] that the ground state of this $J$-$Q$ model with random couplings undergoes a quantum phase transition from the Néel state into a randomness-induced spin-liquid-like state that is a close analogue to the well known random-singlet (RS) state of the random Heisenberg chain. The 2D RS state arises from spinons localized at topological defects. The interacting spinons form a critical state with mean spin-spin correlations decaying with distance $r$ as $r^{-2}$, as in the 1D RS state. The dynamic exponent $z \ge 2$, varying continuously with the model parameters. We here further investigate the properties of the RS state in the $J$-$Q$ model with random $Q$ couplings. We study the temperature dependence of the specific heat and various susceptibilities for large enough systems to reach the thermodynamic limit and also analyze the size dependence of the critical magnetic order parameter and its susceptibility in the ground state. For all these quantities, we find consistency with conventional quantum-critical scaling when the condition implied by the $r^{-2}$ form of the spin correlations is imposed. All quantities can be explained by the same value of the dynamic exponent $z$ at fixed model parameters. We argue that the RS state identified in the $J$-$Q$ model corresponds to a generic renormalization group fixed point that can be reached in many quantum magnets with random couplings, and may already have been observed experimentally.

cond-mat.str-el↗

arXiv2020

Avoid Schnorr randomness

We prove that every finite partition of $ω$ admit an infinite subset that does not compute a Schnorr random real. We use this result to answer two questions of Brendle, Brooke-Taylor, Ng and Nies and strength a result of Khan and Miller.

math.LO↗

arXiv2020

Which DNR can be minimal

Khan and Miller proved that for every computable non decreasing unbounded function $h\in ω^ω$ (henceforth order function), if $h$ is sufficiently large, then there exists a $DNR_h$ that is of minimal degree. Where $h$ has to satisfy $\lim_{n\rightarrow\infty} h(n)/(2^{k\cdot \prod_{m 0$. Their core argument is that we can thin the tree by a factor of $2^j$ to make $j$ Turing functional split. We improve their result by reducing this factor to $j$. Thus we show that for every order function $h$ with $\lim_{n\rightarrow\infty} h(n)/( \prod_{m 0$, there exists a $DNR_h$ of minimal degree. We answer a question of Brendle, Brooke-Taylor, Ng and Nies by showing that there exists a $G\in ω^ω$ such that $G$ is weakly meager covering, $G$ does not compute any Schnorr random real and $G$ does not Schnorr cover REC.

math.LO↗

arXiv2020

Coding power of product of partitions

Given two combinatorial notions $\mathsf{P}_0$ and $\mathsf{P}_1$, can we encode $\mathsf{P}_0$ via $\mathsf{P}_1$. In this talk we address the question where $\mathsf{P}_0$ is 3-coloring of integers and $\mathsf{P}_1$ is product of finitely many 2-colorings of integers. We firstly reduce the question to a lemma which asserts that certain $Π^0_1$ class of colorings admit two members violating a particular combinatorial constraint. Then we took a digression to see how complex does the class has to be so as to maintain the cross constraint. We weaken the two members in the lemma in certain way to address an open question of Cholak, Dzhafarov, Hirschfeldt and Patey, concerning a sort of Weihrauch degree of stable Ramsey's theorem for pairs. It turns out the resulted strengthen of the lemma is a basis theorem for $Π^0_1$ class with additional constraint. We look at several such variants of basis theorem, among them some are unknown. We end up by introducing some results and questions concerning product of infinitely many colorings.

math.LO↗