arXiv subjects

Xiaokang Liu

Publications and source records attributed to Xiaokang Liu.

At least 37 records · Page 2Linked to original sources

arXiv2022

Can Language Models Make Fun? A Case Study in Chinese Comical Crosstalk

Language is the principal tool for human communication, in which humor is one of the most attractive parts. Producing natural language like humans using computers, a.k.a, Natural Language Generation (NLG), has been widely used for dialogue systems, chatbots, machine translation, as well as computer-aid creation e.g., idea generations, scriptwriting. However, the humor aspect of natural language is relatively under-investigated, especially in the age of pre-trained language models. In this work, we aim to preliminarily test whether NLG can generate humor as humans do. We build a new dataset consisting of numerous digitized Chinese Comical Crosstalk scripts (called C$^3$ in short), which is for a popular Chinese performing art called `Xiangsheng' since 1800s. (For convenience for non-Chinese speakers, we called `crosstalk' for `Xiangsheng' in this paper.) We benchmark various generation approaches including training-from-scratch Seq2seq, fine-tuned middle-scale PLMs, and large-scale PLMs (with and without fine-tuning). Moreover, we also conduct a human assessment, showing that 1) large-scale pretraining largely improves crosstalk generation quality; and 2) even the scripts generated from the best PLM is far from what we expect, with only 65% quality of human-created crosstalk. We conclude, humor generation could be largely improved using large-scaled PLMs, but it is still in its infancy. The data and benchmarking code is publicly available in \url{https://github.com/anonNo2/crosstalk-generation}.

cs.CL↗

arXiv2021

A Bayesian Selection Model for Correcting Outcome Reporting Bias With Application to a Meta-analysis on Heart Failure Interventions

Multivariate meta-analysis (MMA) is a powerful tool for jointly estimating multiple outcomes' treatment effects. However, the validity of results from MMA is potentially compromised by outcome reporting bias (ORB), or the tendency for studies to selectively report outcomes. Until recently, ORB has been understudied. Since ORB can lead to biased conclusions, it is crucial to correct the estimates of effect sizes and quantify their uncertainty in the presence of ORB. With this goal, we develop a Bayesian selection model to adjust for ORB in MMA. We further propose a measure for quantifying the impact of ORB on the results from MMA. We evaluate our approaches through a meta-evaluation of 748 bivariate meta-analyses from the Cochrane Database of Systematic Reviews. Our model is motivated by and applied to a meta-analysis of interventions on hospital readmission and quality of life for heart failure patients. In our analysis, the relative risk (RR) of hospital readmission for the intervention group changes from a significant decrease (RR: 0.931, 95% confidence interval [CI]: 0.862-0.993) to a statistically nonsignificant effect (RR: 0.955, 95% CI: 0.876-1.051) after adjusting for ORB. This study demonstrates that failing to account for ORB can lead to different conclusions in a meta-analysis.

stat.AP↗

arXiv2020

Automatic identification of fossils and abiotic grains during carbonate microfacies analysis using deep convolutional neural networks

Petrographic analysis based on microfacies identification in thin sections is widely used in sedimentary environment interpretation and paleoecological reconstruction. Fossil recognition from microfacies is an essential procedure for petrographers to complete this task. Distinguishing the morphological and microstructural diversity of skeletal fragments requires extensive prior knowledge of fossil morphotypes in microfacies and long training sessions under the microscope. This requirement engenders certain challenges for sedimentologists and paleontologists, especially novices. However, a machine classifier can help address this challenge. In this study, we collected a microfacies image dataset comprising both public data from 1,149 references and our own materials (including 30,815 images of 22 fossil and abiotic grain groups). We employed a high-performance workstation to implement four classic deep convolutional neural networks (DCNNs), which have proven to be highly efficient in computer vision over the last several years. Our framework uses a transfer learning technique, which reuses the pre-trained parameters that are trained on a larger ImageNet dataset as initialization for the network to achieve high accuracy with low computing costs. We obtained up to 95% of the top one and 99% of the top three test accuracies in the Inception ResNet v2 architecture. The machine classifier exhibited 0.99 precision on minerals, such as dolomite and pyrite. Although it had some difficulty on samples having similar morphologies, such as the bivalve, brachiopod, and ostracod, it nevertheless obtained 0.88 precision. Our machine learning framework demonstrated high accuracy with reproducibility and bias avoidance that was comparable to those of human classifiers. Its application can thus eliminate much of the tedious, manually intensive efforts by human experts conducting routine identification.

cs.CV↗

arXiv2020

BERT-EMD: Many-to-Many Layer Mapping for BERT Compression with Earth Mover's Distance

Pre-trained language models (e.g., BERT) have achieved significant success in various natural language processing (NLP) tasks. However, high storage and computational costs obstruct pre-trained language models to be effectively deployed on resource-constrained devices. In this paper, we propose a novel BERT distillation method based on many-to-many layer mapping, which allows each intermediate student layer to learn from any intermediate teacher layers. In this way, our model can learn from different teacher layers adaptively for various NLP tasks. %motivated by the intuition that different NLP tasks require different levels of linguistic knowledge contained in the intermediate layers of BERT. In addition, we leverage Earth Mover's Distance (EMD) to compute the minimum cumulative cost that must be paid to transform knowledge from teacher network to student network. EMD enables the effective matching for many-to-many layer mapping. %EMD can be applied to network layers with different sizes and effectively measures semantic distance between the teacher network and student network. Furthermore, we propose a cost attention mechanism to learn the layer weights used in EMD automatically, which is supposed to further improve the model's performance and accelerate convergence time. Extensive experiments on GLUE benchmark demonstrate that our model achieves competitive performance compared to strong competitors in terms of both accuracy and model compression.

cs.CL↗

arXiv2020

Empirical Evaluation of Multi-task Learning in Deep Neural Networks for Natural Language Processing

Multi-Task Learning (MTL) aims at boosting the overall performance of each individual task by leveraging useful information contained in multiple related tasks. It has shown great success in natural language processing (NLP). Currently, a number of MLT architectures and learning mechanisms have been proposed for various NLP tasks. However, there is no systematic exploration and comparison of different MLT architectures and learning mechanisms for their strong performance in-depth. In this paper, we conduct a thorough examination of typical MTL methods on a broad range of representative NLP tasks. Our primary goal is to understand the merits and demerits of existing MTL methods in NLP tasks, thus devising new hybrid architectures intended to combine their strengths.

cs.CL↗

arXiv2020

Multivariate Log-Contrast Regression with Sub-Compositional Predictors: Testing the Association Between Preterm Infants' Gut Microbiome and Neurobehavioral Outcomes

The so-called gut-brain axis has stimulated extensive research on microbiomes. One focus is to assess the association between certain clinical outcomes and the relative abundances of gut microbes, which can be presented as sub-compositional data in conformity with the taxonomic hierarchy of bacteria. Motivated by a study for identifying the microbes in the gut microbiome of preterm infants that impact their later neurobehavioral outcomes, we formulate a constrained integrative multi-view regression, where the neurobehavioral scores form multivariate response, the sub-compositional microbiome data form multi-view feature matrices, and a set of linear constraints on their corresponding sub-coefficient matrices ensures the conformity to the simplex geometry. To enable joint selection and inference of sub-compositions/views, we assume all the sub-coefficient matrices are possibly of low-rank, i.e., the outcomes are associated with the microbiome through different sets of latent sub-compositional factors from different taxa. We propose a scaled composite nuclear norm penalization approach for model estimation and develop a hypothesis testing procedure through de-biasing to assess the significance of different views. Simulation studies confirm the effectiveness of the proposed procedure. In the preterm infant study, the identified microbes are mostly consistent with existing studies and biological understandings. Our approach supports that stressful early life experiences imprint gut microbiome through the regulation of the gut-brain axis.

stat.ME↗

arXiv2020

Multivariate Functional Regression via Nested Reduced-Rank Regularization

We propose a nested reduced-rank regression (NRRR) approach in fitting regression model with multivariate functional responses and predictors, to achieve tailored dimension reduction and facilitate interpretation/visualization of the resulting functional model. Our approach is based on a two-level low-rank structure imposed on the functional regression surfaces. A global low-rank structure identifies a small set of latent principal functional responses and predictors that drives the underlying regression association. A local low-rank structure then controls the complexity and smoothness of the association between the principal functional responses and predictors. Through a basis expansion approach, the functional problem boils down to an interesting integrated matrix approximation task, where the blocks or submatrices of an integrated low-rank matrix share some common row space and/or column space. An iterative algorithm with convergence guarantee is developed. We establish the consistency of NRRR and also show through non-asymptotic analysis that it can achieve at least a comparable error rate to that of the reduced-rank regression. Simulation studies demonstrate the effectiveness of NRRR. We apply NRRR in an electricity demand problem, to relate the trajectories of the daily electricity consumption with those of the daily temperatures.

stat.ME↗

arXiv2019

Phase transition of spacetime: particles as black holes in anti-de Sitter space

In this work, we re-examined the ancient complex metric in the recent quantum picture of black holes as Bose-Einstein condensates of gravitons. Both black holes and particles can be described by the complex Kerr-Newman metric in a 6-D complex space, which appears as a 4-D spacetime for a real or imaginary observer because of the barrier of the horizon. As two kind of complex black holes, particle and black hole are complex conjugated and can convert into each other through a phase transition. From the view of an observer in 3-D real space, an elementary particle with spin appears as an imaginary black hole in an anti-de Sitter space. The self-gravitational interaction of a particle as an imaginary black hole makes it obtain its wave-like nature in 4-D spacetime.

physics.gen-ph↗

arXiv2019

Fluctuation and Inertia

In this work, using Jacobson's idea: '$δ$Q=TdS hold for all the local Rindler causal horizons through each spacetime point', we found that the transitions between the excited and ground state of a particle in a linear acceleration satisfy the fluctuation theorem. The up transition from the ground state to the excited state is an entropy-decreasing process, which requires an external force to contribute equal entropy increase to satisfy the second law of thermodynamics.

gr-qc↗

arXiv2019

Surface tension of the horizon

The idea of treating the horizon of a black hole as a stretched membrane with surface tension has a long history. In this work, we discuss the microscopic origin of the surface tension of the horizon in quantum pictures of spaces, which are Bose-Einstein condensates of gravitons. The horizon is a phase interface of gravitons, the surface tension of which is found to be a result of the difference in the strength of the interaction between the gravitons on its two sides. The gravitational source, such as a Schwarzschild black hole, creates a transitional zone by changing the energy and distribution of its surrounding gravitons. Archimedes' principle for gravity can be expressed as follows: "the gravity on an object is equal to the weight of the gravitons that it displaces."

gr-qc↗

arXiv2018

Black hole as fireplace: limited communications across the horizon

An insightful viewpoint was proposed by Susskind about AMPS firewall: the region behind the firewall does not exist and the firewall is an extension of the singularity. In this work, we provided a possible picture of this idea by combining Newman's complex metric and Dvali-Gomez BEC black holes, which are Bose-Einstein condensates of N gravitons. The inner space behind the horizon is a realized imaginary space encrusted by the real space outside the horizon. In this way, the singularity extents to the horizon to make a firewall for the infalling observer. Some gravitons escape during the fluctuation of the BEC black hole, resulting in a micro-transparent horizon which makes the firewall exposes slightly to an observer outside the horizon. This picture allows limited communications across the horizon.

hep-th↗

arXiv2018

Integrative Multi-View Reduced-Rank Regression: Bridging Group-Sparse and Low-Rank Models

Multi-view data have been routinely collected in various fields of science and engineering. A general problem is to study the predictive association between multivariate responses and multi-view predictor sets, all of which can be of high dimensionality. It is likely that only a few views are relevant to prediction, and the predictors within each relevant view contribute to the prediction collectively rather than sparsely. We cast this new problem under the familiar multivariate regression framework and propose an integrative reduced-rank regression (iRRR), where each view has its own low-rank coefficient matrix. As such, latent features are extracted from each view in a supervised fashion. For model estimation, we develop a convex composite nuclear norm penalization approach, which admits an efficient algorithm via alternating direction method of multipliers. Extensions to non-Gaussian and incomplete data are discussed. Theoretically, we derive non-asymptotic oracle bounds of iRRR under a restricted eigenvalue condition. Our results recover oracle bounds of several special cases of iRRR including Lasso, group Lasso and nuclear norm penalized regression. Therefore, iRRR seamlessly bridges group-sparse and low-rank methods and can achieve substantially faster convergence rate under realistic settings of multi-view learning. Simulation studies and an application in the Longitudinal Studies of Aging further showcase the efficacy of the proposed methods.

stat.ME↗

arXiv2018

CommanderSong: A Systematic Approach for Practical Adversarial Voice Recognition

The popularity of ASR (automatic speech recognition) systems, like Google Voice, Cortana, brings in security concerns, as demonstrated by recent attacks. The impacts of such threats, however, are less clear, since they are either less stealthy (producing noise-like voice commands) or requiring the physical presence of an attack device (using ultrasound). In this paper, we demonstrate that not only are more practical and surreptitious attacks feasible but they can even be automatically constructed. Specifically, we find that the voice commands can be stealthily embedded into songs, which, when played, can effectively control the target system through ASR without being noticed. For this purpose, we developed novel techniques that address a key technical challenge: integrating the commands into a song in a way that can be effectively recognized by ASR through the air, in the presence of background noise, while not being detected by a human listener. Our research shows that this can be done automatically against real world ASR applications. We also demonstrate that such CommanderSongs can be spread through Internet (e.g., YouTube) and radio, potentially affecting millions of ASR users. We further present a new mitigation technique that controls this threat.

cs.CR↗

arXiv2016

Thermodynamics of static: Observers in Gibbs paradox

In this work, Gibbs paradox was discussed from the view of observer. The limitations of real observer are analyzed quantitatively. The entropy of mixing was found to be determined by both the identification ability and the information already in hand of an observer.

physics.class-ph↗

arXiv2016

Carnot's theorem and Szilárd engine

In this work, the relationship between Carnot engine and Szilárd engine was discussed. By defining the available information about the temperature difference between two heat reservoirs, the Carnot engine was found to have a same physical essence with Szilárd engine: lossless conversion of available information. Thus, a generalized Carnot's theorem for wider scope of application can be described as "all the available information is 100% coded into work".

physics.class-ph↗

arXiv2016

Aquaporin-1 can work as a Maxwell's Demon in the Body

Aquaporin-1 (AQP1) is a membrane protein which is selectively permeable to water. Due to its dumbbell shape, AQP1 can sense the size information of solute molecules in osmosis. At the cost of consuming this information, AQP1 can move water against its chemical potential gradient: it is able to work as one kind of Maxwell's Demon. This effect was detected quantitatively by measuring the water osmosis of mice red blood cells. This ability may protect the red blood cells from the eryptosis elicited by osmotic shock when they move in the kidney, where a large gradient of urea is required for the urine concentrating mechanism. This finding anticipates a new beginning of inquiries into the complicated relationships among mass, energy and information in bio-systems.

physics.bio-ph↗

arXiv2016

Modified Kedem-Katchalsky equations for osmosis through nano-pore

This work presents a modified Kedem-Katchalsky equations for osmosis through nano-pore. osmotic reflection coefficient of a solute was found to be chiefly affected by the entrance of the pore while filtration reflection coefficient can be affected by both the entrance and the internal structure of the pore. Using an analytical method, we get the quantitative relationship between osmotic reflection coefficient and the molecule size. The model is verified by comparing the theoretical results with the reported experimental data of aquaporin osmosis. Our work is expected to pave the way for a better understanding of osmosis in bio-system and to give us new ideas in designing new membranes with better performance.

physics.bio-ph↗

arXiv2016

Maxwell's demon and information channel width of a black hole

Using a new generalized second law of thermodynamics, the information and entropy of a black hole and its accretion disk are analyzed respectively. We find the bound of the information channel width of a black hole, which is determined by the variation rate of the horizon temperature and the mass of the black hole's shell(the accretion disk close to the horizon).

quant-ph↗