arXiv subjects

Rui Zhang

Publications and source records attributed to Rui Zhang.

At least 775 records · Page 43Linked to original sources

arXiv2022

A Generalized Doubly Robust Learning Framework for Debiasing Post-Click Conversion Rate Prediction

Post-click conversion rate (CVR) prediction is an essential task for discovering user interests and increasing platform revenues in a range of industrial applications. One of the most challenging problems of this task is the existence of severe selection bias caused by the inherent self-selection behavior of users and the item selection process of systems. Currently, doubly robust (DR) learning approaches achieve the state-of-the-art performance for debiasing CVR prediction. However, in this paper, by theoretically analyzing the bias, variance and generalization bounds of DR methods, we find that existing DR approaches may have poor generalization caused by inaccurate estimation of propensity scores and imputation errors, which often occur in practice. Motivated by such analysis, we propose a generalized learning framework that not only unifies existing DR methods, but also provides a valuable opportunity to develop a series of new debiasing techniques to accommodate different application scenarios. Based on the framework, we propose two new DR methods, namely DR-BIAS and DR-MSE. DR-BIAS directly controls the bias of DR loss, while DR-MSE balances the bias and variance flexibly, which achieves better generalization performance. In addition, we propose a novel tri-level joint learning optimization method for DR-MSE in CVR prediction, and an efficient training algorithm correspondingly. We conduct extensive experiments on both real-world and semi-synthetic datasets, which validate the effectiveness of our proposed methods.

cs.LG↗

arXiv2022

Imbalanced Classification in Medical Imaging via Regrouping

We propose performing imbalanced classification by regrouping majority classes into small classes so that we turn the problem into balanced multiclass classification. This new idea is dramatically different from popular loss reweighting and class resampling methods. Our preliminary result on imbalanced medical image classification shows that this natural idea can substantially boost the classification performance as measured by average precision (approximately area-under-the-precision-recall-curve, or AUPRC), which is more appropriate for evaluating imbalanced classification than other metrics such as balanced accuracy.

cs.CV↗

arXiv2022

XRICL: Cross-lingual Retrieval-Augmented In-Context Learning for Cross-lingual Text-to-SQL Semantic Parsing

In-context learning using large language models has recently shown surprising results for semantic parsing tasks such as Text-to-SQL translation. Prompting GPT-3 or Codex using several examples of question-SQL pairs can produce excellent results, comparable to state-of-the-art finetuning-based models. However, existing work primarily focuses on English datasets, and it is unknown whether large language models can serve as competitive semantic parsers for other languages. To bridge this gap, our work focuses on cross-lingual Text-to-SQL semantic parsing for translating non-English utterances into SQL queries based on an English schema. We consider a zero-shot transfer learning setting with the assumption that we do not have any labeled examples in the target language (but have annotated examples in English). This work introduces the XRICL framework, which learns to retrieve relevant English exemplars for a given query to construct prompts. We also include global translation exemplars for a target language to facilitate the translation process for large language models. To systematically evaluate our model, we construct two new benchmark datasets, XSpider and XKaggle-dbqa, which include questions in Chinese, Vietnamese, Farsi, and Hindi. Our experiments show that XRICL effectively leverages large pre-trained language models to outperform existing baselines. Data and code are publicly available at https://github.com/Impavidity/XRICL.

cs.CL↗

arXiv2022

Au4Mn, a localized ferromagnet with strong spin-orbit coupling, long-range ferromagnetic exchange and high Curie temperature

Metallic Mn-based alloys with a nearest-neighbor Mn-Mn distance greater than 0.4 nm exhibit large, well-localized magnetic moments. Here we investigate the magnetism of tetragonal Au4Mn with a Curie temperature of 385 K, where manganese has a spin moment of 4.1 muB and its orbital moment is quenched. Since 80% of the atoms are gold, the spin orbit interaction is strong and Au4Mn exhibits uniaxial magnetocrystalline anisotropy with surface maze domains at room temperature. The magnetic hardness parameter of 1.0 is sufficient to maintain the magnetization along the c-axis for a sample of any shape. Au also reduces the spin moment of Mn through 5d-3d orbital hybridization. An induced moment of 0.05 muB was found on Au under a pulsed field of 40 T. Density functional theory calculations indicate that the Mn-Mn exchange is mediated by spin-polarized gold 5d and 6p electrons. The distance-dependence shows that it is ferromagnetic or zero for the first ten shells of Mn neighbors out to 1.041 nm (64 atoms), and very weak and oscillatory thereafter.

cond-mat.mtrl-sci↗

arXiv2022

Strategic Decisions Survey, Taxonomy, and Future Directions from Artificial Intelligence Perspective

Strategic Decision-Making is always challenging because it is inherently uncertain, ambiguous, risky, and complex. It is the art of possibility. We develop a systematic taxonomy of decision-making frames that consists of 6 bases, 18 categorical, and 54 frames. We aim to lay out the computational foundation that is possible to capture a comprehensive landscape view of a strategic problem. Compared with traditional models, it covers irrational, non-rational and rational frames c dealing with certainty, uncertainty, complexity, ambiguity, chaos, and ignorance.

cs.AI↗

arXiv2022

ReasTAP: Injecting Table Reasoning Skills During Pre-training via Synthetic Reasoning Examples

Reasoning over tabular data requires both table structure understanding and a broad set of table reasoning skills. Current models with table-specific architectures and pre-training methods perform well on understanding table structures, but they still struggle with tasks that require various table reasoning skills. In this work, we develop ReasTAP to show that high-level table reasoning skills can be injected into models during pre-training without a complex table-specific architecture design. We define 7 table reasoning skills, such as numerical operation, temporal comparison, and conjunction. Each reasoning skill is associated with one example generator, which synthesizes questions over semi-structured tables according to the sampled templates. We model the table pre-training task as a sequence generation task and pre-train ReasTAP to generate precise answers to the synthetic examples. ReasTAP is evaluated on four benchmarks covering three downstream tasks including: 1) WikiSQL and WTQ for Table Question Answering; 2) TabFact for Table Fact Verification; and 3) LogicNLG for Faithful Table-to-Text Generation. Experimental results demonstrate that ReasTAP achieves new state-of-the-art performance on all benchmarks and delivers a significant improvement on low-resource setting. Our code is publicly available at https://github.com/Yale-LILY/ReasTAP.

cs.CL↗

arXiv2022

UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Models

Structured knowledge grounding (SKG) leverages structured knowledge to complete user requests, such as semantic parsing over databases and question answering over knowledge bases. Since the inputs and outputs of SKG tasks are heterogeneous, they have been studied separately by different communities, which limits systematic and compatible research on SKG. In this paper, we overcome this limitation by proposing the UnifiedSKG framework, which unifies 21 SKG tasks into a text-to-text format, aiming to promote systematic SKG research, instead of being exclusive to a single task, domain, or dataset. We use UnifiedSKG to benchmark T5 with different sizes and show that T5, with simple modifications when necessary, achieves state-of-the-art performance on almost all of the 21 tasks. We further demonstrate that multi-task prefix-tuning improves the performance on most tasks, largely improving the overall performance. UnifiedSKG also facilitates the investigation of zero-shot and few-shot learning, and we show that T0, GPT-3, and Codex struggle in zero-shot and few-shot learning for SKG. We also use UnifiedSKG to conduct a series of controlled experiments on structured knowledge encoding variants across SKG tasks. UnifiedSKG is easily extensible to more tasks, and it is open-sourced at https://github.com/hkunlp/unifiedskg.

cs.CL↗

arXiv2022

TransAlign: Fully Automatic and Effective Entity Alignment for Knowledge Graphs

The task of entity alignment between knowledge graphs (KGs) aims to identify every pair of entities from two different KGs that represent the same entity. Many machine learning-based methods have been proposed for this task. However, to our best knowledge, existing methods all require manually crafted seed alignments, which are expensive to obtain. In this paper, we propose the first fully automatic alignment method named TransAlign, which does not require any manually crafted seed alignments. Specifically, for predicate embeddings, TransAlign constructs a predicate-proximity-graph to automatically capture the similarity between predicates across two KGs by learning the attention of entity types. For entity embeddings, TransAlign first computes the entity embeddings of each KG independently using TransE, and then shifts the two KGs' entity embeddings into the same vector space by computing the similarity between entities based on their attributes. Thus, both predicate alignment and entity alignment can be done without manually crafted seed alignments. TransAlign is not only fully automatic, but also highly effective. Experiments using real-world KGs show that TransAlign improves the accuracy of entity alignment significantly compared to state-of-the-art methods.

cs.CL↗

arXiv2022

Causality-driven Hierarchical Structure Discovery for Reinforcement Learning

Hierarchical reinforcement learning (HRL) effectively improves agents' exploration efficiency on tasks with sparse reward, with the guide of high-quality hierarchical structures (e.g., subgoals or options). However, how to automatically discover high-quality hierarchical structures is still a great challenge. Previous HRL methods can hardly discover the hierarchical structures in complex environments due to the low exploration efficiency by exploiting the randomness-driven exploration paradigm. To address this issue, we propose CDHRL, a causality-driven hierarchical reinforcement learning framework, leveraging a causality-driven discovery instead of a randomness-driven exploration to effectively build high-quality hierarchical structures in complicated environments. The key insight is that the causalities among environment variables are naturally fit for modeling reachable subgoals and their dependencies and can perfectly guide to build high-quality hierarchical structures. The results in two complex environments, 2D-Minecraft and Eden, show that CDHRL significantly boosts exploration efficiency with the causality-driven paradigm.

cs.LG↗

arXiv2022

Object-Category Aware Reinforcement Learning

Object-oriented reinforcement learning (OORL) is a promising way to improve the sample efficiency and generalization ability over standard RL. Recent works that try to solve OORL tasks without additional feature engineering mainly focus on learning the object representations and then solving tasks via reasoning based on these object representations. However, none of these works tries to explicitly model the inherent similarity between different object instances of the same category. Objects of the same category should share similar functionalities; therefore, the category is the most critical property of an object. Following this insight, we propose a novel framework named Object-Category Aware Reinforcement Learning (OCARL), which utilizes the category information of objects to facilitate both perception and reasoning. OCARL consists of three parts: (1) Category-Aware Unsupervised Object Discovery (UOD), which discovers the objects as well as their corresponding categories; (2) Object-Category Aware Perception, which encodes the category information and is also robust to the incompleteness of (1) at the same time; (3) Object-Centric Modular Reasoning, which adopts multiple independent and object-category-specific networks when reasoning based on objects. Our experiments show that OCARL can improve both the sample efficiency and generalization in the OORL domain.

cs.LG↗

arXiv2022

Weak convergence of the extremes of branching Lévy processes with regularly varying tails

In this paper, we study the weak convergence of the extremes of supercritical branching Lévy processes $\{\mathbb{X}_t, t \ge0\}$ whose spatial motions are Lévy processes with regularly varying tails. The result is drastically different from the case of branching Brownian motions. We prove that, when properly renormalized, $\mathbb{X}_t$ converges weakly. As a consequence, we obtain a limit theorem for the order statistics of $\mathbb{X}_t$.

math.PR↗

arXiv2022

MIMO Capacity Characterization for Movable Antenna Systems

In this paper, we propose a new multiple-input multiple-output (MIMO) communication system with movable antennas (MAs) to exploit the antenna position optimization for enhancing the capacity. Different from conventional MIMO systems with fixed-position antennas (FPAs), the proposed system can flexibly change the positions of transmit/receive MAs, such that the MIMO channel between them is reconfigured to achieve higher capacity. We aim to characterize the capacity of MA-enabled point-to-point MIMO communication systems, by jointly optimizing the positions of transmit and receive MAs as well as the covariance of transmit signals. First, we develop an efficient alternating optimization algorithm to find a locally optimal solution by iteratively optimizing the transmit covariance matrix and the position of each transmit/receive MA with the other variables being fixed. Next, we propose alternative algorithms of lower complexity for capacity maximization in the low-SNR regime and for the multiple-input single-output (MISO) and single-input multiple-output (SIMO) cases. Numerical results show that our proposed MA systems significantly improve the MIMO channel capacity compared to traditional FPA systems as well as various benchmark schemes, and useful insights are drawn into the capacity gains of MA systems.

cs.IT↗

arXiv2022

Improving Model Robustness with Latent Distribution Locally and Globally

In this work, we consider model robustness of deep neural networks against adversarial attacks from a global manifold perspective. Leveraging both the local and global latent information, we propose a novel adversarial training method through robust optimization, and a tractable way to generate Latent Manifold Adversarial Examples (LMAEs) via an adversarial game between a discriminator and a classifier. The proposed adversarial training with latent distribution (ATLD) method defends against adversarial attacks by crafting LMAEs with the latent manifold in an unsupervised manner. ATLD preserves the local and global information of latent manifold and promises improved robustness against adversarial attacks. To verify the effectiveness of our proposed method, we conduct extensive experiments over different datasets (e.g., CIFAR-10, CIFAR-100, SVHN) with different adversarial attacks (e.g., PGD, CW), and show that our method substantially outperforms the state-of-the-art (e.g., Feature Scattering) in adversarial robustness by a large accuracy margin. The source codes are available at https://github.com/LitterQ/ATLD-pytorch.

cs.LG↗

arXiv2022

Generalised Image Outpainting with U-Transformer

In this paper, we develop a novel transformer-based generative adversarial neural network called U-Transformer for generalised image outpainting problem. Different from most present image outpainting methods conducting horizontal extrapolation, our generalised image outpainting could extrapolate visual context all-side around a given image with plausible structure and details even for complicated scenery, building, and art images. Specifically, we design a generator as an encoder-to-decoder structure embedded with the popular Swin Transformer blocks. As such, our novel neural network can better cope with image long-range dependencies which are crucially important for generalised image outpainting. We propose additionally a U-shaped structure and multi-view Temporal Spatial Predictor (TSP) module to reinforce image self-reconstruction as well as unknown-part prediction smoothly and realistically. By adjusting the predicting step in the TSP module in the testing stage, we can generate arbitrary outpainting size given the input sub-image. We experimentally demonstrate that our proposed method could produce visually appealing results for generalized image outpainting against the state-of-the-art image outpainting approaches.

cs.CV↗

arXiv2022

Selective Annotation Makes Language Models Better Few-Shot Learners

Many recent approaches to natural language tasks are built on the remarkable abilities of large language models. Large language models can perform in-context learning, where they learn a new task from a few task demonstrations, without any parameter updates. This work examines the implications of in-context learning for the creation of datasets for new natural language tasks. Departing from recent in-context learning methods, we formulate an annotation-efficient, two-step framework: selective annotation that chooses a pool of examples to annotate from unlabeled data in advance, followed by prompt retrieval that retrieves task examples from the annotated pool at test time. Based on this framework, we propose an unsupervised, graph-based selective annotation method, voke-k, to select diverse, representative examples to annotate. Extensive experiments on 10 datasets (covering classification, commonsense reasoning, dialogue, and text/code generation) demonstrate that our selective annotation method improves the task performance by a large margin. On average, vote-k achieves a 12.9%/11.4% relative gain under an annotation budget of 18/100, as compared to randomly selecting examples to annotate. Compared to state-of-the-art supervised finetuning approaches, it yields similar performance with 10-100x less annotation cost across 10 tasks. We further analyze the effectiveness of our framework in various scenarios: language models with varying sizes, alternative selective annotation methods, and cases where there is a test data domain shift. We hope that our studies will serve as a basis for data annotations as large language models are increasingly applied to new tasks. Our code is available at https://github.com/HKUNLP/icl-selective-annotation.

cs.CL↗

arXiv2022

The extremal process of super-Brownian motion: a probabilistic approach via skeletons

Recently Ren et al. [Stoch. Proc. Appl., 137 (2021)] have proved that the extremal process of the super-Brownian motion converges in distribution in the limit of large times. Their techniques rely heavily on the study of the convergence of solutions to the Kolmogorov-Petrovsky-Piscounov equation along the lines of [M. Bramson, Mem. Amer. Math. Soc., 44 (1983)]. In this paper we take a different approach. Our approach is based on the skeleton decomposition of super-Brownian motion. The skeleton may be interpreted as immortal particles that determine the large time behaviour of the process. We exploit this fact and carry asymptotic properties from the skeleton over to the super-Brownian motion. Some new results concerning the probabilistic representations of the limiting process are obtained, which cannot be directly obtained through the results of [Y.-X. Ren et al., Stoch. Proc. Appl., 137 (2021)]. Apart from the results, our approach offers insights into the driving force behind the limiting process for super-Brownian motions.

math.PR↗

arXiv2022

Unified gas-kinetic scheme with simplified multi-scale numerical flux for thermodynamic non-equilibrium flow in all flow regimes

In this paper, a unified gas-kinetic scheme (UGKS) with simplified multi-scale numerical flux is proposed for the thermodynamic non-equilibrium flow simulation involving the excitation of molecular vibrational degrees of freedom in all flow regimes. The present UGKS keep the basic conservation laws of the macroscopic flow variables and the microscopic gas distribution function in a discretized space. In order to improve the efficiency of the UGKS, a simplify multi-scale numerical flux is directly constructed from the characteristic difference solution of the kinetic model equation. In addition, a new BGK-type kinetic model for diatomic gases is proposed to describe the high-temperature thermodynamic non-equilibrium effect, which is a phenomenological relaxation model with the continuous distribution modes of rotational and vibrational energies. In present model, the equilibrium distribution functions is constructed by using a multi-dimensional Hermitian expansion around the Maxwellian distribution to achieve the correct Prandtl number and proper relaxation rate of heat fluxes. Furthermore, the application of the unstructured discrete velocity space (DVS) and a simple integration error correction reduce the number of velocity mesh significantly and make the present method be a efficient tool for simulations of flows in all flow regimes. The new scheme are examined in a series of cases, such as Sod's shock tube, high non-equilibrium shock structure, hypersonic flow around a circular cylinder with Knudsen (Kn) number Kn = 0.01, and the rarefied hypersonic flow over a flat plate with a sharp leading edge. The present UGKS results agree well with the benchmark data of DSMC and the other validated methods.

physics.flu-dyn↗

arXiv2022

Performance Analysis of OMP in Super-Resolution

Given a spectrally sparse signal $\mathbf{y} = \sum_{i=1}^s x_i\mathbf{f}(τ_i) \in \mathbb{C}^{2n+1}$ consisting of $s$ complex sinusoids, we consider the super-resolution problem, which is about estimating frequency components $\{τ_i\}_{i=1}^s$ of $\mathbf y$. We consider the OMP-type algorithms for super-resolution, which is more efficient than other approaches based on Semi-Definite Programming. Our analysis shows that a two-stage algorithm with OMP initialization can recover frequency components under the separation condition $nΔ\gtrsim \text{dyn}(\mathbf{x})$ and the dependency on $\text{dyn}(\mathbf{x})$ is inevitable for the vanilla OMP algorithm. We further show that the Sliding-OMP algorithm, a variant of the OMP algorithm with an additional refinement step at each iteration, is provable to recover $\{τ_i\}_{i=1}^s$ under the separation condition $nΔ\geq c$. Moreover, our result can be extended to an incomplete measurement model with $O( s^2\log n)$ measurements.

cs.IT↗