arXiv subjects

Kevin Yang

Publications and source records attributed to Kevin Yang.

At least 55 records · Page 3Linked to original sources

arXiv2022

Addressing Resource and Privacy Constraints in Semantic Parsing Through Data Augmentation

We introduce a novel setup for low-resource task-oriented semantic parsing which incorporates several constraints that may arise in real-world scenarios: (1) lack of similar datasets/models from a related domain, (2) inability to sample useful logical forms directly from a grammar, and (3) privacy requirements for unlabeled natural utterances. Our goal is to improve a low-resource semantic parser using utterances collected through user interactions. In this highly challenging but realistic setting, we investigate data augmentation approaches involving generating a set of structured canonical utterances corresponding to logical forms, before simulating corresponding natural language and filtering the resulting pairs. We find that such approaches are effective despite our restrictive setup: in a low-resource setting on the complex SMCalFlow calendaring dataset (Andreas et al., 2020), we observe 33% relative improvement over a non-data-augmented baseline in top-1 match.

cs.CL↗

arXiv2022

KPZ Equation from non-simple variations on open ASEP

This paper has two main goals. The first is universality of the KPZ equation for fluctuations of dynamic interfaces associated to interacting particle systems in the presence of open boundary. We consider generalizations on the open-ASEP from [Corwin-Shen '16, Parekh '17] but admitting non-simple interactions both at the boundary and within the bulk of the particle system. These variations on open-ASEP are not integrable models, similar to the long-range variations on ASEP considered in [Dembo-Tsai '15, Y '20]. We establish the KPZ equation with the appropriate Robin boundary conditions as scaling limits for height function fluctuations associated to these non-integrable models, providing further evidence for the aforementioned universality of the KPZ equation. We specialize to compact domains and address non-compact domains in a second paper. The procedure that we employ to establish the aforementioned theorem is the second main point of this paper. Invariant measures in the presence of boundary interactions generally lack reasonable descriptions. Thus, global analyses done through the invariant measure, including the theory of energy solutions in [Goncalves-Jara '14, Goncalves-Jara '17, Goncalves-Jara-Sethuraman '15], is immediately obstructed. To circumvent this obstruction, we appeal to the almost entirely local nature of the analysis in [Y '20].

math.PR↗

arXiv2022

Stochastic Burgers Equation via Energy Solutions from Non-Stationary Particle Systems

We prove that the stochastic Burgers equation, which is related to the Kardar-Parisi-Zhang/KPZ equation via weak derivative, is a "critical" scaling limit for density fluctuations for a family of non-integrable and non-stationary interacting particle systems. The models we consider cannot be linearized by a microscopic Cole-Hopf transform or studied directly by the existing energy solution theory of Goncalves-Jara '14. We develop a novel method based on comparison to stationary models, a technique that has not yet been applied to universality of the KPZ equation and nontrivially expands the set of models for which universality is confirmed. We also study crossover fluctuations and prove, for the first time, the full transition and phase diagram from Gaussian to KPZ fluctuations for non-stationary interacting particle systems, which has not even been done yet for integrable models. We emphasize the method developed herein applies to a general class of models/particle systems, but we restrict to a class of zero-range systems whose non-stationary versions have received widespread interest but had not been treated in the context of KPZ until now as well as class of non-simple exclusion processes that we comment on.

math.PR↗

arXiv2022

Learning Space Partitions for Path Planning

Path planning, the problem of efficiently discovering high-reward trajectories, often requires optimizing a high-dimensional and multimodal reward function. Popular approaches like CEM and CMA-ES greedily focus on promising regions of the search space and may get trapped in local maxima. DOO and VOOT balance exploration and exploitation, but use space partitioning strategies independent of the reward function to be optimized. Recently, LaMCTS empirically learns to partition the search space in a reward-sensitive manner for black-box optimization. In this paper, we develop a novel formal regret analysis for when and why such an adaptive region partitioning scheme works. We also propose a new path planning method LaP3 which improves the function value estimation within each sub-region, and uses a latent representation of the search space. Empirically, LaP3 outperforms existing path planning methods in 2D navigation tasks, especially in the presence of difficult-to-escape local optima, and shows benefits when plugged into the planning components of model-based RL such as PETS. These gains transfer to highly multimodal real-world tasks, where we outperform strong baselines in compiler phase ordering by up to 39% on average across 9 tasks, and in molecular design by up to 0.4 on properties on a 0-1 scale. Code is available at https://github.com/yangkevin2/neurips2021-lap3.

cs.AI↗

arXiv2021

Improving Molecular Design by Stochastic Iterative Target Augmentation

Generative models in molecular design tend to be richly parameterized, data-hungry neural models, as they must create complex structured objects as outputs. Estimating such models from data may be challenging due to the lack of sufficient training data. In this paper, we propose a surprisingly effective self-training approach for iteratively creating additional molecular targets. We first pre-train the generative model together with a simple property predictor. The property predictor is then used as a likelihood model for filtering candidate structures from the generative model. Additional targets are iteratively produced and used in the course of stochastic EM iterations to maximize the log-likelihood that the candidate structures are accepted. A simple rejection (re-weighting) sampler suffices to draw posterior samples since the generative model is already reasonable after pre-training. We demonstrate significant gains over strong baselines for both unconditional and conditional molecular design. In particular, our approach outperforms the previous state-of-the-art in conditional molecular design by over 10% in absolute gain. Finally, we show that our approach is useful in other domains as well, such as program synthesis.

cs.LG↗

arXiv2021

A Streaming Approach For Efficient Batched Beam Search

We propose an efficient batching strategy for variable-length decoding on GPU architectures. During decoding, when candidates terminate or are pruned according to heuristics, our streaming approach periodically "refills" the batch before proceeding with a selected subset of candidates. We apply our method to variable-width beam search on a state-of-the-art machine translation model. Our method decreases runtime by up to 71% compared to a fixed-width beam search baseline and 17% compared to a variable-width baseline, while matching baselines' BLEU. Finally, experiments show that our method can speed up decoding in other domains, such as semantic and syntactic parsing.

cs.CL↗

arXiv2021

FUDGE: Controlled Text Generation With Future Discriminators

We propose Future Discriminators for Generation (FUDGE), a flexible and modular method for controlled text generation. Given a pre-existing model G for generating text from a distribution of interest, FUDGE enables conditioning on a desired attribute a (for example, formality) while requiring access only to G's output logits. FUDGE learns an attribute predictor operating on a partial sequence, and uses this predictor's outputs to adjust G's original probabilities. We show that FUDGE models terms corresponding to a Bayesian decomposition of the conditional distribution of G given attribute a. Moreover, FUDGE can easily compose predictors for multiple desired attributes. We evaluate FUDGE on three tasks -- couplet completion in poetry, topic control in language generation, and formality change in machine translation -- and observe gains in all three tasks.

cs.CL↗

arXiv2021

Lower-Order Biases Second Moments of Dirichlet Coefficients in Families of $L$-Functions

Let $\mathcal E: y^2 = x^3 + A(T)x + B(T)$ be a nontrivial one-parameter family of elliptic curves over $\mathbb{Q}(T)$, with $A(T), B(T) \in \mathbb Z(T)$, and consider the $k$\textsuperscript{th} moments $A_{k,\mathcal{E}}(p) := \sum_{t (p)} a_{\mathcal{E}_t}(p)^k$ of the Dirichlet coefficients $a_{\mathcal{E}_t}(p) := p + 1 - |\mathcal{E}_t (\mathbb{F}_p)|$. Rosen and Silverman proved a conjecture of Nagao relating the first moment $A_{1,\mathcal{E}}(p)$ to the rank of the family over $\mathbb{Q}(T)$, and Michel proved that if $j(T)$ is not constant then the second moment is equal to $A_{2,\mathcal{E}}(p) = p^2 + O(p^{3/2})$. Cohomological arguments show that the lower order terms are of sizes $p^{3/2}, p, p^{1/2}$, and $1$. In every case we are able to analyze in closed form, the largest lower order term in the second moment expansion that does not average to zero is on average negative, though numerics suggest this may fail for families of moderate rank. We prove this Bias Conjecture for several large classes of families, including families with rank, complex multiplication, and constant $j(T)$-invariant. We also study the analogous Bias Conjecture for families of Dirichlet characters, holomorphic forms on GL$(2)/\mathbb{Q}$, and their symmetric powers and Rankin-Selberg convolutions. We identify all lower order terms in large classes of families, shedding light on the arithmetic objects controlling these terms. The negative bias in these lower order terms has implications toward the excess rank conjecture and the behavior of zeros near the central point in families of $L$-functions.

math.NT↗

arXiv2020

Uncertainty Quantification Using Neural Networks for Molecular Property Prediction

Uncertainty quantification (UQ) is an important component of molecular property prediction, particularly for drug discovery applications where model predictions direct experimental design and where unanticipated imprecision wastes valuable time and resources. The need for UQ is especially acute for neural models, which are becoming increasingly standard yet are challenging to interpret. While several approaches to UQ have been proposed in the literature, there is no clear consensus on the comparative performance of these models. In this paper, we study this question in the context of regression tasks. We systematically evaluate several methods on five benchmark datasets using multiple complementary performance metrics. Our experiments show that none of the methods we tested is unequivocally superior to all others, and none produces a particularly reliable ranking of errors across multiple datasets. While we believe these results show that existing UQ methods are not sufficient for all common use-cases and demonstrate the benefits of further research, we conclude with a practical recommendation as to which existing techniques seem to perform well relative to others.

cs.LG↗

arXiv2019

Analyzing Learned Molecular Representations for Property Prediction

Advancements in neural machinery have led to a wide range of algorithmic solutions for molecular property prediction. Two classes of models in particular have yielded promising results: neural networks applied to computed molecular fingerprints or expert-crafted descriptors, and graph convolutional neural networks that construct a learned molecular representation by operating on the graph structure of the molecule. However, recent literature has yet to clearly determine which of these two methods is superior when generalizing to new chemical space. Furthermore, prior research has rarely examined these new models in industry research settings in comparison to existing employed models. In this paper, we benchmark models extensively on 19 public and 16 proprietary industrial datasets spanning a wide variety of chemical endpoints. In addition, we introduce a graph convolutional model that consistently matches or outperforms models using fixed molecular descriptors as well as previous graph neural architectures on both public and proprietary datasets. Our empirical findings indicate that while approaches based on these representations have yet to reach the level of experimental reproducibility, our proposed model nevertheless offers significant improvements over models currently used in industrial workflows.

cs.LG↗

arXiv2019

Learning Multimodal Graph-to-Graph Translation for Molecular Optimization

We view molecular optimization as a graph-to-graph translation problem. The goal is to learn to map from one molecular graph to another with better properties based on an available corpus of paired molecules. Since molecules can be optimized in different ways, there are multiple viable translations for each input graph. A key challenge is therefore to model diverse translation outputs. Our primary contributions include a junction tree encoder-decoder for learning diverse graph translations along with a novel adversarial training method for aligning distributions of molecules. Diverse output distributions in our model are explicitly realized by low-dimensional latent vectors that modulate the translation process. We evaluate our model on multiple molecular optimization tasks and show that our model outperforms previous state-of-the-art baselines.

cs.LG↗

arXiv2018

Bulk Eigenvalue Correlation Statistics of Random Biregular Bipartite Graphs

This paper is the second chapter of three of the author's undergraduate thesis. In this paper, we consider the random matrix ensemble given by $(d_b, d_w)$-regular graphs on $M$ black vertices and $N$ white vertices, where $d_b \in [N^γ, N^{2/3 - γ}]$ for any $γ> 0$. We simultaneously prove that the bulk eigenvalue correlation statistics for both normalized adjacency matrices and their corresponding covariance matrices are stable for short times. Combined with an ergodicity analysis of the Dyson Brownian motion in another paper, this proves universality of bulk eigenvalue correlation statistics, matching normalized adjacency matrices with the GOE and the corresponding covariance matrices with the Gaussian Wishart Ensemble.

math.PR↗

arXiv2017

Local Correlation and Gap Statistics under Dyson Brownian Motion for Covariance Matrices

This paper is the third chapter of three of the author's undergraduate thesis. In this paper, we study the convergence of local bulk statistics for linearized covariance matrices under Dyson's Brownian motion. We consider deterministic initial data $V$ approximate the Dyson Brownian motion for linearized covariance matrices by the Wigner flow. Using universality results for the Wigner flow, we deduce universality for the linearized covariance matrices. We deduce bulk universality of averaged bulk correlation functions for both biregular bipartite graphs and honest covariance matrices. We also deduce a weak level repulsion estimate for the Dyson Brownian motion of linearized covariance matrices.

math.PR↗

arXiv2017

Local Marchenko-Pastur Law for Random Bipartite Graphs

This paper is the first chapter of three of the author's undergraduate thesis. We study the random matrix ensemble of covariance matrices arising from random $(d_b, d_w)$-regular bipartite graphs on a set of $M$ black vertices and $N$ white vertices, for $d_b \gg \log^4 N$. We simultaneously prove that the Green's functions of these covariance matrices and the adjacency matrices of the underlying graphs agree with the corresponding limiting law (e.g. Marchenko-Pastur law for covariance matrices) down to the optimal scale. This is an improvement from the previously known mesoscopic results. We obtain eigenvector delocalization for the covariance matrix ensemble as consequence, as well as a weak rigidity estimate.

math.PR↗

arXiv2017

The principle of the mutual energy

Advanced potential solution of Maxwell equations isn't often accepted. We have proven if without advanced potential, it is not possible to satisfy the Maxwell equations. We also shown that it is not the Poynting vector related energy current transferring energy in the space and it is the mutual energy really did that. A important result of the mutual energy theorem is that the advanced potential can suck energy from the transmitter. This energy is equal to the energy received at the receiver. Hence a transmitter can not send any energy out without the receiver. For two remote objects, the energy is transferred only can by the mutual energy of a retarded potential from the source together with an advanced potential from the sink. If the sucked energy is discrete, the summation of mutual energy current of the infinite background atoms or currents, which can be seen as receivers, is a random process. This means that the photon energy sent by the transmitter is actually grabbed by the receiver. Hence the photon from very beginning knows their destination. This receiver send advanced potential to the transmitter. This explanation also avoided the wave function collapse. The retarded potential first reached the receiver, cause the current in the receiver, the current of receiver send a advanced potential to the transmitter with a reversed time, in the same time, a photon minus-time-instantly runs from receiver to transmitter. In our normal feeling, the photon is still runs from the transmitter to the receiver with a positive time. How to transfer superluminal signal using advanced potential is also discussed.Maxwell equations as principle of the theory of the electromagnetic fields is replaced by the mutual energy principle.

physics.class-ph↗

arXiv2016

Random Matrix Ensembles with Split Limiting Behavior

We introduce a new family of $N\times N$ random real symmetric matrix ensembles, the $k$-checkerboard matrices, whose limiting spectral measure has two components which can be determined explicitly. All but $k$ eigenvalues are in the bulk, and their behavior, appropriately normalized, converges to the semi-circle as $N\to\infty$; the remaining $k$ are tightly constrained near $N/k$ and their distribution converges to the $k \times k$ hollow GOE ensemble (this is the density arising by modifying the GOE ensemble by forcing all entries on the main diagonal to be zero). Similar results hold for complex and quaternionic analogues. We isolate the two regimes by using matrix perturbation results and a nonstandard weight function for the eigenvalues, then derive their limiting distributions using a modification of the method of moments and analysis of the resulting combinatorics.

math-ph↗

arXiv2016

The mutual energy current interpretation for quantum mechanics

Quantum physics has the probability interpretation. From the knowledge of light, we know that wave is always spread out, and hence the electron wave should also spread out. That means the electron wave beam should like the light wave beam become diverged from the source. When the electron is received by an atom we thought the wave collapse. The place to collapse is depends on the probability calculated from the square of absolute value of the wave function. The recent new discovery tell us that the light is not just wave, it is a combination of waves, retarded potential and advanced potential. These two potentials together produce the mutual energy current or referred as M-current. Another light energy current is P-current related to Poynting vector. We found P-current doesn't carry any energy for light. The contribution of P-current to energy transfer can be omitted. The light energy is transferred only by M-current. The beam of M-current doesn't like the beam of P-current which is diverged from the source, instead, the M-current beam first diverges from a source and then converged to a sink. Since the M-current at the place to be received is localized at one electron, the concept of wave function collapse is needless. The probability results of light is because that we have use P-current to roughly calculate the M-current. We thought if Schrödinger knew today's light theory, he would for sure also build his wave theory for quantum mechanics similar to the new light theory with M-current. Hence we claim that the M-current theory is not only suitable to the light but also can be applied to the quantum physics. This means all particles are M-current. The M-current is composed of not only the retarded wave, but also the advanced wave. M-current is an inner product of a retarded and an advanced waves.

physics.gen-ph↗

arXiv2016

Some Results in the Theory of Low-lying Zeros: Determining the 1-level density, identifying the group symmetry and the arithmetic of moments of Satake parameters

While Random Matrix Theory has successfully modeled many quantities of families of L-functions, it frequently cannot see the family's arithmetic. In some situations this requires an extended theory that inserts arithmetic factors depending on the family, while in other cases these factors result in contributions which vanish in the limit, and are thus not detected. We review the general theory associated to one of the most important statistics, the n-level density of zeros near the central point. According to the Katz-Sarnak density conjecture, to each family of L-functions there is a corresponding symmetry group such that the behavior of zeros near the central point as the conductors tend to infinity agrees with the behavior of eigenvalues near 1 as the matrix size tends to infinity. We show how these calculations are done, emphasizing the techniques, methods and obstructions to improving the results, by considering in full detail a family of Dirichlet characters. We then describe how we may associate a symmetry constant to each family, and how to determine the symmetry group of a compound family in terms of the symmetries of the constituents. These calculations explain the remarkable universality of behavior, where the main terms are independent of the arithmetic (only the first two moments of the Satake parameters contribute in the limit; similar to the Central Limit Theorem, the higher moments are only felt in the rate of convergence). We end by exploring lower order terms in families of elliptic curves. We present evidence supporting a conjecture that the average second moment in one-parameter families without complex multiplication has, when appropriately viewed, a negative bias, and end with a discussion of the consequences of this bias on the distribution of low-lying zeros, in particular relations between such a bias and the observed excess rank in families.

math.NT↗