arXiv subjects

Yi Ouyang

Publications and source records attributed to Yi Ouyang.

At least 37 records · Page 2Linked to original sources

arXiv2024

On the Algebraic Classification of Non-singular Flexible Kokotsakis Polyhedra

Across various scientific and engineering domains, a growing interest in flexible and deployable structures is becoming evident. These structures facilitate seamless transitions between distinct states of shape and find broad applicability ranging from robotics and solar cells to meta-materials and architecture. In this contribution, we study a class of mechanisms known as Kokotsakis polyhedra with a quadrangular base. These are $3\times3$ quadrilateral meshes whose faces are rigid bodies and joined by hinges at the common edges. Compared to prior work, the quadrilateral faces do not have to be planar. In general, such meshes are not flexible, and the problem of finding and classifying the flexible ones is old, but until now largely unsolved. It appears that the tangent values of the dihedral angles between different faces are algebraically related through polynomials. Specifically, by fixing one angle as a parameter, the others can be parameterized algebraically and hence belong to an extended rational function field of the parameter. We use this approach to characterize shape restrictions resulting in flexible polyhedra.

math.RA↗

arXiv2023

COOPER: Coordinating Specialized Agents towards a Complex Dialogue Goal

In recent years, there has been a growing interest in exploring dialogues with more complex goals, such as negotiation, persuasion, and emotional support, which go beyond traditional service-focused dialogue systems. Apart from the requirement for much more sophisticated strategic reasoning and communication skills, a significant challenge of these tasks lies in the difficulty of objectively measuring the achievement of their goals in a quantifiable way, making it difficult for existing research to directly optimize the dialogue procedure towards them. In our work, we emphasize the multifaceted nature of complex dialogue goals and argue that it is more feasible to accomplish them by comprehensively considering and jointly promoting their different aspects. To this end, we propose a novel dialogue framework, Cooper, which coordinates multiple specialized agents, each dedicated to a specific dialogue goal aspect separately, to approach the complex objective. Through this divide-and-conquer manner, we make complex dialogue goals more approachable and elicit greater intelligence via the collaboration of individual agents. Experiments on persuasion and emotional support dialogues demonstrate the superiority of our method over a set of competitive baselines.

cs.CL↗

arXiv2023

On vectorial functions with maximal number of bent components

We study vectorial functions with maximal number of bent components in this paper. We first study the Walsh transform and nonlinearity of $F(x)=x^{2^e}h(\Tr_{2^{2m}/2^m}(x))$, where $e\geq0$ and $h(x)$ is a permutation over $\F_{2^m}$. If $h(x)$ is monomial, the nonlinearity of $F(x)$ is shown to be at most $ 2^{2m-1}-2^{\lfloor\frac{3m}{2}\rfloor}$ and some non-plateaued and plateaued functions attaining the upper bound are found. This gives a partial answer to the open problems proposed by Pott et al. and Anbar et al. If $h(x)$ is linear, the exact nonlinearity of $F(x)$ is determined. Secondly, we give a construction of vectorial functions with maximal number of bent components from known ones, thus obtain two new classes from the Niho class and the Maiorana-McFarland class. Our construction gives a partial answer to an open problem proposed by Pott et al., and also contains vectorial functions outside the complete Maiorana-McFarland class. Finally, we show that the vectorial function $F: \F_{2^{2m}}\rightarrow \F_{2^{2m}}$, $x\mapsto x^{2^m+1}+x^{2^i+1}$ has maximal number of bent components if and only if $i=0$.

cs.IT↗

arXiv2023

An Approach to Stochastic Dynamic Games with Asymmetric Information and Hidden Actions

We consider in discrete time, a general class of sequential stochastic dynamic games with asymmetric information with the following features. The underlying system has Markovian dynamics controlled by the agents' joint actions. Each agent's instantaneous utility depends on the current system state and the agents' joint actions. At each time instant each agent makes a private noisy observation of the current system state and the agents' actions in the previous time instant. In addition, at each time instant all agents have a common noisy observation of the current system state and their actions in the previous time instant. Each agent's actions are part of his private information. The objective is to determine Bayesian Nash Equilibrium (BNE) strategy profiles that are based on a compressed version of the agents' information and can be sequentially computed; such BNE strategy profiles may not always exist. We present an approach/methodology that achieves the above-stated objective, along with an instance of a game where BNE strategy profiles with the above-mentioned characteristics exist. We show that the methodology also works for the case where the agents have no common observations.

cs.MA↗

arXiv2022

A relaxed technical assumption for posterior sampling-based reinforcement learning for control of unknown linear systems

We revisit the Thompson sampling algorithm to control an unknown linear quadratic (LQ) system recently proposed by Ouyang et al (arXiv:1709.04047). The regret bound of the algorithm was derived under a technical assumption on the induced norm of the closed loop system. In this technical note, we show that by making a minor modification in the algorithm (in particular, ensuring that an episode does not end too soon), this technical assumption on the induced norm can be replaced by a milder assumption in terms of the spectral radius of the closed loop system. The modified algorithm has the same Bayesian regret of $\tilde{\mathcal{O}}(\sqrt{T})$, where $T$ is the time-horizon and the $\tilde{\mathcal{O}}(\cdot)$ notation hides logarithmic terms in~$T$.

eess.SY↗

arXiv2022

Grasping Core Rules of Time Series through Pure Models

Time series underwent the transition from statistics to deep learning, as did many other machine learning fields. Although it appears that the accuracy has been increasing as the model is updated in a number of publicly available datasets, it typically only increases the scale by several times in exchange for a slight difference in accuracy. Through this experiment, we point out a different line of thinking, time series, especially long-term forecasting, may differ from other fields. It is not necessary to use extensive and complex models to grasp all aspects of time series, but to use pure models to grasp the core rules of time series changes. With this simple but effective idea, we created PureTS, a network with three pure linear layers that achieved state-of-the-art in 80% of the long sequence prediction tasks while being nearly the lightest model and having the fastest running speed. On this basis, we discuss the potential of pure linear layers in both phenomena and essence. The ability to understand the core law contributes to the high precision of long-distance prediction, and reasonable fluctuation prevents it from distorting the curve in multi-step prediction like mainstream deep learning models, which is summarized as a pure linear neural network that avoids over-fluctuating. Finally, we suggest the fundamental design standards for lightweight long-step time series tasks: input and output should try to have the same dimension, and the structure avoids fragmentation and complex operations.

cs.LG↗

arXiv2022

Training a Resilient Q-Network against Observational Interference

Deep reinforcement learning (DRL) has demonstrated impressive performance in various gaming simulators and real-world applications. In practice, however, a DRL agent may receive faulty observation by abrupt interferences such as black-out, frozen-screen, and adversarial perturbation. How to design a resilient DRL algorithm against these rare but mission-critical and safety-crucial scenarios is an essential yet challenging task. In this paper, we consider a deep q-network (DQN) framework training with an auxiliary task of observational interferences such as artificial noises. Inspired by causal inference for observational interference, we propose a causal inference based DQN algorithm called causal inference Q-network (CIQ). We evaluate the performance of CIQ in several benchmark DQN environments with different types of interferences as auxiliary labels. Our experimental results show that the proposed CIQ method could achieve higher performance and more resilience against observational interferences.

cs.LG↗

arXiv2022

On abelian $2$-ramification torsion modules of quadratic fields

For a number field $F$ and a prime number $p$, the $\mathbb{Z}_p$-torsion module of the Galois group of the maximal abelian pro-$p$ extension of $F$ unramified outside $p$ over $F$, denoted as $\mathcal{T}_p(F)$, is an important subject in abelian $p$-ramification theory. In this paper we study the group $\mathcal{T}_2(F)=\mathcal{T}_2(m)$ of the quadratic field $F=\mathbb{Q}(\sqrt{ m})$. Firstly, assuming $m>0$, we prove an explicit $4$-rank formula for $\mathcal{T}_2(-m)$. Furthermore, applying this formula, we obtain the $4$-rank density of $\mathcal{T}_2$-groups of imaginary quadratic fields. Secondly, for $l$ an odd prime, we obtain results about the $2$-divisibility of orders of $\mathcal{T}_2(\pm l)$ and $\mathcal{T}_2(\pm 2l)$. In particular we find that $\#\mathcal{T}_2(l)\equiv 2\# \mathcal{T}_2(2l)\equiv h_2(-2l)\bmod{16}$ if $l\equiv 7\bmod{8}$ where $h_2(-2l)$ is the $2$-class number of $\mathbb{Q}(\sqrt{-2l})$. We then obtain density results for $\mathcal{T}_2(\pm l)$ and $\mathcal{T}_2(\pm 2l)$. Finally, based on our density results and numerical data, we propose distribution conjectures about $\mathcal{T}_p(F)$ when $F$ varies over real or imaginary quadratic fields for any prime $p$, and about $\mathcal{T}_2(\pm l)$ and $\mathcal{T}_2(\pm 2 l)$ when $l$ varies, in the spirit of Cohen-Lenstra heuristics. Our conjecture in the $\mathcal{T}_2(l)$ case is closely connected to Shanks-Sime-Washington's speculation on the distributions of the zeros of $2$-adic $L$-functions and to the distributions of the fundamental units.

math.NT↗

arXiv2021

Scalable regret for learning to control network-coupled subsystems with unknown dynamics

We consider the problem of controlling an unknown linear quadratic Gaussian (LQG) system consisting of multiple subsystems connected over a network. Our goal is to minimize and quantify the regret (i.e. loss in performance) of our strategy with respect to an oracle who knows the system model. Viewing the interconnected subsystems globally and directly using existing LQG learning algorithms for the global system results in a regret that increases super-linearly with the number of subsystems. Instead, we propose a new Thompson sampling based learning algorithm which exploits the structure of the underlying network. We show that the expected regret of the proposed algorithm is bounded by $\tilde{\mathcal{O}} \big( n \sqrt{T} \big)$ where $n$ is the number of subsystems, $T$ is the time horizon and the $\tilde{\mathcal{O}}(\cdot)$ notation hides logarithmic terms in $n$ and $T$. Thus, the regret scales linearly with the number of subsystems. We present numerical experiments to illustrate the salient features of the proposed algorithm.

eess.SY↗

arXiv2021

Factorization of Hilbert class polynomials over prime fields

Let $D$ be a negative integer congruent to $0$ or $1\bmod{4}$ and $\mathcal{O}=\mathcal{O}_D$ be the corresponding order of $ K=\mathbb{Q}(\sqrt{D})$. The Hilbert class polynomial $H_D(x)$ is the minimal polynomial of the $j$-invariant $ j_D=j(\mathbb{C}/\mathcal{O})$ of $\mathcal{O}$ over $K$. Let $n_D=(\mathcal{O}_{\mathbb{Q}( j_D)}:\mathbb{Z}[ j_D])$ denote the index of $\mathbb{Z}[ j_D]$ in the ring of integers of $\mathbb{Q}(j_D)$. Suppose $p$ is any prime. We completely determine the factorization of $H_D(x)$ in $\mathbb{F}_p[x]$ if either $p\nmid n_D$ or $p\nmid D$ is inert in $K$ and the $p$-adic valuation $v_p(n_D)\leq 3$. As an application, we analyze the key space of Oriented Supersingular Isogeny Diffie-Hellman (OSIDH) protocol proposed by Colò and Kohel in 2019 which is the roots set of the Hilbert class polynomial in $\mathbb{F}_{p^2}$.

math.NT↗

arXiv2020

Discovering Avoidable Planner Failures of Autonomous Vehicles using Counterfactual Analysis in Behaviorally Diverse Simulation

Automated Vehicles require exhaustive testing in simulation to detect as many safety-critical failures as possible before deployment on public roads. In this work, we focus on the core decision-making component of autonomous robots: their planning algorithm. We introduce a planner testing framework that leverages recent progress in simulating behaviorally diverse traffic participants. Using large scale search, we generate, detect, and characterize dynamic scenarios leading to collisions. In particular, we propose methods to distinguish between unavoidable and avoidable accidents, focusing especially on automatically finding planner-specific defects that must be corrected before deployment. Through experiments in complex multi-agent intersection scenarios, we show that our method can indeed find a wide range of critical planner failures.

cs.LG↗

arXiv2020

Thompson sampling for linear quadratic mean-field teams

We consider optimal control of an unknown multi-agent linear quadratic (LQ) system where the dynamics and the cost are coupled across the agents through the mean-field (i.e., empirical mean) of the states and controls. Directly using single-agent LQ learning algorithms in such models results in regret which increases polynomially with the number of agents. We propose a new Thompson sampling based learning algorithm which exploits the structure of the system model and show that the expected Bayesian regret of our proposed algorithm for a system with agents of $|M|$ different types at time horizon $T$ is $\tilde{\mathcal{O}} \big( |M|^{1.5} \sqrt{T} \big)$ irrespective of the total number of agents, where the $\tilde{\mathcal{O}}$ notation hides logarithmic factors in $T$. We present detailed numerical experiments to illustrate the salient features of the proposed algorithm.

eess.SY↗

arXiv2020

Enhanced Adversarial Strategically-Timed Attacks against Deep Reinforcement Learning

Recent deep neural networks based techniques, especially those equipped with the ability of self-adaptation in the system level such as deep reinforcement learning (DRL), are shown to possess many advantages of optimizing robot learning systems (e.g., autonomous navigation and continuous robot arm control.) However, the learning-based systems and the associated models may be threatened by the risks of intentionally adaptive (e.g., noisy sensor confusion) and adversarial perturbations from real-world scenarios. In this paper, we introduce timing-based adversarial strategies against a DRL-based navigation system by jamming in physical noise patterns on the selected time frames. To study the vulnerability of learning-based navigation systems, we propose two adversarial agent models: one refers to online learning; another one is based on evolutionary learning. Besides, three open-source robot learning and navigation control environments are employed to study the vulnerability under adversarial timing attacks. Our experimental results show that the adversarial timing attacks can lead to a significant performance drop, and also suggest the necessity of enhancing the robustness of robot learning systems.

cs.LG↗

arXiv2020

$\ell$-Class groups of fields in Kummer towers

Let $\ell$ and $p$ be prime numbers and $K_{n,m}=\mathbb{Q}(p^{\frac{1}{\ell^n}},ζ_{2\ell^{m}})$. We study the $\ell$-class group of $K_{n,m}$ in this paper. When $\ell=2$, we determine the structure of the $2$-class group of $K_{n,m}$ for all $(n,m)\in \mathbb{Z}_{\geq 0}^2$ in the case $p=2$ or $p\equiv 3, 5\bmod{8}$, and for $(n,m)=(n,0)$, $(n,1)$ or $(1,m)$ in the case $p\equiv 7\bmod{16}$, eneralizing the results of Parry about the $2$-divisibility of the class number of $K_{2,0}$. We also obtain results about the $\ell$-class group of $K_{n,m}$ when $\ell$ is odd and in particular $\ell=3$. The main tools we use are class field theory, including Chevalley's ambiguous class number formula and its generalization by Gras, and a stationary result about the $\ell$-class groups in the $2$-dimensional Kummer tower $\{K_{n,m}\}$.

math.NT↗

arXiv2020

Regret Bounds for Decentralized Learning in Cooperative Multi-Agent Dynamical Systems

Regret analysis is challenging in Multi-Agent Reinforcement Learning (MARL) primarily due to the dynamical environments and the decentralized information among agents. We attempt to solve this challenge in the context of decentralized learning in multi-agent linear-quadratic (LQ) dynamical systems. We begin with a simple setup consisting of two agents and two dynamically decoupled stochastic linear systems, each system controlled by an agent. The systems are coupled through a quadratic cost function. When both systems' dynamics are unknown and there is no communication among the agents, we show that no learning policy can generate sub-linear in $T$ regret, where $T$ is the time horizon. When only one system's dynamics are unknown and there is one-directional communication from the agent controlling the unknown system to the other agent, we propose a MARL algorithm based on the construction of an auxiliary single-agent LQ problem. The auxiliary single-agent problem in the proposed MARL algorithm serves as an implicit coordination mechanism among the two learning agents. This allows the agents to achieve a regret within $O(\sqrt{T})$ of the regret of the auxiliary single-agent problem. Consequently, using existing results for single-agent LQ regret, our algorithm provides a $\tilde{O}(\sqrt{T})$ regret bound. (Here $\tilde{O}(\cdot)$ hides constants and logarithmic factors). Our numerical experiments indicate that this bound is matched in practice. From the two-agent problem, we extend our results to multi-agent LQ systems with certain communication patterns.

cs.LG↗

arXiv2019

Learning Latent State Spaces for Planning through Reward Prediction

Model-based reinforcement learning methods typically learn models for high-dimensional state spaces by aiming to reconstruct and predict the original observations. However, drawing inspiration from model-free reinforcement learning, we propose learning a latent dynamics model directly from rewards. In this work, we introduce a model-based planning framework which learns a latent reward prediction model and then plans in the latent state-space. The latent representation is learned exclusively from multi-step reward prediction which we show to be the only necessary information for successful planning. With this framework, we are able to benefit from the concise model-free representation, while still enjoying the data-efficiency of model-based algorithms. We demonstrate our framework in multi-pendulum and multi-cheetah environments where several pendulums or cheetahs are shown to the agent but only one of which produces rewards. In these environments, it is important for the agent to construct a concise latent representation to filter out irrelevant observations. We find that our method can successfully learn an accurate latent reward prediction model in the presence of the irrelevant information while existing model-based methods fail. Planning in the learned latent state-space shows strong performance and high sample efficiency over model-free and model-based baselines.

cs.LG↗

arXiv2019

Large-Scale Traffic Signal Offset Optimization

The offset optimization problem seeks to coordinate and synchronize the timing of traffic signals throughout a network in order to enhance traffic flow and reduce stops and delays. Recently, offset optimization was formulated into a continuous optimization problem without integer variables by modeling traffic flow as sinusoidal. In this paper, we present a novel algorithm to solve this new formulation to near-global optimality on a large-scale. Specifically, we solve a convex relaxation of the nonconvex problem using a tree decomposition reduction, and use randomized rounding to recover a near-global solution. We prove that the algorithm always delivers solutions of expected value at least 0.785 times the globally optimal value. Moreover, assuming that the topology of the traffic network is "tree-like", we prove that the algorithm has near-linear time complexity with respect to the number of intersections. These theoretical guarantees are experimentally validated on the Berkeley, Manhattan, and Los Angeles traffic networks. In our numerical results, the empirical time complexity of the algorithm is linear, and the solutions have objectives within 0.99 times the globally optimal value.

math.OC↗

arXiv2019

Online Learning in Planar Pushing with Combined Prediction Model

Pushing is a useful robotic capability for positioning and reorienting objects. The ability to accurately predict the effect of pushes can enable efficient trajectory planning and complicated object manipulation. Physical prediction models for planar pushing have long been established, but their assumptions and requirements usually don't hold in most practical settings. Data-driven approaches can provide accurate predictions for offline data, but they often have generalizability issues. In this paper, we propose a combined prediction model and an online learning framework for planar push prediction. The combined model consists of a neural network module and analytical components with a low-dimensional parameter. We train the neural network offline using pre-collected pushing data. In online situations, the low-dimensional analytical parameter is learned directly from online pushes to quickly adapt to the new environments. We test our combined model and learning framework on real pushing experiments. Our experimental results show that our model is able to quickly adapt to new environments while achieving similar final prediction performance as that of pure neural network models.

cs.RO↗