arXiv subjects

Yuhui Wang

Publications and source records attributed to Yuhui Wang.

At least 73 records · Page 4Linked to original sources

arXiv2023

Guiding Online Reinforcement Learning with Action-Free Offline Pretraining

Offline RL methods have been shown to reduce the need for environment interaction by training agents using offline collected episodes. However, these methods typically require action information to be logged during data collection, which can be difficult or even impossible in some practical cases. In this paper, we investigate the potential of using action-free offline datasets to improve online reinforcement learning, name this problem Reinforcement Learning with Action-Free Offline Pretraining (AFP-RL). We introduce Action-Free Guide (AF-Guide), a method that guides online training by extracting knowledge from action-free offline datasets. AF-Guide consists of an Action-Free Decision Transformer (AFDT) implementing a variant of Upside-Down Reinforcement Learning. It learns to plan the next states from the offline dataset, and a Guided Soft Actor-Critic (Guided SAC) that learns online with guidance from AFDT. Experimental results show that AF-Guide can improve sample efficiency and performance in online training thanks to the knowledge from the action-free offline dataset. Code is available at https://github.com/Vision-CAIR/AF-Guide.

cs.LG↗

arXiv2023

Absence of topological protection of the interface states in $\mathbb{Z}_2$ photonic crystals

Inspired from electronic systems, topological photonics aims to engineer new optical devices with robust properties. In many cases, the ideas from topological phases protected by internal symmetries in fermionic systems are extended to those protected by crystalline symmetries. One such popular photonic crystal model was proposed by Wu and Hu in 2015 for realizing a bosonic $\mathbb{Z}_2$ topological crystalline insulator with robust topological edge states, which led to intense theoretical and experimental studies. However, rigorous relationship between the bulk topology and edge properties for this model, which is central to evaluating its advantage over traditional photonic designs, has never been established. In this work we revisit the expanded and shrunken honeycomb lattice structures proposed by Wu and Hu by using topological quantum chemistry tools and show that they are topologically trivial in the sense that symmetric, localized Wannier functions can be constructed. We show that the $\mathbb{Z}$ and $\mathbb{Z}_2$ type classification of the Wu-Hu model are equivalent to the $C_2T$ protected Euler class and the second Stiefel-Whitney class respectively, with the latter characterizing the full valence bands of Wu-Hu model indicating only a higher order topological insulator (HOTI) phase. We show that the Wu-Hu interface states can be gapped by a uniform topology preserving $C_6$ and $T$ symmetric perturbation, which demonstrates the trivial nature of the interface. Our results reveals that topology is not a necessary condition for the reported helical edge states in many photonics systems and opens new possibilities for interface engineering that may not be constrained to require topological designs.

cond-mat.mtrl-sci↗

arXiv2023

DropIT: Dropping Intermediate Tensors for Memory-Efficient DNN Training

A standard hardware bottleneck when training deep neural networks is GPU memory. The bulk of memory is occupied by caching intermediate tensors for gradient computation in the backward pass. We propose a novel method to reduce this footprint - Dropping Intermediate Tensors (DropIT). DropIT drops min-k elements of the intermediate tensors and approximates gradients from the sparsified tensors in the backward pass. Theoretically, DropIT reduces noise on estimated gradients and therefore has a higher rate of convergence than vanilla-SGD. Experiments show that we can drop up to 90\% of the intermediate tensor elements in fully-connected and convolutional layers while achieving higher testing accuracy for Visual Transformers and Convolutional Neural Networks on various tasks (e.g., classification, object detection, instance segmentation). Our code and models are available at https://github.com/chenjoya/dropit.

cs.CV↗

arXiv2022

Streaming, fast and accurate on-device Inverse Text Normalization for Automatic Speech Recognition

Automatic Speech Recognition (ASR) systems typically yield output in lexical form. However, humans prefer a written form output. To bridge this gap, ASR systems usually employ Inverse Text Normalization (ITN). In previous works, Weighted Finite State Transducers (WFST) have been employed to do ITN. WFSTs are nicely suited to this task but their size and run-time costs can make deployment on embedded applications challenging. In this paper, we describe the development of an on-device ITN system that is streaming, lightweight & accurate. At the core of our system is a streaming transformer tagger, that tags lexical tokens from ASR. The tag informs which ITN category might be applied, if at all. Following that, we apply an ITN-category-specific WFST, only on the tagged text, to reliably perform the ITN conversion. We show that the proposed ITN solution performs equivalent to strong baselines, while being significantly smaller in size and retaining customization capabilities.

cs.CL↗

arXiv2022

Zero Touch Coordinated UAV Network Formation for 360° Views of a Moving Ground Target in Remote VR Applications

Unmanned aerial vehicles (UAVs) with on-board cameras are widely used for remote surveillance and video capturing applications. In remote virtual reality (VR) applications, multiple UAVs can be used to capture different partially overlapping angles of the ground target, which can be stitched together to provide 360° views. This requires coordinated formation of UAVs that is adaptive to movements of the ground target. In this paper, we propose a joint UAV formation and tracking framework to capture 360° angles of the target. The proposed framework uses a zero touch approach for automated and adaptive reconfiguration of multiple UAVs in a coordinated manner without the need for human intervention. This is suited to both military and civilian applications. Simulation results demonstrate the convergence and configuration of the UAVs with arbitrary initial locations and orientations. The performance has been tested for various number of UAVs and different mobility patterns of the ground target.

cs.RO↗

arXiv2022

Proactive and Resilient UAV Orchestration for QoS Driven Connectivity and Coverage of Ground Users

Unmanned aerial vehicles (UAVs) are being successfully used to deliver communication services in applications such as extending the coverage of 5G cellular networks in remote areas, emergency situations, and enhancing the service quality in regions of dense user populations. While optimized placement solutions have been proposed in literature for ensuring quality of service (QoS) of users, they may not be ideal in highly mobile, autonomous, and diverse network scenarios. This paper proposes a proactive and resilient framework for distributed and dynamic orchestration of UAV small cells to provide QoS differentiation in the network. The UAV locations are tailored to end-user locations and service needs while ensuring that UAVs maintain localized backhaul connectivity. Simulation experiments show that under scenarios where physical placement can achieve service differentiation, the developed framework leads to a stable configurations of UAVs satisfying above 90% of user QoS requirements.

cs.NI↗

arXiv2022

An Intellectual Property Entity Recognition Method Based on Transformer and Technological Word Information

Patent texts contain a large amount of entity information. Through named entity recognition, intellectual property entity information containing key information can be extracted from it, helping researchers to understand the patent content faster. Therefore, it is difficult for existing named entity extraction methods to make full use of the semantic information at the word level brought about by professional vocabulary changes. This paper proposes a method for extracting intellectual property entities based on Transformer and technical word information , and provides accurate word vector representation in combination with the BERT language method. In the process of word vector generation, the technical word information extracted by IDCNN is added to improve the understanding of intellectual property entities Representation ability. Finally, the Transformer encoder that introduces relative position encoding is used to learn the deep semantic information of the text from the sequence of word vectors, and realize entity label prediction. Experimental results on public datasets and annotated patent datasets show that the method improves the accuracy of entity recognition.

cs.CL↗

arXiv2022

Resilient UAV Formation for Coverage and Connectivity of Spatially Dispersed Users

Unmanned aerial vehicles (UAVs) are a convenient choice for carrying mobile base stations to rapidly setup communication services for ground users. Unlike terrestrial networks, UAVs do not have fiber optic back-haul connectivity except when they are tethered to the ground, which restricts their mobility. In the absence of back-haul, e.g., in remote areas, emergency situations, or in battlefields, there is a need to ensure connectivity among UAVs in addition to coverage of ground users for creating local area networks. This paper provides a distributed and dynamic approach for UAV formation-based control for coverage and connectivity of spatially dispersed users. We use flocking dynamics as a guide to constructing tailored formations of UAVs on the fly. Simulation results demonstrate that if sufficient aerial base stations are available, the proposed approach results in a strongly connected network of UAVs that is able to provide both a backhaul and fronthaul network. The approach can be further extended to create multi-tier extra-terrestrial networks to cater for large-scale applications.

eess.SY↗

arXiv2022

A SBP-SAT FDTD Subgridding Method Using Staggered Yee's Grids Without Modifying Field Components

A summation-by-parts simultaneous approximation term (SBP-SAT) finite-difference time-domain (FDTD) subgridding method is proposed to model geometrically fine structures in this paper. Compared with our previous work, the proposed SBP-SAT FDTD method uses the staggered Yee's grid without adding or modifying any field components through field extrapolation on the boundaries to make the discrete operators satisfy the SBP property. The accuracy of extrapolation keeps consistency with that of the second-order finite-difference scheme near the boundaries. In addition, the SATs are used to weakly enforce the tangential boundary conditions between multiple mesh blocks with different mesh sizes. With carefully designed interpolation matrices and selected free parameters of the SATs, no dissipation occurs in the whole computational domain. Therefore, its long-time stability is theoretically guaranteed. Three numerical examples are carried out to validate its effectiveness. Results show that the proposed SBP-SAT FDTD subgridding method is stable, accurate, efficient, and easy to implement based on existing FDTD codes with only a few modifications.

cs.CE↗

arXiv2022

A Cooperative-Competitive Multi-Agent Framework for Auto-bidding in Online Advertising

In online advertising, auto-bidding has become an essential tool for advertisers to optimize their preferred ad performance metrics by simply expressing high-level campaign objectives and constraints. Previous works designed auto-bidding tools from the view of single-agent, without modeling the mutual influence between agents. In this paper, we instead consider this problem from a distributed multi-agent perspective, and propose a general $\underline{M}$ulti-$\underline{A}$gent reinforcement learning framework for $\underline{A}$uto-$\underline{B}$idding, namely MAAB, to learn the auto-bidding strategies. First, we investigate the competition and cooperation relation among auto-bidding agents, and propose a temperature-regularized credit assignment to establish a mixed cooperative-competitive paradigm. By carefully making a competition and cooperation trade-off among agents, we can reach an equilibrium state that guarantees not only individual advertiser's utility but also the system performance (i.e., social welfare). Second, to avoid the potential collusion behaviors of bidding low prices underlying the cooperation, we further propose bar agents to set a personalized bidding bar for each agent, and then alleviate the revenue degradation due to the cooperation. Third, to deploy MAAB in the large-scale advertising system with millions of advertisers, we propose a mean-field approach. By grouping advertisers with the same objective as a mean auto-bidding agent, the interactions among the large-scale advertisers are greatly simplified, making it practical to train MAAB efficiently. Extensive experiments on the offline industrial dataset and Alibaba advertising platform demonstrate that our approach outperforms several baseline methods in terms of social welfare and revenue.

cs.MA↗

arXiv2021

Greedy-Step Off-Policy Reinforcement Learning

Most of the policy evaluation algorithms are based on the theories of Bellman Expectation and Optimality Equation, which derive two popular approaches - Policy Iteration (PI) and Value Iteration (VI). However, multi-step bootstrapping is often at cross-purposes with and off-policy learning in PI-based methods due to the large variance of multi-step off-policy correction. In contrast, VI-based methods are naturally off-policy but subject to one-step learning.In this paper, we deduce a novel multi-step Bellman Optimality Equation by utilizing a latent structure of multi-step bootstrapping with the optimal value function. Via this new equation, we derive a new multi-step value iteration method that converges to the optimal value function with exponential contraction rate $\mathcal{O}(γ^n)$ but only linear computational complexity. Moreover, it can naturally derive a suite of multi-step off-policy algorithms that can safely utilize data collected by arbitrary policies without correction.Experiments reveal that the proposed methods are reliable, easy to implement and achieve state-of-the-art performance on a series of standard benchmark datasets.

cs.LG↗

arXiv2021

Mimic: An adaptive algorithm for multivariate time series classification

Time series data are valuable but are often inscrutable. Gaining trust in time series classifiers for finance, healthcare, and other critical applications may rely on creating interpretable models. Researchers have previously been forced to decide between interpretable methods that lack predictive power and deep learning methods that lack transparency. In this paper, we propose a novel Mimic algorithm that retains the predictive accuracy of the strongest classifiers while introducing interpretability. Mimic mirrors the learning method of an existing multivariate time series classifier while simultaneously producing a visual representation that enhances user understanding of the learned model. Experiments on 26 time series datasets support Mimic's ability to imitate a variety of time series classifiers visually and accurately.

cs.LG↗

arXiv2021

A Stable FDTD Subgridding Scheme with SBP-SAT for Transient Electromagnetic Analysis

We proposed a provably stable FDTD subgridding method for accurate and efficient transient electromagnetic analysis. In the proposed method, several field components are properly added to the boundaries of Yee's grid to make sure that the discrete operators meet the summation-by-parts (SBP) property. Then, by incorporating the simultaneous approximation terms (SATs) into the finite-difference time-domain (FDTD) method, the proposed FDTD subgridding method mimics the energy estimate of the continuous Maxwell's equations at the semi-discrete level to guarantee its stability. Further, to couple multiple mesh blocks with different mesh sizes, the interpolation matrices are also derived. The proposed FDTD subgridding method is accurate, efficient, easy to implement and be integrated into the existing FDTD codes with only simple modifications. At last, three numerical examples with fine structures are carried out to validate the effectiveness of the proposed method.

cs.CE↗

arXiv2020

SMIX($λ$): Enhancing Centralized Value Functions for Cooperative Multi-Agent Reinforcement Learning

Learning a stable and generalizable centralized value function (CVF) is a crucial but challenging task in multi-agent reinforcement learning (MARL), as it has to deal with the issue that the joint action space increases exponentially with the number of agents in such scenarios. This paper proposes an approach, named SMIX($λ$), to address the issue using an efficient off-policy centralized training method within a flexible learner search space. As importance sampling for such off-policy training is both computationally costly and numerically unstable, we proposed to use the $λ$-return as a proxy to compute the TD error. With this new loss function objective, we adopt a modified QMIX network structure as the base to train our model. By further connecting it with the ${Q(λ)}$ approach from an unified expectation correction viewpoint, we show that the proposed SMIX($λ$) is equivalent to ${Q(λ)}$ and hence shares its convergence properties, while without being suffered from the aforementioned curse of dimensionality problem inherent in MARL. Experiments on the StarCraft Multi-Agent Challenge (SMAC) benchmark demonstrate that our approach not only outperforms several state-of-the-art MARL methods by a large margin, but also can be used as a general tool to improve the overall performance of other CTDE-type algorithms by enhancing their CVFs.

cs.MA↗

arXiv2020

The Limit of the Batch Size

Large-batch training is an efficient approach for current distributed deep learning systems. It has enabled researchers to reduce the ImageNet/ResNet-50 training from 29 hours to around 1 minute. In this paper, we focus on studying the limit of the batch size. We think it may provide a guidance to AI supercomputer and algorithm designers. We provide detailed numerical optimization instructions for step-by-step comparison. Moreover, it is important to understand the generalization and optimization performance of huge batch training. Hoffer et al. introduced "ultra-slow diffusion" theory to large-batch training. However, our experiments show contradictory results with the conclusion of Hoffer et al. We provide comprehensive experimental results and detailed analysis to study the limitations of batch size scaling and "ultra-slow diffusion" theory. For the first time we scale the batch size on ImageNet to at least a magnitude larger than all previous work, and provide detailed studies on the performance of many state-of-the-art optimization schemes under this setting. We propose an optimization recipe that is able to improve the top-1 test accuracy by 18% compared to the baseline.

cs.LG↗

arXiv2020

Truly Proximal Policy Optimization

Proximal policy optimization (PPO) is one of the most successful deep reinforcement-learning methods, achieving state-of-the-art performance across a wide range of challenging tasks. However, its optimization behavior is still far from being fully understood. In this paper, we show that PPO could neither strictly restrict the likelihood ratio as it attempts to do nor enforce a well-defined trust region constraint, which means that it may still suffer from the risk of performance instability. To address this issue, we present an enhanced PPO method, named Truly PPO. Two critical improvements are made in our method: 1) it adopts a new clipping function to support a rollback behavior to restrict the difference between the new policy and the old one; 2) the triggering condition for clipping is replaced with a trust region-based one, such that optimizing the resulted surrogate objective function provides guaranteed monotonic improvement of the ultimate policy performance. It seems, by adhering more truly to making the algorithm proximal - confining the policy within the trust region, the new algorithm improves the original PPO on both sample efficiency and performance.

cs.LG↗

arXiv2020

Z_2 Photonic topological insulators in the visible wavelength range for robust nanoscale photonics

Topological photonics provides an ideal platform for demonstrating novel band topology concepts, which are also promising for robust waveguiding, communication and computation applications. However, many challenges such as extremely large device footprint and functionality at short wavelengths remain to be solved which are required to make practical and useful devices that can also couple to electronic excitations in many important organic and inorganic semiconductors. In this letter, we report an experimental realization of Z_2 photonic topological insulators with their topological edge state energies spanning across the visible wavelength range including in the sub-500 nm regime. The photonic structures are based on deformed hexagonal lattices with preserved six-fold rotational symmetry patterned on suspended SiNx membranes. The experimentally measured energy-momentum dispersion of the topological lattices directly show topological band inversion by the swapping of the brightness of the bulk energy bands, and also the helical edge states when the measurement is taken near the topological interface. The robust topological transport of the helical edge modes in real space is demonstrated by successfully guiding circularly polarized light beams unidirectionally through sharp kinks without major signal loss. This work paves the way for small footprint photonic topological devices working in the short wavelength range that can also be utilized to couple to excitons for unconventional light-matter interactions at the nanoscale.

physics.optics↗

arXiv2019

Trust Region-Guided Proximal Policy Optimization

Proximal policy optimization (PPO) is one of the most popular deep reinforcement learning (RL) methods, achieving state-of-the-art performance across a wide range of challenging tasks. However, as a model-free RL method, the success of PPO relies heavily on the effectiveness of its exploratory policy search. In this paper, we give an in-depth analysis on the exploration behavior of PPO, and show that PPO is prone to suffer from the risk of lack of exploration especially under the case of bad initialization, which may lead to the failure of training or being trapped in bad local optima. To address these issues, we proposed a novel policy optimization method, named Trust Region-Guided PPO (TRGPPO), which adaptively adjusts the clipping range within the trust region. We formally show that this method not only improves the exploration ability within the trust region but enjoys a better performance bound compared to the original PPO as well. Extensive experiments verify the advantage of the proposed method.

cs.LG↗