arXiv subjects

Sheng Zhang

Publications and source records attributed to Sheng Zhang.

At least 217 records · Page 12Linked to original sources

arXiv2021

Transform the Non-linear Programming Problem to the Initial-value Problem to Solve

A dynamic method to solve the Non-linear Programming (NLP) problem with Equality Constraints (ECs) and Inequality Constraints (IECs) is proposed. Inspired by the Lyapunov continuous-time dynamics stability theory in the control field, the optimal solution is analogized to the stable equilibrium point of a finite-dimensional dynamic system and it is solved in an asymptotic manner. Under the premise that the Karush-Kuhn-Tucker (KKT) optimality condition exists, the Dynamic Optimization Equation (DOE), which has the same dimension to that of the optimization parameter vector, is established and its solution will converge to the optimal solution of the NLP globally with a theoretical guarantee. Using the matrix pseudo-inverse, the DOE is valid even without the linearly independent regularity requirement on the nonlinear constraints. In addition, the analytic expressions of the Lagrange multipliers and KKT multipliers, which adjoin the ECs and the IECs respectively during the entire optimization process, are also derived. Via the proposed method, the NLP may be transformed to the Initial-value Problem (IVP) to be solved, with mature Ordinary Differential Equation (ODE) integration methods. Illustrative examples are solved and it is shown that the dynamic method developed may produce the right numerical solutions with high efficiency.

math.OC↗

arXiv2021

Modular Self-Supervision for Document-Level Relation Extraction

Extracting relations across large text spans has been relatively underexplored in NLP, but it is particularly important for high-value domains such as biomedicine, where obtaining high recall of the latest findings is crucial for practical applications. Compared to conventional information extraction confined to short text spans, document-level relation extraction faces additional challenges in both inference and learning. Given longer text spans, state-of-the-art neural architectures are less effective and task-specific self-supervision such as distant supervision becomes very noisy. In this paper, we propose decomposing document-level relation extraction into relation detection and argument resolution, taking inspiration from Davidsonian semantics. This enables us to incorporate explicit discourse modeling and leverage modular self-supervision for each sub-problem, which is less noise-prone and can be further refined end-to-end via variational EM. We conduct a thorough evaluation in biomedical machine reading for precision oncology, where cross-paragraph relation mentions are prevalent. Our method outperforms prior state of the art, such as multi-scale learning and graph neural networks, by over 20 absolute F1 points. The gain is particularly pronounced among the most challenging relation instances whose arguments never co-occur in a paragraph.

cs.CL↗

arXiv2021

Spatially homogeneous few-cycle compression of Yb lasers via all-solid-state free-space soliton management

The high power and variable repetition rate of Yb femtosecond lasers make them very attractive for ultrafast science. However, for capturing sub-200 fs dynamics, efficient, high-fidelity, and high-stability pulse compression techniques are essential. Spectral broadening using an all-solid-state free-space geometry is particularly attractive, as it is simple, robust, and low-cost. However, spatial and temporal losses caused by spatio-spectral inhomogeneities have been a major challenge to date, due to coupled space-time dynamics associated with unguided nonlinear propagation. In this work, we use all-solid-state free-space compressors to demonstrate compression of 170 fs pulses at a wavelength of 1030nm from a Yb:KGW laser to ~9.2 fs, with a highly spatially homogeneous mode. This is achieved by ensuring that the nonlinear beam propagation in periodic layered Kerr media occurs in soliton modes and confining the nonlinear phase through each material layer to less than 1.0 rad. A remarkable spatio-spectral homogeneity of ~0.87 can be realized, which yields a high efficiency of >50% for few-cycle compression. The universality of the method is demonstrated by implementing high-quality pulse compression under a wide range of laser conditions. The high spatiotemporal quality and the exceptional stability of the compressed pulses are further verified by high-harmonic generation. This work represents the highest efficiency and the best spatio-spectral quality ever achieved by an all-solid-state free-space pulse compressor for few-cycle-pulse generation.

physics.optics↗

arXiv2021

Real-time Keypoints Detection for Autonomous Recovery of the Unmanned Ground Vehicle

The combination of a small unmanned ground vehicle (UGV) and a large unmanned carrier vehicle allows more flexibility in real applications such as rescue in dangerous scenarios. The autonomous recovery system, which is used to guide the small UGV back to the carrier vehicle, is an essential component to achieve a seamless combination of the two vehicles. This paper proposes a novel autonomous recovery framework with a low-cost monocular vision system to provide accurate positioning and attitude estimation of the UGV during navigation. First, we introduce a light-weight convolutional neural network called UGV-KPNet to detect the keypoints of the small UGV from the images captured by a monocular camera. UGV-KPNet is computationally efficient with a small number of parameters and provides pixel-level accurate keypoints detection results in real-time. Then, six degrees of freedom pose is estimated using the detected keypoints to obtain positioning and attitude information of the UGV. Besides, we are the first to create a large-scale real-world keypoints dataset of the UGV. The experimental results demonstrate that the proposed system achieves state-of-the-art performance in terms of both accuracy and speed on UGV keypoint detection, and can further boost the 6-DoF pose estimation for the UGV.

cs.CV↗

arXiv2021

DeepFaceLab: Integrated, flexible and extensible face-swapping framework

Deepfake defense not only requires the research of detection but also requires the efforts of generation methods. However, current deepfake methods suffer the effects of obscure workflow and poor performance. To solve this problem, we present DeepFaceLab, the current dominant deepfake framework for face-swapping. It provides the necessary tools as well as an easy-to-use way to conduct high-quality face-swapping. It also offers a flexible and loose coupling structure for people who need to strengthen their pipeline with other features without writing complicated boilerplate code. We detail the principles that drive the implementation of DeepFaceLab and introduce its pipeline, through which every aspect of the pipeline can be modified painlessly by users to achieve their customization purpose. It is noteworthy that DeepFaceLab could achieve cinema-quality results with high fidelity. We demonstrate the advantage of our system by comparing our approach with other face-swapping methods.For more information, please visit:https://github.com/iperov/DeepFaceLab/.

cs.CV↗

arXiv2021

Pattern Transfer Learning for Reinforcement Learning in Order Dispatching

Order dispatch is one of the central problems to ride-sharing platforms. Recently, value-based reinforcement learning algorithms have shown promising performance on this problem. However, in real-world applications, the non-stationarity of the demand-supply system poses challenges to re-utilizing data generated in different time periods to learn the value function. In this work, motivated by the fact that the relative relationship between the values of some states is largely stable across various environments, we propose a pattern transfer learning framework for value-based reinforcement learning in the order dispatch problem. Our method efficiently captures the value patterns by incorporating a concordance penalty. The superior performance of the proposed method is supported by experiments.

cs.LG↗

arXiv2021

Augmented Gaussian Random Field: Theory and Computation

We propose the novel augmented Gaussian random field (AGRF), which is a universal framework incorporating the data of observable and derivatives of any order. Rigorous theory is established. We prove that under certain conditions, the observable and its derivatives of any order are governed by a single Gaussian random field, which is the aforementioned AGRF. As a corollary, the statement ``the derivative of a Gaussian process remains a Gaussian process'' is validated, since the derivative is represented by a part of the AGRF. Moreover, a computational method corresponding to the universal AGRF framework is constructed. Both noiseless and noisy scenarios are considered. Formulas of the posterior distributions are deduced in a nice closed form. A significant advantage of our computational method is that the universal AGRF framework provides a natural way to incorporate arbitrary order derivatives and deal with missing data. We use four numerical examples to demonstrate the effectiveness of the computational method. The numerical examples are composite function, damped harmonic oscillator, Korteweg-De Vries equation, and Burgers' equation.

math.ST↗

arXiv2021

DouZero: Mastering DouDizhu with Self-Play Deep Reinforcement Learning

Games are abstractions of the real world, where artificial agents learn to compete and cooperate with other agents. While significant achievements have been made in various perfect- and imperfect-information games, DouDizhu (a.k.a. Fighting the Landlord), a three-player card game, is still unsolved. DouDizhu is a very challenging domain with competition, collaboration, imperfect information, large state space, and particularly a massive set of possible actions where the legal actions vary significantly from turn to turn. Unfortunately, modern reinforcement learning algorithms mainly focus on simple and small action spaces, and not surprisingly, are shown not to make satisfactory progress in DouDizhu. In this work, we propose a conceptually simple yet effective DouDizhu AI system, namely DouZero, which enhances traditional Monte-Carlo methods with deep neural networks, action encoding, and parallel actors. Starting from scratch in a single server with four GPUs, DouZero outperformed all the existing DouDizhu AI programs in days of training and was ranked the first in the Botzone leaderboard among 344 AI agents. Through building DouZero, we show that classic Monte-Carlo methods can be made to deliver strong results in a hard domain with a complex action space. The code and an online demo are released at https://github.com/kwai/DouZero with the hope that this insight could motivate future work.

cs.AI↗

arXiv2021

Active spintronic-metasurface terahertz emitters with tunable chirality

The ability to manipulate the electric-field vector of broadband terahertz waves is essential for applications of terahertz technologies in many areas, and can open up new possibilities for nonlinear terahertz spectroscopy and coherent control. Here, we propose a novel laser-driven terahertz emitter, consisting of metasurface-patterned magnetic multilayer heterostructures. Such hybrid terahertz emitters can combine the advantages of spintronic emitters for being ultrabroadband, efficient and flexible, as well as those of metasurfaces for the unique capability to manipulate terahertz waves with high precision and degree of freedom. Taking a stripe-patterned metasurface as an example, we demonstrate the generation of broadband terahertz waves with tunable chirality. Based on experimental and theoretical studies, the interplay between the laser-induced spintronic-origin currents and the metasurface-induced transient charges/currents are investigated, revealing the strong influence on the device functionality originated from both the light-matter interactions in individual metasurface units and the dynamic coupling between them. Our work not only offers a flexible, reliable and cost-effective solution for chiral terahertz wave generation and manipulation, but also opens a new pathway to metasurface-tailored spintronic devices for efficient vector-control of electromagnetic waves in the terahertz regime.

physics.optics↗

arXiv2021

TransHash: Transformer-based Hamming Hashing for Efficient Image Retrieval

Deep hamming hashing has gained growing popularity in approximate nearest neighbour search for large-scale image retrieval. Until now, the deep hashing for the image retrieval community has been dominated by convolutional neural network architectures, e.g. \texttt{Resnet}\cite{he2016deep}. In this paper, inspired by the recent advancements of vision transformers, we present \textbf{Transhash}, a pure transformer-based framework for deep hashing learning. Concretely, our framework is composed of two major modules: (1) Based on \textit{Vision Transformer} (ViT), we design a siamese vision transformer backbone for image feature extraction. To learn fine-grained features, we innovate a dual-stream feature learning on top of the transformer to learn discriminative global and local features. (2) Besides, we adopt a Bayesian learning scheme with a dynamically constructed similarity matrix to learn compact binary hash codes. The entire framework is jointly trained in an end-to-end manner.~To the best of our knowledge, this is the first work to tackle deep hashing learning problems without convolutional neural networks (\textit{CNNs}). We perform comprehensive experiments on three widely-studied datasets: \textbf{CIFAR-10}, \textbf{NUSWIDE} and \textbf{IMAGENET}. The experiments have evidenced our superiority against the existing state-of-the-art deep hashing methods. Specifically, we achieve 8.2\%, 2.6\%, 12.7\% performance gains in terms of average \textit{mAP} for different hash bit lengths on three public datasets, respectively.

cs.CV↗

arXiv2021

Joint Universal Syntactic and Semantic Parsing

While numerous attempts have been made to jointly parse syntax and semantics, high performance in one domain typically comes at the price of performance in the other. This trade-off contradicts the large body of research focusing on the rich interactions at the syntax-semantics interface. We explore multiple model architectures which allow us to exploit the rich syntactic and semantic annotations contained in the Universal Decompositional Semantics (UDS) dataset, jointly parsing Universal Dependencies and UDS to obtain state-of-the-art results in both formalisms. We analyze the behaviour of a joint model of syntax and semantics, finding patterns supported by linguistic theory at the syntax-semantics interface. We then investigate to what degree joint modeling generalizes to a multilingual setting, where we find similar trends across 8 languages.

cs.CL↗

arXiv2021

The Direct Shooting Method is a Complete Method

The direct shooting method is a classic approach for the solution of Optimal Control Problems (OCPs). It parameterizes the control variables and transforms the OCP to the Nonlinear Programming (NLP) problem to solve. This method is easy to use and it often introduces less parameters compared with all-variable parameterization method like the Pseudo-spectral (PS) method. However, it is long believed that its solution is not guaranteed to satisfy the optimality conditions of the OCP and the costates are not available in using this method. In this paper, we show that the direct shooting method may also provide the costate information, and it is proved that both the state and the costate solutions converge to the optimal as long as the control variable tends to the optimal, while the parameterized control may approach the optimal control with reasonable parameterization. This gives us the credit for the optimal control computation when employing the direct shooting method.

math.OC↗

arXiv2021

L-SNet: from Region Localization to Scale Invariant Medical Image Segmentation

Coarse-to-fine models and cascade segmentation architectures are widely adopted to solve the problem of large scale variations in medical image segmentation. However, those methods have two primary limitations: the first-stage segmentation becomes a performance bottleneck; the lack of overall differentiability makes the training process of two stages asynchronous and inconsistent. In this paper, we propose a differentiable two-stage network architecture to tackle these problems. In the first stage, a localization network (L-Net) locates Regions of Interest (RoIs) in a detection fashion; in the second stage, a segmentation network (S-Net) performs fine segmentation on the recalibrated RoIs; a RoI recalibration module between L-Net and S-Net eliminating the inconsistencies. Experimental results on the public dataset show that our method outperforms state-of-the-art coarse-to-fine models with negligible computation overheads.

cs.CV↗

arXiv2021

A submetric characterization of Rolewicz's property ($β$)

The main result is a submetric characterization of the class of Banach spaces admitting an equivalent norm with Rolewicz's property ($β$). As applications we prove that up to renorming, property ($β$) is stable under coarse Lipschitz embeddings and coarse quotients.

math.FA↗

arXiv2020

Variation Evolving for Optimal Control Computation, a Compact Way

A compact version of the variation evolving method (VEM) is developed in the primal variable space for optimal control computation. Following the idea that originates from the Lyapunov continuous-time dynamics stability theory in the control field, the optimal solution is analogized to the stable equilibrium point of a dynamic system and obtained asymptotically through the variation motion. With the introduction of a virtual dimension, namely the variation time, the evolution partial differential equation (EPDE), which seeks the optimal solution with a theoretical guarantee, is developed for the optimal control problem (OCP) with free terminal states, and the equivalent optimality conditions with no employment of costates are established in the primal space. These conditions show that the optimal feedback control law is generally not analytically available because the optimal control is related to the future states. Since the derived EPDE is suitable to be computed with the semi-discrete method in the field of PDE numerical calculation, the optimal solution may be obtained by solving the resulting finite-dimensional initial-value problem (IVP).

eess.SY↗

arXiv2020

SubTSBR to tackle high noise and outliers for data-driven discovery of differential equations

Data-driven discovery of differential equations has been an emerging research topic. We propose a novel algorithm subsampling-based threshold sparse Bayesian regression (SubTSBR) to tackle high noise and outliers. The subsampling technique is used for improving the accuracy of the Bayesian learning algorithm. It has two parameters: subsampling size and the number of subsamples. When the subsampling size increases with fixed total sample size, the accuracy of our algorithm goes up and then down. When the number of subsamples increases, the accuracy of our algorithm keeps going up. We demonstrate how to use our algorithm step by step and compare our algorithm with threshold sparse Bayesian regression (TSBR) for the discovery of differential equations. We show that our algorithm produces better results. We also discuss the merits of discovering differential equations from data and demonstrate how to discover models with random initial and boundary condition as well as models with bifurcations. The numerical examples are: (1) predator-prey model with noise, (2) shallow water equations with outliers, (3) heat diffusion with random initial and boundary condition, and (4) fish-harvesting problem with bifurcations.

stat.ML↗

arXiv2020

Knowledge-guided Open Attribute Value Extraction with Reinforcement Learning

Open attribute value extraction for emerging entities is an important but challenging task. A lot of previous works formulate the problem as a \textit{question-answering} (QA) task. While the collections of articles from web corpus provide updated information about the emerging entities, the retrieved texts can be noisy, irrelevant, thus leading to inaccurate answers. Effectively filtering out noisy articles as well as bad answers is the key to improving extraction accuracy. Knowledge graph (KG), which contains rich, well organized information about entities, provides a good resource to address the challenge. In this work, we propose a knowledge-guided reinforcement learning (RL) framework for open attribute value extraction. Informed by relevant knowledge in KG, we trained a deep Q-network to sequentially compare extracted answers to improve extraction accuracy. The proposed framework is applicable to different information extraction system. Our experimental results show that our method outperforms the baselines by 16.5 - 27.8\%.

cs.CL↗

arXiv2020

Worst-Case-Aware Curriculum Learning for Zero and Few Shot Transfer

Multi-task transfer learning based on pre-trained language encoders achieves state-of-the-art performance across a range of tasks. Standard approaches implicitly assume the tasks, for which we have training data, are equally representative of the tasks we are interested in, an assumption which is often hard to justify. This paper presents a more agnostic approach to multi-task transfer learning, which uses automated curriculum learning to minimize a new family of worst-case-aware losses across tasks. Not only do these losses lead to better performance on outlier tasks; they also lead to better performance in zero-shot and few-shot transfer settings.

cs.CL↗