Searcharxiv⌕ Search

arXiv subjects

Yan Zhang

Publications and source records attributed to Yan Zhang.

At least 487 records · Page 27Linked to original sources
arXiv2021

Asynchronous Zeroth-Order Distributed Optimization with Residual Feedback

We consider a zeroth-order distributed optimization problem, where the global objective function is a black-box function and, as such, its gradient information is inaccessible to the local agents. Instead, the local agents can only use the values of the objective function to estimate the gradient and update their local decision variables. In this paper, we also assume that these updates are done asynchronously. To solve this problem, we propose an asynchronous zeroth-order distributed optimization method that relies on a one-point residual feedback to estimate the unknown gradient. We show that this estimator is unbiased under asynchronous updating, and theoretically analyze the convergence of the proposed method. We also present numerical experiments that demonstrate that our method outperforms two-point methods under asynchronous updating. To the best of our knowledge, this is the first asynchronous zeroth-order distributed optimization method that is also supported by theoretical guarantees.

math.OC↗
arXiv2021

r-GAT: Relational Graph Attention Network for Multi-Relational Graphs

Graph Attention Network (GAT) focuses on modelling simple undirected and single relational graph data only. This limits its ability to deal with more general and complex multi-relational graphs that contain entities with directed links of different labels (e.g., knowledge graphs). Therefore, directly applying GAT on multi-relational graphs leads to sub-optimal solutions. To tackle this issue, we propose r-GAT, a relational graph attention network to learn multi-channel entity representations. Specifically, each channel corresponds to a latent semantic aspect of an entity. This enables us to aggregate neighborhood information for the current aspect using relation features. We further propose a query-aware attention mechanism for subsequent tasks to select useful aspects. Extensive experiments on link prediction and entity classification tasks show that our r-GAT can model multi-relational graphs effectively. Also, we show the interpretability of our approach by case study.

cs.AI↗
arXiv2021

A New One-Point Residual-Feedback Oracle For Black-Box Learning and Control

Zeroth-order optimization (ZO) algorithms have been recently used to solve black-box or simulation-based learning and control problems, where the gradient of the objective function cannot be easily computed but can be approximated using the objective function values. Many existing ZO algorithms adopt two-point feedback schemes due to their fast convergence rate compared to one-point feedback schemes. However, two-point schemes require two evaluations of the objective function at each iteration, which can be impractical in applications where the data are not all available a priori, e.g., in online optimization. In this paper, we propose a novel one-point feedback scheme that queries the function value once at each iteration and estimates the gradient using the residual between two consecutive points. When optimizing a deterministic Lipschitz function, we show that the query complexity of ZO with the proposed one-point residual feedback matches that of ZO with the existing two-point schemes. Moreover, the query complexity of the proposed algorithm can be improved when the objective function has Lipschitz gradient. Then, for stochastic bandit optimization problems where only noisy objective function values are given, we show that ZO with one-point residual feedback achieves the same convergence rate as that of two-point scheme with uncontrollable data samples. We demonstrate the effectiveness of the proposed one-point residual feedback via extensive numerical experiments.

math.OC↗
arXiv2021

Dual-constrained Deep Semi-Supervised Coupled Factorization Network with Enriched Prior

Nonnegative matrix factorization is usually powerful for learning the "shallow" parts-based representation, but it clearly fails to discover deep hierarchical information within both the basis and representation spaces. In this paper, we technically propose a new enriched prior based Dual-constrained Deep Semi-Supervised Coupled Factorization Network, called DS2CF-Net, for learning the hierarchical coupled representations. To ex-tract hidden deep features, DS2CF-Net is modeled as a deep-structure and geometrical structure-constrained neural network. Specifically, DS2CF-Net designs a deep coupled factorization architecture using multi-layers of linear transformations, which coupled updates the bases and new representations in each layer. To improve the discriminating ability of learned deep representations and deep coefficients, our network clearly considers enriching the supervised prior by the joint deep coefficients-regularized label prediction, and incorporates enriched prior information as additional label and structure constraints. The label constraint can enable the samples of the same label to have the same coordinate in the new feature space, while the structure constraint forces the coefficient matrices in each layer to be block-diagonal so that the enhanced prior using the self-expressive label propagation are more accurate. Our network also integrates the adaptive dual-graph learning to retain the local manifold structures of both the data manifold and feature manifold by minimizing the reconstruction errors in each layer. Extensive experiments on several real databases demonstrate that our DS2CF-Net can obtain state-of-the-art performance for representation learning and clustering.

cs.LG↗
arXiv2021

Learning Motion Priors for 4D Human Body Capture in 3D Scenes

Recovering high-quality 3D human motion in complex scenes from monocular videos is important for many applications, ranging from AR/VR to robotics. However, capturing realistic human-scene interactions, while dealing with occlusions and partial views, is challenging; current approaches are still far from achieving compelling results. We address this problem by proposing LEMO: LEarning human MOtion priors for 4D human body capture. By leveraging the large-scale motion capture dataset AMASS, we introduce a novel motion smoothness prior, which strongly reduces the jitters exhibited by poses recovered over a sequence. Furthermore, to handle contacts and occlusions occurring frequently in body-scene interactions, we design a contact friction term and a contact-aware motion infiller obtained via per-instance self-supervised training. To prove the effectiveness of the proposed motion priors, we combine them into a novel pipeline for 4D human body capture in 3D scenes. With our pipeline, we demonstrate high-quality 4D human body capture, reconstructing smooth motions and physically plausible body-scene interactions. The code and data are available at https://sanweiliti.github.io/LEMO/LEMO.html.

cs.CV↗
arXiv2021

Detecting Mixing Services via Mining Bitcoin Transaction Network with Hybrid Motifs

As the first decentralized peer-to-peer (P2P) cryptocurrency system allowing people to trade with pseudonymous addresses, Bitcoin has become increasingly popular in recent years. However, the P2P and pseudonymous nature of Bitcoin make transactions on this platform very difficult to track, thus triggering the emergence of various illegal activities in the Bitcoin ecosystem. Particularly, mixing services in Bitcoin, originally designed to enhance transaction anonymity, have been widely employed for money laundry to complicate trailing illicit fund. In this paper, we focus on the detection of the addresses belonging to mixing services, which is an important task for anti-money laundering in Bitcoin. Specifically, we provide a feature-based network analysis framework to identify statistical properties of mixing services from three levels, namely, network level, account level and transaction level. To better characterize the transaction patterns of different types of addresses, we propose the concept of Attributed Temporal Heterogeneous motifs (ATH motifs). Moreover, to deal with the issue of imperfect labeling, we tackle the mixing detection task as a Positive and Unlabeled learning (PU learning) problem and build a detection model by leveraging the considered features. Experiments on real Bitcoin datasets demonstrate the effectiveness of our detection model and the importance of hybrid motifs including ATH motifs in mixing detection.

cs.SI↗
arXiv2021

Evidence of a hidden flux phase in the topological kagome metal CsV$_3$Sb$_5$

Phase transitions governed by spontaneous time reversal symmetry breaking (TRSB) have long been sought in many quantum systems, including materials with anomalous Hall effect (AHE), cuprate high temperature superconductors, Iridates and so on. However, experimentally identifying such a phase transition is extremely challenging because the transition is hidden from many experimental probes. Here, using zero-field muon spin relaxation (ZF-$μ$SR) technique, we observe strong TRSB signals below 70 K in the newly discovered kagome superconductor CsV$_3$Sb$_5$. The TRSB state emerges from the 2 x 2 charge density wave (CDW) phase present below ~ 95 K. By carrying out optical second-harmonic generation (SHG) experiments, we also find that inversion symmetry is maintained in the temperature range of interest. Combining all the experimental results and symmetry constraints, we conclude that the interlayer coupled chiral flux phase (CFP) is the most promising candidate for the TRSB state among all theoretical proposals of orbital current orders. Thus, this prototypical kagome metal CsV3Sb5 can be a platform to establish a TRSB current-ordered state and explore its relationship with CDW, giant AHE, and superconductivity.

cond-mat.supr-con↗
arXiv2021

Visualization for Dichotomous Variables, the Independence and Markov chains

In probability theory, the independence is a very fundamental concept, but with a little mystery. People can always easily manipulate it logistically but not geometrically, especially when it comes to the independence relationships among more that two variables, which may also involve conditional independence. Here I am particularly interested in visualizing Markov chains which have the well known memoryless property. I am not talking about drawing the transition graph, instead, I will draw all events of the Markov process in a single plot. Here, to simplify the question, this work will only consider dichotomous variables, but all the methods actually can be generalized to arbitrary set of discrete variables.

math.HO↗
arXiv2021

Dynamic Urban Planning: an Agent-Based Model Coupling Mobility Mode and Housing Choice. Use case Kendall Square

As cities become increasingly populated, urban planning plays a key role in ensuring the equitable and inclusive development of metropolitan areas. MIT City Science group created a data-driven tangible platform, CityScope, to help different stakeholders, such as government representatives, urban planners, developers, and citizens, collaboratively shape the urban scenario through the real-time impact analysis of different urban interventions. This paper presents an agent-based model that characterizes citizens' behavioural patterns with respect to housing and mobility choice that will constitute the first step in the development of a dynamic incentive system for an open interactive governance process. The realistic identification and representation of the criteria that affect this decision-making process will help understand and evaluate the impacts of potential housing incentives that aim to promote urban characteristics such as equality, diversity, walkability, and efficiency. The calibration and validation of the model have been performed in a well-known geographic area for the Group: Kendall Square in Cambridge, MA.

cs.MA↗
arXiv2021

Blockchain and 6G: The Future of Secure and Ubiquitous Communication

The future communication will be characterized by ubiquitous connectivity and security. These features will be essential requirements for the efficient functioning of the futuristic applications. In this paper, in order to highlight the impact of blockchain and 6G on the future communication systems, we categorize these application requirements into two broad groups. In the first category, called Requirement Group I \mbox{(RG-I)}, we include the performance-related needs on data rates, latency, reliability and massive connectivity, while in the second category, called Requirement Group II \mbox{(RG-II)}, we include the security-related needs on data integrity, non-repudiability, and auditability. With blockchain and 6G, the network decentralization and resource sharing would minimize resource under-utilization thereby facilitating RG-I targets. Furthermore, through appropriate selection of blockchain type and consensus algorithms, RG-II needs of 6G applications can also be readily addressed. Through this study, the combination of blockchain and 6G emerges as an elegant solution for secure and ubiquitous future communication.

cs.CR↗
arXiv2021

Converged Reconfigurable Intelligent Surface and Mobile Edge Computing for Space Information Networks

Space information networks (SIN) are facing an ever-increasing thirst for high-speed and high-capacity seamless data transmission due to the integration of ground, air, and space communications. However, this imposes a new paradigm on the architecture design of the integrated SIN. Recently, reconfigurable intelligent surfaces (RISs) and mobile edge computing (MEC) are the most promising techniques, conceived to improve communication and computation capability by reconfiguring the wireless propagation environment and offloading. Hence, converging RISs and MEC in SIN is becoming an effort to reap the double benefits of computation and communication. In this article, we propose an RIS-assisted collaborative MEC architecture for SIN and discuss its implementation. Then we present its potential benefits, major challenges, and feasible applications. Subsequently, we study different cases to evaluate the system data rate and latency. Finally, we conclude with a list of open issues in this research area.

cs.NI↗
arXiv2021

Learning without Knowing: Unobserved Context in Continuous Transfer Reinforcement Learning

In this paper, we consider a transfer Reinforcement Learning (RL) problem in continuous state and action spaces, under unobserved contextual information. For example, the context can represent the mental view of the world that an expert agent has formed through past interactions with this world. We assume that this context is not accessible to a learner agent who can only observe the expert data. Then, our goal is to use the context-aware expert data to learn an optimal context-unaware policy for the learner using only a few new data samples. Such problems are typically solved using imitation learning that assumes that both the expert and learner agents have access to the same information. However, if the learner does not know the expert context, using the expert data alone will result in a biased learner policy and will require many new data samples to improve. To address this challenge, in this paper, we formulate the learning problem as a causal bound-constrained Multi-Armed-Bandit (MAB) problem. The arms of this MAB correspond to a set of basis policy functions that can be initialized in an unsupervised way using the expert data and represent the different expert behaviors affected by the unobserved context. On the other hand, the MAB constraints correspond to causal bounds on the accumulated rewards of these basis policy functions that we also compute from the expert data. The solution to this MAB allows the learner agent to select the best basis policy and improve it online. And the use of causal bounds reduces the exploration variance and, therefore, improves the learning rate. We provide numerical experiments on an autonomous driving example that show that our proposed transfer RL method improves the learner's policy faster compared to existing imitation learning methods and enjoys much lower variance during training.

cs.LG↗
arXiv2021

DynaEval: Unifying Turn and Dialogue Level Evaluation

A dialogue is essentially a multi-turn interaction among interlocutors. Effective evaluation metrics should reflect the dynamics of such interaction. Existing automatic metrics are focused very much on the turn-level quality, while ignoring such dynamics. To this end, we propose DynaEval, a unified automatic evaluation framework which is not only capable of performing turn-level evaluation, but also holistically considers the quality of the entire dialogue. In DynaEval, the graph convolutional network (GCN) is adopted to model a dialogue in totality, where the graph nodes denote each individual utterance and the edges represent the dependency between pairs of utterances. A contrastive loss is then applied to distinguish well-formed dialogues from carefully constructed negative samples. Experiments show that DynaEval significantly outperforms the state-of-the-art dialogue coherence model, and correlates strongly with human judgements across multiple dialogue evaluation aspects at both turn and dialogue level.

cs.CL↗
arXiv2021

Naked emergence of an anti-Hale active region I. Overall evolution and magnetic properties

{In order to understand the emergence of the active region, we investigate the emerging process and magnetic properties of a naked anti-Hale active region during the period between August 24 to 25, 2018.} {Using the data from Helioseismic and Magnetic Imager on board the Soar Dynamic Observatory and the New Vacuum Solar Telescope, we calculated different evolving parameters (such as pole separation, tilt angle) and magnetic parameters (such as vertical electric current, force-free parameter, relative magnetic helicity) during the emergence of the active region. With these calculated parameters and some reasonable assumptions, we use two different methods to estimate the twist of the active region.} {The magnetic flux and pole separation continue increasing while the tilt angle exhibits a decreasing pattern during the emergence of the active region. The increase of the pole separation is mainly contributed as a result of the enhancement in the longitude direction. A power-law relationship between pole separation and total flux is found during the emergence of the active region. On the other hand, it is found that both the positive and negative electric currents increased equivalently and the average flux-weighted force-free parameter $\tilde α$ remains almost consistently positive, on the order of $\sim$ 10$^{-8}$ m$^{-1}$. The relative magnetic helicity is mainly contributed by the shear term, while the relative magnetic helicity injection flux of the shear term changes its sign at the latter stage of the emergence. The twist number of the whole active region remains on the order of 10$^{-1}$ turns during the emergence of the active region.} {We find that the magnetic flux tube with low twist also could emerge into the solar atmosphere.}

astro-ph.SR↗
arXiv2021

PanGu-$α$: Large-scale Autoregressive Pretrained Chinese Language Models with Auto-parallel Computation

Large-scale Pretrained Language Models (PLMs) have become the new paradigm for Natural Language Processing (NLP). PLMs with hundreds of billions parameters such as GPT-3 have demonstrated strong performances on natural language understanding and generation with \textit{few-shot in-context} learning. In this work, we present our practice on training large-scale autoregressive language models named PanGu-$α$, with up to 200 billion parameters. PanGu-$α$ is developed under the MindSpore and trained on a cluster of 2048 Ascend 910 AI processors. The training parallelism strategy is implemented based on MindSpore Auto-parallel, which composes five parallelism dimensions to scale the training task to 2048 processors efficiently, including data parallelism, op-level model parallelism, pipeline model parallelism, optimizer model parallelism and rematerialization. To enhance the generalization ability of PanGu-$α$, we collect 1.1TB high-quality Chinese data from a wide range of domains to pretrain the model. We empirically test the generation ability of PanGu-$α$ in various scenarios including text summarization, question answering, dialogue generation, etc. Moreover, we investigate the effect of model scales on the few-shot performances across a broad range of Chinese NLP tasks. The experimental results demonstrate the superior capabilities of PanGu-$α$ in performing various tasks under few-shot or zero-shot settings.

cs.CL↗
arXiv2021

A Joint Energy and Latency Framework for Transfer Learning over 5G Industrial Edge Networks

In this paper, we propose a transfer learning (TL)-enabled edge-CNN framework for 5G industrial edge networks with privacy-preserving characteristic. In particular, the edge server can use the existing image dataset to train the CNN in advance, which is further fine-tuned based on the limited datasets uploaded from the devices. With the aid of TL, the devices that are not participating in the training only need to fine-tune the trained edge-CNN model without training from scratch. Due to the energy budget of the devices and the limited communication bandwidth, a joint energy and latency problem is formulated, which is solved by decomposing the original problem into an uploading decision subproblem and a wireless bandwidth allocation subproblem. Experiments using ImageNet demonstrate that the proposed TL-enabled edge-CNN framework can achieve almost 85% prediction accuracy of the baseline by uploading only about 1% model parameters, for a compression ratio of 32 of the autoencoder.

cs.NI↗
arXiv2021

LEAP: Learning Articulated Occupancy of People

Substantial progress has been made on modeling rigid 3D objects using deep implicit representations. Yet, extending these methods to learn neural models of human shape is still in its infancy. Human bodies are complex and the key challenge is to learn a representation that generalizes such that it can express body shape deformations for unseen subjects in unseen, highly-articulated, poses. To address this challenge, we introduce LEAP (LEarning Articulated occupancy of People), a novel neural occupancy representation of the human body. Given a set of bone transformations (i.e. joint locations and rotations) and a query point in space, LEAP first maps the query point to a canonical space via learned linear blend skinning (LBS) functions and then efficiently queries the occupancy value via an occupancy network that models accurate identity- and pose-dependent deformations in the canonical space. Experiments show that our canonicalized occupancy estimation with the learned LBS functions greatly improves the generalization capability of the learned occupancy representation across various human shapes and poses, outperforming existing solutions in all settings.

cs.CV↗