arXiv subjects

Yang Shi

Publications and source records attributed to Yang Shi.

At least 163 records · Page 9Linked to original sources

arXiv2022

A Robust Distributed Model Predictive Control Framework for Consensus of Multi-Agent Systems with Input Constraints and Varying Delays

This paper studies the consensus problem of general linear discrete-time multi-agent systems (MAS) with input constraints and bounded time-varying communication delays. We propose a robust distributed model predictive control (DMPC) consensus protocol that integrates the offline consensus design with online DMPC optimization to exploit their respective advantages. More precisely, each agent is equipped with an offline consensus protocol, which is a priori designed, depending on its immediate neighbors' estimated states. Further, the estimation errors propagated over time due to inexact neighboring information are proved bounded under mild technical assumptions, based on which a robust DMPC strategy is deliberately designed to achieve robust consensus while satisfying input constraints. Moreover, it is shown that, with the suitably designed cost function and constraints, the feasibility of the associated optimization problem can be recursively ensured. We further provide the consensus convergence result of the constrained MAS in the presence of bounded varying delays. Finally, two numerical examples are given to verify the effectiveness of the proposed distributed consensus algorithm.

eess.SY↗

arXiv2022

Achieving Model Fairness in Vertical Federated Learning

Vertical federated learning (VFL) has attracted greater and greater interest since it enables multiple parties possessing non-overlapping features to strengthen their machine learning models without disclosing their private data and model parameters. Similar to other machine learning algorithms, VFL faces demands and challenges of fairness, i.e., the learned model may be unfairly discriminatory over some groups with sensitive attributes. To tackle this problem, we propose a fair VFL framework in this work. First, we systematically formulate the problem of training fair models in VFL, where the learning task is modelled as a constrained optimization problem. To solve it in a federated and privacy-preserving manner, we consider the equivalent dual form of the problem and develop an asynchronous gradient coordinate-descent ascent algorithm, where some active data parties perform multiple parallelized local updates per communication round to effectively reduce the number of communication rounds. The messages that the server sends to passive parties are deliberately designed such that the information necessary for local updates is released without intruding on the privacy of data and sensitive attributes. We rigorously study the convergence of the algorithm when applied to general nonconvex-concave min-max problems. We prove that the algorithm finds a $δ$-stationary point of the dual objective in $\mathcal{O}(δ^{-4})$ communication rounds under mild conditions. Finally, the extensive experiments on three benchmark datasets demonstrate the superior performance of our method in training fair models.

cs.LG↗

arXiv2022

Erato: Cooperative Data Story Editing via Fact Interpolation

As an effective form of narrative visualization, visual data stories are widely used in data-driven storytelling to communicate complex insights and support data understanding. Although important, they are difficult to create, as a variety of interdisciplinary skills, such as data analysis and design, are required. In this work, we introduce Erato, a human-machine cooperative data story editing system, which allows users to generate insightful and fluent data stories together with the computer. Specifically, Erato only requires a number of keyframes provided by the user to briefly describe the topic and structure of a data story. Meanwhile, our system leverages a novel interpolation algorithm to help users insert intermediate frames between the keyframes to smooth the transition. We evaluated the effectiveness and usefulness of the Erato system via a series of evaluations including a Turing test, a controlled user study, a performance validation, and interviews with three expert users. The evaluation results showed that the proposed interpolation technique was able to generate coherent story content and help users create data stories more efficiently.

cs.HC↗

arXiv2022

Accelerated Dual Averaging Methods for Decentralized Constrained Optimization

In this work, we study decentralized convex constrained optimization problems in networks. We focus on the dual averaging-based algorithmic framework that is well-documented to be superior in handling constraints and complex communication environments simultaneously. Two new decentralized dual averaging (DDA) algorithms are proposed. In the first one, a second-order dynamic average consensus protocol is tailored for DDA-type algorithms, which equips each agent with a provably more accurate estimate of the global dual variable than conventional schemes. We rigorously prove that the proposed algorithm attains $\mathcal{O}(1/t)$ convergence for general convex and smooth problems, for which existing DDA methods were only known to converge at $\mathcal{O}(1/\sqrt{t})$ prior to our work. In the second one, we use the extrapolation technique to accelerate the convergence of DDA. Compared to existing accelerated algorithms, where typically two different variables are exchanged among agents at each time, the proposed algorithm only seeks consensus on local gradients. Then, the extrapolation is performed based on two sequences of primal variables which are determined by the accumulations of gradients at two consecutive time instants, respectively. The algorithm is proved to converge at $\mathcal{O}(1)\left(\frac{1}{t^2}+\frac{1}{t(1-β)^2}\right)$, where $β$ denotes the second largest singular value of the mixing matrix. We remark that the condition for the algorithmic parameter to guarantee convergence does not rely on the spectrum of the mixing matrix, making itself easy to satisfy in practice. Finally, numerical results are presented to demonstrate the efficiency of the proposed methods.

math.OC↗

arXiv2022

Code-DKT: A Code-based Knowledge Tracing Model for Programming Tasks

Knowledge tracing (KT) models are a popular approach for predicting students' future performance at practice problems using their prior attempts. Though many innovations have been made in KT, most models including the state-of-the-art Deep KT (DKT) mainly leverage each student's response either as correct or incorrect, ignoring its content. In this work, we propose Code-based Deep Knowledge Tracing (Code-DKT), a model that uses an attention mechanism to automatically extract and select domain-specific code features to extend DKT. We compared the effectiveness of Code-DKT against Bayesian and Deep Knowledge Tracing (BKT and DKT) on a dataset from a class of 50 students attempting to solve 5 introductory programming assignments. Our results show that Code-DKT consistently outperforms DKT by 3.07-4.00% AUC across the 5 assignments, a comparable improvement to other state-of-the-art domain-general KT models over DKT. Finally, we analyze problem-specific performance through a set of case studies for one assignment to demonstrate when and how code features improve Code-DKT's predictions.

cs.SE↗

arXiv2022

Learning Similarity Preserving Binary Codes for Recommender Systems

Hashing-based Recommender Systems (RSs) are widely studied to provide scalable services. The existing methods for the systems combine three modules to achieve efficiency: feature extraction, interaction modeling, and binarization. In this paper, we study an unexplored module combination for the hashing-based recommender systems, namely Compact Cross-Similarity Recommender (CCSR). Inspired by cross-modal retrieval, CCSR utilizes Maximum a Posteriori similarity instead of matrix factorization and rating reconstruction to model interactions between users and items. We conducted experiments on MovieLens1M, Amazon product review, Ichiba purchase dataset and confirmed CCSR outperformed the existing matrix factorization-based methods. On the Movielens1M dataset, the absolute performance improvements are up to 15.69% in NDCG and 4.29% in Recall. In addition, we extensively studied three binarization modules: $sign$, scaled tanh, and sign-scaled tanh. The result demonstrated that although differentiable scaled tanh is popular in recent discrete feature learning literature, a huge performance drop occurs when outputs of scaled $tanh$ are forced to be binary.

cs.IR↗

arXiv2022

Direct visualization of percolating metal-insulator transition in V2O3 using scanning microwave impedance microscopy

Using the extensively studied V2O3 as a prototype system, we investigate the role of percolation in metal-insulator transition (MIT). We apply scanning microwave impedance microscopy to directly determine the metallic phase fraction p and relate it to the macroscopic conductance G, which shows a sudden jump when p reaches the percolation threshold. Interestingly, the conductance G exhibits a hysteretic behavior against p, suggesting two different percolating processes upon cooling and warming. Based on our image analysis and model simulation, we ascribe such hysteretic behavior to different domain nucleation and growth processes between cooling and warming, which is likely caused by the decoupled structural and electronic transitions in V2O3 during MIT. Our work provides a microscopic view of how the interplay of structural and electronic degrees of freedom affects MIT in strongly correlated systems.

cond-mat.str-el↗

arXiv2022

Self-triggered min-max DMPC for asynchronous multi-agent systems with communication delays

This paper studies the formation stabilization problem of asynchronous nonlinear multi-agent systems (MAS) subject to parametric uncertainties, external disturbances and bounded time-varying communication delays. A self-triggered min-max distributed model predictive control (DMPC) approach is proposed to handle these practical issues. At triggering instants, each agent solves a local min-max optimization problem based on local system states and predicted system states of neighbors, determines its next triggering instant and broadcasts its predicted state trajectory to its neighbors. As a result, the communication load is greatly alleviated while retaining robustness and comparable control performance compared to periodic algorithms. In order to handle time-varying delays, a novel consistency constraint is incorporated into each local optimization problem to restrict the deviation between the newest predicted states and previously broadcast predicted states. Consequently, each agent can utilize previously predicted states of its neighbors to achieve cooperation in the presence of time-varying delays and asynchronous communication induced by the distributed triggered scheduler. The recursive feasibility of the proposed algorithm and the closed-loop stability of MAS at triggering time instants are proven. Finally, numerical simulations are conducted to verify the efficiency of the proposed control method.

eess.SY↗

arXiv2022

Efficient Cross-Modal Retrieval via Deep Binary Hashing and Quantization

Cross-modal retrieval aims to search for data with similar semantic meanings across different content modalities. However, cross-modal retrieval requires huge amounts of storage and retrieval time since it needs to process data in multiple modalities. Existing works focused on learning single-source compact features such as binary hash codes that preserve similarities between different modalities. In this work, we propose a jointly learned deep hashing and quantization network (HQ) for cross-modal retrieval. We simultaneously learn binary hash codes and quantization codes to preserve semantic information in multiple modalities by an end-to-end deep learning architecture. At the retrieval step, binary hashing is used to retrieve a subset of items from the search space, then quantization is used to re-rank the retrieved items. We theoretically and empirically show that this two-stage retrieval approach provides faster retrieval results while preserving accuracy. Experimental results on the NUS-WIDE, MIR-Flickr, and Amazon datasets demonstrate that HQ achieves boosts of more than 7% in precision compared to supervised neural network-based compact coding models.

cs.IR↗

arXiv2021

Composite Adaptive Control for Anti-Unwinding Attitude Maneuvers: An Exponential Stability Result Without Persistent Excitation

This paper provides an exponential stability result for the adaptive anti-unwinding attitude tracking control problem of a rigid body with uncertain but constant inertia parameters, without requiring the satisfaction of persistent excitation (PE) condition. Specifically, a composite immersion and invariance (I&I) adaptive controller is derived by integrating a prediction-error-driven learning law into the dynamically scaled I&I adaptive control framework, wherein we modify the scaling factor so that the algorithm design does not involve any dynamic gains. To avoid the unwinding problem, a barrier function is introduced as the attitude error function, along with the tactful establishment of two crucial algebra properties for exponential stability analysis. The regressor filtering method is adopted in combination with the dynamic regressor extension and mixing (DREM) procedure to acquire the prediction error using only easily obtainable signals. In particular, aiding by a constructive liner time-varying filter, the scalar regressor of DREM is extended to generate a new exciting counterpart. In this way, the derived controller is shown to permit closed-loop exponential stability without PE, in the sense that both output-tracking and parameter estimation errors exponentially converge to zero. Further, the composite learning law is augmented with a power term to achieve synchronized finite/fixed-time parameter convergence. Numerical simulations are performed to verify the theoretical findings.

eess.SY↗

arXiv2021

A Survey on Federated Learning and its Applications for Accelerating Industrial Internet of Things

Federated learning (FL) brings collaborative intelligence into industries without centralized training data to accelerate the process of Industry 4.0 on the edge computing level. FL solves the dilemma in which enterprises wish to make the use of data intelligence with security concerns. To accelerate industrial Internet of things with the further leverage of FL, existing achievements on FL are developed from three aspects: 1) define terminologies and elaborate a general framework of FL for accommodating various scenarios; 2) discuss the state-of-the-art of FL on fundamental researches including data partitioning, privacy preservation, model optimization, local model transportation, personalization, motivation mechanism, platform & tools, and benchmark; 3) discuss the impacts of FL from the economic perspective. To attract more attention from industrial academia and practice, a FL-transformed manufacturing paradigm is presented, and future research directions of FL are given and possible immediate applications in Industry 4.0 domain are also proposed.

cs.DC↗

arXiv2021

Toward Semi-Automatic Misconception Discovery Using Code Embeddings

Understanding students' misconceptions is important for effective teaching and assessment. However, discovering such misconceptions manually can be time-consuming and laborious. Automated misconception discovery can address these challenges by highlighting patterns in student data, which domain experts can then inspect to identify misconceptions. In this work, we present a novel method for the semi-automated discovery of problem-specific misconceptions from students' program code in computing courses, using a state-of-the-art code classification model. We trained the model on a block-based programming dataset and used the learned embedding to cluster incorrect student submissions. We found these clusters correspond to specific misconceptions about the problem and would not have been easily discovered with existing approaches. We also discuss potential applications of our approach and how these misconceptions inform domain-specific insights into students' learning processes.

cs.LG↗

arXiv2020

Optimal Power Control for DoS Attack over Fading Channel: A Game-Theoretic Approach

In this paper, we investigate remote state estimation against an intelligent denial-of-service (DoS) attack over a vulnerable wireless network whose channel undergoes attenuation and distortion caused by fading. We use the sensor to observe system states and transmit its local state estimates to the remote center. Meanwhile, the attacker injects a jamming signal to destroy the packet accepted by the remote center and causes the performance degradation. Most of the existing works are built on a time-invariant channel state information (CSI) model in which the channel fading is stationary. However, the wireless communication channels are more prone to dynamic changes. To capture this time-variant property in the channel quality of the real-world networks, we study the fading channel network whose channel model is characterized by a generalized finite-state Markov chain. With the goals of two players in infinite-time horizon, we describe the conflicting characteristic between the attacker and the sensor with a general-sum stochastic game. Moreover, the Q-learning techniques are applied to obtain an optimal strategy pair at a Nash equilibrium. Also the monotone structure of the optimal stationary strategy is constructed under a sufficient condition. Besides, when channel gain is known a priori, except for the full Channel State Information (CSI), we also investigate the partial CSI, where Bayesian games are employed. Based on the player's own channel information and the belief on the channel distribution of other players, the energy strategy at a Nash equilibrium is obtained.

eess.SY↗

arXiv2020

Adaptive Model Predictive Control for A Class of Constrained Linear Systems with Parametric Uncertainties

This paper investigates adaptive model predictive control (MPC) for a class of constrained linear systems with unknown model parameters. This is also posed as the dual control problem consisting of system identification and regulation. We first propose an online strategy for simultaneous unknown parameter identification and uncertainty set estimation based on the recursive least square technique. The designed strategy provides a contractive sequence of uncertain parameter sets, and the convergence of parameter estimates is achieved under certain conditions. Second, by integrating tube MPC with proposed estimation routine, the developed adaptive MPC provides a less conservative solution to handle multiplicative uncertainties. This is made possible by constructing the polytopic tube based on the consistently updated nominal system and uncertain parameter set. In addition, the proposed method is extended with reduced computational complexity by sacrificing some degrees of optimality. We theoretically show that both designed adaptive MPC algorithms are recursively feasible, and the perturbed closed-loop system is asymptotically stable under standard assumptions. Finally, numerical simulations and comparison are given to illustrate the proposed method.

math.OC↗

arXiv2020

Calliope: Automatic Visual Data Story Generation from a Spreadsheet

Visual data stories shown in the form of narrative visualizations such as a poster or a data video, are frequently used in data-oriented storytelling to facilitate the understanding and memorization of the story content. Although useful, technique barriers, such as data analysis, visualization, and scripting, make the generation of a visual data story difficult. Existing authoring tools rely on users' skills and experiences, which are usually inefficient and still difficult. In this paper, we introduce a novel visual data story generating system, Calliope, which creates visual data stories from an input spreadsheet through an automatic process and facilities the easy revision of the generated story based on an online story editor. Particularly, Calliope incorporates a new logic-oriented Monte Carlo tree search algorithm that explores the data space given by the input spreadsheet to progressively generate story pieces (i.e., data facts) and organize them in a logical order. The importance of data facts is measured based on information theory, and each data fact is visualized in a chart and captioned by an automatically generated description. We evaluate the proposed technique through three example stories, two controlled experiments, and a series of interviews with 10 domain experts. Our evaluation shows that Calliope is beneficial to efficient visual data story generation.

cs.HC↗

arXiv2020

Chinese Lexical Simplification

Lexical simplification has attracted much attention in many languages, which is the process of replacing complex words in a given sentence with simpler alternatives of equivalent meaning. Although the richness of vocabulary in Chinese makes the text very difficult to read for children and non-native speakers, there is no research work for Chinese lexical simplification (CLS) task. To circumvent difficulties in acquiring annotations, we manually create the first benchmark dataset for CLS, which can be used for evaluating the lexical simplification systems automatically. In order to acquire more thorough comparison, we present five different types of methods as baselines to generate substitute candidates for the complex word that include synonym-based approach, word embedding-based approach, pretrained language model-based approach, sememe-based approach, and a hybrid approach. Finally, we design the experimental evaluation of these baselines and discuss their advantages and disadvantages. To our best knowledge, this is the first study for CLS task.

cs.CL↗

arXiv2020

TEST_POSITIVE at W-NUT 2020 Shared Task-3: Joint Event Multi-task Learning for Slot Filling in Noisy Text

The competition of extracting COVID-19 events from Twitter is to develop systems that can automatically extract related events from tweets. The built system should identify different pre-defined slots for each event, in order to answer important questions (e.g., Who is tested positive? What is the age of the person? Where is he/she?). To tackle these challenges, we propose the Joint Event Multi-task Learning (JOELIN) model. Through a unified global learning framework, we make use of all the training data across different events to learn and fine-tune the language model. Moreover, we implement a type-aware post-processing procedure using named entity recognition (NER) to further filter the predictions. JOELIN outperforms the BERT baseline by 17.2% in micro F1.

cs.CL↗

arXiv2020

Towards an $O(\frac{1}{t})$ convergence rate for distributed dual averaging

Recently, distributed dual averaging has received increasing attention due to its superiority in handling constraints and dynamic networks in multiagent optimization. However, all distributed dual averaging methods reported so far considered nonsmooth problems and have a convergence rate of $O(\frac{1}{\sqrt{t}})$. To achieve an improved convergence guarantee for smooth problems, this work proposes a second-order consensus scheme that assists each agent to locally track the global dual variable more accurately. This new scheme in conjunction with smoothness of the objective ensures that the accumulation of consensus error over time caused by incomplete global information is bounded from above. Then, a rigorous investigation of dual averaging with inexact gradient oracles is carried out to compensate the consensus error and achieve an $O(\frac{1}{t})$ convergence rate. The proposed method is examined in a large-scale LASSO problem.

math.OC↗