arXiv subjects

Ming Jin

Publications and source records attributed to Ming Jin.

At least 181 records · Page 10Linked to original sources

arXiv2022

Multivariate Time Series Forecasting with Dynamic Graph Neural ODEs

Multivariate time series forecasting has long received significant attention in real-world applications, such as energy consumption and traffic prediction. While recent methods demonstrate good forecasting abilities, they have three fundamental limitations. (i) Discrete neural architectures: Interlacing individually parameterized spatial and temporal blocks to encode rich underlying patterns leads to discontinuous latent state trajectories and higher forecasting numerical errors. (ii) High complexity: Discrete approaches complicate models with dedicated designs and redundant parameters, leading to higher computational and memory overheads. (iii) Reliance on graph priors: Relying on predefined static graph structures limits their effectiveness and practicability in real-world applications. In this paper, we address all the above limitations by proposing a continuous model to forecast $\textbf{M}$ultivariate $\textbf{T}$ime series with dynamic $\textbf{G}$raph neural $\textbf{O}$rdinary $\textbf{D}$ifferential $\textbf{E}$quations ($\texttt{MTGODE}$). Specifically, we first abstract multivariate time series into dynamic graphs with time-evolving node features and unknown graph structures. Then, we design and solve a neural ODE to complement missing graph topologies and unify both spatial and temporal message passing, allowing deeper graph propagation and fine-grained temporal information aggregation to characterize stable and precise latent spatial-temporal dynamics. Our experiments demonstrate the superiorities of $\texttt{MTGODE}$ from various perspectives on five time series benchmark datasets.

cs.LG↗

arXiv2022

Towards Graph Self-Supervised Learning with Contrastive Adjusted Zooming

Graph representation learning (GRL) is critical for graph-structured data analysis. However, most of the existing graph neural networks (GNNs) heavily rely on labeling information, which is normally expensive to obtain in the real world. Although some existing works aim to effectively learn graph representations in an unsupervised manner, they suffer from certain limitations, such as the heavy reliance on monotone contrastiveness and limited scalability. To overcome the aforementioned problems, we introduce a novel self-supervised graph representation learning algorithm via Graph Contrastive Adjusted Zooming, namely G-Zoom, to learn node representations by leveraging the proposed adjusted zooming scheme. Specifically, this mechanism enables G-Zoom to explore and extract self-supervision signals from a graph from multiple scales: micro (i.e., node-level), meso (i.e., neighborhood-level), and macro (i.e., subgraph-level). Firstly, we generate two augmented views of the input graph via two different graph augmentations. Then, we establish three different contrastiveness on the above three scales progressively, from node, neighboring, to subgraph level, where we maximize the agreement between graph representations across scales. While we can extract valuable clues from a given graph on the micro and macro perspectives, the neighboring-level contrastiveness offers G-Zoom the capability of a customizable option based on our adjusted zooming scheme to manually choose an optimal viewpoint that lies between the micro and macro perspectives to better understand the graph data. Additionally, to make our model scalable to large graphs, we employ a parallel graph diffusion approach to decouple model training from the graph size. We have conducted extensive experiments on real-world datasets, and the results demonstrate that our proposed model outperforms state-of-the-art methods consistently.

cs.LG↗

arXiv2022

Non-stationary Risk-sensitive Reinforcement Learning: Near-optimal Dynamic Regret, Adaptive Detection, and Separation Design

We study risk-sensitive reinforcement learning (RL) based on an entropic risk measure in episodic non-stationary Markov decision processes (MDPs). Both the reward functions and the state transition kernels are unknown and allowed to vary arbitrarily over time with a budget on their cumulative variations. When this variation budget is known a prior, we propose two restart-based algorithms, namely Restart-RSMB and Restart-RSQ, and establish their dynamic regrets. Based on these results, we further present a meta-algorithm that does not require any prior knowledge of the variation budget and can adaptively detect the non-stationarity on the exponential value functions. A dynamic regret lower bound is then established for non-stationary risk-sensitive RL to certify the near-optimality of the proposed algorithms. Our results also show that the risk control and the handling of the non-stationarity can be separately designed in the algorithm if the variation budget is known a prior, while the non-stationary detection mechanism in the adaptive algorithm depends on the risk parameter. This work offers the first non-asymptotic theoretical analyses for the non-stationary risk-sensitive RL in the literature.

cs.LG↗

arXiv2022

Hyper-Parameter Auto-Tuning for Sparse Bayesian Learning

Choosing the values of hyper-parameters in sparse Bayesian learning (SBL) can significantly impact performance. However, the hyper-parameters are normally tuned manually, which is often a difficult task. Most recently, effective automatic hyper-parameter tuning was achieved by using an empirical auto-tuner. In this work, we address the issue of hyper-parameter auto-tuning using neural network (NN)-based learning. Inspired by the empirical auto-tuner, we design and learn a NN-based auto-tuner, and show that considerable improvement in convergence rate and recovery performance can be achieved.

eess.SP↗

arXiv2022

Recognizing Nested Entities from Flat Supervision: A New NER Subtask, Feasibility and Challenges

Many recent named entity recognition (NER) studies criticize flat NER for its non-overlapping assumption, and switch to investigating nested NER. However, existing nested NER models heavily rely on training data annotated with nested entities, while labeling such data is costly. This study proposes a new subtask, nested-from-flat NER, which corresponds to a realistic application scenario: given data annotated with flat entities only, one may still desire the trained model capable of recognizing nested entities. To address this task, we train span-based models and deliberately ignore the spans nested inside labeled entities, since these spans are possibly unlabeled entities. With nested entities removed from the training data, our model achieves 54.8%, 54.2% and 41.1% F1 scores on the subset of spans within entities on ACE 2004, ACE 2005 and GENIA, respectively. This suggests the effectiveness of our approach and the feasibility of the task. In addition, the model's performance on flat entities is entirely unaffected. We further manually annotate the nested entities in the test set of CoNLL 2003, creating a nested-from-flat NER benchmark. Analysis results show that the main challenges stem from the data and annotation inconsistencies between the flat and nested entities.

cs.CL↗

arXiv2022

Variational Bayesian Inference Clustering Based Joint User Activity and Data Detection for Grant-Free Random Access in mMTC

Tailor-made for massive connectivity and sporadic access, grant-free random access has become a promising candidate access protocol for massive machine-type communications (mMTC). Compared with conventional grant-based protocols, grant-free random access skips the exchange of scheduling information to reduce the signaling overhead, and facilitates sharing of access resources to enhance access efficiency. However, some challenges remain to be addressed in the receiver design, such as unknown identity of active users and multi-user interference (MUI) on shared access resources. In this work, we deal with the problem of joint user activity and data detection for grant-free random access. Specifically, the approximate message passing (AMP) algorithm is first employed to mitigate MUI and decouple the signals of different users. Then, we extend the data symbol alphabet to incorporate the null symbols from inactive users. In this way, the joint user activity and data detection problem is formulated as a clustering problem under the Gaussian mixture model. Furthermore, in conjunction with the AMP algorithm, a variational Bayesian inference based clustering (VBIC) algorithm is developed to solve this clustering problem. Simulation results show that, compared with state-of-art solutions, the proposed AMP-combined VBIC (AMP-VBIC) algorithm achieves a significant performance gain in detection accuracy.

eess.SP↗

arXiv2022

Graph Self-Supervised Learning: A Survey

Deep learning on graphs has attracted significant interests recently. However, most of the works have focused on (semi-) supervised learning, resulting in shortcomings including heavy label reliance, poor generalization, and weak robustness. To address these issues, self-supervised learning (SSL), which extracts informative knowledge through well-designed pretext tasks without relying on manual labels, has become a promising and trending learning paradigm for graph data. Different from SSL on other domains like computer vision and natural language processing, SSL on graphs has an exclusive background, design ideas, and taxonomies. Under the umbrella of graph self-supervised learning, we present a timely and comprehensive review of the existing approaches which employ SSL techniques for graph data. We construct a unified framework that mathematically formalizes the paradigm of graph SSL. According to the objectives of pretext tasks, we divide these approaches into four categories: generation-based, auxiliary property-based, contrast-based, and hybrid approaches. We further describe the applications of graph SSL across various research fields and summarize the commonly used datasets, evaluation benchmark, performance comparison and open-source codes of graph SSL. Finally, we discuss the remaining challenges and potential future directions in this research field.

cs.LG↗

arXiv2022

A faithful solid-state spin-wave quantum memory for polarization qubits

Polarization-encoded qubits are particularly useful in quantum information tasks due to the easy transportation in a single spatial and temporal mode, the accurate qubit manipulation and the high robustness against decoherence. Reliable storage of polarization-encoded qubits is essential for the construction of large-scale quantum networks. Here we demonstrate a faithful quantum memory for photonic polarization qubits using the noiseless photon echo protocol implemented in a rare-earth-ion doped crystal (151Eu3+:Y2SiO5). Based on a detailed spectroscopic investigation on the 151Eu3+ ions at the site 2 of Y2SiO5 crystals, the qubit memory is implemented using a single piece of crystal which provides a near-uniform absorption for two orthogonal polarization states. A process fidelity of 0.919(24) is obtained for the storage of qubits carried by single-photon-level coherent pulses, which is beyond the maximal fidelity that can be achieved using the classical measure-and-prepare strategy. This compact device shall provide a useful solution for the construction of a long-lived transportable quantum memory and the memory-based quantum networks.

quant-ph↗

arXiv2022

Learning Neural Networks under Input-Output Specifications

In this paper, we examine an important problem of learning neural networks that certifiably meet certain specifications on input-output behaviors. Our strategy is to find an inner approximation of the set of admissible policy parameters, which is convex in a transformed space. To this end, we address the key technical challenge of convexifying the verification condition for neural networks, which is derived by abstracting the nonlinear specifications and activation functions with quadratic constraints. In particular, we propose a reparametrization scheme of the original neural network based on loop transformation, which leads to a convex condition that can be enforced during learning. This theoretical construction is validated in an experiment that specifies reachable sets for different regions of inputs.

eess.SY↗

arXiv2022

Fully on-chip microwave photonics system

Microwave photonics (MWP), harnessing the tremendous bandwidth of light to generate, process and measure wideband microwave signals, are poised to spark a new revolution for the information and communication fields. Within the past decade, new opportunity for MWP has emerged driven by the advances of integrated photonics. However, despite significant progress made in terms of integration level, a fully on-chip MWP functional system comprising all the necessary photonic and electronic components, is yet to be demonstrated. Here, we break the status quo and provide a complete on-chip solution for MWP system, by exploiting hybrid integration of indium phosphide, silicon photonics and complementary metal-oxide-semiconductor (CMOS) electronics platforms. Applying this hybrid integration methodology, a fully chip-based MWP microwave instantaneous frequency measurement (IFM) system is experimentally demonstrated. The unprecedented integration level brings great promotion to the compactness, reliability, and performances of the overall MWP IFM system, including a wide frequency measurement range (2-34 GHz), ultralow estimation errors (10.85 MHz) and a fast response speed (0.3 ns). Furthermore, we deploy the chip-scale MWP IFM system into realistic application tasks, where diverse microwave signals with rapid-varying frequencies at X-band (8-12 GHz) are accurately identified in real-time. This demonstration marks a milestone for the development of integrated MWP, by providing the technology basis for the miniaturization and massive implementations of various MWP functional systems.

physics.app-ph↗

arXiv2022

Adversarial Unlearning of Backdoors via Implicit Hypergradient

We propose a minimax formulation for removing backdoors from a given poisoned model based on a small set of clean data. This formulation encompasses much of prior work on backdoor removal. We propose the Implicit Bacdoor Adversarial Unlearning (I-BAU) algorithm to solve the minimax. Unlike previous work, which breaks down the minimax into separate inner and outer problems, our algorithm utilizes the implicit hypergradient to account for the interdependence between inner and outer optimization. We theoretically analyze its convergence and the generalizability of the robustness gained by solving minimax on clean data to unseen test data. In our evaluation, we compare I-BAU with six state-of-art backdoor defenses on seven backdoor attacks over two datasets and various attack settings, including the common setting where the attacker targets one class as well as important but underexplored settings where multiple classes are targeted. I-BAU's performance is comparable to and most often significantly better than the best baseline. Particularly, its performance is more robust to the variation on triggers, attack settings, poison ratio, and clean data size. Moreover, I-BAU requires less computation to take effect; particularly, it is more than $13\times$ faster than the most efficient baseline in the single-target attack setting. Furthermore, it can remain effective in the extreme case where the defender can only access 100 clean samples -- a setting where all the baselines fail to produce acceptable results.

cs.LG↗

arXiv2022

Generative and Contrastive Self-Supervised Learning for Graph Anomaly Detection

Anomaly detection from graph data has drawn much attention due to its practical significance in many critical applications including cybersecurity, finance, and social networks. Existing data mining and machine learning methods are either shallow methods that could not effectively capture the complex interdependency of graph data or graph autoencoder methods that could not fully exploit the contextual information as supervision signals for effective anomaly detection. To overcome these challenges, in this paper, we propose a novel method, Self-Supervised Learning for Graph Anomaly Detection (SL-GAD). Our method constructs different contextual subgraphs (views) based on a target node and employs two modules, generative attribute regression and multi-view contrastive learning for anomaly detection. While the generative attribute regression module allows us to capture the anomalies in the attribute space, the multi-view contrastive learning module can exploit richer structure information from multiple subgraphs, thus abling to capture the anomalies in the structure space, mixing of structure, and attribute information. We conduct extensive experiments on six benchmark datasets and the results demonstrate that our method outperforms state-of-the-art methods by a large margin.

cs.LG↗

arXiv2022

On-demand Integrated Quantum Memory for Polarization Qubits

Photonic polarization qubits are widely used in quantum computation and quantum communication due to the robustness in transmission and the easy qubit manipulation. An integrated quantum memory for polarization qubits is a fundamental building block for large-scale integrated quantum networks. However, on-demand storing polarization qubits in an integrated quantum memory is a long-standing challenge due to the anisotropic absorption of solids and the polarization-dependent features of microstructures. Here we demonstrate a reliable on-demand quantum memory for polarization qubits, using a depressed-cladding waveguide fabricated in a 151Eu3+: Y2SiO5 crystal. The site-2 151Eu3+ ions in Y2SiO5 crystal provides a near-uniform absorption for arbitrary polarization states and a new pump sequence is developed to prepare a wideband and enhanced absorption profile. A fidelity of 99.4\pm0.6% is obtained for the qubit storage process with an input of 0.32 photons per pulse, together with a storage bandwidth of 10 MHz. This reliable integrated quantum memory for polarization qubits reveals the potential for use in the construction of integrated quantum networks.

quant-ph↗

arXiv2021

Sub-milliwatt, widely-tunable coherent microcomb generation with feedback-free operation

Microcombs are revolutionizing optoelectronics by providing parallelized, mutually coherent wavelength channels for time-frequency metrology and information processing. To implement this essential function in integrated photonic systems, it is desirable to drive microcombs directly with an on-chip laser in a simple and flexible way. However, two major difficulties are preventing this goal: 1) generating mode-locked comb states usually requires a significant amount of pump power and 2) the requirement to align laser and resonator frequency significantly complicates operation and limits the tunability of the comb lines. Here, we address these problems by using microresonators on an AlGaAs on-insulator platform to generate dark-pulse microcombs. This highly nonlinear platform dramatically relaxes fabrication requirements and leads to a record-low pump power of less than 1 mW for coherent comb generation. Dark-pulse microcombs facilitated by thermally-controlled avoided mode-crossings are accessed by direct DFB laser pumping. Without any feedback or control circuitries, the comb shows good coherence and stability. This approach also leads to an unprecedented wide chirping range of all the comb lines. Our work provides a route to realize power-efficient, simple and reconfigurable microcombs that can be seamlessly integrated with a wide range of photonic systems.

physics.optics↗

arXiv2021

Recurrent Neural Network Controllers Synthesis with Stability Guarantees for Partially Observed Systems

Neural network controllers have become popular in control tasks thanks to their flexibility and expressivity. Stability is a crucial property for safety-critical dynamical systems, while stabilization of partially observed systems, in many cases, requires controllers to retain and process long-term memories of the past. We consider the important class of recurrent neural networks (RNN) as dynamic controllers for nonlinear uncertain partially-observed systems, and derive convex stability conditions based on integral quadratic constraints, S-lemma and sequential convexification. To ensure stability during the learning and control process, we propose a projected policy gradient method that iteratively enforces the stability conditions in the reparametrized space taking advantage of mild additional information on system dynamics. Numerical experiments show that our method learns stabilizing controllers while using fewer samples and achieving higher final performance compared with policy gradient.

eess.SY↗

arXiv2021

Controlling Smart Inverters using Proxies: A Chance-Constrained DNN-based Approach

Coordinating inverters at scale under uncertainty is the desideratum for integrating renewables in distribution grids. Unless load demands and solar generation are telemetered frequently, controlling inverters given approximate grid conditions or proxies thereof becomes a key specification. Although deep neural networks (DNNs) can learn optimal inverter schedules, guaranteeing feasibility is largely elusive. Rather than training DNNs to imitate already computed optimal power flow (OPF) solutions, this work integrates DNN-based inverter policies into the OPF. The proposed DNNs are trained through two OPF alternatives that confine voltage deviations on the average and as a convex restriction of chance constraints. The trained DNNs can be driven by partial, noisy, or proxy descriptors of the current grid conditions. This is important when OPF has to be solved for an unobservable feeder. DNN weights are trained via back-propagation and upon differentiating the AC power flow equations assuming the network model is known. Otherwise, a gradient-free variant is put forth. The latter is relevant when inverters are controlled by an aggregator having access only to a power flow solver or a digital twin of the feeder. Numerical tests compare the DNN-based inverter control schemes with the optimal inverter setpoints in terms of optimality and feasibility.

math.OC↗

arXiv2021

Bridging microcombs and silicon photonic engines for optoelectronics systems

Microcombs have sparked a surge of applications over the last decade, ranging from optical communications to metrology. Despite their diverse deployment, most microcomb-based systems rely on a tremendous amount of bulk equipment to fulfill their desired functions, which is rather complicated, expensive and power-consuming. On the other hand, foundry-based silicon photonics (SiPh) has had remarkable success in providing versatile functionality in a scalable and low-cost manner, but its available chip-based light sources lack the capacity for parallelization, which limits the scope of SiPh applications. Here, we bridge these two technologies by using a power-efficient and operationally-simple AlGaAs on insulator microcomb source to drive CMOS SiPh engines. We present two important chip-scale photonic systems for optical data transmissions and microwave photonics respectively: The first microcomb-based integrated photonic data link is demonstrated, based on a pulse-amplitude 4-level modulation scheme with 2 Tbps aggregate rate, and a highly reconfigurable microwave photonic filter with unprecedented integration level is constructed, using a time stretch scheme. Such synergy of microcomb and SiPh integrated components is an essential step towards the next generation of fully integrated photonic systems.

physics.app-ph↗

arXiv2021

Power up! Robust Graph Convolutional Network via Graph Powering

Graph convolutional networks (GCNs) are powerful tools for graph-structured data. However, they have been recently shown to be vulnerable to topological attacks. To enhance adversarial robustness, we go beyond spectral graph theory to robust graph theory. By challenging the classical graph Laplacian, we propose a new convolution operator that is provably robust in the spectral domain and is incorporated in the GCN architecture to improve expressivity and interpretability. By extending the original graph to a sequence of graphs, we also propose a robust training paradigm that encourages transferability across graphs that span a range of spatial and spectral characteristics. The proposed approaches are demonstrated in extensive experiments to simultaneously improve performance in both benign and adversarial situations.

cs.LG↗