Searcharxiv⌕ Search

arXiv subjects

Xinyi Xu

Publications and source records attributed to Xinyi Xu.

At least 55 records · Page 3Linked to original sources
arXiv2021

Incentivizing Collaboration in Machine Learning via Synthetic Data Rewards

This paper presents a novel collaborative generative modeling (CGM) framework that incentivizes collaboration among self-interested parties to contribute data to a pool for training a generative model (e.g., GAN), from which synthetic data are drawn and distributed to the parties as rewards commensurate to their contributions. Distributing synthetic data as rewards (instead of trained models or money) offers task- and model-agnostic benefits for downstream learning tasks and is less likely to violate data privacy regulation. To realize the framework, we firstly propose a data valuation function using maximum mean discrepancy (MMD) that values data based on its quantity and quality in terms of its closeness to the true data distribution and provide theoretical results guiding the kernel choice in our MMD-based data valuation function. Then, we formulate the reward scheme as a linear optimization problem that when solved, guarantees certain incentives such as fairness in the CGM framework. We devise a weighted sampling algorithm for generating synthetic data to be distributed to each party as reward such that the value of its data and the synthetic data combined matches its assigned reward value by the reward scheme. We empirically show using simulated and real-world datasets that the parties' synthetic data rewards are commensurate to their contributions.

cs.LG↗
arXiv2021

Molecule3D: A Benchmark for Predicting 3D Geometries from Molecular Graphs

Graph neural networks are emerging as promising methods for modeling molecular graphs, in which nodes and edges correspond to atoms and chemical bonds, respectively. Recent studies show that when 3D molecular geometries, such as bond lengths and angles, are available, molecular property prediction tasks can be made more accurate. However, computing of 3D molecular geometries requires quantum calculations that are computationally prohibitive. For example, accurate calculation of 3D geometries of a small molecule requires hours of computing time using density functional theory (DFT). Here, we propose to predict the ground-state 3D geometries from molecular graphs using machine learning methods. To make this feasible, we develop a benchmark, known as Molecule3D, that includes a dataset with precise ground-state geometries of approximately 4 million molecules derived from DFT. We also provide a set of software tools for data processing, splitting, training, and evaluation, etc. Specifically, we propose to assess the error and validity of predicted geometries using four metrics. We implement two baseline methods that either predict the pairwise distance between atoms or atom coordinates in 3D space. Experimental results show that, compared with generating 3D geometries with RDKit, our method can achieve comparable prediction accuracy but with much smaller computational costs. Our Molecule3D is available as a module of the MoleculeX software library (https://github.com/divelab/MoleculeX).

cs.LG↗
arXiv2021

Towards Improved and Interpretable Deep Metric Learning via Attentive Grouping

Grouping has been commonly used in deep metric learning for computing diverse features. However, current methods are prone to overfitting and lack interpretability. In this work, we propose an improved and interpretable grouping method to be integrated flexibly with any metric learning framework. Our method is based on the attention mechanism with a learnable query for each group. The query is fully trainable and can capture group-specific information when combined with the diversity loss. An appealing property of our method is that it naturally lends itself interpretability. The attention scores between the learnable query and each spatial position can be interpreted as the importance of that position. We formally show that our proposed grouping method is invariant to spatial permutations of features. When used as a module in convolutional neural networks, our method leads to translational invariance. We conduct comprehensive experiments to evaluate our method. Our quantitative results indicate that the proposed method outperforms prior methods consistently and significantly across different datasets, evaluation metrics, base models, and loss functions. For the first time to the best of our knowledge, our interpretation results clearly demonstrate that the proposed method enables the learning of distinct and diverse features across groups. The code is available on https://github.com/XinyiXuXD/DGML-master.

cs.CV↗
arXiv2021

A Reputation Mechanism Is All You Need: Collaborative Fairness and Adversarial Robustness in Federated Learning

Federated learning (FL) is an emerging practical framework for effective and scalable machine learning among multiple participants, such as end users, organizations and companies. However, most existing FL or distributed learning frameworks have not well addressed two important issues together: collaborative fairness and adversarial robustness (e.g. free-riders and malicious participants). In conventional FL, all participants receive the global model (equal rewards), which might be unfair to the high-contributing participants. Furthermore, due to the lack of a safeguard mechanism, free-riders or malicious adversaries could game the system to access the global model for free or to sabotage it. In this paper, we propose a novel Robust and Fair Federated Learning (RFFL) framework to achieve collaborative fairness and adversarial robustness simultaneously via a reputation mechanism. RFFL maintains a reputation for each participant by examining their contributions via their uploaded gradients (using vector similarity) and thus identifies non-contributing or malicious participants to be removed. Our approach differentiates itself by not requiring any auxiliary/validation dataset. Extensive experiments on benchmark datasets show that RFFL can achieve high fairness and is very robust to different types of adversaries while achieving competitive predictive accuracy.

cs.LG↗
arXiv2021

Group Contrastive Self-Supervised Learning on Graphs

We study self-supervised learning on graphs using contrastive methods. A general scheme of prior methods is to optimize two-view representations of input graphs. In many studies, a single graph-level representation is computed as one of the contrastive objectives, capturing limited characteristics of graphs. We argue that contrasting graphs in multiple subspaces enables graph encoders to capture more abundant characteristics. To this end, we propose a group contrastive learning framework in this work. Our framework embeds the given graph into multiple subspaces, of which each representation is prompted to encode specific characteristics of graphs. To learn diverse and informative representations, we develop principled objectives that enable us to capture the relations among both intra-space and inter-space representations in groups. Under the proposed framework, we further develop an attention-based representor function to compute representations that capture different substructures of a given graph. Built upon our framework, we extend two current methods into GroupCL and GroupIG, equipped with the proposed objective. Comprehensive experimental results show our framework achieves a promising boost in performance on a variety of datasets. In addition, our qualitative results show that features generated from our representor successfully capture various specific characteristics of graphs.

cs.LG↗
arXiv2021

Industry Practice of Coverage-Guided Enterprise-Level DBMS Fuzzing

As an infrastructure for data persistence and analysis, Database Management Systems (DBMSs) are the cornerstones of modern enterprise software. To improve their correctness, the industry has been applying blackbox fuzzing for decades. Recently, the research community achieved impressive fuzzing gains using coverage guidance. However, due to the complexity and distributed nature of enterprise-level DBMSs, seldom are these researches applied to the industry. In this paper, we apply coverage-guided fuzzing to enterprise-level DBMSs from Huawei and Bloomberg LP. In our practice of testing GaussDB and Comdb2, we found major challenges in all three testing stages. The challenges are collecting precise coverage, optimizing fuzzing performance, and analyzing root causes. In search of a general method to overcome these challenges, we propose Ratel, a coverage-guided fuzzer for enterprise-level DBMSs. With its industry-oriented design, Ratel improves the feedback precision, enhances the robustness of input generation, and performs an on-line investigation on the root cause of bugs. As a result, Ratel outperformed other fuzzers in terms of coverage and bugs. Compared to industrial black box fuzzers SQLsmith and SQLancer, as well as coverage-guided academic fuzzer Squirrel, Ratel covered 38.38%, 106.14%, 583.05% more basic blocks than the best results of other three fuzzers in GaussDB, PostgreSQL, and Comdb2, respectively. More importantly, Ratel has discovered 32, 42, and 5 unknown bugs in GaussDB, Comdb2, and PostgreSQL.

cs.SE↗
arXiv2020

Hierarchical Reinforcement Learning in StarCraft II with Human Expertise in Subgoals Selection

This work is inspired by recent advances in hierarchical reinforcement learning (HRL) (Barto and Mahadevan 2003; Hengst 2010), and improvements in learning efficiency from heuristic-based subgoal selection, experience replay (Lin 1993; Andrychowicz et al. 2017), and task-based curriculum learning (Bengio et al. 2009; Zaremba and Sutskever 2014). We propose a new method to integrate HRL, experience replay and effective subgoal selection through an implicit curriculum design based on human expertise to support sample-efficient learning and enhance interpretability of the agent's behavior. Human expertise remains indispensable in many areas such as medicine (Buch, Ahmed, and Maruthappu 2018) and law (Cath 2018), where interpretability, explainability and transparency are crucial in the decision making process, for ethical and legal reasons. Our method simplifies the complex task sets for achieving the overall objectives by decomposing them into subgoals at different levels of abstraction. Incorporating relevant subjective knowledge also significantly reduces the computational resources spent in exploration for RL, especially in high speed, changing, and complex environments where the transition dynamics cannot be effectively learned and modelled in a short time. Experimental results in two StarCraft II (SC2) (Vinyals et al. 2017) minigames demonstrate that our method can achieve better sample efficiency than flat and end-to-end RL methods, and provides an effective method for explaining the agent's performance.

cs.AI↗
arXiv2020

Collaborative Fairness in Federated Learning

In current deep learning paradigms, local training or the Standalone framework tends to result in overfitting and thus poor generalizability. This problem can be addressed by Distributed or Federated Learning (FL) that leverages a parameter server to aggregate model updates from individual participants. However, most existing Distributed or FL frameworks have overlooked an important aspect of participation: collaborative fairness. In particular, all participants can receive the same or similar models, regardless of their contributions. To address this issue, we investigate the collaborative fairness in FL, and propose a novel Collaborative Fair Federated Learning (CFFL) framework which utilizes reputation to enforce participants to converge to different models, thus achieving fairness without compromising the predictive performance. Extensive experiments on benchmark datasets demonstrate that CFFL achieves high fairness, delivers comparable accuracy to the Distributed framework, and outperforms the Standalone framework.

cs.LG↗
arXiv2020

Nonlinear twistoptics at symmetry-broken interfaces

Broken symmetries induce strong nonlinear optical responses in materials and at interfaces. Twist angle can give complete control over the presence or lack of inversion symmetry at a crystal interface, and is thus an appealing knob for tuning nonlinear optical systems. In contrast to conventional nonlinear crystals with rigid lattices, the weak interlayer coupling in van der Waals (vdW) heterostructures allows for arbitrary selection of twist angle, making nanomechanical manipulation of fundamental interfacial symmetry possible within a single device. Here we report highly tunable second harmonic generation (SHG) from nanomechanically rotatable stacks of bulk hexagonal boron nitride (BN) crystals, and introduce the term twistoptics to describe studies of optical properties in dynamically twistable vdW systems. We observe SHG intensity modulated by a factor of more than 50, polarization patterns determined by moiré interface symmetry, and enhanced conversion efficiency for bulk crystals by stacking multiple pieces of BN joined by symmetry-broken interfaces. Our study provides a foundation for compact twistoptics architectures aimed at efficient, scalable, and tunable frequency-conversion, and demonstrates SHG as a robust probe of buried vdW interfaces.

cond-mat.mes-hall↗
arXiv2020

Spin Wave Generation via Localized Spin-Orbit Torque in an Antiferromagnet-Topological Insulator Heterostructure

The spin-orbit torque induced by a topological insulator (TI) is theoretically examined for spin wave generation in a neighboring antiferromagnetic thin film. The investigation is based on the micromagnetic simulation of Néel vector dynamics and the analysis of transport properties in the TI. The results clearly illustrate that propagating spin waves can be achieved in the antiferromagnetic thin-film strip through localized excitation, traveling over a long distance. The oscillation amplitude gradually decays due to the non-zero damping as the Néel vector precesses around the magnetic easy axis with a fixed frequency. The frequency is also found to be tunable via the strength of the driving electrical current density. While both the bulk and the surface states of the TI contribute to induce the effective torque, the calculation indicates that the surface current plays a dominant role over the bulk counterpart except in the heavily degenerate cases. Compared to the more commonly applied heavy metals, the use of a TI can substantially reduce the threshold current density to overcome the magnetic anisotropy, making it an efficient choice for spin wave generation. The Néel vector dynamics in the nano-oscillator geometry are examined as well.

cond-mat.mtrl-sci↗
arXiv2019

Broadband optical parametric amplification by two-dimensional semiconductors

Optical parametric amplification is a second-order nonlinear process whereby an optical signal is amplified by a pump via the generation of an idler field. It is the key ingredient of tunable sources of radiation that play an important role in several photonic applications. This mechanism is inherently related to spontaneous parametric down-conversion that currently constitutes the building block for entangled photon pair generation, which has been exploited in modern quantum technologies ranging from computing to communications and cryptography. Here we demonstrate single-pass optical parametric amplification at the ultimate thickness limit; using semiconducting transition-metal dichalcogenides, we show that amplification can be attained over a propagation through a single atomic layer. Such a second-order nonlinear interaction at the 2D limit bypasses phase-matching requirements and achieves ultrabroad amplification bandwidths. The amplification process is independent on the in-plane polarization of the impinging signal and pump fields. First-principle calculations confirm the observed polarization invariance and linear relationship between idler and pump powers. Our results pave the way for the development of atom-sized tunable sources of radiation with applications in nanophotonics and quantum information technology.

physics.optics↗
arXiv2019

Electrical generation and propagation of spin waves in antiferromagnetic thin films

Electrical generation of THz spin waves is theoretically explored in an antiferromangetic nanostrip via the current-induced spin-orbit torque. The analysis based on micromagnetic simulations clearly illustrates that the Neel-vector oscillations excited at one end of the magnetic strip can propagate in the form of a traveling wave when the nanostrip axis aligns with the magnetic easy-axis. A sizable threshold is observed in the driving current density or the torque to overcome the unfavorable anisotropy as expected. The generated spin waves are found to travel over a long distance while the angle of rotation undergoes continuous decay in the presence of non-zero damping. The oscillation frequency is tunable via the strength of the spin-orbit torque, reaching the THz regime. Other key characteristics of the spin waves such as the phase and the chirality can also be modulated actively. The simulation results further indicate the possibility of wave-like superposition between the excited spin oscillations, illustrating its application as an efficient source of spin-wave signals for information processing.

cond-mat.mes-hall↗
arXiv2019

Controllable dispersion of domain wall movement in antiferromagnetic thin films at finite temperatures

The dynamics of a 90$^{\circ }$ domain wall in an antiferromagnetic nanostrip driven by the current-induced spin-orbital torque are theoretically examined in the presence of random thermal fluctuations. A soliton-type equation of motion is developed on the basis of energy balance between the driving forces and dissipative processes in terms of the domain wall velocity. Comparison with micromagnetic simulations in the deterministic conditions shows good agreement in both the transient and steady-state transport. When the effects of random thermal fluctuations are included via a stochastic treatment, the results clearly indicate that the dispersion in the domain wall position can be controlled electrically by tailoring the strength and duration of the driving current mediating the spin orbital torque in the antiferromagnet. More specifically, the standard deviation of the probability distribution function for the domain wall movement can be tuned widely while maintaining the average position unaffected. Potential applications of this unusual functionality include the probabilistic computing such as Bayesian learning.

cond-mat.mtrl-sci↗
arXiv2019

Prediction of Silicate Glasses' Stiffness by High-Throughput Molecular Dynamics Simulations and Machine Learning

The development by machine learning of models predicting materials' properties usually requires the use of a large number of consistent data for training. However, quality experimental datasets are not always available or self-consistent. Here, as an alternative route, we combine machine learning with high-throughput molecular dynamics simulations to predict the Young's modulus of silicate glasses. We demonstrate that this combined approach offers excellent predictions over the entire compositional domain. By comparing the performance of select machine learning algorithms, we discuss the nature of the balance between accuracy, simplicity, and interpretability in machine learning.

cond-mat.mtrl-sci↗
arXiv2016

Designing for Situation Awareness of Future Power Grids: An Indicator System Based on Linear Eigenvalue Statistics of Large Random Matrices

Future power grids are fundamentally different from current ones, both in size and in complexity; this trend imposes challenges for situation awareness (SA) based on classical indicators, which are usually model-based and deterministic. As an alternative, this paper proposes a statistical indicator system based on linear eigenvalue statistics (LESs) of large random matrices: 1) from a data modeling viewpoint, we build, starting from power flows equations, the random matrix models (RMMs) only using the real-time data flow in a statistical manner; 2) for a data analysis that is fully driven from RMMs, we put forward the high-dimensional indicators, called LESs that have some unique statistical features such as Gaussian properties; and 3) we develop a three-dimensional (3D) power-map to visualize the system, respectively, from a high-dimensional viewpoint and a low-dimensional one. Therefore, a statistical methodology of SA is employed; it conducts SA with a model-free and data-driven procedure, requiring no knowledge of system topologies, units operation/control models, causal relationship, etc. This methodology has numerous advantages, such as sensitivity, universality, speed, and flexibility. In particular, its robustness against bad data is highlighted, with potential advantages in cyber security. The theory of big data based stability for on-line operations may prove feasible along with this line of work, although this critical development will be reported elsewhere.

stat.ME↗
arXiv2016

A Correlation Analysis Method for Power Systems Based on Random Matrix Theory

The operating status of power systems is influenced by growing varieties of factors, resulting from the developing sizes and complexity of power systems; in this situation, the modelbased methods need be revisited. A data-driven method, as the novel alternative, on the other hand, is proposed in this paper: it reveals the correlations between the factors and the system status through statistical properties of data. An augmented matrix, as the data source, is the key trick for this method; it is formulated by two parts: 1) status data as the basic part, and 2) factor data as the augmented part. The random matrix theory (RMT) is applied as the mathematical framework. The linear eigenvalue statistics (LESs), such as the mean spectral radius (MSR), are defined to study data correlations through large random matrices. Compared with model-based methods, the proposed method is inspired by a pure statistical approach, without a prior knowledge of operation and interaction mechanism models for power systems and factors. In general, this method is direct in analysis, robust against bad data, universal to various factors, and applicable for real-time analysis. A case study, based on the standard IEEE 118-bus system, validates the proposed method.

stat.ME↗
arXiv2015

Multi-photon Absorption in Optical Pumping of Rubidium

In optical pumping of rubidium, a new kind of absorption occurs with a higher amplitude of radio frequency current. From measurement of the corresponding magnetic field value where this absorption occurs, there is a conclusion that it is multi-photon absorption. Both the degeneracy and energy of photons contribute to the intensity.

physics.optics↗