arXiv subjects

Yi Zhou

Publications and source records attributed to Yi Zhou.

At least 379 records · Page 21Linked to original sources

arXiv2022

Plasma Image Classification Using Cosine Similarity Constrained CNN

Plasma jets are widely investigated both in the laboratory and in nature. Astrophysical objects such as black holes, active galactic nuclei, and young stellar objects commonly emit plasma jets in various forms. With the availability of data from plasma jet experiments resembling astrophysical plasma jets, classification of such data would potentially aid in investigating not only the underlying physics of the experiments but the study of astrophysical jets. In this work we use deep learning to process all of the laboratory plasma images from the Caltech Spheromak Experiment spanning two decades. We found that cosine similarity can aid in feature selection, classify images through comparison of feature vector direction, and be used as a loss function for the training of AlexNet for plasma image classification. We also develop a simple vector direction comparison algorithm for binary and multi-class classification. Using our algorithm we demonstrate 93% accurate binary classification to distinguish unstable columns from stable columns and 92% accurate five-way classification of a small, labeled data set which includes three classes corresponding to varying levels of kink instability.

physics.plasm-ph↗

arXiv2022

DDDM: a Brain-Inspired Framework for Robust Classification

Despite their outstanding performance in a broad spectrum of real-world tasks, deep artificial neural networks are sensitive to input noises, particularly adversarial perturbations. On the contrary, human and animal brains are much less vulnerable. In contrast to the one-shot inference performed by most deep neural networks, the brain often solves decision-making with an evidence accumulation mechanism that may trade time for accuracy when facing noisy inputs. The mechanism is well described by the Drift-Diffusion Model (DDM). In the DDM, decision-making is modeled as a process in which noisy evidence is accumulated toward a threshold. Drawing inspiration from the DDM, we propose the Dropout-based Drift-Diffusion Model (DDDM) that combines test-phase dropout and the DDM for improving the robustness for arbitrary neural networks. The dropouts create temporally uncorrelated noises in the network that counter perturbations, while the evidence accumulation mechanism guarantees a reasonable decision accuracy. Neural networks enhanced with the DDDM tested in image, speech, and text classification tasks all significantly outperform their native counterparts, demonstrating the DDDM as a task-agnostic defense against adversarial attacks.

cs.NE↗

arXiv2022

SHAPE: An Unified Approach to Evaluate the Contribution and Cooperation of Individual Modalities

As deep learning advances, there is an ever-growing demand for models capable of synthesizing information from multi-modal resources to address the complex tasks raised from real-life applications. Recently, many large multi-modal datasets have been collected, on which researchers actively explore different methods of fusing multi-modal information. However, little attention has been paid to quantifying the contribution of different modalities within the proposed models. In this paper, we propose the {\bf SH}apley v{\bf A}lue-based {\bf PE}rceptual (SHAPE) scores that measure the marginal contribution of individual modalities and the degree of cooperation across modalities. Using these scores, we systematically evaluate different fusion methods on different multi-modal datasets for different tasks. Our experiments suggest that for some tasks where different modalities are complementary, the multi-modal models still tend to use the dominant modality alone and ignore the cooperation across modalities. On the other hand, models learn to exploit cross-modal cooperation when different modalities are indispensable for the task. In this case, the scores indicate it is better to fuse different modalities at relatively early stages. We hope our scores can help improve the understanding of how the present multi-modal models operate on different modalities and encourage more sophisticated methods of integrating multiple modalities.

cs.LG↗

arXiv2022

Data Sampling Affects the Complexity of Online SGD over Dependent Data

Conventional machine learning applications typically assume that data samples are independently and identically distributed (i.i.d.). However, practical scenarios often involve a data-generating process that produces highly dependent data samples, which are known to heavily bias the stochastic optimization process and slow down the convergence of learning. In this paper, we conduct a fundamental study on how different stochastic data sampling schemes affect the sample complexity of online stochastic gradient descent (SGD) over highly dependent data. Specifically, with a $ϕ$-mixing model of data dependence, we show that online SGD with proper periodic data-subsampling achieves an improved sample complexity over the standard online SGD in the full spectrum of the data dependence level. Interestingly, even subsampling a subset of data samples can accelerate the convergence of online SGD over highly dependent data. Moreover, we show that online SGD with mini-batch sampling can further substantially improve the sample complexity over online SGD with periodic data-subsampling over highly dependent data. Numerical experiments validate our theoretical results.

cs.LG↗

arXiv2022

Delving into the Estimation Shift of Batch Normalization in a Network

Batch normalization (BN) is a milestone technique in deep learning. It normalizes the activation using mini-batch statistics during training but the estimated population statistics during inference. This paper focuses on investigating the estimation of population statistics. We define the estimation shift magnitude of BN to quantitatively measure the difference between its estimated population statistics and expected ones. Our primary observation is that the estimation shift can be accumulated due to the stack of BN in a network, which has detriment effects for the test performance. We further find a batch-free normalization (BFN) can block such an accumulation of estimation shift. These observations motivate our design of XBNBlock that replace one BN with BFN in the bottleneck block of residual-style networks. Experiments on the ImageNet and COCO benchmarks show that XBNBlock consistently improves the performance of different architectures, including ResNet and ResNeXt, by a significant margin and seems to be more robust to distribution shift.

cs.CV↗

arXiv2022

Inhomogeneous superconducting states in two weakly linked superconducting ultra thin films

A sufficiently large parallel magnetic field will generate staggered supercurrent loops and superfluid density wave in two weakly linked superconducting (SC) ultrathin films, resulting in an inhomogeneous Fulde-Ferrell-Larkin-Ovchinnikov (FFLO) state. The SC order parameter of such an FFLO state is characterized by Bloch wave functions, called the "Bloch SC state". The staggered supercurrent loops form an array of Josephson vortex-antivortex pairs, instead of the usual Josephson vortex lattice. Enclosing a unit cell of the array, the London's fluxoid is quantized as $Φ^{\prime}=Φ_0=hc/2e$, while the net orbital magnetization caused by the staggered supercurrent is zero. Meanwhile, a small parallel magnetic field gives rise to an Fulde-Ferrell (FF) state that has uniform superfluid density. The phase transition between the Bloch SC state and the FF state belongs to the universality class of two-dimensional commensurate-incommensurate transitions. An analytical solution in terms of Jacobian elliptic functions is found to be an excellent approximation to the Bloch SC order parameter.

cond-mat.supr-con↗

arXiv2022

Sense Embeddings are also Biased--Evaluating Social Biases in Static and Contextualised Sense Embeddings

Sense embedding learning methods learn different embeddings for the different senses of an ambiguous word. One sense of an ambiguous word might be socially biased while its other senses remain unbiased. In comparison to the numerous prior work evaluating the social biases in pretrained word embeddings, the biases in sense embeddings have been relatively understudied. We create a benchmark dataset for evaluating the social biases in sense embeddings and propose novel sense-specific bias evaluation measures. We conduct an extensive evaluation of multiple static and contextualised sense embeddings for various types of social biases using the proposed measures. Our experimental results show that even in cases where no biases are found at word-level, there still exist worrying levels of social biases at sense-level, which are often ignored by the word-level bias evaluation measures.

cs.CL↗

arXiv2022

A likelihood based sensitivity analysis for publication bias on summary ROC in meta-analysis of diagnostic test accuracy

In meta-analysis of diagnostic test accuracy, summary receiver operating characteristic (SROC) is a recommended method to summarize the discriminant capacity of a diagnostic test in the presence of study-specific cutoff values and the area under the SROC (SAUC) gives the aggregate measure of test accuracy. SROC or SAUC can be estimated by bivariate modelling of pairs of sensitivity and specificity over the primary diagnostic studies. However, publication bias is a major threat to the validity of estimates in meta-analysis. To address this issue, we propose to adopt sensitivity analysis to make an objective inference for the impact of publication bias on SROC or SAUC. We extend Copas likelihood based sensitivity analysis to the bivariate normal model used for meta-analysis of diagnostic test accuracy to evaluate how much SROC or SAUC would change with different selection probabilities under several selective publication mechanisms dependent on sensitivity and/or specificity. The selection probability is modelled by a selection function on $t$-type statistic for the linear combination of logit-transformed sensitivity and specificity, allowing the selective publication of each study to be influenced by the cutoff-dependent $p$-value for sensitivity, specificity, or diagnostic odds ratio. By embedding the selection function into the bivariate normal model, the conditional likelihood is proposed and the bias-corrected SROC or SAUC can be estimated by maximizing the likelihood. We illustrate the proposed sensitivity analysis by reanalyzing a meta-analysis of test accuracy for intravascular device related infection. Simulation studies are conducted to investigate the performance of proposed methods.

stat.ME↗

arXiv2022

Extended Load Flexibility of Industrial P2H Plants: A Process Constraint-Aware Scheduling Approach

The operational flexibility of industrial power-to-hydrogen (P2H) plants enables admittance of volatile renewable power and provides auxiliary regulatory services for the power grid. Aiming to extend the flexibility of the P2H plant further, this work presents a scheduling method by considering detailed process constraints of the alkaline electrolyzers. Unlike existing works that assume constant load range, the presented scheduling framework fully exploits the dynamic processes of the electrolyzer, including temperature and hydrogen-to-oxygen (HTO) crossover, to improve operational flexibility. Varying energy conversion efficiency under different load levels and temperature is also considered. The scheduling model is solved by proper mathematical transformation as a mixed-integer linear programming (MILP), which determines the on-off-standby states and power levels of different electrolyzers in the P2H plant for daily operation. With experiment-verified constraints, a case study show that compared to the existing scheduling approach, the improved flexibility leads to a 1.627% profit increase when the P2H plant is directly coupled to the photovoltaic power.

eess.SY↗

arXiv2022

Unveiling a critical stripy state in the triangular-lattice SU(4) spin-orbital model

The simplest spin-orbital model can host a nematic spin-orbital liquid state on the triangular lattice. We provide clear evidence that the ground state of the SU(4) Kugel-Khomskii model on the triangular lattice can be well described by a "single" Gutzwiller projected wave function with an emergent parton Fermi surface, despite it exhibits strong finite-size effect in quasi-one-dimensional cylinders. The finite-size effect can be resolved by the fact that the parton Fermi surface consists of open orbits in the reciprocal space. Thereby, a stripy liquid state is expected in the two-dimensional limit, which preserves the SU(4) symmetry while breaks the translational symmetry by doubling the unit cell along one of the lattice vector directions. It is indicative that these stripes are critical and the central charge is $c=3$, in agreement with the SU(4)$_1$ Wess-Zumino-Witten conformal field theory. All these results are consistent with the Lieb-Schultz-Mattis-Oshikawa-Hastings theorem.

cond-mat.str-el↗

arXiv2022

Listing Maximal k-Plexes in Large Real-World Graphs

Listing dense subgraphs in large graphs plays a key task in varieties of network analysis applications like community detection. Clique, as the densest model, has been widely investigated. However, in practice, communities rarely form as cliques for various reasons, e.g., data noise. Therefore, $k$-plex, -- graph with each vertex adjacent to all but at most $k$ vertices, is introduced as a relaxed version of clique. Often, to better simulate cohesive communities, an emphasis is placed on connected $k$-plexes with small $k$. In this paper, we continue the research line of listing all maximal $k$-plexes and maximal $k$-plexes of prescribed size. Our first contribution is algorithm ListPlex that lists all maximal $k$-plexes in $O^*(γ^D)$ time for each constant $k$, where $γ$ is a value related to $k$ but strictly smaller than 2, and $D$ is the degeneracy of the graph that is far less than the vertex number $n$ in real-word graphs. Compared to the trivial bound of $2^n$, the improvement is significant, and our bound is better than all previously known results. In practice, we further use several techniques to accelerate listing $k$-plexes of a given size, such as structural-based prune rules, cache-efficient data structures, and parallel techniques. All these together result in a very practical algorithm. Empirical results show that our approach outperforms the state-of-the-art solutions by up to orders of magnitude.

cs.DS↗

arXiv2022

Desingularization and p-Curvature of Recurrence Operators

Linear recurrence operators in characteristic $p$ are classified by their $p$-curvature. For a recurrence operator $L$, denote by $χ(L)$ the characteristic polynomial of its $p$-curvature. We can obtain information about the factorization of $L$ by factoring $χ(L)$. The main theorem of this paper gives an unexpected relation between $χ(L)$ and the true singularities of $L$. An application is to speed up a fast algorithm for computing $χ(L)$ by desingularizing $L$ first. Another contribution of this paper is faster desingularization.

cs.SC↗

arXiv2022

Single-shot Hyper-parameter Optimization for Federated Learning: A General Algorithm & Analysis

We address the relatively unexplored problem of hyper-parameter optimization (HPO) for federated learning (FL-HPO). We introduce Federated Loss SuRface Aggregation (FLoRA), a general FL-HPO solution framework that can address use cases of tabular data and any Machine Learning (ML) model including gradient boosting training algorithms and therefore further expands the scope of FL-HPO. FLoRA enables single-shot FL-HPO: identifying a single set of good hyper-parameters that are subsequently used in a single FL training. Thus, it enables FL-HPO solutions with minimal additional communication overhead compared to FL training without HPO. We theoretically characterize the optimality gap of FL-HPO, which explicitly accounts for the heterogeneous non-IID nature of the parties' local data distributions, a dominant characteristic of FL systems. Our empirical evaluation of FLoRA for multiple ML algorithms on seven OpenML datasets demonstrates significant model accuracy improvements over the considered baseline, and robustness to increasing number of parties involved in FL-HPO training.

cs.LG↗

arXiv2022

UNISON: Unpaired Cross-lingual Image Captioning

Image captioning has emerged as an interesting research field in recent years due to its broad application scenarios. The traditional paradigm of image captioning relies on paired image-caption datasets to train the model in a supervised manner. However, creating such paired datasets for every target language is prohibitively expensive, which hinders the extensibility of captioning technology and deprives a large part of the world population of its benefit. In this work, we present a novel unpaired cross-lingual method to generate image captions without relying on any caption corpus in the source or the target language. Specifically, our method consists of two phases: (i) a cross-lingual auto-encoding process, which utilizing a sentence parallel (bitext) corpus to learn the mapping from the source to the target language in the scene graph encoding space and decode sentences in the target language, and (ii) a cross-modal unsupervised feature mapping, which seeks to map the encoded scene graph features from image modality to language modality. We verify the effectiveness of our proposed method on the Chinese image caption generation task. The comparisons against several existing methods demonstrate the effectiveness of our approach.

cs.CL↗

arXiv2022

Sample and Communication-Efficient Decentralized Actor-Critic Algorithms with Finite-Time Analysis

Actor-critic (AC) algorithms have been widely adopted in decentralized multi-agent systems to learn the optimal joint control policy. However, existing decentralized AC algorithms either do not preserve the privacy of agents or are not sample and communication-efficient. In this work, we develop two decentralized AC and natural AC (NAC) algorithms that are private, and sample and communication-efficient. In both algorithms, agents share noisy information to preserve privacy and adopt mini-batch updates to improve sample and communication efficiency. Particularly for decentralized NAC, we develop a decentralized Markovian SGD algorithm with an adaptive mini-batch size to efficiently compute the natural policy gradient. Under Markovian sampling and linear function approximation, we prove the proposed decentralized AC and NAC algorithms achieve the state-of-the-art sample complexities $\mathcal{O}\big(ε^{-2}\ln(ε^{-1})\big)$ and $\mathcal{O}\big(ε^{-3}\ln(ε^{-1})\big)$, respectively, and the same small communication complexity $\mathcal{O}\big(ε^{-1}\ln(ε^{-1})\big)$. Numerical experiments demonstrate that the proposed algorithms achieve lower sample and communication complexities than the existing decentralized AC algorithm.

cs.LG↗

arXiv2022

Matrix product states for Hartree-Fock-Bogoliubov wave functions

We provide an efficient and accurate method for converting Hartree-Fock-Bogoliubov wave functions into matrix product states (MPSs). These wave functions, also known as Bogoliubov vacua, exhibit a peculiar entanglement structure that the eigenvectors of the reduced density matrix are also Bogoliubov vacua. We exploit this important feature to obtain their optimal MPS approximation and derive an explicit formula for corresponding MPS matrices. The performance of our method is benchmarked with the Kitaev chain and the Majorana-Hubbard model on the honeycomb lattice. The approach facilitates the applications of Hartree-Fock-Bogoliubov wave functions and is ideally suited for combining with the density-matrix renormalization group method.

cond-mat.str-el↗

arXiv2022

Coordinated Frequency Control through Safe Reinforcement Learning

With widespread deployment of renewables, the electric power grids are experiencing increasing dynamics and uncertainties, with its secure operation being threatened. Existing frequency control schemes based on day-ahead offline analysis and minute-level online sensitivity calculations are difficult to adapt to rapidly changing system states. In particular, they are unable to facilitate coordinated control of system frequency and power flows. A refined approach and tools are urgently needed to assist system operators to make timely decisions. This paper proposes a novel model-free coordinated frequency control framework based on safe reinforcement learning, with multiple control objectives considered. The load frequency control problem is modeled as a constrained Markov decision process, which can be solved by an AI agent continuously interacting with the grid to achieve sub-second decision making. Extensive numerical experiments conducted at East China Power Grid demonstrate the effectiveness and promise of the proposed method.

eess.SY↗

arXiv2022

Specificity-preserving RGB-D Saliency Detection

Salient object detection (SOD) on RGB and depth images has attracted more and more research interests, due to its effectiveness and the fact that depth cues can now be conveniently captured. Existing RGB-D SOD models usually adopt different fusion strategies to learn a shared representation from the two modalities (\ie, RGB and depth), while few methods explicitly consider how to preserve modality-specific characteristics. In this study, we propose a novel framework, termed SPNet} (Specificity-preserving network), which benefits SOD performance by exploring both the shared information and modality-specific properties (\eg, specificity). Specifically, we propose to adopt two modality-specific networks and a shared learning network to generate individual and shared saliency prediction maps, respectively. To effectively fuse cross-modal features in the shared learning network, we propose a cross-enhanced integration module (CIM) and then propagate the fused feature to the next layer for integrating cross-level information. Moreover, to capture rich complementary multi-modal information for boosting the SOD performance, we propose a multi-modal feature aggregation (MFA) module to integrate the modality-specific features from each individual decoder into the shared decoder. By using a skip connection, the hierarchical features between the encoder and decoder layers can be fully combined. Extensive experiments demonstrate that our~\ours~outperforms cutting-edge approaches on six popular RGB-D SOD and three camouflaged object detection benchmarks. The project is publicly available at: https://github.com/taozh2017/SPNet.

cs.CV↗