Searcharxiv⌕ Search

arXiv subjects

Bin Deng

Publications and source records attributed to Bin Deng.

33 records · Page 2Linked to original sources
arXiv2023

A Spin-dependent Machine Learning Framework for Transition Metal Oxide Battery Cathode Materials

Owing to the trade-off between the accuracy and efficiency, machine-learning-potentials (MLPs) have been widely applied in the battery materials science, enabling atomic-level dynamics description for various critical processes. However, the challenge arises when dealing with complex transition metal (TM) oxide cathode materials, as multiple possibilities of d-orbital electrons localization often lead to convergence to different spin states (or equivalently local minimums with respect to the spin configurations) after ab initio self-consistent-field calculations, which causes a significant obstacle for training MLPs of cathode materials. In this work, we introduce a solution by incorporating an additional feature - atomic spins - into the descriptor, based on the pristine deep potential (DP) model, to address the above issue by distinguishing different spin states of TM ions. We demonstrate that our proposed scheme provides accurate descriptions for the potential energies of a variety of representative cathode materials, including the traditional Li$_x$TMO$_2$ (TM=Ni, Co, Mn, $x$=0.5 and 1.0), Li-Ni anti-sites in Li$_x$NiO$_2$ ($x$=0.5 and 1.0), cobalt-free high-nickel Li$_x$Ni$_{1.5}$Mn$_{0.5}$O$_4$ ($x$=1.5 and 0.5), and even a ternary cathode material Li$_x$Ni$_{1/3}$Co$_{1/3}$Mn$_{1/3}$O$_2$ ($x$=1.0 and 0.67). We highlight that our approach allows the utilization of all ab initio results as a training dataset, regardless of the system being in a spin ground state or not. Overall, our proposed approach paves the way for efficiently training MLPs for complex TM oxide cathode materials.

cond-mat.mtrl-sci↗
arXiv2023

Adversarial Style Augmentation for Domain Generalization

It is well-known that the performance of well-trained deep neural networks may degrade significantly when they are applied to data with even slightly shifted distributions. Recent studies have shown that introducing certain perturbation on feature statistics (\eg, mean and standard deviation) during training can enhance the cross-domain generalization ability. Existing methods typically conduct such perturbation by utilizing the feature statistics within a mini-batch, limiting their representation capability. Inspired by the domain generalization objective, we introduce a novel Adversarial Style Augmentation (ASA) method, which explores broader style spaces by generating more effective statistics perturbation via adversarial training. Specifically, we first search for the most sensitive direction and intensity for statistics perturbation by maximizing the task loss. By updating the model against the adversarial statistics perturbation during training, we allow the model to explore the worst-case domain and hence improve its generalization performance. To facilitate the application of ASA, we design a simple yet effective module, namely AdvStyle, which instantiates the ASA method in a plug-and-play manner. We justify the efficacy of AdvStyle on tasks of cross-domain classification and instance retrieval. It achieves higher mean accuracy and lower performance fluctuation. Especially, our method significantly outperforms its competitors on the PACS dataset under the single source generalization setting, \eg, boosting the classification accuracy from 61.2\% to 67.1\% with a ResNet50 backbone. Our code will be available at \url{https://github.com/YBZh/AdvStyle}.

cs.CV↗
arXiv2023

Counterfactual Supervision-based Information Bottleneck for Out-of-Distribution Generalization

Learning invariant (causal) features for out-of-distribution (OOD) generalization has attracted extensive attention recently, and among the proposals invariant risk minimization (IRM) is a notable solution. In spite of its theoretical promise for linear regression, the challenges of using IRM in linear classification problems remain. By introducing the information bottleneck (IB) principle into the learning of IRM, IB-IRM approach has demonstrated its power to solve these challenges. In this paper, we further improve IB-IRM from two aspects. First, we show that the key assumption of support overlap of invariant features used in IB-IRM is strong for the guarantee of OOD generalization and it is still possible to achieve the optimal solution without this assumption. Second, we illustrate two failure modes that IB-IRM (and IRM) could fail for learning the invariant features, and to address such failures, we propose a \textit{Counterfactual Supervision-based Information Bottleneck (CSIB)} learning algorithm that provably recovers the invariant features. By requiring counterfactual inference, CSIB works even when accessing data from a single environment. Empirical experiments on several datasets verify our theoretical results.

cs.LG↗
arXiv2021

Non-degeneracy and quantitative stability of half-harmonic maps from ${\mathbb R}$ to ${\mathbb S}$

We consider half-harmonic maps from $\mathbb{R}$ (or $\mathbb{S}$) to $\mathbb{S}$. We prove that all (finite energy) half-harmonic maps are non-degenerate. In other words, they are integrable critical points of the energy functional. A full description of the kernel of the linearized operator around each half-harmonic map is given. The second part of this paper devotes to studying the quantitative stability of half-harmonic maps. When its degree is $\pm 1$, we prove that the deviation of any map $\boldsymbol{u}:\mathbb{R}\to \mathbb{S}$ from Möbius transformations can be controlled uniformly by $\|\boldsymbol{u}\|_{\dot H^{1/2}(\mathbb{R})}^2-deg \boldsymbol{u}$. This result resembles the quantitative rigidity estimate of degree $\pm 1$ harmonic maps $\mathbb{R}^2\to \mathbb{S}^2$ which is proved recently. Furthermore, we address the quantitative stability for half-harmonic maps of higher degree. We prove that if $\boldsymbol{u}$ is already near to a Blaschke product, then the deviation of $\boldsymbol{u}$ to Blaschke products can be controlled by $\|\boldsymbol{u}\|_{\dot H^{1/2}(\mathbb{R})}^2-deg \boldsymbol{u}$. Additionally, a striking example is given to show that such quantitative estimate can not be true uniformly for all $\boldsymbol{u}$ of degree 2. We conjecture similar things happen for harmonic maps ${\mathbb R}^2\to {\mathbb S}^2$.

math.AP↗
arXiv2021

Gradual Domain Adaptation via Self-Training of Auxiliary Models

Domain adaptation becomes more challenging with increasing gaps between source and target domains. Motivated from an empirical analysis on the reliability of labeled source data for the use of distancing target domains, we propose self-training of auxiliary models (AuxSelfTrain) that learns models for intermediate domains and gradually combats the distancing shifts across domains. We introduce evolving intermediate domains as combinations of decreasing proportion of source data and increasing proportion of target data, which are sampled to minimize the domain distance between consecutive domains. Then the source model could be gradually adapted for the use in the target domain by self-training of auxiliary models on evolving intermediate domains. We also introduce an enhanced indicator for sample selection via implicit ensemble and extend the proposed method to semi-supervised domain adaptation. Experiments on benchmark datasets of unsupervised and semi-supervised domain adaptation verify its efficacy.

cs.LG↗
arXiv2021

Semi-supervised Models are Strong Unsupervised Domain Adaptation Learners

Unsupervised domain adaptation (UDA) and semi-supervised learning (SSL) are two typical strategies to reduce expensive manual annotations in machine learning. In order to learn effective models for a target task, UDA utilizes the available labeled source data, which may have different distributions from unlabeled samples in the target domain, while SSL employs few manually annotated target samples. Although UDA and SSL are seemingly very different strategies, we find that they are closely related in terms of task objectives and solutions, and SSL is a special case of UDA problems. Based on this finding, we further investigate whether SSL methods work on UDA tasks. By adapting eight representative SSL algorithms on UDA benchmarks, we show that SSL methods are strong UDA learners. Especially, state-of-the-art SSL methods significantly outperform existing UDA methods on the challenging UDA benchmark of DomainNet, and state-of-the-art UDA methods could be further enhanced with SSL techniques. We thus promote that SSL methods should be employed as baselines in future UDA studies and expect that the revealed relationship between UDA and SSL could shed light on future UDA development. Codes are available at \url{https://github.com/YBZh}.

cs.LG↗
arXiv2021

Sharp quantitative estimates of Struwe's Decomposition

Suppose $u\in \dot{H}^1(\mathbb{R}^n)$. In a seminal work, Struwe proved that if $u\geq 0$ and $\|Δu+u^{\frac{n+2}{n-2}}\|_{H^{-1}}:=Γ(u)\to 0$ then $dist(u,\mathcal{T})\to 0$, where $dist(u,\mathcal{T})$ denotes the $\dot{H}^1(\mathbb{R}^n)$-distance of $u$ from the manifold of sums of Talenti bubbles. Ciraolo, Figalli and Maggi obtained the first quantitative version of Struwe's decomposition with one bubble in all dimensions, namely $δ(u) \leq C Γ(u)$. For Struwe's decomposition with two or more bubbles, Figalli and Glaudo showed a striking dimensional dependent quantitative estimate, namely $δ(u)\leq C Γ(u)$ when $3\leq n\leq 5$ while this is false for $ n\geq 6$. In this paper, we show that \[dist (u,\mathcal{T})\leq C\begin{cases} Γ(u)\left|\log Γ(u)\right|^{\frac{1}{2}}\quad&\text{if }n=6, |Γ(u)|^{\frac{n+2}{2(n-2)}}\quad&\text{if }n\geq 7.\end{cases}\] Furthermore, we show that this inequality is sharp.

math.AP↗
arXiv2021

On Universal Black-Box Domain Adaptation

In this paper, we study an arguably least restrictive setting of domain adaptation in a sense of practical deployment, where only the interface of source model is available to the target domain, and where the label-space relations between the two domains are allowed to be different and unknown. We term such a setting as Universal Black-Box Domain Adaptation (UB$^2$DA). The great promise that UB$^2$DA makes, however, brings significant learning challenges, since domain adaptation can only rely on the predictions of unlabeled target data in a partially overlapped label space, by accessing the interface of source model. To tackle the challenges, we first note that the learning task can be converted as two subtasks of in-class\footnote{In this paper we use in-class (out-class) to describe the classes observed (not observed) in the source black-box model.} discrimination and out-class detection, which can be respectively learned by model distillation and entropy separation. We propose to unify them into a self-training framework, regularized by consistency of predictions in local neighborhoods of target samples. Our framework is simple, robust, and easy to be optimized. Experiments on domain adaptation benchmarks show its efficacy. Notably, by accessing the interface of source model only, our framework outperforms existing methods of universal domain adaptation that make use of source data and/or source models, with a newly proposed (and arguably more reasonable) metric of H-score, and performs on par with them with the metric of averaged class accuracy.

cs.LG↗
arXiv2020

Unsupervised Multi-Class Domain Adaptation: Theory, Algorithms, and Practice

In this paper, we study the formalism of unsupervised multi-class domain adaptation (multi-class UDA), which underlies a few recent algorithms whose learning objectives are only motivated empirically. Multi-Class Scoring Disagreement (MCSD) divergence is presented by aggregating the absolute margin violations in multi-class classification, and this proposed MCSD is able to fully characterize the relations between any pair of multi-class scoring hypotheses. By using MCSD as a measure of domain distance, we develop a new domain adaptation bound for multi-class UDA; its data-dependent, probably approximately correct bound is also developed that naturally suggests adversarial learning objectives to align conditional feature distributions across source and target domains. Consequently, an algorithmic framework of Multi-class Domain-adversarial learning Networks (McDalNets) is developed, and its different instantiations via surrogate learning objectives either coincide with or resemble a few recently popular methods, thus (partially) underscoring their practical effectiveness. Based on our identical theory for multi-class UDA, we also introduce a new algorithm of Domain-Symmetric Networks (SymmNets), which is featured by a novel adversarial strategy of domain confusion and discrimination. SymmNets affords simple extensions that work equally well under the problem settings of either closed set, partial, or open set UDA. We conduct careful empirical studies to compare different algorithms of McDalNets and our newly introduced SymmNets. Experiments verify our theoretical analysis and show the efficacy of our proposed SymmNets. In addition, we have made our implementation code publicly available.

cs.LG↗
arXiv2020

Classification of Blow-ups and Monotonicity Formula for Half Laplacian Nonlinear Heat Equation

We consider the nonlinear half laplacian heat equation $$ u_t+(-Δ)^{\frac{1}{2}} u-|u|^{p-1}u=0,\quad \mathbb{R}^n\times (0, T). $$ We prove that all blows-up are type I, provided that $n \leq 4$ and $ 1<p<p_{*} (n)$ where $ p_{*} (n)$ is an explicit exponent which is below $\frac{n+1}{n-1}$, the critical Sobolev exponent. Central to our proof is a Giga-Kohn type monotonicity formula for half laplacian and a Liouville type theorem for self-similar nonlinear heat equation. This is the first instance of a monotonicity formula at the level of the nonlocal equation, without invoking the extension to the half-space.

math.AP↗
arXiv2020

Label Propagation with Augmented Anchors: A Simple Semi-Supervised Learning baseline for Unsupervised Domain Adaptation

Motivated by the problem relatedness between unsupervised domain adaptation (UDA) and semi-supervised learning (SSL), many state-of-the-art UDA methods adopt SSL principles (e.g., the cluster assumption) as their learning ingredients. However, they tend to overlook the very domain-shift nature of UDA. In this work, we take a step further to study the proper extensions of SSL techniques for UDA. Taking the algorithm of label propagation (LP) as an example, we analyze the challenges of adopting LP to UDA and theoretically analyze the conditions of affinity graph/matrix construction in order to achieve better propagation of true labels to unlabeled instances. Our analysis suggests a new algorithm of Label Propagation with Augmented Anchors (A$^2$LP), which could potentially improve LP via generation of unlabeled virtual instances (i.e., the augmented anchors) with high-confidence label predictions. To make the proposed A$^2$LP useful for UDA, we propose empirical schemes to generate such virtual instances. The proposed schemes also tackle the domain-shift challenge of UDA by alternating between pseudo labeling via A$^2$LP and domain-invariant feature learning. Experiments show that such a simple SSL extension improves over representative UDA methods of domain-invariant feature learning, and could empower two state-of-the-art methods on benchmark UDA datasets. Our results show the value of further investigation on SSL techniques for UDA problems.

cs.LG↗
arXiv2019

The Monge-Ampère equation for strictly $(n-1)$-convex functions with Neumann condition

A $C^2$ function on $\mathbb{R}^n$ is called strictly $(n-1)$-convex if the sum of any $n-1$ eigenvalues of its Hessian is positive. In this paper, we establish a global $C^2$ estimates to the Monge-Ampère equation for strictly $(n-1)$-convex functions with Neumann condition. By the method of continuity, we prove an existence theorem for strictly $(n-1)$-convex solutions of the Neumann problems.

math.AP↗
arXiv2018

Fast Super-resolution 3D SAR Imaging Using an Unfolded Deep Network

For 3D Synthetic Aperture Radar (SAR) imaging, one typical approach is to achieve the cross-track 1D focusing for each range-azimuth pixel after obtaining a stack of 2D complex-valued images. The cross-track focusing is the main difficulty as its aperture length is limited and the antenna positions are usually non-uniformly distributed. Sparsity regularization methods are widely used to tackle these problems. However, these methods are of obvious limitations. The most well-known ones are their heavy computational burdens and unsatisfied stabilities. In this letter, an efficient deep network-based cross-track imaging method is proposed. When trained, the imaging process, i.e. the forward propagation of the network, is made up of simple matrix-vector calculations and element-wise nonlinearity operations, which significantly speed up the imaging. Also, we find that the deep network is of good robustness against noise and model errors. Comprehensive simulations and experiments have been carried out, and the superiority of the proposed method can be clearly seen.

eess.SP↗
arXiv2018

Enhanced Radar Imaging Using a Complex-valued Convolutional Neural Network

Convolutional neural networks (CNN) have been successfully employed to tackle several remote sensing tasks such as image classification and show better performance than previous techniques. For the radar imaging community, a natural question is: Can CNN be introduced to radar imaging and enhance its performance? The presented letter gives an affirmative answer to this question. We firstly propose a processing framework by which a complex-valued CNN (CV-CNN) is used to enhance radar imaging. Then we introduce two modifications to the CV-CNN to adapt it to radar imaging tasks. Subsequently, the method to generate training data is shown and some implementation details are presented. Finally, simulations and experiments are carried out, and both results show the superiority of the proposed method on imaging quality and computational efficiency.

eess.SP↗