arXiv subjects

Bo Han

Publications and source records attributed to Bo Han.

At least 343 records · Page 19Linked to original sources

arXiv2021

Imaging Seebeck drift of excitons and trions in MoSe2 monolayers

Hyperspectral imaging at cryogenic temperatures is used to investigate exciton and trion propagation in MoSe$_2$ monolayers encapsulated with hexagonal boron nitride (hBN). Under a tightly focused, continuous-wave laser excitation, the spatial distribution of neutral excitons and charged trions strongly differ at high excitation densities. Remarkably, in this regime the trion distribution develops a halo shape, similar to that previously observed in WS2 monolayers at room temperature and under pulsed excitation. In contrast, the exciton distribution only presents a moderate broadening without the appereance of a halo. Spatially and spectrally resolved luminescence spectra reveal the buildup of a significant temperature gradient at high excitation power, that is attributed to the energy relaxation of photoinduced hot carriers. We show, via a numerical resolution of the transport equations for excitons and trions, that the halo can be interpreted as thermal drift of trions due to a Seebeck term in the particle current. The model shows that the difference between trion and exciton profiles is simply understood in terms of the very different lifetimes of these two quasiparticles.

cond-mat.mtrl-sci↗

arXiv2021

Learning with Group Noise

Machine learning in the context of noise is a challenging but practical setting to plenty of real-world applications. Most of the previous approaches in this area focus on the pairwise relation (casual or correlational relationship) with noise, such as learning with noisy labels. However, the group noise, which is parasitic on the coarse-grained accurate relation with the fine-grained uncertainty, is also universal and has not been well investigated. The challenge under this setting is how to discover true pairwise connections concealed by the group relation with its fine-grained noise. To overcome this issue, we propose a novel Max-Matching method for learning with group noise. Specifically, it utilizes a matching mechanism to evaluate the relation confidence of each object w.r.t. the target, meanwhile considering the Non-IID characteristics among objects in the group. Only the most confident object is considered to learn the model, so that the fine-grained noise is mostly dropped. The performance on arange of real-world datasets in the area of several learning paradigms demonstrates the effectiveness of Max-Matching

cs.LG↗

arXiv2021

Confidence Scores Make Instance-dependent Label-noise Learning Possible

In learning with noisy labels, for every instance, its label can randomly walk to other classes following a transition distribution which is named a noise model. Well-studied noise models are all instance-independent, namely, the transition depends only on the original label but not the instance itself, and thus they are less practical in the wild. Fortunately, methods based on instance-dependent noise have been studied, but most of them have to rely on strong assumptions on the noise models. To alleviate this issue, we introduce confidence-scored instance-dependent noise (CSIDN), where each instance-label pair is equipped with a confidence score. We find with the help of confidence scores, the transition distribution of each instance can be approximately estimated. Similarly to the powerful forward correction for instance-independent noise, we propose a novel instance-level forward correction for CSIDN. We demonstrate the utility and effectiveness of our method through multiple experiments under synthetic label noise and real-world unknown noise.

cs.LG↗

arXiv2021

A Survey of Label-noise Representation Learning: Past, Present and Future

Classical machine learning implicitly assumes that labels of the training data are sampled from a clean distribution, which can be too restrictive for real-world scenarios. However, statistical-learning-based methods may not train deep learning models robustly with these noisy labels. Therefore, it is urgent to design Label-Noise Representation Learning (LNRL) methods for robustly training deep models with noisy labels. To fully understand LNRL, we conduct a survey study. We first clarify a formal definition for LNRL from the perspective of machine learning. Then, via the lens of learning theory and empirical study, we figure out why noisy labels affect deep models' performance. Based on the theoretical guidance, we categorize different LNRL methods into three directions. Under this unified taxonomy, we provide a thorough discussion of the pros and cons of different categories. More importantly, we summarize the essential components of robust LNRL, which can spark new directions. Lastly, we propose possible research directions within LNRL, such as new datasets, instance-dependent LNRL, and adversarial LNRL. We also envision potential directions beyond LNRL, such as learning with feature-noise, preference-noise, domain-noise, similarity-noise, graph-noise and demonstration-noise.

cs.LG↗

arXiv2021

Butterfly: One-step Approach towards Wildly Unsupervised Domain Adaptation

In unsupervised domain adaptation (UDA), classifiers for the target domain (TD) are trained with clean labeled data from the source domain (SD) and unlabeled data from TD. However, in the wild, it is difficult to acquire a large amount of perfectly clean labeled data in SD given limited budget. Hence, we consider a new, more realistic and more challenging problem setting, where classifiers have to be trained with noisy labeled data from SD and unlabeled data from TD -- we name it wildly UDA (WUDA). We show that WUDA ruins all UDA methods if taking no care of label noise in SD, and to this end, we propose a Butterfly framework, a powerful and efficient solution to WUDA. Butterfly maintains four deep networks simultaneously, where two take care of all adaptations (i.e., noisy-to-clean, labeled-to-unlabeled, and SD-to-TD-distributional) and then the other two can focus on classification in TD. As a consequence, Butterfly possesses all the conceptually necessary components for solving WUDA. Experiments demonstrate that, under WUDA, Butterfly significantly outperforms existing baseline methods.

cs.LG↗

arXiv2021

Understanding the Interaction of Adversarial Training with Noisy Labels

Noisy labels (NL) and adversarial examples both undermine trained models, but interestingly they have hitherto been studied independently. A recent adversarial training (AT) study showed that the number of projected gradient descent (PGD) steps to successfully attack a point (i.e., find an adversarial example in its proximity) is an effective measure of the robustness of this point. Given that natural data are clean, this measure reveals an intrinsic geometric property -- how far a point is from its class boundary. Based on this breakthrough, in this paper, we figure out how AT would interact with NL. Firstly, we find if a point is too close to its noisy-class boundary (e.g., one step is enough to attack it), this point is likely to be mislabeled, which suggests to adopt the number of PGD steps as a new criterion for sample selection for correcting NL. Secondly, we confirm AT with strong smoothing effects suffers less from NL (without NL corrections) than standard training (ST), which suggests AT itself is an NL correction. Hence, AT with NL is helpful for improving even the natural accuracy, which again illustrates the superiority of AT as a general-purpose robust learning criterion.

cs.LG↗

arXiv2020

Part-dependent Label Noise: Towards Instance-dependent Label Noise

Learning with the \textit{instance-dependent} label noise is challenging, because it is hard to model such real-world noise. Note that there are psychological and physiological evidences showing that we humans perceive instances by decomposing them into parts. Annotators are therefore more likely to annotate instances based on the parts rather than the whole instances, where a wrong mapping from parts to classes may cause the instance-dependent label noise. Motivated by this human cognition, in this paper, we approximate the instance-dependent label noise by exploiting \textit{part-dependent} label noise. Specifically, since instances can be approximately reconstructed by a combination of parts, we approximate the instance-dependent \textit{transition matrix} for an instance by a combination of the transition matrices for the parts of the instance. The transition matrices for parts can be learned by exploiting anchor points (i.e., data points that belong to a specific class almost surely). Empirical evaluations on synthetic and real-world datasets demonstrate our method is superior to the state-of-the-art approaches for learning from the instance-dependent label noise.

cs.LG↗

arXiv2020

SemiNLL: A Framework of Noisy-Label Learning by Semi-Supervised Learning

Deep learning with noisy labels is a challenging task. Recent prominent methods that build on a specific sample selection (SS) strategy and a specific semi-supervised learning (SSL) model achieved state-of-the-art performance. Intuitively, better performance could be achieved if stronger SS strategies and SSL models are employed. Following this intuition, one might easily derive various effective noisy-label learning methods using different combinations of SS strategies and SSL models, which is, however, reinventing the wheel in essence. To prevent this problem, we propose SemiNLL, a versatile framework that combines SS strategies and SSL models in an end-to-end manner. Our framework can absorb various SS strategies and SSL backbones, utilizing their power to achieve promising performance. We also instantiate our framework with different combinations, which set the new state of the art on benchmark-simulated and real-world datasets with noisy labels.

cs.LG↗

arXiv2020

Extended T: Learning with Mixed Closed-set and Open-set Noisy Labels

The label noise transition matrix $T$, reflecting the probabilities that true labels flip into noisy ones, is of vital importance to model label noise and design statistically consistent classifiers. The traditional transition matrix is limited to model closed-set label noise, where noisy training data has true class labels within the noisy label set. It is unfitted to employ such a transition matrix to model open-set label noise, where some true class labels are outside the noisy label set. Thus when considering a more realistic situation, i.e., both closed-set and open-set label noise occurs, existing methods will undesirably give biased solutions. Besides, the traditional transition matrix is limited to model instance-independent label noise, which may not perform well in practice. In this paper, we focus on learning under the mixed closed-set and open-set label noise. We address the aforementioned issues by extending the traditional transition matrix to be able to model mixed label noise, and further to the cluster-dependent transition matrix to better approximate the instance-dependent label noise in real-world applications. We term the proposed transition matrix as the cluster-dependent extended transition matrix. An unbiased estimator (i.e., extended $T$-estimator) has been designed to estimate the cluster-dependent extended transition matrix by only exploiting the noisy data. Comprehensive synthetic and real experiments validate that our method can better model the mixed label noise, following its more robust performance than the prior state-of-the-art label-noise learning methods.

cs.LG↗

arXiv2020

Confusable Learning for Large-class Few-Shot Classification

Few-shot image classification is challenging due to the lack of ample samples in each class. Such a challenge becomes even tougher when the number of classes is very large, i.e., the large-class few-shot scenario. In this novel scenario, existing approaches do not perform well because they ignore confusable classes, namely similar classes that are difficult to distinguish from each other. These classes carry more information. In this paper, we propose a biased learning paradigm called Confusable Learning, which focuses more on confusable classes. Our method can be applied to mainstream meta-learning algorithms. Specifically, our method maintains a dynamically updating confusion matrix, which analyzes confusable classes in the dataset. Such a confusion matrix helps meta learners to emphasize on confusable classes. Comprehensive experiments on Omniglot, Fungi, and ImageNet demonstrate the efficacy of our method over state-of-the-art baselines.

cs.CV↗

arXiv2020

Provably Consistent Partial-Label Learning

Partial-label learning (PLL) is a multi-class classification problem, where each training example is associated with a set of candidate labels. Even though many practical PLL methods have been proposed in the last two decades, there lacks a theoretical understanding of the consistency of those methods-none of the PLL methods hitherto possesses a generation process of candidate label sets, and then it is still unclear why such a method works on a specific dataset and when it may fail given a different dataset. In this paper, we propose the first generation model of candidate label sets, and develop two novel PLL methods that are guaranteed to be provably consistent, i.e., one is risk-consistent and the other is classifier-consistent. Our methods are advantageous, since they are compatible with any deep network or stochastic optimizer. Furthermore, thanks to the generation model, we would be able to answer the two questions above by testing if the generation model matches given candidate label sets. Experiments on benchmark and real-world datasets validate the effectiveness of the proposed generation model and two PLL methods.

cs.LG↗

arXiv2020

SIGUA: Forgetting May Make Learning with Noisy Labels More Robust

Given data with noisy labels, over-parameterized deep networks can gradually memorize the data, and fit everything in the end. Although equipped with corrections for noisy labels, many learning methods in this area still suffer overfitting due to undesired memorization. In this paper, to relieve this issue, we propose stochastic integrated gradient underweighted ascent (SIGUA): in a mini-batch, we adopt gradient descent on good data as usual, and learning-rate-reduced gradient ascent on bad data; the proposal is a versatile approach where data goodness or badness is w.r.t. desired or undesired memorization given a base learning method. Technically, SIGUA pulls optimization back for generalization when their goals conflict with each other; philosophically, SIGUA shows forgetting undesired memorization can reinforce desired memorization. Experiments demonstrate that SIGUA successfully robustifies two typical base learning methods, so that their performance is often significantly improved.

cs.LG↗

arXiv2020

Long-time behavior for the Cauchy problem of the 3-component Manakov system

In this work, the Riemann-Hilbert problem for the 3-component Manakov system is formulated on the basis of the corresponding $4\times 4$ matrix spectral problem. Furthermore, by applying the nonlinear steepest descent techniques to an associated $4\times 4$ matrix valued Riemann-Hilbert problem, we can find leading-order asymptotics for the Cauchy problem of the 3-component Manakov system.

math.AP↗

arXiv2020

Searching to Exploit Memorization Effect in Learning from Corrupted Labels

Sample selection approaches are popular in robust learning from noisy labels. However, how to properly control the selection process so that deep networks can benefit from the memorization effect is a hard problem. In this paper, motivated by the success of automated machine learning (AutoML), we model this issue as a function approximation problem. Specifically, we design a domain-specific search space based on general patterns of the memorization effect and propose a novel Newton algorithm to solve the bi-level optimization problem efficiently. We further provide theoretical analysis of the algorithm, which ensures a good approximation to critical points. Experiments are performed on benchmark data sets. Results demonstrate that the proposed method is much better than the state-of-the-art noisy-label-learning approaches, and also much more efficient than existing AutoML algorithms.

cs.LG↗

arXiv2020

GeneticKNN: A Weighted KNN Approach Supported by Genetic Algorithm for Photometric Redshift Estimation of Quasars

We combine K-Nearest Neighbors (KNN) with genetic algorithm (GA) for photometric redshift estimation of quasars, short for GeneticKNN, which is a weighted KNN approach supported by GA. This approach has two improvements compared to KNN: one is the feature weighted by GA; another is that the predicted redshift is not the redshift average of K neighbors but the weighted average of median and mean of redshifts for K neighbors, i.e. $p\times z_{median} + (1-p)\times z_{mean}$. Based on the SDSS and SDSS-WISE quasar samples, we explore the performance of GeneticKNN for photometric redshift estimation, comparing with the other six traditional machine learning methods, i.e. Least absolute shrinkage and selection operator (LASSO), support vector regression (SVR), Multi Layer Perceptrons (MLP), XGBoost, KNN and random forest. KNN and random forest show their superiority. Considering the easy implementation of KNN, we make improvement on KNN as GeneticKNN and apply GeneticKNN on photometric redshift estimation of quasars. Finally the performance of GeneticKNN is better than that of LASSO, SVR, MLP, XGBoost, KNN and random forest for all cases. Moreover the accuracy is better with the additional WISE magnitudes for the same method.

astro-ph.IM↗

arXiv2020

Attacks Which Do Not Kill Training Make Adversarial Learning Stronger

Adversarial training based on the minimax formulation is necessary for obtaining adversarial robustness of trained models. However, it is conservative or even pessimistic so that it sometimes hurts the natural generalization. In this paper, we raise a fundamental question---do we have to trade off natural generalization for adversarial robustness? We argue that adversarial training is to employ confident adversarial data for updating the current model. We propose a novel approach of friendly adversarial training (FAT): rather than employing most adversarial data maximizing the loss, we search for least adversarial (i.e., friendly adversarial) data minimizing the loss, among the adversarial data that are confidently misclassified. Our novel formulation is easy to implement by just stopping the most adversarial data searching algorithms such as PGD (projected gradient descent) early, which we call early-stopped PGD. Theoretically, FAT is justified by an upper bound of the adversarial risk. Empirically, early-stopped PGD allows us to answer the earlier question negatively---adversarial robustness can indeed be achieved without compromising the natural generalization.

cs.LG↗

arXiv2020

Classification of $SL_2$ deformed Floquet Conformal Field Theories

Classification of the non-equilibrium quantum many-body dynamics is a challenging problem in condensed matter physics and statistical mechanics. In this work, we study the basic question that whether a (1+1) dimensional conformal field theory (CFT) is stable or not under a periodic driving with $N$ non-commuting Hamiltonians. Previous works showed that a Floquet (or periodically driven) CFT driven by certain $SL_2$ deformed Hamiltonians exhibit both non-heating (stable) and heating (unstable) phases. In this work, we show that the phase diagram depends on the types of driving Hamiltonians. In general, the heating phase is generic, but the non-heating phase may be absent in the phase diagram. For the existence of the non-heating phases, we give sufficient and necessary conditions for $N=2$, and sufficient conditions for $N>2$. These conditions are composed of $N$ layers of data, with each layer determined by the types of driving Hamiltonians. Our results also apply to the single quantum quench problem with $N=1$.

cond-mat.stat-mech↗

arXiv2020

Entanglement Entropy of Generalized Moore-Read Fractional Quantum Hall State Interfaces

Topologically ordered phases of matter can be characterized by the presence of a universal, constant contribution to the entanglement entropy known as the topological entanglement entropy (TEE). The TEE can been calculated for Abelian phases via a "cut-and-glue" approach by treating the entanglement cut as a physical cut, coupling the resulting gapless edges with explicit tunneling terms, and computing the entanglement between the two edges. We provide a first step towards extending this methodology to non-Abelian topological phases, focusing on the generalized Moore-Read (MR) fractional quantum Hall states at filling fractions $ν=1/n$. We consider interfaces between different MR states, write down explicit gapping interactions, which we motivate using an anyon condensation picture, and compute the entanglement entropy for an entanglement cut lying along the interface. Our work provides new insight towards understanding the connections between anyon condensation, gapped interfaces of non-Abelian phases, and TEE.

cond-mat.str-el↗