arXiv subjects

Donghui Yan

Publications and source records attributed to Donghui Yan.

22 records · Page 2Linked to original sources

arXiv2018

The Turtleback Diagram for Conditional Probability

We elaborate on an alternative representation of conditional probability to the usual tree diagram. We term the representation `turtleback diagram' for its resemblance to the pattern on turtle shells. Adopting the set theoretic view of events and the sample space, the turtleback diagram uses elements from Venn diagrams---set intersection, complement and partition---for conditioning, with the additional notion that the area of a set indicates probability whereas the ratio of areas for conditional probability. Once parts of the diagram are drawn and properly labeled, the calculation of conditional probability involves only simple arithmetic on the area of relevant sets. We discuss turtleback diagrams in relation to other visual representations of conditional probability, and detail several scenarios in which turtleback diagrams prove useful. By the equivalence of recursive space partition and the tree, the turtleback diagram is seen to be equally expressive as the tree diagram for representing abstract concepts. We also provide empirical data on the use of turtleback diagrams with undergraduate students in elementary statistics or probability courses.

stat.AP↗

arXiv2013

Cluster Forests

With inspiration from Random Forests (RF) in the context of classification, a new clustering ensemble method---Cluster Forests (CF) is proposed. Geometrically, CF randomly probes a high-dimensional data cloud to obtain "good local clusterings" and then aggregates via spectral clustering to obtain cluster assignments for the whole dataset. The search for good local clusterings is guided by a cluster quality measure kappa. CF progressively improves each local clustering in a fashion that resembles the tree growth in RF. Empirical studies on several real-world datasets under two different performance metrics show that CF compares favorably to its competitors. Theoretical analysis reveals that the kappa measure makes it possible to grow the local clustering in a desirable way---it is "noise-resistant". A closed-form expression is obtained for the mis-clustering rate of spectral clustering under a perturbation model, which yields new insights into some aspects of spectral clustering.

stat.ME↗

arXiv2012

Statistical methods for tissue array images - algorithmic scoring and co-training

Recent advances in tissue microarray technology have allowed immunohistochemistry to become a powerful medium-to-high throughput analysis tool, particularly for the validation of diagnostic and prognostic biomarkers. However, as study size grows, the manual evaluation of these assays becomes a prohibitive limitation; it vastly reduces throughput and greatly increases variability and expense. We propose an algorithm - Tissue Array Co-Occurrence Matrix Analysis (TACOMA) - for quantifying cellular phenotypes based on textural regularity summarized by local inter-pixel relationships. The algorithm can be easily trained for any staining pattern, is absent of sensitive tuning parameters and has the ability to report salient pixels in an image that contribute to its score. Pathologists' input via informative training patches is an important aspect of the algorithm that allows the training for any specific marker or cell type. With co-training, the error rate of TACOMA can be reduced substantially for a very small training sample (e.g., with size 30). We give theoretical insights into the success of co-training via thinning of the feature set in a high-dimensional setting when there is "sufficient" redundancy among the features. TACOMA is flexible, transparent and provides a scoring process that can be evaluated with clarity and confidence. In a study based on an estrogen receptor (ER) marker, we show that TACOMA is comparable to, or outperforms, pathologists' performance in terms of accuracy and repeatability.

stat.ME↗

arXiv2012

Classification under Data Contamination with Application to Remote Sensing Image Mis-registration

This work is motivated by the problem of image mis-registration in remote sensing and we are interested in determining the resulting loss in the accuracy of pattern classification. A statistical formulation is given where we propose to use data contamination to model and understand the phenomenon of image mis-registration. This model is widely applicable to many other types of errors as well, for example, measurement errors and gross errors etc. The impact of data contamination on classification is studied under a statistical learning theoretical framework. A closed-form asymptotic bound is established for the resulting loss in classification accuracy, which is less than $ε/(1-ε)$ for data contamination of an amount of $ε$. Our bound is sharper than similar bounds in the domain adaptation literature and, unlike such bounds, it applies to classifiers with an infinite Vapnik-Chervonekis (VC) dimension. Extensive simulations have been conducted on both synthetic and real datasets under various types of data contamination, including label flipping, feature swapping and the replacement of feature values with data generated from a random source such as a Gaussian or Cauchy distribution. Our simulation results show that the bound we derive is fairly tight.

stat.ME↗