arXiv subjects

Linda Ness

Publications and source records attributed to Linda Ness.

5 recordsLinked to original sources

arXiv2020

GitEvolve: Predicting the Evolution of GitHub Repositories

Software development is becoming increasingly open and collaborative with the advent of platforms such as GitHub. Given its crucial role, there is a need to better understand and model the dynamics of GitHub as a social platform. Previous work has mostly considered the dynamics of traditional social networking sites like Twitter and Facebook. We propose GitEvolve, a system to predict the evolution of GitHub repositories and the different ways by which users interact with them. To this end, we develop an end-to-end multi-task sequential deep neural network that given some seed events, simultaneously predicts which user-group is next going to interact with a given repository, what the type of the interaction is, and when it happens. To facilitate learning, we use graph based representation learning to encode relationship between repositories. We map users to groups by modelling common interests to better predict popularity and to generalize to unseen users during inference. We introduce an artificial event type to better model varying levels of activity of repositories in the dataset. The proposed multi-task architecture is generic and can be extended to model information diffusion in other social networks. In a series of experiments, we demonstrate the effectiveness of the proposed model, using multiple metrics and baselines. Qualitative analysis of the model's ability to predict popularity and forecast trends proves its applicability.

cs.SI↗

arXiv2020

Product Formalisms for Measures on Spaces with Binary Tree Structures: Representation, Visualization, and Multiscale Noise

In this paper we present a theoretical foundation for a representation of a data set as a measure in a very large hierarchically parametrized family of positive measures, whose parameters can be computed explicitly (rather than estimated by optimization), and illustrate its applicability to a wide range of data types. The pre-processing step then consists of representing data sets as simple measures. The theoretical foundation consists of a dyadic product formula representation lemma, a visualization theorem. We also define an additive multiscale noise model which can be used to sample from dyadic measures and a more general multiplicative multiscale noise model which can be used to perturb continuous functions, Borel measures, and dyadic measures. The first two results are based on theorems. The representation uses the very simple concept of a dyadic tree, and hence is widely applicable, easily understood, and easily computed. Since the data sample is represented as a measure, subsequent analysis can exploit statistical and measure theoretic concepts and theories. Because the representation uses the very simple concept of a dyadic tree defined on the universe of a data set and the parameters are simply and explicitly computable and easily interpretable and visualizable, we hope that this approach will be broadly useful to mathematicians, statisticians, and computer scientists who are intrigued by or involved in data science including its mathematical foundations.

math.CA↗

arXiv2018

Heuristic Framework for Multi-Scale Testing of the Multi-Manifold Hypothesis

When analyzing empirical data, we often find that global linear models overestimate the number of parameters required. In such cases, we may ask whether the data lies on or near a manifold or a set of manifolds (a so-called multi-manifold) of lower dimension than the ambient space. This question can be phrased as a (multi-) manifold hypothesis. The identification of such intrinsic multiscale features is a cornerstone of data analysis and representation and has given rise to a large body of work on manifold learning. In this work, we review key results on multi-scale data analysis and intrinsic dimension followed by the introduction of a heuristic, multiscale framework for testing the multi-manifold hypothesis. Our method implements a hypothesis test on a set of spline-interpolated manifolds constructed from variance-based intrinsic dimensions. The workflow is suitable for empirical data analysis as we demonstrate on two use cases.

stat.ML↗

arXiv2017

Inference of a Dyadic Measure and its Simplicial Geometry from Binary Feature Data and Application to Data Quality

We propose a new method for representing data sets with a set of binary feature functions. We compute both the dyadic set structure determined by an order on the binary features together with the canonical product coefficient parameters for the associated dyadic measure and a variant of a nerve simplicial complex determined by the support of the dyadic measure together with its betti numbers. The product coefficient parameters characterize the relative skewness of the dyadic measure at dyadic scales and localities. The more abstract betti number statistics summarize the simplicial geometry of the support of the measure and satisfy a differential privacy property. Both types of statistics can be computed algorithmically from the binary feature representation of the data. This representation provides a new method for pre-processing data into automatically generated features which explicitly characterize the dyadic statistics and geometry of the data. useful for statistical fusion, decision-making, inference, multi-scale hypothesis testing and visualization. We illustrated the methods on a data quality data set. We exploit a representation lemma for dyadic measures on the unit interval (Fefferman, Kenig and Pipher) reformulated for measures on dyadic sets by Bassu, Jones, Ness and Shallcross. We prove that dyadic sets with dyadic measures have a canonical set of binary features and determine canonical nerve simplicial complexes. We compare our methods with other results for measures on sets with tree structures, recent multi-resolution theory, and persistent homology and suggest links to differential privacy, Bayesian reasoning and algebraic statistics.

math.CA↗

arXiv2014

Multi-Scale Local Shape Analysis and Feature Selection in Machine Learning Applications

We introduce a method called multi-scale local shape analysis, or MLSA, for extracting features that describe the local structure of points within a dataset. The method uses both geometric and topological features at multiple levels of granularity to capture diverse types of local information for subsequent machine learning algorithms operating on the dataset. Using synthetic and real dataset examples, we demonstrate significant performance improvement of classification algorithms constructed for these datasets with correspondingly augmented features.

cs.CG↗