arXiv subjects

Bruno Ribeiro

Publications and source records attributed to Bruno Ribeiro.

At least 55 records · Page 3Linked to original sources

arXiv2021

Set Twister for Single-hop Node Classification

Node classification is a central task in relational learning, with the current state-of-the-art hinging on two key principles: (i) predictions are permutation-invariant to the ordering of a node's neighbors, and (ii) predictions are a function of the node's $r$-hop neighborhood topology and attributes, $r \geq 2$. Both graph neural networks and collective inference methods (e.g., belief propagation) rely on information from up to $r$-hops away. In this work, we study if the use of more powerful permutation-invariant functions can sometimes avoid the need for classifiers to collect information beyond $1$-hop. Towards this, we introduce a new architecture, the Set Twister, which generalizes DeepSets (Zaheer et al., 2017), a simple and widely-used permutation-invariant representation. Set Twister theoretically increases expressiveness of DeepSets, allowing it to capture higher-order dependencies, while keeping its simplicity and low computational cost. Empirically, we see accuracy improvements of Set Twister over DeepSets as well as a variety of graph neural networks and collective inference schemes in several tasks, while showcasing its implementation simplicity and computational efficiency.

cs.LG↗

arXiv2021

Reconstruction for Powerful Graph Representations

Graph neural networks (GNNs) have limited expressive power, failing to represent many graph classes correctly. While more expressive graph representation learning (GRL) alternatives can distinguish some of these classes, they are significantly harder to implement, may not scale well, and have not been shown to outperform well-tuned GNNs in real-world tasks. Thus, devising simple, scalable, and expressive GRL architectures that also achieve real-world improvements remains an open challenge. In this work, we show the extent to which graph reconstruction -- reconstructing a graph from its subgraphs -- can mitigate the theoretical and practical problems currently faced by GRL architectures. First, we leverage graph reconstruction to build two new classes of expressive graph representations. Secondly, we show how graph reconstruction boosts the expressive power of any GNN architecture while being a (provably) powerful inductive bias for invariances to vertex removals. Empirically, we show how reconstruction can boost GNN's expressive power -- while maintaining its invariance to permutations of the vertices -- by solving seven graph property tasks not solvable by the original GNN. Further, we demonstrate how it boosts state-of-the-art GNN's performance across nine real-world benchmark datasets.

cs.LG↗

arXiv2021

Contextual Unsupervised Outlier Detection in Sequences

This work proposes an unsupervised learning framework for trajectory (sequence) outlier detection that combines ranking tests with user sequence models. The overall framework identifies sequence outliers at a desired false positive rate (FPR), in an otherwise parameter-free manner. We evaluate our methodology on a collection of real and simulated datasets based on user actions at the websites last.fm and msnbc.com, where we know ground truth, and demonstrate improved accuracy over existing approaches. We also apply our approach to a large real-world dataset of Pinterest and Facebook users, where we find that users tend to re-share Pinterest posts of Facebook friends significantly more than other types of users, pointing to a potential influence of Facebook friendship on sharing behavior on Pinterest.

cs.LG↗

arXiv2021

Size-Invariant Graph Representations for Graph Classification Extrapolations

In general, graph representation learning methods assume that the train and test data come from the same distribution. In this work we consider an underexplored area of an otherwise rapidly developing field of graph representation learning: The task of out-of-distribution (OOD) graph classification, where train and test data have different distributions, with test data unavailable during training. Our work shows it is possible to use a causal model to learn approximately invariant representations that better extrapolate between train and test data. Finally, we conclude with synthetic and real-world dataset experiments showcasing the benefits of representations that are invariant to train/test distribution shifts.

cs.LG↗

arXiv2021

The evolution of the UV luminosity and stellar mass functions of Lyman-alpha emitters from z~2 to z~6

We measure the evolution of the rest-frame UV luminosity function (LF) and the stellar mass function (SMF) of Lyman-alpha (Lya) emitters (LAEs) from z~2 to z~6 by exploring ~4000 LAEs from the SC4K sample. We find a correlation between Lya luminosity (LLya) and rest-frame UV (M_UV), with best-fit M_UV=-1.6+-0.2 log10(LLya/erg/s)+47+-12 and a shallower relation between LLya and stellar mass (Mstar), with best-fit log10( Mstar/Msun)=0.9+-0.1 log10(LLya/erg/s)-28+-4.0. An increasing LLya cut predominantly lowers the number density of faint M_UV and low Mstar LAEs. We estimate a proxy for the full UV LFs and SMFs of LAEs with simple assumptions of the faint end slope. For the UV LF, we find a brightening of the characteristic UV luminosity (M_UV*) with increasing redshift and a decrease of the characteristic number density (Phi*). For the SMF, we measure a characteristic stellar mass (Mstar*/Msun) increase with increasing redshift, and a Phi* decline. However, if we apply a uniform luminosity cut of log10 (LLya/erg/s) >= 43.0, we find much milder to no evolution in the UV and SMF of LAEs. The UV luminosity density (rho_UV) of the full sample of LAEs shows moderate evolution and the stellar mass density (rho_M) decreases, with both being always lower than the total rho_UV and rho_M of more typical galaxies but slowly approaching them with increasing redshift. Overall, our results indicate that both rho_UV and rho_M of LAEs slowly approach the measurements of continuum-selected galaxies at z>6, which suggests a key role of LAEs in the epoch of reionisation.

astro-ph.GA↗

arXiv2021

Neural Networks for Learning Counterfactual G-Invariances from Single Environments

Despite -- or maybe because of -- their astonishing capacity to fit data, neural networks are believed to have difficulties extrapolating beyond training data distribution. This work shows that, for extrapolations based on finite transformation groups, a model's inability to extrapolate is unrelated to its capacity. Rather, the shortcoming is inherited from a learning hypothesis: Examples not explicitly observed with infinitely many training examples have underspecified outcomes in the learner's model. In order to endow neural networks with the ability to extrapolate over group transformations, we introduce a learning framework counterfactually-guided by the learning hypothesis that any group invariance to (known) transformation groups is mandatory even without evidence, unless the learner deems it inconsistent with the training data. Unlike existing invariance-driven methods for (counterfactual) extrapolations, this framework allows extrapolations from a single environment. Finally, we introduce sequence and image extrapolation tasks that validate our framework and showcase the shortcomings of traditional approaches.

cs.LG↗

arXiv2021

Sequential Stratified Regeneration: MCMC for Large State Spaces with an Application to Subgraph Count Estimation

This work considers the general task of estimating the sum of a bounded function over the edges of a graph, given neighborhood query access and where access to the entire network is prohibitively expensive. To estimate this sum, prior work proposes Markov chain Monte Carlo (MCMC) methods that use random walks started at some seed vertex and whose equilibrium distribution is the uniform distribution over all edges, eliminating the need to iterate over all edges. Unfortunately, these existing estimators are not scalable to massive real-world graphs. In this paper, we introduce Ripple, an MCMC-based estimator that achieves unprecedented scalability by stratifying the Markov chain state space into ordered strata with a new technique that we denote {\em sequential stratified regenerations}. We show that the Ripple estimator is consistent, highly parallelizable, and scales well. We empirically evaluate our method by applying Ripple to the task of estimating connected, induced subgraph counts given some input graph. Therein, we demonstrate that Ripple is accurate and can estimate counts of up to $12$-node subgraphs, which is a task at a scale that has been considered unreachable, not only by prior MCMC-based methods but also by other sampling approaches. For instance, in this target application, we present results in which the Markov chain state space is as large as $10^{43}$, for which Ripple computes estimates in less than $4$ hours, on average.

cs.SI↗

arXiv2021

Membership Inference Attacks and Defenses in Classification Models

We study the membership inference (MI) attack against classifiers, where the attacker's goal is to determine whether a data instance was used for training the classifier. Through systematic cataloging of existing MI attacks and extensive experimental evaluations of them, we find that a model's vulnerability to MI attacks is tightly related to the generalization gap -- the difference between training accuracy and test accuracy. We then propose a defense against MI attacks that aims to close the gap by intentionally reduces the training accuracy. More specifically, the training process attempts to match the training and validation accuracies, by means of a new {\em set regularizer} using the Maximum Mean Discrepancy between the softmax output empirical distributions of the training and validation sets. Our experimental results show that combining this approach with another simple defense (mix-up training) significantly improves state-of-the-art defense against MI attacks, with minimal impact on testing accuracy.

cs.CR↗

arXiv2020

Efficient modeling of higher-order dependencies in networks: from algorithm to application for anomaly detection

Complex systems, represented as dynamic networks, comprise of components that influence each other via direct and/or indirect interactions. Recent research has shown the importance of using Higher-Order Networks (HONs) for modeling and analyzing such complex systems, as the typical Markovian assumption in developing the First Order Network (FON) can be limiting. This higher-order network representation not only creates a more accurate representation of the underlying complex system, but also leads to more accurate network analysis. In this paper, we first present a scalable and accurate model, BuildHON+, for higher-order network representation of data derived from a complex system with various orders of dependencies. Then, we show that this higher-order network representation modeled by BuildHON+ is significantly more accurate in identifying anomalies than FON, demonstrating a need for the higher-order network representation and modeling of complex systems for deriving meaningful conclusions.

cs.SI↗

arXiv2020

Graph Pattern Mining and Learning through User-defined Relations (Extended Version)

In this work we propose R-GPM, a parallel computing framework for graph pattern mining (GPM) through a user-defined subgraph relation. More specifically, we enable the computation of statistics of patterns through their subgraph classes, generalizing traditional GPM methods. R-GPM provides efficient estimators for these statistics by employing a MCMC sampling algorithm combined with several optimizations. We provide both theoretical guarantees and empirical evaluations of our estimators in application scenarios such as stochastic optimization of deep high-order graph neural network models and pattern (motif) counting. We also propose and evaluate optimizations that enable improvements of our estimators accuracy, while reducing their computational costs in up to 3-orders-of-magnitude. Finally,we show that R-GPM is scalable, providing near-linear speedups on 44 cores in all of our tests.

cs.LG↗

arXiv2020

Unsupervised Joint $k$-node Graph Representations with Compositional Energy-Based Models

Existing Graph Neural Network (GNN) methods that learn inductive unsupervised graph representations focus on learning node and edge representations by predicting observed edges in the graph. Although such approaches have shown advances in downstream node classification tasks, they are ineffective in jointly representing larger $k$-node sets, $k{>}2$. We propose MHM-GNN, an inductive unsupervised graph representation approach that combines joint $k$-node representations with energy-based models (hypergraph Markov networks) and GNNs. To address the intractability of the loss that arises from this combination, we endow our optimization with a loss upper bound using a finite-sample unbiased Markov Chain Monte Carlo estimator. Our experiments show that the unsupervised MHM-GNN representations of MHM-GNN produce better unsupervised representations than existing approaches from the literature.

cs.LG↗

arXiv2020

A Collective Learning Framework to Boost GNN Expressiveness

Graph Neural Networks (GNNs) have recently been used for node and graph classification tasks with great success, but GNNs model dependencies among the attributes of nearby neighboring nodes rather than dependencies among observed node labels. In this work, we consider the task of inductive node classification using GNNs in supervised and semi-supervised settings, with the goal of incorporating label dependencies. Because current GNNs are not universal (i.e., most-expressive) graph representations, we propose a general collective learning approach to increase the representation power of any existing GNN. Our framework combines ideas from collective classification with self-supervised learning, and uses a Monte Carlo approach to sampling embeddings for inductive learning across graphs. We evaluate performance on five real-world network datasets and demonstrate consistent, significant improvement in node classification accuracy, for a variety of state-of-the-art GNNs.

cs.LG↗

arXiv2020

On the Equivalence between Positional Node Embeddings and Structural Graph Representations

This work provides the first unifying theoretical framework for node (positional) embeddings and structural graph representations, bridging methods like matrix factorization and graph neural networks. Using invariant theory, we show that the relationship between structural representations and node embeddings is analogous to that of a distribution and its samples. We prove that all tasks that can be performed by node embeddings can also be performed by structural representations and vice-versa. We also show that the concept of transductive and inductive learning is unrelated to node embeddings and graph representations, clearing another source of confusion in the literature. Finally, we introduce new practical guidelines to generating and using node embeddings, which fixes significant shortcomings of standard operating procedures used today.

cs.LG↗

arXiv2020

Deceptive Deletions for Protecting Withdrawn Posts on Social Platforms

Over-sharing poorly-worded thoughts and personal information is prevalent on online social platforms. In many of these cases, users regret posting such content. To retrospectively rectify these errors in users' sharing decisions, most platforms offer (deletion) mechanisms to withdraw the content, and social media users often utilize them. Ironically and perhaps unfortunately, these deletions make users more susceptible to privacy violations by malicious actors who specifically hunt post deletions at large scale. The reason for such hunting is simple: deleting a post acts as a powerful signal that the post might be damaging to its owner. Today, multiple archival services are already scanning social media for these deleted posts. Moreover, as we demonstrate in this work, powerful machine learning models can detect damaging deletions at scale. Towards restraining such a global adversary against users' right to be forgotten, we introduce Deceptive Deletion, a decoy mechanism that minimizes the adversarial advantage. Our mechanism injects decoy deletions, hence creating a two-player minmax game between an adversary that seeks to classify damaging content among the deleted posts and a challenger that employs decoy deletions to masquerade real damaging deletions. We formalize the Deceptive Game between the two players, determine conditions under which either the adversary or the challenger provably wins the game, and discuss the scenarios in-between these two extremes. We apply the Deceptive Deletion mechanism to a real-world task on Twitter: hiding damaging tweet deletions. We show that a powerful global adversary can be beaten by a powerful challenger, raising the bar significantly and giving a glimmer of hope in the ability to be really forgotten on social platforms.

cs.SI↗

arXiv2020

Towards Studying Hierarchical Assembly in Real Time: A Milky Way Progenitor Galaxy at z = 2.36 under the Microscope

We use Hubble Space Telescope (HST) imaging and near-infrared spectroscopy from Keck/MOSFIRE to study the sub-structure around the progenitor of a Milky Way-mass galaxy in the Hubble Frontier Fields (HFF). Specifically, we study an $r_e = 40^{+70}_{-30}$pc, $M_{\star} \sim 10^{8.2} M_{\odot}$ rest-frame ultra-violet luminous "clump" at a projected distance of $\sim$100~pc from a $M_{\star} \sim 10^{9.8}$M$_{\odot}$ galaxy at $z = 2.36$ with a magnification $μ= 5.21$. We measure the star formation history of the clump and galaxy by jointly modeling the broadband spectral energy distribution from HST photometry and H$α$ from MOSFIRE spectroscopy. Given our inferred properties (e.g., mass, metallicity, dust) of the clump and galaxy, we explore scenarios in which the clump formed \emph{in-situ} (e.g., a star forming complex) or \emph{ex-situ} (e.g., a dwarf galaxy being accreted). If it formed \emph{in-situ}, we conclude that the clump is likely a single entity as opposed to a aggregation of smaller star clusters, making it one of the most dense star clusters cataloged. If it formed \emph{ex-situ}, then we are witnessing an accretion event with a 1:40 stellar mass ratio. However, our data alone are not informative enough to distinguish between \emph{in-situ} and \emph{ex-situ} scenarios to a high level of significance. We posit that the addition of high-fidelity metallicity information, such as [OIII]4363Å, which can be detected at modest S/N with only a few hours of JWST/NIRSpec time, may be a powerful discriminant. We suggest that studying larger samples of moderately lensed sub-structures across cosmic time can provide unique insight into the hierarchical formation of galaxies like the Milky Way.

astro-ph.GA↗

arXiv2020

Infinity Learning: Learning Markov Chains from Aggregate Steady-State Observations

We consider the task of learning a parametric Continuous Time Markov Chain (CTMC) sequence model without examples of sequences, where the training data consists entirely of aggregate steady-state statistics. Making the problem harder, we assume that the states we wish to predict are unobserved in the training data. Specifically, given a parametric model over the transition rates of a CTMC and some known transition rates, we wish to extrapolate its steady state distribution to states that are unobserved. A technical roadblock to learn a CTMC from its steady state has been that the chain rule to compute gradients will not work over the arbitrarily long sequences necessary to reach steady state ---from where the aggregate statistics are sampled. To overcome this optimization challenge, we propose $\infty$-SGD, a principled stochastic gradient descent method that uses randomly-stopped estimators to avoid infinite sums required by the steady state computation, while learning even when only a subset of the CTMC states can be observed. We apply $\infty$-SGD to a real-world testbed and synthetic experiments showcasing its accuracy, ability to extrapolate the steady state distribution to unobserved states under unobserved conditions (heavy loads, when training under light loads), and succeeding in difficult scenarios where even a tailor-made extension of existing methods fails.

cs.LG↗

arXiv2020

ALMA reveals the molecular gas properties of 5 star-forming galaxies across the main sequence at 3 < z < 3.5

We present the detection of CO(5-4) with S/N> 7 - 13 and a lower CO transition with S/N > 3 (CO(4-3) for 4 galaxies, and CO(3-2) for one) with ALMA in band 3 and 4 in five main sequence star-forming galaxies with stellar masses 3-6x10^10 M/M_sun at 3 < z < 3.5. We find a good correlation between the total far-infrared luminosity LFIR and the luminosity of the CO(5-4) transition L'CO(5-4), where L'CO(5-4) increases with SFR, indicating that CO(5-4) is a good tracer of the obscured SFR in these galaxies. The two galaxies that lie closer to the star-forming main sequence have CO SLED slopes that are comparable to other star-forming populations, such as local SMGs and BzK star-forming galaxies; the three objects with higher specific star formation rates (sSFR) have far steeper CO SLEDs, which possibly indicates a more concentrated episode of star formation. By exploiting the CO SLED slopes to extrapolate the luminosity of the CO(1-0) transition, and using a classical conversion factor for main sequence galaxies of alpha_CO = 3.8 M_sun(K km s^-1 pc^-2)^-1, we find that these galaxies are very gas rich, with molecular gas fractions between 60 and 80%, and quite long depletion times, between 0.2 and 1 Gyr. Finally, we obtain dynamical masses that are comparable with the sum of stellar and gas mass (at least for four out of five galaxies), allowing us to put a first constraint on the alpha_CO parameter for main sequence galaxies at an unprecedented redshift.

astro-ph.GA↗

arXiv2020

Random Spiking and Systematic Evaluation of Defenses Against Adversarial Examples

Image classifiers often suffer from adversarial examples, which are generated by strategically adding a small amount of noise to input images to trick classifiers into misclassification. Over the years, many defense mechanisms have been proposed, and different researchers have made seemingly contradictory claims on their effectiveness. We present an analysis of possible adversarial models, and propose an evaluation framework for comparing different defense mechanisms. As part of the framework, we introduce a more powerful and realistic adversary strategy. Furthermore, we propose a new defense mechanism called Random Spiking (RS), which generalizes dropout and introduces random noises in the training process in a controlled manner. Evaluations under our proposed framework suggest RS delivers better protection against adversarial examples than many existing schemes.

cs.LG↗