arXiv subjects

Pavlos Protopapas

Publications and source records attributed to Pavlos Protopapas.

At least 91 records · Page 5Linked to original sources

arXiv2019

Neural Network Models for the Anisotropic Reynolds Stress Tensor in Turbulent Channel Flow

Reynolds-averaged Navier-Stokes (RANS) equations are presently one of the most popular models for simulating turbulence. Performing RANS simulation requires additional modeling for the anisotropic Reynolds stress tensor, but traditional Reynolds stress closure models lead to only partially reliable predictions. Recently, data-driven turbulence models for the Reynolds anisotropy tensor involving novel machine learning techniques have garnered considerable attention and have been rapidly developed. Focusing on modeling the Reynolds stress closure for the specific case of turbulent channel flow, this paper proposes three modifications to a standard neural network to account for the no-slip boundary condition of the anisotropy tensor, the Reynolds number dependence, and spatial non-locality. The modified models are shown to provide increased predicative accuracy compared to the standard neural network when they are trained and tested on channel flow at different Reynolds numbers. The best performance is yielded by the model combining the boundary condition enforcement and Reynolds number injection. This model also outperforms the Tensor Basis Neural Network (Ling et al., 2016) on the turbulent channel flow dataset.

physics.flu-dyn↗

arXiv2019

A Full Probabilistic Model for Yes/No Type Crowdsourcing in Multi-Class Classification

Crowdsourcing has become widely used in supervised scenarios where training sets are scarce and difficult to obtain. Most crowdsourcing models in the literature assume labelers can provide answers to full questions. In classification contexts, full questions require a labeler to discern among all possible classes. Unfortunately, discernment is not always easy in realistic scenarios. Labelers may not be experts in differentiating all classes. In this work, we provide a full probabilistic model for a shorter type of queries. Our shorter queries only require "yes" or "no" responses. Our model estimates a joint posterior distribution of matrices related to labelers' confusions and the posterior probability of the class of every object. We developed an approximate inference approach, using Monte Carlo Sampling and Black Box Variational Inference, which provides the derivation of the necessary gradients. We built two realistic crowdsourcing scenarios to test our model. The first scenario queries for irregular astronomical time-series. The second scenario relies on the image classification of animals. We achieved results that are comparable with those of full query crowdsourcing. Furthermore, we show that modeling labelers' failures plays an important role in estimating true classes. Finally, we provide the community with two real datasets obtained from our crowdsourcing experiments. All our code is publicly available.

cs.LG↗

arXiv2019

Improving Image Classification Robustness through Selective CNN-Filters Fine-Tuning

Image quality plays a big role in CNN-based image classification performance. Fine-tuning the network with distorted samples may be too costly for large networks. To solve this issue, we propose a transfer learning approach optimized to keep into account that in each layer of a CNN some filters are more susceptible to image distortion than others. Our method identifies the most susceptible filters and applies retraining only to the filters that show the highest activation maps distance between clean and distorted images. Filters are ranked using the Borda count election method and then only the most affected filters are fine-tuned. This significantly reduces the number of parameters to retrain. We evaluate this approach on the CIFAR-10 and CIFAR-100 datasets, testing it on two different models and two different types of distortion. Results show that the proposed transfer learning technique recovers most of the lost performance due to input data distortion, at a considerably faster pace with respect to existing methods, thanks to the reduced number of parameters to fine-tune. When few noisy samples are provided for training, our filter-level fine tuning performs particularly well, also outperforming state of the art layer-level transfer learning approaches.

cs.CV↗

arXiv2019

Efficient Optimization of Echo State Networks for Time Series Datasets

Echo State Networks (ESNs) are recurrent neural networks that only train their output layer, thereby precluding the need to backpropagate gradients through time, which leads to significant computational gains. Nevertheless, a common issue in ESNs is determining its hyperparameters, which are crucial in instantiating a well performing reservoir, but are often set manually or using heuristics. In this work we optimize the ESN hyperparameters using Bayesian optimization which, given a limited budget of function evaluations, outperforms a grid search strategy. In the context of large volumes of time series data, such as light curves in the field of astronomy, we can further reduce the optimization cost of ESNs. In particular, we wish to avoid tuning hyperparameters per individual time series as this is costly; instead, we want to find ESNs with hyperparameters that perform well not just on individual time series but rather on groups of similar time series without sacrificing predictive performance significantly. This naturally leads to a notion of clusters, where each cluster is represented by an ESN tuned to model a group of time series of similar temporal behavior. We demonstrate this approach both on synthetic datasets and real world light curves from the MACHO survey. We show that our approach results in a significant reduction in the number of ESN models required to model a whole dataset, while retaining predictive performance for the series in each cluster.

cs.NE↗

arXiv2019

An Algorithm for the Visualization of Relevant Patterns in Astronomical Light Curves

Within the last years, the classification of variable stars with Machine Learning has become a mainstream area of research. Recently, visualization of time series is attracting more attention in data science as a tool to visually help scientists to recognize significant patterns in complex dynamics. Within the Machine Learning literature, dictionary-based methods have been widely used to encode relevant parts of image data. These methods intrinsically assign a degree of importance to patches in pictures, according to their contribution in the image reconstruction. Inspired by dictionary-based techniques, we present an approach that naturally provides the visualization of salient parts in astronomical light curves, making the analogy between image patches and relevant pieces in time series. Our approach encodes the most meaningful patterns such that we can approximately reconstruct light curves by just using the encoded information. We test our method in light curves from the OGLE-III and StarLight databases. Our results show that the proposed model delivers an automatic and intuitive visualization of relevant light curve parts, such as local peaks and drops in magnitude.

astro-ph.IM↗

arXiv2019

T-CGAN: Conditional Generative Adversarial Network for Data Augmentation in Noisy Time Series with Irregular Sampling

In this paper we propose a data augmentation method for time series with irregular sampling, Time-Conditional Generative Adversarial Network (T-CGAN). Our approach is based on Conditional Generative Adversarial Networks (CGAN), where the generative step is implemented by a deconvolutional NN and the discriminative step by a convolutional NN. Both the generator and the discriminator are conditioned on the sampling timestamps, to learn the hidden relationship between data and timestamps, and consequently to generate new time series. We evaluate our model with synthetic and real-world datasets. For the synthetic data, we compare the performance of a classifier trained with T-CGAN-generated data, against the performance of the same classifier trained on the original data. Results show that classifiers trained on T-CGAN-generated data perform the same as classifiers trained on real data, even with very short time series and small training sets. For the real world datasets, we compare our method with other techniques of data augmentation for time series, such as time slicing and time warping, over a classification problem with unbalanced datasets. Results show that our method always outperforms the other approaches, both in case of regularly sampled and irregularly sampled time series. We achieve particularly good performance in case with a small training set and short, noisy, irregularly-sampled time series.

cs.LG↗

arXiv2018

Deep Variational Transfer: Transfer Learning through Semi-supervised Deep Generative Models

In real-world applications, it is often expensive and time-consuming to obtain labeled examples. In such cases, knowledge transfer from related domains, where labels are abundant, could greatly reduce the need for extensive labeling efforts. In this scenario, transfer learning comes in hand. In this paper, we propose Deep Variational Transfer (DVT), a variational autoencoder that transfers knowledge across domains using a shared latent Gaussian mixture model. Thanks to the combination of a semi-supervised ELBO and parameters sharing across domains, we are able to simultaneously: (i) align all supervised examples of the same class into the same latent Gaussian Mixture component, independently from their domain; (ii) predict the class of unsupervised examples from different domains and use them to better model the occurring shifts. We perform tests on MNIST and USPS digits datasets, showing DVT's ability to perform transfer learning across heterogeneous datasets. Additionally, we present DVT's top classification performances on the MNIST semi-supervised learning challenge. We further validate DVT on a astronomical datasets. DVT achieves states-of-the-art classification performances, transferring knowledge across real stars surveys datasets, EROS, MACHO and HiTS, . In the worst performance, we double the achieved F1-score for rare classes. These experiments show DVT's ability to tackle all major challenges posed by transfer learning: different covariate distributions, different and highly imbalanced class distributions and different feature spaces.

cs.LG↗

arXiv2018

Deep learning for turbulent channel flow

Turbulence modeling is a classical approach to address the multiscale nature of fluid turbulence. Instead of resolving all scales of motion, which is currently mathematically and numerically intractable, reduced models that capture the large-scale behavior are derived. One of the most popular reduced models is the Reynolds averaged Navier-Stokes (RANS) equations. The goal is to solve the RANS equations for the mean velocity and pressure field. However, the RANS equations contain a term called the Reynolds stress tensor, which is not known in terms of the mean velocity field. Many RANS turbulence models have been proposed to model the Reynolds stress tensor in terms of the mean velocity field, but are usually not suitably general for all flow fields of interest. Data-driven turbulence models have recently garnered considerable attention and have been rapidly developed. In a seminal work, Ling et al (2016) developed the tensor basis neural network (TBNN), which was used to learn a general Galilean invariant model for the Reynolds stress tensor. The TBNN was applied to a variety of flow fields with encouraging results. In the present study, the TBNN is applied to the turbulent channel flow. Its performance is compared with classical turbulence models as well as a neural network model that does not preserve Galilean invariance. A sensitivity study on the TBNN reveals that the network attempts to adjust to the dataset, but is limited by the mathematical form that guarantees Galilean invariance.

physics.flu-dyn↗

arXiv2018

Deep Learning for Image Sequence Classification of Astronomical Events

We propose a new sequential classification model for astronomical objects based on a recurrent convolutional neural network (RCNN) which uses sequences of images as inputs. This approach avoids the computation of light curves or difference images. This is the first time that sequences of images are used directly for the classification of variable objects in astronomy. The second contribution of this work is the image simulation process. We generate synthetic image sequences that take into account the instrumental and observing conditions, obtaining a realistic, set of movies for each astronomical object. The simulated dataset is used to train our RCNN classifier. This approach allows us to generate datasets to train and test our RCNN model for different astronomical surveys and telescopes. We aim at building a simulated dataset whose distribution is close enough to the real dataset, so that a fine tuning could match the distributions between real and simulated dataset. To test the RCNN classifier trained with the synthetic dataset, we used real-world data from the High cadence Transient Survey (HiTS) obtaining an average recall of 85%, improved to 94% after performing fine tuning with 10 real samples per class. We compare the results of our model with those of a light curve random forest classifier. The proposed RCNN with fine tuning has a similar performance on the HiTS dataset compared to the light curve classifier, trained on an augmented training set with 10 real samples per class. The RCNN approach presents several advantages in an alert stream classification scenario, such as a reduction of the data pre-processing, faster online evaluation and easier performance improvement using a few real data samples. These results encourage us to use this method for alert brokers systems that will process alert streams generated by new telescopes such as the Large Synoptic Survey Telescope.

astro-ph.IM↗

arXiv2018

Multiband galaxy morphologies for CLASH: a convolutional neural network transferred from CANDELS

We present visual-like morphologies over 16 photometric bands, from ultra-violet to near infrared, for 8,412 galaxies in the Cluster Lensing And Supernova survey with Hubble (CLASH) obtained by a convolutional neural network (CNN) model. Our model follows the CANDELS main morphological classification scheme, obtaining the probability for each galaxy at each CLASH band of being spheroid, disk, irregular, point source, or unclassifiable. Our catalog contains morphologies for each galaxy with Hmag < 24.5 in every filter where the galaxy is observed. We trained an initial CNN model using approximately 7,500 expert eyeball labels from The Cosmic Assembly Near-IR Deep Extragalactic Legacy Survey (CANDELS). We created eyeball labels for 100 randomly selected galaxies per each of the 16-filters set of CLASH (1,600 galaxy images in total), where each image was classified by at least five of us. We use these labels to fine-tune the network in order to accurately predict labels for the CLASH data and to evaluate the performance of our model. We achieve a root-mean-square error of 0.0991 on the test set. We show that our proposed fine-tuning technique reduces the number of labeled images needed for training, as compared to directly training over the CLASH data, and achieves a better performance. This approach is very useful to minimize eyeball labeling efforts when classifying unlabeled data from new surveys. This will become particularly useful for massive datasets such as the ones coming from near future surveys such as EUCLID or the LSST. Our catalog consists of prediction of probabilities for each galaxy by morphology in their different bands and is made publicly available at http://www.inf.udec.cl/~guille/data/Deep-CLASH.csv.

astro-ph.IM↗

arXiv2018

The High Cadence Transient Survey (HITS): Compilation and characterization of light-curve catalogs

The High Cadence Transient Survey (HiTS) aims to discover and study transient objects with characteristic timescales between hours and days, such as pulsating, eclipsing and exploding stars. This survey represents a unique laboratory to explore large etendue observations from cadences of about 0.1 days and to test new computational tools for the analysis of large data. This work follows a fully \textit{Data Science} approach: from the raw data to the analysis and classification of variable sources. We compile a catalog of ${\sim}15$ million object detections and a catalog of ${\sim}2.5$ million light-curves classified by variability. The typical depth of the survey is $24.2$, $24.3$, $24.1$ and $23.8$ in $u$, $g$, $r$ and $i$ bands, respectively. We classified all point-like non-moving sources by first extracting features from their light-curves and then applying a Random Forest classifier. For the classification, we used a training set constructed using a combination of cross-matched catalogs, visual inspection, transfer/active learning and data augmentation. The classification model consists of several Random Forest classifiers organized in a hierarchical scheme. The classifier accuracy estimated on a test set is approximately $97\%$. In the unlabeled data, $3\,485$ sources were classified as variables, of which $1\,321$ were classified as periodic. Among the periodic classes we discovered with high confidence, 1 $δ$-scutti, 39 eclipsing binaries, 48 rotational variables and 90 RR-Lyrae and for the non-periodic classes we discovered 1 cataclysmic variables, 630 QSO, and 1 supernova candidates. The first data release can be accessed in the project archive of HiTS.

astro-ph.IM↗

arXiv2018

Unraveling the Spectral Energy Distributions of Clustered YSOs

Stars form in clustered environments, but how they form when the available resources are shared is still not well understood. A related question is whether the IMF is in fact universal across galactic environments, a galactic initial mass function (IGIMF), or whether it is an average of local IMFs. One of the long-standing problems in resolving this question and in the study of young clusters is observational: the emission from multiple sources is frequently seen as blended because at different wavelengths or with different telescopes the beam sizes are different. The confusion hinders our ability to fully characterize clustered star formation. Here we present a new method that uses a genetic algorithm and Bayesian inference to fit the blended SEDs and images of individual YSOs in confused clusters. We apply this method to the infrared photometry of a sample comprising 70 Spitzer-selected, low-mass ($M_{\rm{cl}}<100~\rm{M}_{\odot}$) young clusters in the galactic plane, and use the derived physical parameters to investigate the distributions of masses and evolutionary stages of clustered YSOs, and the implications of those distributions for studies of the IMF and the different models of star formation. We find that for low-mass clusters composed of class I and class II YSOs, there exists a non-trivial relationship between the total stellar mass of the cluster ($M_{\rm{cl}}$) and the mass of its most massive member ($m_{\rm{max}}$). The properties of the derived correlation are most compatible with the random sampling of a Kroupa IMF, with a fundamental high-mass limit of $150~\rm{M}_{\odot}$. Our results are also compatible with SPH models that predict a dynamical termination of the accretion in protostars, with massive stars undergoing this stopping at later times in their evolution.

astro-ph.SR↗

arXiv2018

Uncertain classification of Variable Stars: handling observational GAPS and noise

Automatic classification methods applied to sky surveys have revolutionized the astronomical target selection process. Most surveys generate a vast amount of time series, or \quotes{lightcurves}, that represent the brightness variability of stellar objects in time. Unfortunately, lightcurves' observations take several years to be completed, producing truncated time series that generally remain without the application of automatic classifiers until they are finished. This happens because state of the art methods rely on a variety of statistical descriptors or features that present an increasing degree of dispersion when the number of observations decreases, which reduces their precision. In this paper we propose a novel method that increases the performance of automatic classifiers of variable stars by incorporating the deviations that scarcity of observations produces. Our method uses Gaussian Process Regression to form a probabilistic model of each lightcurve's observations. Then, based on this model, bootstrapped samples of the time series features are generated. Finally a bagging approach is used to improve the overall performance of the classification. We perform tests on the MACHO and OGLE catalogs, results show that our method classifies effectively some variability classes using a small fraction of the original observations. For example, we found that RR Lyrae stars can be classified with around 80\% of accuracy just by observing the first 5\% of the whole lightcurves' observations in MACHO and OGLE catalogs. We believe these results prove that, when studying lightcurves, it is important to consider the features' error and how the measurement process impacts it.

astro-ph.IM↗

arXiv2018

Automatic Survey-Invariant Variable Star Classification

Machine learning techniques have been successfully used to classify variable stars on widely-studied astronomical surveys. These datasets have been available to astronomers long enough, thus allowing them to perform deep analysis over several variable sources and generating useful catalogs with identified variable stars. The products of these studies are labeled data that enable supervised learning models to be trained successfully. However, when these models are blindly applied to data from new sky surveys their performance drops significantly. Furthermore, unlabeled data becomes available at a much higher rate than its labeled counterpart, since labeling is a manual and time-consuming effort. Domain adaptation techniques aim to learn from a domain where labeled data is available, the \textit{source domain}, and through some adaptation perform well on a different domain, the \textit{target domain}. We propose a full probabilistic model that represents the joint distribution of features from two surveys as well as a probabilistic transformation of the features between one survey to the other. This allows us to transfer labeled data to a study where it is not available and to effectively run a variable star classification model in a new survey. Our model represents the features of each domain as a Gaussian mixture and models the transformation as a translation, rotation and scaling of each separate component. We perform tests using three different variability catalogs: EROS, MACHO, and HiTS, presenting differences among them, such as the amount of observations per star, cadence, observational time and optical bands observed, among others.

astro-ph.IM↗

arXiv2017

A dwarf planet class object in the 21:5 resonance with Neptune

We report the discovery of a $H_r = 3.4\pm0.1$ dwarf planet candidate by the Pan-STARRS Outer Solar System Survey. 2010 JO$_{179}$ is red with $(g-r)=0.88 \pm 0.21$, roughly round, and slowly rotating, with a period of $30.6$ hr. Estimates of its albedo imply a diameter of 600--900~km. Observations sampling the span between 2005--2016 provide an exceptionally well-determined orbit for 2010 JO$_{179}$, with a semi-major axis of $78.307\pm0.009$ au, distant orbits known to this precision are rare. We find that 2010 JO$_{179}$ librates securely within the 21:5 mean-motion resonance with Neptune on hundred-megayear time scales, joining the small but growing set of known distant dwarf planets on metastable resonant orbits. These imply a substantial trans-Neptunian population that shifts between stability in high-order resonances, the detached population, and the eroding population of the scattering disk.

astro-ph.EP↗

arXiv2017

Robust period estimation using mutual information for multi-band light curves in the synoptic survey era

The Large Synoptic Survey Telescope (LSST) will produce an unprecedented amount of light curves using six optical bands. Robust and efficient methods that can aggregate data from multidimensional sparsely-sampled time series are needed. In this paper we present a new method for light curve period estimation based on the quadratic mutual information (QMI). The proposed method does not assume a particular model for the light curve nor its underlying probability density and it is robust to non-Gaussian noise and outliers. By combining the QMI from several bands the true period can be estimated even when no single-band QMI yields the period. Period recovery performance as a function of average magnitude and sample size is measured using 30,000 synthetic multi-band light curves of RR Lyrae and Cepheid variables generated by the LSST Operations and Catalog simulators. The results show that aggregating information from several bands is highly beneficial in LSST sparsely-sampled time series, obtaining an absolute increase in period recovery rate up to 50%. We also show that the QMI is more robust to noise and light curve length (sample size) than the multiband generalizations of the Lomb Scargle and Analysis of Variance periodograms, recovering the true period in 10-30% more cases than its competitors. A python package containing efficient Cython implementations of the QMI and other methods is provided.

astro-ph.IM↗

arXiv2016

Detection of Time Lags Between Quasar Continuum Emission Bands based on Pan-STARRS Light-curves

We study the time lags between the continuum emission of quasars at different wavelengths, based on more than four years of multi-band ($g$, $r$, $i$, $z$) light-curves in the Pan-STARRS Medium Deep Fields. As photons from different bands emerge from different radial ranges in the accretion disk, the lags constrain the sizes of the accretion disks. We select 240 quasars with redshifts $z \approx 1$ or $z \approx 0.3$ that are relatively emission line free. The light curves are sampled from day to month timescales, which makes it possible to detect lags on the scale of the light crossing time of the accretion disks. With the code JAVELIN, we detect typical lags of several days in the rest frame between the $g$ band and the $riz$ bands. The detected lags are $\sim 2-3$ times larger than the light crossing time estimated from the standard thin disk model, consistent with the recently measured lag in NGC5548 and micro-lensing measurements of quasars. The lags in our sample are found to increase with increasing luminosity. Furthermore, the increase in lags going from $g-r$ to $g-i$ and then to $g-z$ is slower than predicted in the thin disk model, particularly for high luminosity quasars. The radial temperature profile in the disk must be different from what is assumed. We also find evidence that the lags decrease with increasing line ratios between ultraviolet FeII lines and MgII, which may point to changes in the accretion disk structure at higher metallicity.

astro-ph.HE↗

arXiv2016

Clustering Based Feature Learning on Variable Stars

The success of automatic classification of variable stars strongly depends on the lightcurve representation. Usually, lightcurves are represented as a vector of many statistical descriptors designed by astronomers called features. These descriptors commonly demand significant computational power to calculate, require substantial research effort to develop and do not guarantee good performance on the final classification task. Today, lightcurve representation is not entirely automatic; algorithms that extract lightcurve features are designed by humans and must be manually tuned up for every survey. The vast amounts of data that will be generated in future surveys like LSST mean astronomers must develop analysis pipelines that are both scalable and automated. Recently, substantial efforts have been made in the machine learning community to develop methods that prescind from expert-designed and manually tuned features for features that are automatically learned from data. In this work we present what is, to our knowledge, the first unsupervised feature learning algorithm designed for variable stars. Our method first extracts a large number of lightcurve subsequences from a given set of photometric data, which are then clustered to find common local patterns in the time series. Representatives of these patterns, called exemplars, are then used to transform lightcurves of a labeled set into a new representation that can then be used to train an automatic classifier. The proposed algorithm learns the features from both labeled and unlabeled lightcurves, overcoming the bias generated when the learning process is done only with labeled data. We test our method on MACHO and OGLE datasets; the results show that the classification performance we achieve is as good and in some cases better than the performance achieved using traditional features, while the computational cost is significantly lower.

astro-ph.SR↗