arXiv subjects

Lior Shamir

Publications and source records attributed to Lior Shamir.

At least 55 records · Page 3Linked to original sources

arXiv2021

Analysis of the alignment of non-random patterns of spin directions in populations of spiral galaxies

Observations of non-random distribution of galaxies with opposite spin directions have recently attracted considerable attention. Here, a method for identifying cosine-dependence in a dataset of galaxies annotated by their spin directions is described in the light of different aspects that can impact the statistical analysis of the data. These aspects include the presence of duplicate objects in a dataset, errors in the galaxy annotation process, and non-random distribution of the asymmetry that does not necessarily form a dipole or quadrupole axes. The results show that duplicate objects in the dataset can artificially increase the likelihood of cosine dependence detected in the data, but a very high number of duplicate objects is required to lead to a false detection of an axis. Inaccuracy in galaxy annotations has relatively minor impact on the identification of cosine dependence when the error is randomly distributed between clockwise and counterclockwise galaxies. However, when the error is not random, even a small bias of 1% leads to a statistically significant cosine dependence that peaks at the celestial pole. Experiments with artificial datasets in which the distribution was not random showed strong cosine dependence even when the data did not form a full dipole axis alignment. The analysis when using the unmodified data shows asymmetry profile similar to the profile shown in multiple previous studies using several different telescopes.

astro-ph.CO↗

arXiv2021

Automatic identification of outliers in Hubble Space Telescope galaxy images

Rare extragalactic objects can carry substantial information about the past, present, and future universe. Given the size of astronomical databases in the information era it can be assumed that very many outlier galaxies are included in existing and future astronomical databases. However, manual search for these objects is impractical due to the required labor, and therefore the ability to detect such objects largely depends on computer algorithms. This paper describes an unsupervised machine learning algorithm for automatic detection of outlier galaxy images, and its application to several Hubble Space Telescope fields. The algorithm does not require training, and therefore is not dependent on the preparation of clean training sets. The application of the algorithm to a large collection of galaxies detected a variety of outlier galaxy images. The algorithm is not perfect in the sense that not all objects detected by the algorithm are indeed considered outliers, but it reduces the dataset by two orders of magnitude to allow practical manual identification. The catalogue contains 147 objects that would be very difficult to identify without using automation.

astro-ph.GA↗

arXiv2020

Multipole alignment in the large-scale distribution of spin direction of spiral galaxies

Previous observations have suggested non-random distribution of spin directions of galaxies at scales far larger than the size of a supercluster. Here I use $\sim1.7\cdot10^5$ spiral galaxies from SDSS and $3.3\cdot10^4$ spiral galaxies from Pan-STARRS to analyze the distribution of galaxy spin patterns of spiral galaxies as observed from Earth. The analysis shows in both SDSS and Pan-STARRS that the distribution of galaxy spin directions forms a non-random pattern, and can be fitted to a dipole axis in probability much higher than mere chance. These observations agree with previous findings, but are based on more data and two different telescopes. The analysis also shows that the distribution of galaxy spin directions fits a large-scale multipole alignment, with best fit to quadrupole alignment with probability of $\sim6.9σ$ to have such distribution by chance. Comparison of two separate datasets from SDSS and Pan-STARRS such that the galaxies in both datasets have similar redshift distribution provides nearly identical quadrupole patterns.

astro-ph.GA↗

arXiv2020

Galaxy spin direction distribution in HST and SDSS show similar large-scale asymmetry

Several recent observations using large datasets of galaxies showed non-random distribution of the spin directions of spiral galaxies, even when the galaxies are too far from each other to have gravitational interaction. Here, a dataset of $\sim8.7\cdot10^3$ spiral galaxies imaged by Hubble Space Telescope is used to test and profile a possible asymmetry between galaxy spin directions. The asymmetry between galaxies with opposite spin directions is compared to the asymmetry of galaxies from the Sloan Digital Sky Survey. The two datasets contain different galaxies at different redshift ranges, and each dataset was annotated using a different annotation method. The results show that both datasets show a similar asymmetry in the COSMOS field, which is covered by both telescopes. Fitting the asymmetry of the galaxies to cosine dependence shows a dipole axis with probabilities of $\sim2.8σ$ and $\sim7.38σ$ in HST and SDSS, respectively. The most likely dipole axis identified in the HST galaxies is at $(α=78^o,δ=47^o)$, and is well within the $1σ$ error range compared to the location of the most likely dipole axis in the SDSS galaxies with $z>0.15$, identified at $(α=71^o,δ=61^o)$.

astro-ph.CO↗

arXiv2020

A catalog of broad morphology of Pan-STARRS galaxies based on deep learning

Autonomous digital sky surveys such as Pan-STARRS have the ability to image a very large number of galactic and extra-galactic objects, and the large and complex nature of the image data reinforces the use of automation. Here we describe the design and implementation of a data analysis process for automatic broad morphology annotation of galaxies, and applied it to the data of Pan-STARRS DR1. The process is based on filters followed by a two-step convolutional neural network (CNN) classification. Training samples are generated by using an augmented and balanced set of manually classified galaxies. Results are evaluated for accuracy by comparison to the annotation of Pan-STARRS included in a previous broad morphology catalog of SDSS galaxies. Our analysis shows that a CNN combined with several filters is an effective approach for annotating the galaxies and removing unclean images. The catalog contains morphology labels for 1,662,190 galaxies with ~95% accuracy. The accuracy can be further improved by selecting labels above certain confidence thresholds. The catalog is publicly available.

astro-ph.GA↗

arXiv2020

Patterns of galaxy spin directions in SDSS and Pan-STARRS show parity violation and multipoles

The distribution of spin directions of $\sim6.4\cdot10^4$ SDSS spiral galaxies with spectra was examined, and compared to the distribution of $\sim3.3\cdot10^4$ Pan-STARRS galaxies. The analysis shows a statistically significant asymmetry between the number of SDSS galaxies with opposite spin directions, and the magnitude and direction of the asymmetry changes with the direction of observation and with the redshift. The redshift dependence shows that the distribution of the spin direction of SDSS galaxies becomes more asymmetric as the redshift gets higher. Fitting the distribution of the galaxy spin directions to a quadrupole alignment provides fitness with statistical significance >5$σ$, which grows to >8$σ$ when just galaxies with z>0.15 are used. Similar analysis with Pan-STARRS galaxies provides dipole and quadrupole alignments nearly identical to the analysis of SDSS galaxies, showing that the source of the asymmetry is not necessarily a certain unknown flaw in a specific telescope system. While these observations are clearly provocative, there is no known error that could exhibit itself in such form. The data analysis process is fully automatic, and uses deterministic and symmetric algorithms with defined rules. It does not involve either manual analysis that can lead to human perceptual bias, or machine learning that can capture human biases or other subtle differences that are difficult to identify due to the complex nature of machine learning processes. Also, an error in the galaxy annotation process is expected to show consistent bias in all parts of the sky, rather than change with the direction of observation to form a clear and definable pattern.

astro-ph.CO↗

arXiv2020

Computer Analysis of Architecture Using Automatic Image Understanding

In the past few years, computer vision and pattern recognition systems have been becoming increasingly more powerful, expanding the range of automatic tasks enabled by machine vision. Here we show that computer analysis of building images can perform quantitative analysis of architecture, and quantify similarities between city architectural styles in a quantitative fashion. Images of buildings from 18 cities and three countries were acquired using Google StreetView, and were used to train a machine vision system to automatically identify the location of the imaged building based on the image visual content. Experimental results show that the automatic computer analysis can automatically identify the geographical location of the StreetView image. More importantly, the algorithm was able to group the cities and countries and provide a phylogeny of the similarities between architectural styles as captured by StreetView images. These results demonstrate that computer vision and pattern recognition algorithms can perform the complex cognitive task of analyzing images of buildings, and can be used to measure and quantify visual similarities and differences between different styles of architectures. This experiment provides a new paradigm for studying architecture, based on a quantitative approach that can enhance the traditional manual observation and analysis. The source code used for the analysis is open and publicly available.

cs.CV↗

arXiv2020

Large-scale asymmetry between clockwise and counterclockwise galaxies revisited

The ability of digital sky surveys to collect and store very large amounts of data provides completely new ways to study the local universe. Perhaps one of the most provocative observations reported with such tools is the asymmetry between galaxies with clockwise and counterclockwise spin patterns. Here I use $\sim1.7\cdot10^5$ spiral galaxies from SDSS and sort them by their spin patterns (clockwise or counterclockwise) to identify and profile a possible large-scale pattern of the distribution of galaxy spin patterns as observed from Earth. The analysis shows asymmetry between the number of clockwise and counterclockwise spiral galaxies imaged by SDSS, and a dipole axis. These findings largely agree with previous reports using smaller datasets. The probability of the differences between the number of galaxies to occur by chance is (P<4*10^-9), and the probability of an asymmetry axis to occur by mere chance is (P<1.4*10^-5).

astro-ph.GA↗

arXiv2020

Eliminating self-selection: Using data science for authentic undergraduate research in a first-year introductory course

Research experience and mentoring has been identified as an effective intervention for increasing student engagement and retention in the STEM fields, with high impact on students from undeserved populations. However, one-on-one mentoring is limited by the number of available faculty, and in certain cases also by the availability of funding for stipend. One-on-one mentoring is further limited by the selection and self-selection of students. Since research positions are often competitive, they are often taken by the best-performing students. More importantly, many students who do not see themselves as the top students of their class, or do not identify themselves as researchers might not apply, and that self selection can have the highest impact on non-traditional students. To address the obstacles of scalability, selection, and self-selection, we designed a data science research experience for undergraduates as part of an introductory computer science course. Through the intervention, the students are exposed to authentic research as early as their first semester. The intervention is inclusive in the sense that all students registered to the course participate in the research, with no process of selection or self-selection. The research is focused on analytics of large text databases. Using discovery-enabling software tools, the students analyze a corpus of congressional speeches, and identify patterns of differences between democratic speeches and republican speeches, differences between speeches for and against certain bills, and differences between speeches about bills that passed and bills that did not pass. In the beginning of the research experience all student follow the same protocol and use the same data, and then each group of students work on their own research project as part of their final project of the course.

physics.ed-ph↗

arXiv2020

Asymmetry between galaxies with different spin patterns: A comparison between COSMOS, SDSS, and Pan-STARRS

Previous observations of a large number of galaxies show differences between the photometry of spiral galaxies with clockwise spin patterns and spiral galaxies with counterclockwise spin patterns. In this study the mean magnitude of a large number of clockwise galaxies is compared to the mean magnitude of a large number of counterclockwise galaxies. The observed difference between clockwise and counterclockwise spiral galaxies imaged by the space-based COSMOS survey is compared to the differences between clockwise and counterclockwise galaxies imaged by the Earth-based SDSS and Pan-STARRS around the same field. The annotation of clockwise and counterclockwise galaxies is a fully automatic process that does not involve human intervention, and in all experiments both clockwise and counterclockwise galaxies are separated from the same fields. The comparison shows that the same asymmetry was identified by all three telescopes, providing strong evidence that the rotation direction of a spiral galaxy is linked to its luminosity as measured from Earth. Analysis of the luminosity difference using a large number of galaxies from different parts of the sky shows that the difference between clockwise and counterclockwise galaxies changes with the direction of observation, and oriented around an axis.

astro-ph.GA↗

arXiv2019

Large-scale patterns of galaxy spin rotation show cosmological-scale parity violation and multipoles

The distribution of spin direction of ~6.4*10^4 spiral galaxies with spectra was examined. The analysis shows a statistically significant cosmological-scale asymmetry between galaxies with opposite spin direction. The data also reveals that the asymmetry changes with the direction of observation, and with the redshift. The redshift dependence shows that the distribution of the spin direction of galaxies becomes more homogeneous as the redshift gets higher. The data also show photometric differences between galaxies with opposite spin patterns. When normalizing the data by the redshift, the photometric asymmetry is eliminated. However, when normalizing the data by the magnitude, statistically significant differences in the redshift remain. These evidence suggest a violation of the cosmological isotropy and homogeneity assumptions. Fitting the distribution of the galaxy spin directions to a quadrupole alignment provides fitness with statistical significance >5 sigma, which grows to >8 sigma when just galaxies with $z>0.15$ are used. The data analysis process is fully automatic, and it is based on deterministic and symmetric algorithms with defined rules. It does not involve neither manual analysis of the data that can lead to human perceptual bias, nor machine learning that can capture human biases or other subtle differences that are difficult to identify due to the complex and non-intuitive nature of machine learning processes.

astro-ph.GA↗

arXiv2019

Automatic detection of full ring galaxy candidates in SDSS

A full ring is a form of galaxy morphology that is not associated with a specific stage on the Hubble sequence. Digital sky surveys can collect many millions of galaxy images, and therefore even rare forms of galaxies are expected to be present in relatively large numbers in image databases created by digital sky surveys. Sloan Digital Sky Survey (SDSS) data release (DR) 14 contains ~2.6*10^6 objects with spectra identified as galaxies. The method described in this paper applied automatic detection to identify a set of 443 ring galaxy candidates, 104 of them were already included in the Buta + 17 catalogue of ring galaxies in SDSS, but the majority of the galaxies are not included in previous catalogues. Machine analysis cannot yet match the superior pattern recognition abilities of the human brain, and even a small false positive rate makes automatic analysis impractical when scanning through millions of galaxies. Reducing the false positive rate also increases the true negative rate, and therefore the catalogue of ring galaxy candidates is not exhaustive. However, due to its clear advantage in speed, it can provide a large collection of galaxies that can be used for follow-up observations of objects with ring morphology.

astro-ph.GA↗

arXiv2019

Algorithms and Statistical Models for Scientific Discovery in the Petabyte Era

The field of astronomy has arrived at a turning point in terms of size and complexity of both datasets and scientific collaboration. Commensurately, algorithms and statistical models have begun to adapt --- e.g., via the onset of artificial intelligence --- which itself presents new challenges and opportunities for growth. This white paper aims to offer guidance and ideas for how we can evolve our technical and collaborative frameworks to promote efficient algorithmic development and take advantage of opportunities for scientific discovery in the petabyte era. We discuss challenges for discovery in large and complex data sets; challenges and requirements for the next stage of development of statistical methodologies and algorithmic tool sets; how we might change our paradigms of collaboration and education; and the ethical implications of scientists' contributions to widely applicable algorithms and computational modeling. We start with six distinct recommendations that are supported by the commentary following them. This white paper is related to a larger corpus of effort that has taken place within and around the Petabytes to Science Workshops (https://petabytestoscience.github.io/).

astro-ph.IM↗

arXiv2019

Astro2020 APC White Paper: Elevating the Role of Software as a Product of the Research Enterprise

Software is a critical part of modern research, and yet there are insufficient mechanisms in the scholarly ecosystem to acknowledge, cite, and measure the impact of research software. The majority of academic fields rely on a one-dimensional credit model whereby academic articles (and their associated citations) are the dominant factor in the success of a researcher's career. In the petabyte era of astronomical science, citing software and measuring its impact enables academia to retain and reward researchers that make significant software contributions. These highly skilled researchers must be retained to maximize the scientific return from petabyte-scale datasets. Evolving beyond the one-dimensional credit model requires overcoming several key challenges, including the current scholarly ecosystem and scientific culture issues. This white paper will present these challenges and suggest practical solutions for elevating the role of software as a product of the research enterprise.

astro-ph.IM↗

arXiv2018

A hybrid approach to machine learning annotation of large galaxy image databases

Modern astronomy relies on massive databases collected by robotic telescopes and digital sky surveys, acquiring data in a much faster pace than what manual analysis can support. Among other data, these sky surveys collect information about millions and sometimes billions of extra-galactic objects. Since the very large number of objects makes manual observation impractical, automatic methods that can analyze and annotate extra-galactic objects are required to fully utilize the discovery power of these databases. Machine learning methods for annotation of celestial objects can be separated broadly into methods that use the photometric information collected by digital sky surveys, and methods that analyze the image of the object. Here we describe a hybrid method that combines photometry and image data to annotate galaxies by their morphology, and a method that uses that information to identify objects that are visually similar to a query object (query-by-example). The results are compared to using just photometric information from SDSS, and to using just the morphological descriptors extracted directly from the images. The comparison shows that for automatic classification the image data provide marginal addition to the information provided by the photometry data. For query-by-example, however, the analysis of the image data provides more information that improves the automatic detection substantially. The source code and binaries of the method can be downloaded through the Astrophysics Source Code Library.

astro-ph.IM↗

arXiv2018

A catalog of photometric redshift and the distribution of broad galaxy morphologies

We created a catalog of photometric redshift of ~3,000,000 SDSS galaxies annotated by their broad morphology. The photometric redshift was optimized by testing and comparing several pattern recognition algorithms and variable selection strategies, trained and tested on a subset of the galaxies in the catalog that had spectra. The galaxies in the catalog have i magnitude brighter than 18 and Petrosian radius greater than 5.5''. The majority of these objects are not included in previous SDSS photometric redshift catalogs such as the photoz table of SDSS DR12. Analysis of the catalog shows that the number of galaxies in the catalog that are visually spiral increases until redshift of ~0.085, where it peaks and starts to decrease. It also shows that the number of spiral galaxies compared to elliptical galaxies drops as the redshift increases. The catalog is publicly available at https://figshare.com/articles/Morphology_and_photometric_redshift_catalog/4833593

astro-ph.GA↗

arXiv2018

Best Practices for a Future Open Code Policy: Experiences and Vision of the Astrophysics Source Code Library

We are members of the Astrophysics Source Code Library's Advisory Committee and its editor-in-chief. The Astrophysics Source Code Library (ASCL, ascl.net) is a successful initiative that advocates for open research software and provides an infrastructure for registering, discovering, sharing, and citing this software. Started in 1999, the ASCL has been expanding in recent years, with an average of over 200 codes added each year, and now houses over 1,600 code entries.

astro-ph.IM↗

arXiv2017

The Astrophysics Source Code Library: What's new, what's coming

The Astrophysics Source Code Library (ASCL, ascl.net), established in 1999, is a citable online registry of source codes used in research that are available for download; the ASCL's main purpose is to improve the transparency, reproducibility, and falsifiability of research. In 2017, improvements to the resource included real-time data backup for submissions and newly-published entries, improved cross-matching of research papers with software entries in ADS, and expansion of preferred citation information for the software in the ASCL.

astro-ph.IM↗