arXiv subjects

Lior Shamir

Publications and source records attributed to Lior Shamir.

At least 19 recordsLinked to original sources

arXiv2026

Amplifying the imaging power of digital sky surveys with space telescopes data and generative AI

While Digital sky surveys provide excellent throughput of image data and can cover a large footprint, their imaging power is normally inferior to that of space-based telescopes. Space-based telescopes, on the other hand, provide excellent imaging power and can image the deep Universe, but cannot provide the same throughput as advanced ground-based sky surveys. Here, we utilize generative AI to elevate the quality of galaxy images taken by ground-based telescopes to the level of details enabled by space telescopes. The solution is based on the nature of galaxy shapes, allowing generative AI trained on space-based images to convert weak signal into detailed and clear galaxy images. The method allows for combining the high throughput of ground-based sky surveys with the image quality of space-based telescopes. The source code for the method is available, as well as paired training data and a catalog of 63,202 galaxy images enhanced by the proposed method. We also provide a software tool that encapsulates the entire pipeline and the custom generative AI model to generate galaxy images with enhanced quality.

astro-ph.IM↗

arXiv2026

GRAZE: Grounded Refinement and Motion-Aware Zero-Shot Event Localization

American football practice generates video at scale, yet the interaction of interest occupies only a brief window of each long, untrimmed clip. Reliable biomechanical analysis, therefore, depends on spatiotemporal localization that identifies both the interacting entities and the onset of contact. We study First Point of Contact (FPOC), defined as the first frame in which a player physically touches a tackle dummy, in unconstrained practice footage with camera motion, clutter, multiple similarly equipped athletes, and rapid pose changes around impact. We present GRAZE, a training-free pipeline for FPOC localization that requires no labeled tackle-contact examples. GRAZE uses Grounding DINO to discover candidate player-dummy interactions, refines them with motion-aware temporal reasoning, and uses SAM2 as an explicit pixel-level verifier of contact rather than relying on detection confidence alone. This separation between candidate discovery and contact confirmation makes the approach robust to cluttered scenes and unstable grounding near impact. On 738 tackle-practice videos, GRAZE produces valid outputs for 97.4% of clips and localizes FPOC within $\pm$ 10 frames on 77.5% of all clips and within $\pm$ 20 frames on 82.7% of all clips. These results show that frame-accurate contact onset localization in real-world practice footage is feasible without task-specific training.

cs.CV↗

arXiv2026

Unmasking Biases and Reliability Concerns in Convolutional Neural Networks Analysis of Cancer Pathology Images

Convolutional Neural Networks have shown promising effectiveness in identifying different types of cancer from radiographs. However, the opaque nature of CNNs makes it difficult to fully understand the way they operate, limiting their assessment to empirical evaluation. Here we study the soundness of the standard practices by which CNNs are evaluated for the purpose of cancer pathology. Thirteen highly used cancer benchmark datasets were analyzed, using four common CNN architectures and different types of cancer, such as melanoma, carcinoma, colorectal cancer, and lung cancer. We compared the accuracy of each model with that of datasets made of cropped segments from the background of the original images that do not contain clinically relevant content. Because the rendered datasets contain no clinical information, the null hypothesis is that the CNNs should provide mere chance-based accuracy when classifying these datasets. The results show that the CNN models provided high accuracy when using the cropped segments, sometimes as high as 93\%, even though they lacked biomedical information. These results show that some CNN architectures are more sensitive to bias than others. The analysis shows that the common practices of machine learning evaluation might lead to unreliable results when applied to cancer pathology. These biases are very difficult to identify, and might mislead researchers as they use available benchmark datasets to test the efficacy of CNN methods.

eess.IV↗

arXiv2025

CornViT: A Multi-Stage Convolutional Vision Transformer Framework for Hierarchical Corn Kernel Analysis

Accurate grading of corn kernels is critical for seed certification, directional seeding, and breeding, yet it is still predominantly performed by manual inspection. This work introduces CornViT, a three-stage Convolutional Vision Transformer (CvT) framework that emulates the hierarchical reasoning of human seed analysts for single-kernel evaluation. Three sequential CvT-13 classifiers operate on 384x384 RGB images: Stage 1 distinguishes pure from impure kernels; Stage 2 categorizes pure kernels into flat and round morphologies; and Stage 3 determines the embryo orientation (up vs. down) for pure, flat kernels. Starting from a public corn seed image collection, we manually relabeled and filtered images to construct three stage-specific datasets: 7265 kernels for purity, 3859 pure kernels for morphology, and 1960 pure-flat kernels for embryo orientation, all released as benchmarks. Head-only fine-tuning of ImageNet-22k pretrained CvT-13 backbones yields test accuracies of 93.76% for purity, 94.11% for shape, and 91.12% for embryo-orientation detection. Under identical training conditions, ResNet-50 reaches only 76.56 to 81.02 percent, whereas DenseNet-121 attains 86.56 to 89.38 percent accuracy. These results highlight the advantages of convolution-augmented self-attention for kernel analysis. To facilitate adoption, we deploy CornViT in a Flask-based web application that performs stage-wise inference and exposes interpretable outputs through a browser interface. Together, the CornViT framework, curated datasets, and web application provide a deployable solution for automated corn kernel quality assessment in seed quality workflows. Source code and data are publicly available.

cs.CV↗

arXiv2025

Galaxy image simplification using Generative AI

Modern digital sky surveys have been acquiring images of billions of galaxies. While these images often provide sufficient details to analyze the shape of the galaxies, accurate analysis of such high volumes of images requires effective automation. Current solutions often rely on machine learning annotation of the galaxy images based on a set of pre-defined classes. Here we introduce a new approach to galaxy image analysis that is based on generative AI. The method simplifies the galaxy images and automatically converts them into a ``skeletonized" form. The simplified images allow accurate measurements of the galaxy shapes and analysis that is not limited to a certain pre-defined set of classes. We demonstrate the method by applying it to galaxy images acquired by the DESI Legacy Survey. The code and data are publicly available. The method was applied to 125,000 DESI Legacy Survey images, and the catalog of the simplified images is publicly available.

astro-ph.GA↗

arXiv2025

An open dataset of neural networks for hypernetwork research

Despite the transformative potential of AI, the concept of neural networks that can produce other neural networks by generating model weights (hypernetworks) has been largely understudied. One of the possible reasons is the lack of available research resources that can be used for the purpose of hypernetwork research. Here we describe a dataset of neural networks, designed for the purpose of hypernetworks research. The dataset includes $10^4$ LeNet-5 neural networks trained for binary image classification separated into 10 classes, such that each class contains 1,000 different neural networks that can identify a certain ImageNette V2 class from all other classes. A computing cluster of over $10^4$ cores was used to generate the dataset. Basic classification results show that the neural networks can be classified with accuracy of 72.0%, indicating that the differences between the neural networks can be identified by supervised machine learning algorithms. The ultimate purpose of the dataset is to enable hypernetworks research. The dataset and the code that generates it are open and accessible to the public.

cs.LG↗

arXiv2025

Explainable identification of similarities between entities for discovery in large text

With the availability of virtually infinite number text documents in digital format, automatic comparison of textual data is essential for extracting meaningful insights that are difficult to identify manually. Many existing tools, including AI and large language models, struggle to provide precise and explainable insights into textual similarities. In many cases they determine the similarity between documents as reflected by the text, rather than the similarities between the subjects being discussed in these documents. This study addresses these limitations by developing an n-gram analysis framework designed to compare documents automatically and uncover explainable similarities. A scoring formula is applied to assigns each of the n-grams with a weight, where the weight is higher when the n-grams are more frequent in both documents, but is penalized when the n-grams are more frequent in the English language. Visualization tools like word clouds enhance the representation of these patterns, providing clearer insights. The findings demonstrate that this framework effectively uncovers similarities between text documents, offering explainable insights that are often difficult to identify manually. This non-parametric approach provides a deterministic solution for identifying similarities across various fields, including biographies, scientific literature, historical texts, and more. Code for the method is publicly available.

cs.IR↗

arXiv2025

The distribution of galaxy rotation in JWST Advanced Deep Extragalactic Survey

JWST provides a view of the Universe never seen before, and specifically fine details of galaxies in deep space. JWST Advanced Deep Extragalactic Survey (JADES) is a deep field survey, providing unprecedentedly detailed view of galaxies in the early Universe. The field is also in relatively close proximity to the Galactic pole. Analysis of spiral galaxies by their direction of rotation in JADES shows that the number of galaxies in that field that rotate in the opposite direction relative to the Milky Way galaxy is ~50% higher than the number of galaxies that rotate in the same direction relative to the Milky Way. The analysis is done using a computer-aided quantitative method, but the difference is so extreme that it can be noticed and inspected even by the unaided human eye. These observations are in excellent agreement with deep fields taken at around the same footprint by HST and JWST. The reason for the difference may be related to the structure of the early Universe, but it can also be related to the physics of galaxy rotation and the internal structure of galaxies. In that case the observation can provide possible explanations to other puzzling anomalies such as the Ho tension and the observation of massive mature galaxies at very high redshifts.

astro-ph.CO↗

arXiv2024

Identifying Bias in Deep Neural Networks Using Image Transforms

CNNs have become one of the most commonly used computational tool in the past two decades. One of the primary downsides of CNNs is that they work as a ``black box", where the user cannot necessarily know how the image data are analyzed, and therefore needs to rely on empirical evaluation to test the efficacy of a trained CNN. This can lead to hidden biases that affect the performance evaluation of neural networks, but are difficult to identify. Here we discuss examples of such hidden biases in common and widely used benchmark datasets, and propose techniques for identifying dataset biases that can affect the standard performance evaluation metrics. One effective approach to identify dataset bias is to perform image classification by using merely blank background parts of the original images. However, in some situations a blank background in the images is not available, making it more difficult to separate foreground or contextual information from the bias. To overcome this, we propose a method to identify dataset bias without the need to crop background information from the images. That method is based on applying several image transforms to the original images, including Fourier transform, wavelet transforms, median filter, and their combinations. These transforms were applied to recover background bias information that CNNs use to classify images. This transformations affect the contextual visual information in a different manner than it affects the systemic background bias. Therefore, the method can distinguish between contextual information and the bias, and alert on the presence of background bias even without the need to separate sub-images parts from the blank background of the original images. Code used in the experiments is publicly available.

cs.CV↗

arXiv2024

Asymmetry in the distribution of HSC galaxy spin directions: comment on arXiv:2410.18884v1

In the past decade, an asymmetry in the large-scale distribution of galaxy spin directions has been observed in data from all relevant digital sky surveys, all showing a higher number of galaxies rotating in the opposite direction relative to the Milky Way as observed from Earth. Additionally, JWST deep fields have shown that the asymmetry is clear and obvious, and can be sensed even by the naked human eye. These experiments were performed using two separate statistical methods: standard binomial distribution and simple $\chi^2$ statistics. Stiskalek \& Desmond (2024) suggested that the asymmetry in the distribution of galaxy spin directions is due to the use of binomial or $\chi^2$ statistics. Instead, they developed a new complex ad-hoc statistical method that shows random distribution in galaxy spin directions, and specifically in data from HSC. Source code for the method was also made available. The primary downside of the new method is that it is not able to identify asymmetry in the distribution of galaxy spin directions. Even when the new method is provided with synthetic data with extreme and obvious asymmetry, it still reports a null-hypothesis Universe with random distribution. That shows empirically that the method cannot sense asymmetry in the distribution of the directions of rotation of galaxies. While this further concludes that the distribution of galaxy spin direction as observed from Earth is not symmetric, it is not necessarily an indication of an anomaly in the large-scale structure. The excessive number of galaxies that rotate in the opposite direction relative to the Milky Way can also be driven by the internal structure of galaxies and the physics of galaxy rotation. The phenomenon can be related to other puzzling anomalies such the Ho tension. Data are publicly available, and no code is needed to reproduce the results since only conventional statistics is used.

astro-ph.CO↗

arXiv2024

Asymmetry in galaxy spin directions: a fully reproducible experiment using HSC data

The asymmetry in the large-scale distribution of the directions towards spiral galaxies rotate has been observed by multiple telescopes, all show a consistent asymmetry in the distribution of galaxy spin directions as observed from Earth. Here, galaxies with redshift from HSC DR3 are annotated by their direction of rotation, and their distribution is analyzed. The results show that galaxies that rotate in the opposite direction relative to the Milky Way as observed from Earth are significantly more prevalent compared to galaxies that rotate in the same direction relative to the Milky Way. The asymmetry also forms a dipole axis that becomes stronger when the redshift gets higher. These results are aligned with observations from virtually all premier digital sky surveys, as well as space telescopes such as HST and JWST. That shows that the distribution of galaxy spin directions as observed from Earth is not symmetric, and has a possible link to the rotational velocity of the Milky Way. The experiment is provided with data, code, and a full protocol that allows to easily reproduce the results in a transparent manner. That practice is used to overcome the ``reproducibility crisis" in science.

astro-ph.CO↗

arXiv2024

An empirical consistent redshift bias: A possible direct reproducible observation of Zwicky's TL theory

Recent advancements have shown tensions between observations and our current understanding of the Universe. Such observations may include the $H_o$ tension and massive galaxies at high redshifts that are older than what traditional galaxy formation models predicted. Since these observations are based on the redshift as the primary distance indicator, a bias in the redshift may explain these tensions. While the redshift follows an established model, when applied to astronomy it is based on the assumption that the rotational velocity of the Milky Way galaxy relative to the observed galaxies has a negligible effect on the redshift. But given the mysterious nature of the physics of galaxy rotation, that assumption should be tested. The test is done by comparing the redshift of galaxies rotating in the same direction relative to the Milky Way to the redshift of galaxies rotating in the opposite direction relative to the Milky Way. The results show that the mean redshift of galaxies that rotate in the same direction relative to the Milky Way is higher than the mean redshift of galaxies that rotate in the opposite direction. Additionally, the redshift difference becomes larger as the redshift gets higher. The consistency of the analysis was verified by comparing data collected by three different telescopes, annotated using four different methods, released by three different research teams, and cover both the Northern and Southern ends of the galactic pole. All datasets are in excellent agreement with each other, showing consistency in the observed redshift bias. Given the "reproducibility crisis" in science, all datasets used in this study are publicly available, and the results can be easily reproduced. The observation could be a first direct empirical reproducible observation for the Zwicky's "tired-light" model.

astro-ph.CO↗

arXiv2024

Analysis and prevention of AI-based phishing email attacks

Phishing email attacks are among the most common and most harmful cybersecurity attacks. With the emergence of generative AI, phishing attacks can be based on emails generated automatically, making it more difficult to detect them. That is, instead of a single email format sent to a large number of recipients, generative AI can be used to send each potential victim a different email, making it more difficult for cybersecurity systems to identify the scam email before it reaches the recipient. Here we describe a corpus of AI-generated phishing emails. We also use different machine learning tools to test the ability of automatic text analysis to identify AI-generated phishing emails. The results are encouraging, and show that machine learning tools can identify an AI-generated phishing email with high accuracy compared to regular emails or human-generated scam email. By applying descriptive analytic, the specific differences between AI-generated emails and manually crafted scam emails are profiled, and show that AI-generated emails are different in their style from human-generated phishing email scams. Therefore, automatic identification tools can be used as a warning for the user. The paper also describes the corpus of AI-generated phishing emails that is made open to the public, and can be used for consequent studies. While the ability of machine learning to detect AI-generated phishing email is encouraging, AI-generated phishing emails are different from regular phishing emails, and therefore it is important to train machine learning systems also with AI-generated emails in order to repel future phishing attacks that are powered by generative AI.

cs.CR↗

arXiv2024

Reproducible empirical evidence of cosmological-scale asymmetry in galaxy spin directions: comment on arXiv:2404.06617

The distribution of the spin directions of galaxies has been a question in the past decade, with numerous Earth-based and space-based experiments showing that the distribution is not necessarily random. These experiments were based on different statistical methods, one of them was a simple and empirically verified open source $\chi^2$ method. Patel & Desmond (2024) proposed that previous experiments showing non-random distribution are flawed since they assume Gaussian distribution. To address that, they apply a new complex ad-hoc statistical method to several datasets, none of them except for one were used in the past to claim for a dipole axis. The new method showed that all datasets except for one exhibit isotropy. This paper discusses the soundness of the contention that Gaussian distribution cannot be assumed for galaxy spin directions. More importantly, simple empirical analyses show that the new statistical method is not fully responsive to asymmetry in the distribution of galaxy spin directions, and does not identify non-random distribution even in situations where a dipole axis clearly exists in the data, or when an artificial bias is added to the data to create an extremely non-random dataset. Results using Monte Carlo simulation show substantial differences between the results of the simulation and the results of the new statistical method. Code and data to reproduce the experiments are available, and released in a manner that is easily reproducible. Possible reasons leading to the results are also discussed. The claims that the actual results are different from the results reported in the papers are examined in an open and transparent manner. These claims are found to be inaccurate, as the previous literature results are fully reproducible. These findings further reinforce the need to study astrophysical or cosmological explanations for the non-random distribution.

astro-ph.CO↗

arXiv2024

Galaxy spin direction asymmetry in JWST deep fields

The unprecedented imaging power of JWST provides new abilities to observe the shapes of objects in the early Universe in a way that has not been possible before. Recently, JWST acquired a deep field image inside the same field imaged in the past as the HST Ultra Deep Field. Computer-based quantitative analysis of spiral galaxies in that field shows that among 34 galaxies for which their rotation of direction can be determined by the shapes of the arms, 24 rotate clockwise, and just 10 rotate counterclockwise. The one-tailed binomial distribution probability to have asymmetry equal or stronger than the observed asymmetry by chance is $\sim$0.012. While the analysis is limited by the small size of the data, the observed asymmetry is aligned with all relevant previous large-scale analyses from all premier digital sky surveys, all show a higher number of galaxies rotating clockwise in that part of the sky, and the magnitude of the asymmetry increases as the redshift gets higher. This paper also provides data and analysis to reproduce previous experiments suggesting that the distribution of galaxy rotation in the Universe is random, to show that the exact same data used in these studies in fact show non-random distribution, and in excellent agreement with the results shown here. These findings reinforce consideration of the possibility that the directions of rotation of spiral galaxies as observed from Earth are not necessarily randomly distributed. The explanation can be related to the large-scale structure of the Universe, but can also be related to a possible anomaly in the physics of galaxy rotation.

astro-ph.CO↗

arXiv2023

Large-scale asymmetry in the distribution of galaxy spin directions -- analysis and reproduction

Recent independent observations using several different telescope systems an analysis methods have provided evidence of parity violation between the number of galaxies that spin in opposite directions. On the other hand, other studies argued that no parity violation can be identified. This paper provides detailed analysis, statistical inference, and reproduction of previous reports that show no preferred spin direction. Code and data used for the reproduction are publicly available. The results show that the data used in all of these studies agrees with the observation of a preferred direction as observed from Earth. In some of these studies the datasets were too small, or the statistical analysis was incomplete. In other papers the results were impacted by experimental design decisions that lead directly to show non-preferred direction. In some of these cases these decisions are not stated in the papers, but were revealed after further investigation in cases where the reproduction of the work did not match the results reported in the papers. These results show that the data used in all of these previous studies in fact agree with the contention that galaxies as observed from Earth have a preferred spin direction, and the distribution of galaxy spin directions as observed from Earth form a cosmological-scale dipole axis. This study also shows that the reason for the observations is not necessarily an anomaly in the large-scale structure, and can also be related to internal structure of galaxies.

astro-ph.CO↗

arXiv2023

A simple direct empirical observation of systematic bias of the redshift as a distance indicator

Recent puzzling observations such as the $H_o$ tension, large-scale anisotropies, and massive disk galaxies at high redshifts have been challenging the standard cosmological model. While one possible explanation is that the standard model is incomplete, other theories are based on the contention that the redshift model as a distance indicator might be biased. While these theories can explain the recent observations, they are challenged by the absence of a direct empirical reproducible observation that the redshift model can indeed be inconsistent. Here I describe a simple experiment that shows that the spectra of galaxies depend on their rotational velocity relative to the rotational velocity of the Milky Way. Moreover, it shows that the redshift of galaxies that rotate in the same direction relative to the Milky Way is significantly different from the redshift of galaxies that rotate in the opposite direction relative to the Milky Way (P$<0.006$). Three different datasets are used independently, each one was prepared in a different manner, all of them show similar redshift bias. A fourth dataset of galaxies from the Southern Galactic pole was also analyzed, and shows similar results. All four datasets are publicly available. While a maximum average $\Delta z$ of $\sim$0.012 observed with galaxies of relatively low redshift (z$<$0.25) might not seem dramatic, the bias is consistent, and can explain puzzling observations such as the $H_o$ tension.

astro-ph.CO↗

arXiv2023

Outlier galaxy images in the Dark Energy Survey and their identification with unsupervised machine learning

The Dark Energy Survey is able to collect image data of an extremely large number of extragalactic objects, and it can be reasonably assumed that many unusual objects of high scientific interest are hidden inside these data. Due to the extreme size of DES data, identifying these objects among many millions of other celestial objects is a challenging task. The problem of outlier detection is further magnified by the presence of noisy or saturated images. When the number of tested objects is extremely high, even a small rate of noise or false positives leads to a very large number of false detections, making an automatic system impractical. This study applies an automatic method for automatic detection of outlier objects in the first data release of the Dark Energy Survey. By using machine learning-based outlier detection, the algorithm is able to identify objects that are visually different from the majority of the other objects in the database. An important feature of the algorithm is that it allows to control the false-positive rate, and therefore can be used for practical outlier detection. The algorithm does not provide perfect accuracy in the detection of outlier objects, but it reduces the data substantially to allow practical outlier detection. For instance, the selection of the top 250 objects after applying the algorithm to more than $2\cdot10^6$ DES images provides a collection of uncommon galaxies. Such collection would have been extremely time-consuming to compile by using manual inspection of the data.

astro-ph.GA↗