arXiv subjects

John R. Smith

Publications and source records attributed to John R. Smith.

11 recordsLinked to original sources

arXiv2023

Accelerating Material Design with the Generative Toolkit for Scientific Discovery

With the growing availability of data within various scientific domains, generative models hold enormous potential to accelerate scientific discovery. They harness powerful representations learned from datasets to speed up the formulation of novel hypotheses with the potential to impact material discovery broadly. We present the Generative Toolkit for Scientific Discovery (GT4SD). This extensible open-source library enables scientists, developers, and researchers to train and use state-of-the-art generative models to accelerate scientific discovery focused on material design.

cs.LG↗

arXiv2020

A Broader Study of Cross-Domain Few-Shot Learning

Recent progress on few-shot learning largely relies on annotated data for meta-learning: base classes sampled from the same domain as the novel classes. However, in many applications, collecting data for meta-learning is infeasible or impossible. This leads to the cross-domain few-shot learning problem, where there is a large shift between base and novel class domains. While investigations of the cross-domain few-shot scenario exist, these works are limited to natural images that still contain a high degree of visual similarity. No work yet exists that examines few-shot learning across different imaging methods seen in real world scenarios, such as aerial and medical imaging. In this paper, we propose the Broader Study of Cross-Domain Few-Shot Learning (BSCD-FSL) benchmark, consisting of image data from a diverse assortment of image acquisition methods. This includes natural images, such as crop disease images, but additionally those that present with an increasing dissimilarity to natural images, such as satellite images, dermatology images, and radiology images. Extensive experiments on the proposed benchmark are performed to evaluate state-of-art meta-learning approaches, transfer learning approaches, and newer methods for cross-domain few-shot learning. The results demonstrate that state-of-art meta-learning methods are surprisingly outperformed by earlier meta-learning approaches, and all meta-learning methods underperform in relation to simple fine-tuning by 12.8% average accuracy. Performance gains previously observed with methods specialized for cross-domain few-shot learning vanish in this more challenging benchmark. Finally, accuracy of all methods tend to correlate with dataset similarity to natural images, verifying the value of the benchmark to better represent the diversity of data seen in practice and guiding future research.

cs.CV↗

arXiv2020

Covering the News with (AI) Style

We introduce a multi-modal discriminative and generative frame-work capable of assisting humans in producing visual content re-lated to a given theme, starting from a collection of documents(textual, visual, or both). This framework can be used by edit or to generate images for articles, as well as books or music album covers. Motivated by a request from the The New York Times (NYT) seeking help to use AI to create art for their special section on Artificial Intelligence, we demonstrated the application of our system in producing such image.

eess.IV↗

arXiv2019

Diversity in Faces

Face recognition is a long standing challenge in the field of Artificial Intelligence (AI). The goal is to create systems that accurately detect, recognize, verify, and understand human faces. There are significant technical hurdles in making these systems accurate, particularly in unconstrained settings due to confounding factors related to pose, resolution, illumination, occlusion, and viewpoint. However, with recent advances in neural networks, face recognition has achieved unprecedented accuracy, largely built on data-driven deep learning methods. While this is encouraging, a critical aspect that is limiting facial recognition accuracy and fairness is inherent facial diversity. Every face is different. Every face reflects something unique about us. Aspects of our heritage - including race, ethnicity, culture, geography - and our individual identify - age, gender, and other visible manifestations of self-expression, are reflected in our faces. We expect face recognition to work equally accurately for every face. Face recognition needs to be fair. As we rely on data-driven methods to create face recognition technology, we need to ensure necessary balance and coverage in training data. However, there are still scientific questions about how to represent and extract pertinent facial features and quantitatively measure facial diversity. Towards this goal, Diversity in Faces (DiF) provides a data set of one million annotated human face images for advancing the study of facial diversity. The annotations are generated using ten well-established facial coding schemes from the scientific literature. The facial coding schemes provide human-interpretable quantitative measures of facial features. We believe that by making the extracted coding schemes available on a large set of faces, we can accelerate research and development towards creating more fair and accurate facial recognition systems.

cs.CV↗

arXiv2018

ABALONE Photosensors for the IceCube Experiment

The ABALONE Photosensor Technology (U.S. Pat. 9064678) is a modern, scalable technology specifically invented for cost effective mass production, robustness, and high performance. We present the performance of advanced fused silica ABALONE Photosensors, developed specifically for the potential extension of the Ice Cube neutrino experiment, and stress tested for 120 days. The resulting performance makes a significant difference: intrinsic gain in the high 100 million range, total afterpulsing rate of only 0.005 ions per photoelectron, subnanosecond timing resolution, single photon sensitivity, and unique radiopurity and UV sensitivity, thanks to the fused silica components, at no additional cost to the assembly process.

physics.ins-det↗

arXiv2018

Collaborative Human-AI (CHAI): Evidence-Based Interpretable Melanoma Classification in Dermoscopic Images

Automated dermoscopic image analysis has witnessed rapid growth in diagnostic performance. Yet adoption faces resistance, in part, because no evidence is provided to support decisions. In this work, an approach for evidence-based classification is presented. A feature embedding is learned with CNNs, triplet-loss, and global average pooling, and used to classify via kNN search. Evidence is provided as both the discovered neighbors, as well as localized image regions most relevant to measuring distance between query and neighbors. To ensure that results are relevant in terms of both label accuracy and human visual similarity for any skill level, a novel hierarchical triplet logic is implemented to jointly learn an embedding according to disease labels and non-expert similarity. Results are improved over baselines trained on disease labels alone, as well as standard multiclass loss. Quantitative relevance of results, according to non-expert similarity, as well as localized image regions, are also significantly improved.

cs.CV↗

arXiv2017

Automatic Curation of Golf Highlights using Multimodal Excitement Features

The production of sports highlight packages summarizing a game's most exciting moments is an essential task for broadcast media. Yet, it requires labor-intensive video editing. We propose a novel approach for auto-curating sports highlights, and use it to create a real-world system for the editorial aid of golf highlight reels. Our method fuses information from the players' reactions (action recognition such as high-fives and fist pumps), spectators (crowd cheering), and commentator (tone of the voice and word analysis) to determine the most interesting moments of a game. We accurately identify the start and end frames of key shot highlights with additional metadata, such as the player's name and the hole number, allowing personalized content summarization and retrieval. In addition, we introduce new techniques for learning our classifiers with reduced manual training data annotation by exploiting the correlation of different modalities. Our work has been demonstrated at a major golf tournament, successfully extracting highlights from live video streams over four consecutive days.

cs.CV↗

arXiv2016

Deep Learning Ensembles for Melanoma Recognition in Dermoscopy Images

Melanoma is the deadliest form of skin cancer. While curable with early detection, only highly trained specialists are capable of accurately recognizing the disease. As expertise is in limited supply, automated systems capable of identifying disease could save lives, reduce unnecessary biopsies, and reduce costs. Toward this goal, we propose a system that combines recent developments in deep learning with established machine learning approaches, creating ensembles of methods that are capable of segmenting skin lesions, as well as analyzing the detected area and surrounding tissue for melanoma detection. The system is evaluated using the largest publicly available benchmark dataset of dermoscopic images, containing 900 training and 379 testing images. New state-of-the-art performance levels are demonstrated, leading to an improvement in the area under receiver operating characteristic curve of 7.5% (0.843 vs. 0.783), in average precision of 4% (0.649 vs. 0.624), and in specificity measured at the clinically relevant 95% sensitivity operating point 2.9 times higher than the previous state-of-the-art (36.8% specificity compared to 12.5%). Compared to the average of 8 expert dermatologists on a subset of 100 test images, the proposed system produces a higher accuracy (76% vs. 70.5%), and specificity (62% vs. 59%) evaluated at an equivalent sensitivity (82%).

cs.CV↗

arXiv2016

Oracle performance for visual captioning

The task of associating images and videos with a natural language description has attracted a great amount of attention recently. Rapid progress has been made in terms of both developing novel algorithms and releasing new datasets. Indeed, the state-of-the-art results on some of the standard datasets have been pushed into the regime where it has become more and more difficult to make significant improvements. Instead of proposing new models, this work investigates the possibility of empirically establishing performance upper bounds on various visual captioning datasets without extra data labelling effort or human evaluation. In particular, it is assumed that visual captioning is decomposed into two steps: from visual inputs to visual concepts, and from visual concepts to natural language descriptions. One would be able to obtain an upper bound when assuming the first step is perfect and only requiring training a conditional language model for the second step. We demonstrate the construction of such bounds on MS-COCO, YouTube2Text and LSMDC (a combination of M-VAD and MPII-MD). Surprisingly, despite of the imperfect process we used for visual concept extraction in the first step and the simplicity of the language model for the second step, we show that current state-of-the-art models fall short when being compared with the learned upper bounds. Furthermore, with such a bound, we quantify several important factors concerning image and video captioning: the number of visual concepts captured by different models, the trade-off between the amount of visual elements captured and their accuracy, and the intrinsic difficulty and blessing of different datasets.

cs.CV↗

arXiv2013

Automatic Methods for Handling Nearly Singular Covariance Structures Using the Cholesky Decomposition of an Indefinite Matrix

Linear models have found widespread use in statistical investigations. For every linear model there exists a matrix representation for which the ReML (Restricted Maximum Likelihood) can be constructed from the elements of the corresponding matrix. This method works in the standard manner when the covariance structure is non-singular. It can also be used in the case where the covariance structure is singular, because the method identifies particular non-stochastic linear combinations of the observations which must be constrained to zero. In order to use this method, the Cholesky decomposition has to be generalized to symmetric and indefinite matrices using complex arithmetic methods. This method is applied to the problem of determining the spatial size (vertex) for the Higgs Boson decay in the Higgs -> 4 lepton channel. A comparison based on the Chi-Square variable from the vertex fit for Higgs signal and t-tbar background is presented and shows that the background can be greatly suppressed using the Chi-Square variable. One of the major advantages of this novel method over the currently adopted technique of b-tagging is that it is not affected by multiple interactions (pile up).

hep-ex↗

arXiv2012

Speckle Patterns and 2-Dimensional Spatial Models

The result of 2-dimensional Gaussian lattice fit to a speckle intensity pattern based on a linear model that includes nearest-neighbor interactions is presented. We also include a Monte Carlo simulation of the same spatial speckle pattern that takes the nearest-neighbor interactions into account. These nearest-neighbor interactions lead to a spatial variance structure on the lattice. The resulting spatial pattern fluctuates in value from point to point in a manner characteristic of a stationary stochastic process. The value at a lattice point in the simulation is interpreted as an inten-sity level and the difference in values in neighboring cells produces a fluctuating intensity pattern on the lattice. Changing the size of the mesh changes the relative size of the speckles. Increasing the mesh size tends to average out the intensity in the direction of the mean of the stationary process.

physics.optics↗