arXiv subjects

Elad Yom-Tov

Publications and source records attributed to Elad Yom-Tov.

At least 19 recordsLinked to original sources

arXiv2025

Disparate Conditional Prediction in Multiclass Classifiers

We propose methods for auditing multiclass classifiers for fairness under multiclass equalized odds,by estimating the deviation from equalized odds when the classifier is not completely fair. We generalize to multiclass classifiers the measure of Disparate Conditional Prediction (DCP), originally suggested by Sabato & Yom-Tov (2020) for binary classifiers. DCP is defined as the fraction of the population for which the classifier predicts with conditional prediction probabilities that differ from the closest common baseline. We provide new local-optimization methods for estimating the multiclass DCPunder two different regimes,one in which the conditional confusion matrices for each protected sub-population are known, and one in which these cannot be estimated, for instance, because the classifier is inaccessible or because good-quality individual-level data is not available. These methods can be used to detect classifiers that likely treat a significant fraction of the population unfairly. Experiments demonstrate the accuracy of the methods. Code is provided at https://github.com/sivansabato/DCPmulticlass.

cs.LG↗

arXiv2025

Tracking COVID-19 using online search

Previous research has demonstrated that various properties of infectious diseases can be inferred from online search behaviour. In this work we use time series of online search query frequencies to gain insights about the prevalence of COVID-19 in multiple countries. We first develop unsupervised modelling techniques based on associated symptom categories identified by the United Kingdom's National Health Service and Public Health England. We then attempt to minimise an expected bias in these signals caused by public interest --as opposed to infections-- using the proportion of news media coverage devoted to COVID-19 as a proxy indicator. Our analysis indicates that models based on online searches precede the reported confirmed cases and deaths by 16.7 (10.2 - 23.2) and 22.1 (17.4 - 26.9) days, respectively. We also investigate transfer learning techniques for mapping supervised models from countries where the spread of disease has progressed extensively to countries that are in earlier phases of their respective epidemic curves. Furthermore, we compare time series of online search activity against confirmed COVID-19 cases or deaths jointly across multiple countries, uncovering interesting querying patterns, including the finding that rarer symptoms are better predictors than common ones. Finally, we show that web searches improve the short-term forecasting accuracy of autoregressive models for COVID-19 deaths. Our work provides evidence that online search data can be used to develop complementary public health surveillance methods to help inform the COVID-19 response in conjunction with more established approaches.

cs.SI↗

arXiv2024

Generating medical screening questionnaires through analysis of social media data

Screening questionnaires are used in medicine as a diagnostic aid. Creating them is a long and expensive process, which could potentially be improved through analysis of social media posts related to symptoms and behaviors prior to diagnosis. Here we show a preliminary investigation into the feasibility of generating screening questionnaires for a given medical condition from social media postings. The method first identifies a cohort of relevant users through their posts in dedicated patient groups and a control group of users who reported similar symptoms but did not report being diagnosed with the condition of interest. Posts made prior to diagnosis are used to generate decision rules to differentiate between the different groups, by clustering symptoms mentioned by these users and training a decision tree to differentiate between the two groups. We validate the generated rules by correlating them with scores given by medical doctors to matching hypothetical cases. We demonstrate the proposed method by creating questionnaires for three conditions (endometriosis, lupus, and gout) using the data of several hundreds of users from Reddit. These questionnaires were then validated by medical doctors. The average Pearson's correlation between the latter's scores and the decision rules were 0.58 (endometriosis), 0.40 (lupus) and 0.27 (gout). Our results suggest that the process of questionnaire generation can be, at least partly, automated. These questionnaires are advantageous in that they are based on real-world experience but are currently lacking in their ability to capture the context, duration, and timing of symptoms.

cs.LG↗

arXiv2021

COVID-19 Datathon Based on Deidentified Governmental Data as an Approach for Solving Policy Challenges, Increasing Trust, and Building a Community: Case Study

Triggered by the COVID-19 crisis, Israel's Ministry of Health (MoH) held a virtual Datathon based on deidentified governmental data. Organized by a multidisciplinary committee, Israel's research community was invited to offer insights to COVID-19 policy challenges. The Datathon was designed to (1) develop operationalizable data-driven models to address COVID-19 health-policy challenges and (2) build a community of researchers from academia, industry, and government and rebuild their trust in the government. Three specific challenges were defined based on their relevance (significance, data availability, and potential to anonymize the data): immunization policies, special needs of the young population, and populations whose rate of compliance with COVID-19 testing is low. The MoH team extracted diverse, reliable, up-to-date, and deidentified governmental datasets for each challenge. Secure remote-access research environments with relevant data science tools were set on Amazon Web. The MoH screened the applicants and accepted around 80 participants, teaming them to balance areas of expertise as well as represent all sectors of the community. One week following the event, anonymous surveys for participants and mentors were distributed to assess overall usefulness and points for improvement. The 48-hour Datathon and pre-event sessions included 18 multidisciplinary teams, mentored by 20 data scientists, 6 epidemiologists, 5 presentation mentors, and 12 judges. The insights developed by the 3 winning teams are currently considered by the MoH as potential data science methods relevant for national policies. The most positive results were increased trust in the MoH and greater readiness to work with the government on these or future projects. Detailed feedback offered concrete lessons for improving the structure and organization of future government-led datathons.

cs.CY↗

arXiv2021

Initial growth rates of malware epidemics fail to predict their reach

Empirical studies show that epidemiological models based on an epidemic's initial spread rate often fail to predict the true scale of that epidemic. Most epidemics with a rapid early rise die out before affecting a significant fraction of the population, whereas the early pace of some pandemics is rather modest. Recent models suggest that this could be due to the heterogeneity of the target population's susceptibility. We study a computer malware ecosystem exhibiting spread mechanisms resembling those of biological systems while offering details unavailable for human epidemics. Rather than comparing models, we directly estimate reach from a new and vastly more complete data from a parallel domain, that offers superior details and insight as concerns biological outbreaks. We find a highly heterogeneous distribution of computer susceptibilities, with nearly all outbreaks initially over-affecting the tail of the distribution, then collapsing quickly once this tail is depleted. This mechanism restricts the correlation between an epidemic's initial growth rate and its total reach, thus preventing the majority of epidemics, including initially fast-growing outbreaks, from reaching a macroscopic fraction of the population. The few pervasive malwares distinguish themselves early on via the following key trait: they avoid infecting the tail, while preferentially targeting computers unaffected by typical malware.

physics.soc-ph↗

arXiv2020

Providing early indication of regional anomalies in COVID19 case counts in England using search engine queries

COVID19 was first reported in England at the end of January 2020, and by mid-June over 150,000 cases were reported. We assume that, similarly to influenza-like illnesses, people who suffer from COVID19 may query for their symptoms prior to accessing the medical system (or in lieu of it). Therefore, we analyzed searches to Bing from users in England, identifying cases where unexpected rises in relevant symptom searches occurred at specific areas of the country. Our analysis shows that searches for "fever" and "cough" were the most correlated with future case counts, with searches preceding case counts by 16-17 days. Unexpected rises in search patterns were predictive of future case counts multiplying by 2.5 or more within a week, reaching an Area Under Curve (AUC) of 0.64. Similar rises in mortality were predicted with an AUC of approximately 0.61 at a lead time of 3 weeks. Thus, our metric provided Public Health England with an indication which could be used to plan the response to COVID19 and could possibly be utilized to detect regional anomalies of other pathogens.

cs.IR↗

arXiv2020

Algorithmic Copywriting: Automated Generation of Health-Related Advertisements to Improve their Performance

Search advertising, a popular method for online marketing, has been employed to improve health by eliciting positive behavioral change. However, writing effective advertisements requires expertise and experimentation, which may not be available to health authorities wishing to elicit such changes, especially when dealing with public health crises such as epidemic outbreaks. Here we develop a framework, comprised of two neural networks models, that automatically generate ads. First, it employs a generator model, which create ads from web pages. It then employs a translation model, which transcribes ads to improve performance. We trained the networks using 114K health-related ads shown on Microsoft Advertising. We measure ads performance using the click-through rates (CTR). Our experiments show that the generated advertisements received approximately the same CTR as human-authored ads. The marginal contribution of the generator model was, on average, 28\% lower than that of human-authored ads, while the translator model received, on average, 32\% more clicks than human-authored ads. Our analysis shows that the translator model produces ads reflecting higher values of psychological attributes associated with a user action, including higher valance and arousal, and more calls-to-actions. In contrast, levels of these attributes in ads produced by the generator model are similar to those of human-authored ads. Our results demonstrate the ability to automatically generate useful advertisements for the health domain. We believe that our work offers health authorities an improved ability to nudge people towards healthier behaviors while saving the time and cost needed to build effective advertising campaigns.

cs.IR↗

arXiv2020

Privacy, Altruism, and Experience: Estimating the Perceived Value of Internet Data for Medical Uses

People increasingly turn to the Internet when they have a medical condition. The data they create during this process is a valuable source for medical research and for future health services. However, utilizing these data could come at a cost to user privacy. Thus, it is important to balance the perceived value that users assign to these data with the value of the services derived from them. Here we describe experiments where methods from Mechanism Design were used to elicit a truthful valuation from users for their Internet data and for services to screen people for medical conditions. In these experiments, 880 people from around the world were asked to participate in an auction to provide their data for uses differing in their contribution to the participant, to society, and in the disease they addressed. Some users were offered monetary compensation for their participation, while others were asked to pay to participate. Our findings show that 99\% of people were willing to contribute their data in exchange for monetary compensation and an analysis of their data, while 53\% were willing to pay to have their data analyzed. The average perceived value users assigned to their data was estimated at US\$49. Their value to screen them for a specific cancer was US\$22 while the value of this service offered to the general public was US\$22. Participants requested higher compensation when notified that their data would be used to analyze a more severe condition. They were willing to pay more to have their data analyzed when the condition was more severe, when they had higher education or if they had recently experienced a serious medical condition.

cs.GT↗

arXiv2019

Modeling infection methods of computer malware in the presence of vaccinations using epidemiological models: An analysis of real-world data

Computer malware and biological pathogens often use similar mechanisms of infections. For this reason, it has been suggested to model malware spread using epidemiological models developed to characterize the spread of biological pathogens. However, most work examining the similarities between malware and pathogens using such methods was based on theoretical analysis and simulation. Here we extend the classical Susceptible-Infected-Recovered (SIR) epidemiological model to describe two of the most common infection methods used by malware. We fit the proposed model to malware collected over a period of one year from a major anti-malware vendor. We show that by fitting the proposed model it is possible to identify the method of transmission used by the malware, its rate of infection, and the number of machines which will be infected unless blocked by anti-virus software. The Spearman correlation between the number of actual and predicted infected machines is 0.84. Examining cases where an anti-malware "signature" was transmitted to susceptible computers by the anti-virus provider, we show that the time to remove the malware will be short and independent of the number of infected computers if fewer than approximately 60% of susceptible computers have been infected. If more computers were infected, the time to removal will be approximately 3.2 greater, and will depend on the fraction of infected computers. Our results show that the application of epidemiological models of infection to malware can provide anti-virus providers with information on malware spread and its potential damage. We further propose that similarities between computer malware and biological pathogens, the availability of data on the former and the dearth of data on the latter, make malware an extremely useful model for testing interventions which could later be applied to improve medicine.

cs.CR↗

arXiv2019

On the use of multiple compartment epidemiological models to describe the dynamics of influenza in Europe

We develop a multiple compartment Susceptible-Infected-Recovered (SIR) model to analyze the spread of several infectious diseases through different geographic areas. Additionally, we propose a data-quality sensitive optimization framework for fitting this model to observed data. We fit the model to the temporal profile of the number of people infected by one of six influenza strains in Europe over $7$ influenza seasons. In addition to describing the temporal and spatial spread of influenza, the model provides an estimate of the inter-country and intra-country infection and recovery rates of each strain and in each season. We find that disease parameters remain relatively stable, with a correlation greater than $0.5$ over seasons and stains. Clustering of influenza strains by the inferred disease parameters is consistent with genome sub-types. Surprisingly, our analysis suggests that inter-country human mobility plays a negligible role in the spread of influenza in Europe. Finally, we show that the model allows the estimation of disease load in countries with poor or none existent data from the disease load in adjacent countries. Our findings reveal information on the spreading mechanism of influenza and on disease parameters. These can be used to assist in disease surveillance and in control of influenza as well as of other infectious pathogens in a heterogenic environment.

q-bio.PE↗

arXiv2019

Multi-Season Analysis Reveals the Spatial Structure of Disease Spread

Understanding the dynamics of infectious disease spread in a heterogeneous population is an important factor in designing control strategies. Here, we develop a novel tensor-driven multi-compartment version of the classic Susceptible-Infected-Recovered (SIR) model and apply it to Internet data to reveal information about the complex spatial structure of disease spread. The model is used to analyze state-level Google search data from the US pertaining to two viruses, Respiratory Syncytial Virus (RSV), and West Nile Virus (WNV). We fit the data with correlations of $R^2=0.70$, and $0.52$ for RSV and WNV, respectively. Although no prior assumptions on spatial structure are made, human movement patterns in the US explain 27-30\% of the estimated inter-state transmission rates. The transmission rates within states are correlated with known demographic indicators, such as population density and average age. Finally, we show that the patterns of disease load for subsequent seasons can be predicted using the model parameters estimated for previous seasons and as few as $7$ weeks of data from the current season. Our results are applicable to other countries and similar viruses, allowing the identification of disease spread parameters and prediction of disease load for seasonal viruses earlier in season.

q-bio.PE↗

arXiv2018

Screening for cancer using a learning Internet advertising system

Studies have shown that the traces people leave when browsing the internet may indicate the onset of diseases such as cancer. Here we show that the adaptive engines of advertising systems in conjunction with clinically verified questionnaires can be used to identify people who are suspected of having one of three types of solid tumor cancers. In the first study, 308 people were recruited through ads shown on the Bing search engine to complete a clinically verified risk questionnaire. A classifier trained to predict questionnaire response using only past queries on Bing reached an Area Under the Curve of 0.64 for all three cancer types, verifying that past searches could be used to identify people with suspected cancer. The second study was conducted using the Google ads system in the same configuration as in the first study. However, in this study the ads system was set to automatically learn to identify people with suspected cancer. A total of 70,586 people were shown the ads, and 6,484 clicked and were referred to complete the clinical questionnaires. People from countries with higher Internet access and lower life expectancy tended to click more on the ads. Over time the advertisement system learned to identify people who were likely to have symptoms consistent with suspected cancer, such that the percentage of people completing the questionnaires and found to have suspected cancer reached approximately 11\% at the end of the experiment. These results demonstrate the utility of using search engine queries to screen for possible cancer and the application of modern advertising systems to help identify people who are likely suffering from serious medical conditions. This is especially true in countries where medical services are less developed.

cs.CY↗

arXiv2018

Demographic differences in search engine use with implications for cohort selection

The correlation between the demographics of users and the text they write has been investigated through literary texts and, more recently, social media. However, differences pertaining to language use in search engines has not been thoroughly analyzed, especially for age and gender differences. Such differences are important especially due to the growing use of search engine data in the study of human health, where queries are used to identify patient populations. Using data from multiple general-purpose Internet search engines gathered over a period of one month we investigate the correlation between demography (age, gender, and income) and the text of queries submitted to search engines. Our results show that females and younger people use longer queries. This difference is such that females make approximately 25% more queries with 10 or more words. In the case of queries which identify users as having specific medical conditions we find that females make 50% more queries than expected, and that this results in patient cohorts which are highly skewed in gender and age, compared to known gender balance. Our results indicate that studies where demographic representation is important, such as in the study of health aspect of users or when search engines are evaluated for fairness, care should be taken in the selection of search engine data so as to create a representative dataset.

cs.CY↗

arXiv2018

Detecting Parkinson's Disease from interactions with a search engine: Is expert knowledge sufficient?

Parkinson's disease (PD) is a slowly progressing neurodegenerative disease with early manifestation of motor signs. Recently, there has been a growing interest in developing automatic tools that can assess motor function in PD patients. Here we show that mouse tracking data collected during people's interaction with a search engine can be used to distinguish PD patients from similar, non-diseased users and present a methodology developed for the diagnosis of PD from these data. A main challenge we address is the extraction of informative features from raw mouse tracking data. We do so in two complementary ways: First, we manually construct expert-recommended informative features, aiming to identify abnormalities in motor behaviors. Second, we use an unsupervised representation learning technique to map these raw data to high-level features. Using all the extracted features, a Random Forest classifier is then used to distinguish PD patients from controls, achieving an AUC of 0.92, while results using only expert-generated or auto-generated features are 0.87 and 0.83, respectively. Our results indicate that mouse tracking data can help in detecting users at early stages of the disease, and that both expert-generated features and unsupervised techniques for feature generation are required to achieve the best possible performance

cs.IR↗

arXiv2018

Characterizing Efficient Referrals in Social Networks

Users of social networks often focus on specific areas of that network, leading to the well-known "filter bubble" effect. Connecting people to a new area of the network in a way that will cause them to become active in that area could help alleviate this effect and improve social welfare. Here we present preliminary analysis of network referrals, that is, attempts by users to connect peers to other areas of the network. We classify these referrals by their efficiency, i.e., the likelihood that a referral will result in a user becoming active in the new area of the network. We show that by using features describing past experience of the referring author and the content of their messages we are able to predict whether referral will be effective, reaching an AUC of 0.87 for those users most experienced in writing efficient referrals. Our results represent a first step towards algorithmically constructing efficient referrals with the goal of mitigating the "filter bubble" effect pervasive in on line social networks.

cs.SI↗

arXiv2018

Discriminative Learning of Prediction Intervals

In this work we consider the task of constructing prediction intervals in an inductive batch setting. We present a discriminative learning framework which optimizes the expected error rate under a budget constraint on the interval sizes. Most current methods for constructing prediction intervals offer guarantees for a single new test point. Applying these methods to multiple test points can result in a high computational overhead and degraded statistical guarantees. By focusing on expected errors, our method allows for variability in the per-example conditional error rates. As we demonstrate both analytically and empirically, this flexibility can increase the overall accuracy, or alternatively, reduce the average interval size. While the problem we consider is of a regressive flavor, the loss we use is combinatorial. This allows us to provide PAC-style, finite-sample guarantees. Computationally, we show that our original objective is NP-hard, and suggest a tractable convex surrogate. We conclude with a series of experimental evaluations.

cs.LG↗

arXiv2018

Microsoft Malware Classification Challenge

The Microsoft Malware Classification Challenge was announced in 2015 along with a publication of a huge dataset of nearly 0.5 terabytes, consisting of disassembly and bytecode of more than 20K malware samples. Apart from serving in the Kaggle competition, the dataset has become a standard benchmark for research on modeling malware behaviour. To date, the dataset has been cited in more than 50 research papers. Here we provide a high-level comparison of the publications citing the dataset. The comparison simplifies finding potential research directions in this field and future performance evaluation of the dataset.

cs.CR↗

arXiv2017

Evidence from web-based dietary search patterns to the role of B12 deficiency in chronic pain

Profound vitamin B12 deficiency is a known cause of disease, but the role of low or intermediate levels of B12 in the development of neuropathy and other neuropsychiatric symptoms as well as the relationship of eating meat and B12 levels is unclear. Here we use food-related internet search patterns from a sample of 8.5 million US-based people as a proxy to B12 intake and correlate these searches with internet searches related to possible effects of B12 deficiency. Food-related search patterns are highly correlated with known consumption and food-related searches (Spearman 0.69). Awareness of B12 deficiency was associated with a higher consumption of B12-rich foods and with queries for B12 supplements. Searches for terms related to neurological disorders were correlated with searches for B12-poor foods, in contrast with control terms. Popular medicines, those having fewer indications, and those which are predominantly used to treat pain are more strongly correlated with the ability to predict neuropathic pain queries using the B12 contents of food. Our findings provide evidence for the utility of using Internet search patterns to investigate health questions in large populations and suggest that low B12 intake may be associated with a broader spectrum of neurological disorders than currently appreciated.

cs.IR↗