SEARCH · Searcharxiv
Search indexed arXiv papers on artificial intelligence, large language models, computer vision and robotics. Read source abstracts and follow links to arXiv.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
New in the probability theory and eventology theory, the concept of Kopula (eventological copula) is introduced. The theorem on the characterization of the sets of events by Kopula is proved, which serves as the eventological pre-image of the well-known Sclar's theorem on copulas (1959). The Kopulas of doublets and triplets of events are given, as well as of some N-sets of events.
This paper assesses the empirical content of one of the most prevalent assumptions in the economics of networks literature, namely the assumption that decision makers have full knowledge about the networks they interact on. Using network data from 75 villages, we ask 4,554 individuals to assess whether five randomly chosen pairs of households in their village are linked through financial, social, and informational relationships. We find that network knowledge is low and highly localized, declining steeply with the pair's network distance to the respondent. 46% of respondents are not even able to offer a guess about the status of a potential link between a given pair of individuals. Even when willing to offer a guess, respondents can only correctly identify the links 37% of the time. We also find that a one-step increase in the social distance to the pair corresponds to a 10pp increase in the probability of misidentifying the link. We then investigate the theoretical implications of this assumption by showing that the predictions of various models change substantially if agents behave under the more realistic assumption of incomplete knowledge about the network. Taken together, our results suggest that the assumption of full network knowledge (i) may serve as a poor approximation to the real world and (ii) is not innocuous: allowing for incomplete network knowledge may have first-order implications for a range of qualitative and quantitative results in various contexts.
Objective: To evaluate unsupervised clustering methods for identifying individual-level behavioral-clinical phenotypes that relate personal biomarkers and behavioral traits in type 2 diabetes (T2DM) self-monitoring data. Materials and Methods: We used hierarchical clustering (HC) to identify groups of meals with similar nutrition and glycemic impact for 6 individuals with T2DM who collected self-monitoring data. We evaluated clusters on: 1) correspondence to gold standards generated by certified diabetes educators (CDEs) for 3 participants; 2) face validity, rated by CDEs, and 3) impact on CDEs' ability to identify patterns for another 3 participants. Results: Gold standard (GS) included 9 patterns across 3 participants. Of these, all 9 were re-discovered using HC: 4 GS patterns were consistent with patterns identified by HC (over 50% of meals in a cluster followed the pattern); another 5 were included as sub-groups in broader clusers. 50% (9/18) of clusters were rated over 3 on 5-point Likert scale for validity, significance, and being actionable. After reviewing clusters, CDEs identified patterns that were more consistent with data (70% reduction in contradictions between patterns and participants' records). Discussion: Hierarchical clustering of blood glucose and macronutrient consumption appears suitable for discovering behavioral-clinical phenotypes in T2DM. Most clusters corresponded to gold standard and were rated positively by CDEs for face validity. Cluster visualizations helped CDEs identify more robust patterns in nutrition and glycemic impact, creating new possibilities for visual analytic solutions. Conclusion: Machine learning methods can use diabetes self-monitoring data to create personalized behavioral-clinical phenotypes, which may prove useful for delivering personalized medicine.
The recent simulation-based inference (SBI) movement in algebra-based introductory statistics courses (Stat 101) has provided preliminary evidence of improved student conceptual understanding and retention. However, little is known about whether these positive effects are preferentially distributed across types of students entering the course. We consider how two metrics of Stat 101 student preparation (pre-course performance on concept inventory and math ACT score) may or may not be associated with end of course student performance on conceptual inventories. Students across all preparation levels tended to show improvement in Stat 101, but more improvement was observed across all student preparation levels in early versions of a SBI course. Furthermore, students' gains tended to be similar regardless of whether students entered the course with more preparation or less. Recent data on a sample of students using a current version of an SBI course showed similar results, though direct comparison with non-SBI students was not possible. Overall, our analysis provides additional evidence that SBI curricula are effective at improving students' conceptual understanding of statistical ideas post-course regardless student preparation. Further work is needed to better understand nuances of student improvement based on other student demographics, prior coursework, as well as instructor and institutional variables.
Significance tests are probably the most extended form of inference in empirical research, and significance is often interpreted as providing greater informational content than non-significance. In this article we show, however, that rejection of a point null often carries very little information, while failure to reject may be highly informative. This is particularly true in empirical contexts where data sets are large and where there are rarely reasons to put substantial prior probability on a point null. Our results challenge the usual practice of conferring point null rejections a higher level of scientific significance than non-rejections. In consequence, we advocate a visible reporting and discussion of non-significant results in empirical practice.
Games can be a powerful tool for learning about statistical methodology. Effective game design involves a fine balance between caricature and realism, to simultaneously illustrate salient concepts in a controlled setting and serve as a testament to real-world applicability. Striking that balance is particularly challenging in response surface and design domains, where real-world scenarios often play out over long time scales, during which theories are revised, model and inferential techniques are improved, and knowledge is updated. Here I present a game, borrowing liberally from one first played over forty years ago, that attempts to achieve that balance while reinforcing a cascade of topics in modern nonparametric response surfaces, sequential design and optimization. The game embeds a blackbox simulation within a shiny app whose interface is designed to simulate a realistic information-availability setting, while offering a stimulating, competitive environment wherein students can try out new methodology, and ultimately appreciate its power and limitations. Interface, rules, timing with course material, and evaluation are described, along with a "case study" involving a cohort of students at Virginia Tech.
In this paper, we propose a locally optimum detection (LOD) scheme for detecting a weak radioactive source buried in background clutter. We develop a decentralized algorithm, based on alternating direction method of multipliers (ADMM), for implementing the proposed scheme in autonomous sensor networks. Results show that algorithm performance approaches the centralized clairvoyant detection algorithm in the low SNR regime, and exhibits excellent convergence rate and scaling behavior (w.r.t. number of nodes). We also devise a low-overhead, robust ADMM algorithm for Byzantine-resilient detection, and demonstrate its robustness to data falsification attacks.
We demonstrate that large texts, representing human (English, Russian, Ukrainian) and artificial (C++, Java) languages, display quantitative patterns characterized by the Benford-like and Zipf laws. The frequency of a word following the Zipf law is inversely proportional to its rank, whereas the total numbers of a certain word appearing in the text generate the uneven Benford-like distribution of leading numbers. Excluding the most popular words essentially improves the correlation of actual textual data with the Zipfian distribution, whereas the Benford distribution of leading numbers (arising from the overall amount of a certain word) is insensitive to the same elimination procedure. The calculated values of the moduli of slopes of double logarithmical plots for artificial languages (C++, Java) are markedly larger than those for human ones.
Manny Parzen passed away in February 2016, and this article is written partly as a memorial and appreciation. Manny made important contributions to several areas, but the two that influenced me most were his contributions to kernel density estimation and to Reproducing Kernel Hilbert Spaces, the two kernels of the title. Some fond memories of Manny as a PhD advisor begin this memorial, followed by a discussion of Manny's influence on density estimation and RKHS methods. A picture gallery of trips comes next, followed by the technical part of the article. Here our goal is to show how risk models can be built using RKHS penalized likelihood methods where subjects have personal (sample) densities which can be used as {\it attributes} in such models.
Suppose you are told that taking a statin will reduce your risk of a heart attack or stroke by 3% in the next ten years, or that women have better emotional intelligence than men. You may wonder how accurate the 3% is, or how confident we should be about the assertion about women's emotional intelligence, bearing in mind that these conclusions are only based on samples of data? My aim here is to present two statistical approaches to questions like these. Approach 1 is often called null hypothesis testing but I prefer the phrase "baseline hypothesis": this is the standard approach in many areas of inquiry but is fraught with problems. Approach 2 can be viewed as a generalisation of the idea of confidence intervals, or as the application of Bayes' theorem. Unlike Approach 1, Approach 2 provides a tentative estimate of the probability of hypotheses of interest. For both approaches, I explain, from first principles, building only on "common sense" statistical concepts like averages and randomness, both how to derive answers, and the rationale behind the answers. This is achieved by using computer simulation methods (resampling and bootstrapping using a spreadsheet available on the web) which avoid the use of probability distributions (t, normal, etc). Such a minimalist, but reasonably rigorous, analysis is particularly useful in a discipline like statistics which is widely used by people who are not specialists. My intended audience includes both statisticians, and users of statistical methods who are not statistical experts.
A verbal autopsy (VA) consists of a survey with a relative or close contact of a person who has recently died. VA surveys are commonly used to infer likely causes of death for individuals when deaths happen outside of hospitals or healthcare facilities. Several statistical and algorithmic methods are available to assign cause of death using VA surveys. Each of these methods require as inputs some information about the joint distribution of symptoms and causes. In this note, we examine the generalizability of this symptom-cause information by comparing different automated coding methods using various combinations of inputs and evaluation data. VA algorithm performance is affected by both the specific SCI themselves and the logic of a given algorithm. Using a variety of performance metrics for all existing VA algorithms, we demonstrate that in general the adequacy of the information about the joint distribution between symptoms and cause affects performance at least as much or more than algorithm logic.
We consider the specification of effects of numerical actor attributes in statistical models for directed social networks. A fundamental mechanism is homophily or assortativity, where actors have a higher likelihood to be tied with others having similar values of the variable under study. But there are other mechanisms that may also play a role in how the attribute values of two actors influence the likelihood of a tie. We discuss three additional mechanisms: aspiration to send ties to others having high values; conformity in the sense of sending more ties to others whose values are close to what may be considered the `social norm'; and sociability, where those having higher values will tend to send more ties generally. These mechanisms may operate jointly, and then their effects will be confounded. We present a specification representing these effects simultaneously by a four-parameter quadratic function of the values of sender and receiver. Greater flexibility can be obtained by a five-parameter extension. We argue that empirical researchers often overlook the possibility that homophily may be confounded with these other mechanisms, and that for actor attributes that have important effects on directed networks, these specifications may provide an improvement. An illustration is given of the dependence of advice ties on academic grades in a network of MBA students, analyzed by the Stochastic Actor-oriented Model.
For more than a century, fingerprints have been used with considerable success to identify criminals or verify the identity of individuals. The categorical conclusion scheme used by fingerprint examiners, and more generally the inference process followed by forensic scientists, have been heavily criticised in the scientific and legal literature. Instead, scholars have proposed to characterise the weight of forensic evidence using the Bayes factor as the key element of the inference process. In forensic science, quantifying the magnitude of support is equally as important as determining which model is supported. Unfortunately, the complexity of fingerprint patterns render likelihood-based inference impossible. In this paper, we use an Approximate Bayesian Computation model selection algorithm to quantify the weight of fingerprint evidence. We supplement the ABC algorithm using a Receiver Operating Characteristic curve to mitigate the effect of the curse of dimensionality. Our modified algorithm is computationally efficient and makes it easier to monitor convergence as the number of simulations increase. We use our method to quantify the weight of fingerprint evidence in forensic science, but we note that it can be applied to any other forensic pattern evidence.
We propose a model for recalculating the target score in rain affected matches based on empirical data. During the development of the current stage of the Cricket, different methods have been introduced to recalculate the target scores in interpreted games. Currently, the International Cricket Council (ICC) uses the Duckworth-Lewis method and have in the past strongly considered changing to the VJD method. Here, we introduce a simple approach to calculate target scores in interrupted games by considering the area under a run rate curve. To calculate the target we have analysed over a decades worth of empirical data using various statistical methods. As in the case of Duckworth- Lewis method, we also have two parameters in our model, that is overs and wickets in combination. We also found that in the one day international cricket (ODI) wickets play a crucial role whereas in T20 cricket they do not effect the run rate of the games to the same degree. Using empirical and mathematical arguments we show that the run scoring distributions are independent of the innings.
We present and evaluate the Fast (conditional) Independence Test (FIT) -- a nonparametric conditional independence test. The test is based on the idea that when $P(X \mid Y, Z) = P(X \mid Y)$, $Z$ is not useful as a feature to predict $X$, as long as $Y$ is also a regressor. On the contrary, if $P(X \mid Y, Z) \neq P(X \mid Y)$, $Z$ might improve prediction results. FIT applies to thousand-dimensional random variables with a hundred thousand samples in a fraction of the time required by alternative methods. We provide an extensive evaluation that compares FIT to six extant nonparametric independence tests. The evaluation shows that FIT has low probability of making both Type I and Type II errors compared to other tests, especially as the number of available samples grows. Our implementation of FIT is publicly available.
Sequential, multiple assignment, randomized trial (SMART) designs have become increasingly popular in the field of precision medicine by providing a means for comparing sequences of treatments tailored to the individual patient, i.e., dynamic treatment regime (DTR). The construction of evidence-based DTRs promises a replacement to adhoc one-size-fits-all decisions pervasive in patient care. However, there are substantial statistical challenges in sizing SMART designs due to the complex correlation structure between the DTRs embedded in the design. Since the primary goal of SMARTs is the construction of an optimal DTR, investigators are interested in sizing SMARTs based on the ability to screen out DTRs inferior to the optimal DTR by a given amount which cannot be done using existing methods. In this paper, we fill this gap by developing a rigorous power analysis framework that leverages multiple comparisons with the best methodology. Our method employs Monte Carlo simulation in order to compute the minimum number of individuals to enroll in an arbitrary SMART. We will evaluate our method through extensive simulation studies. We will illustrate our method by retrospectively computing the power in the Extending Treatment Effectiveness of Naltrexone SMART study.
Patients who are seriously ill may ask doctors to treat them with unapproved medication, about which not much is known, or else with known medication in a high dosage. Apart from strict legal constraints such cases may involve difficult ethical questions as e.g. how long a series of treatments of different patients should be continued. Similar questions also arise in less serious situations. A physician trusts that a certain combination of freely available drugs are efficient against a specific disease and tries to help patients and to follow at the same time the primum-non-nocere principle. The objective of this paper is to contribute to the research on such questions in the form of mathematical models. Arguing in a step-to-step approach, we will show that certain sequential optimisation problems comply in a natural way with the true spirit of major ethical principles in medicine. We then suggest protocols and associate algorithms to find optimal, or approximately optimal, treatment strategies. Although the contribution may sometimes be difficult to apply in medical practice, the author thinks that the rational behind the approach offers a valuable alternative for finding decision support and should attract attention.
We introduce a framework for updating large scale geospatial processes using a model-data synthesis method based on Bayesian hierarchical modelling. Two major challenges come from updating large-scale Gaussian process and modelling non-stationarity. To address the first, we adopt the SPDE approach that uses a sparse Gaussian Markov random fields (GMRF) approximation to reduce the computational cost and implement the Bayesian inference by using the INLA method. For non-stationary global processes, we propose two general models that accommodate commonly-seen geospatial problems. Finally, we show an example of updating an estimate of global glacial isostatic adjustment (GIA) using GPS measurements.