arXiv subjects

Gregory Camilli

Publications and source records attributed to Gregory Camilli.

4 recordsLinked to original sources

arXiv2025

Modeling Issues with Eye Tracking Data

I describe and compare procedures for binary eye-tracking (ET) data. The basic GLM model is a logistic mixed model combined with random effects for persons and items. Additional models address error correlation in eye-tracking serial observations. In particular, three novel approaches are illustrated that address serial without the use of an observed lag-1 predictor: a first-order autoregressive model and a first-order moving average models obtained with generalized estimating equations, and a recurrent two-state survival model used with run-length encoded data. Altogether, the results of five different analyses point to unresolved issues in the analysis of eye-tracking data and new directions for analytic development. A more traditional model incorporating a lag-1 observed outcome for serial correlation is also included.

stat.ME↗

arXiv2025

The "Days of Learning" Metric for Education Evaluations

The third National Charter School Study (NCSS III) aimed to test whether charter school were effective and to highlight outcomes on academic progress. The authors reported that typical charter school students outperformed similar students in non-charter public schools by 6 days in mathematics and 16 days in reading. This "days of learning" metric used to claim relatively higher performance in charter schools than in comparable public schools. This logic of this metric is critiqued in this paper, and an alternative method of reporting outcomes is proposed.

econ.GN↗

arXiv2024

NLP Cluster Analysis of Common Core State Standards and NAEP Item Specifications

Camilli (2024) proposed a methodology using natural language processing (NLP) to map the relationship of a set of content standards to item specifications. This study provided evidence that NLP can be used to improve the mapping process. As part of this investigation, the nominal classifications of standards and items specifications were used to examine construct equivalence. In the current paper, we determine the strength of empirical support for the semantic distinctiveness of these classifications, which are known as "domains" for Common Core standards, and "strands" for National Assessment of Educational Progress (NAEP) item specifications. This is accomplished by separate k-means clustering for standards and specifications of their corresponding embedding vectors. We then briefly illustrate an application of these findings.

cs.CY↗

arXiv2024

An NLP Crosswalk Between the Common Core State Standards and NAEP Item Specifications

Natural language processing (NLP) is rapidly developing for applications in educational assessment. In this paper, I describe an NLP-based procedure that can be used to support subject matter experts in establishing a crosswalk between item specifications and content standards. This paper extends recent work by proposing and demonstrating the use of multivariate similarity based on embedding vectors for sentences or texts. In particular, a hybrid regression procedure is demonstrated for establishing the match of each content standard to multiple item specifications. The procedure is used to evaluate the match of the Common Core State Standards (CCSS) for mathematics at grade 4 to the corresponding item specifications for the 2026 National Assessment of Educational Progress (NAEP).

cs.CL↗