Searcharxiv⌕ Search

arXiv subjects

Ronaldo C. Prati

Publications and source records attributed to Ronaldo C. Prati.

4 recordsLinked to original sources

A Temporal Algebraic Framework for Data Stream Evaluation Under Delayed Labels and Distribution Shifts

In data streams, labels often arrive late or only for some instances, class prevalence drifts, and rare classes vanish from short windows. Standard evaluation then confuses a degraded decision boundary with a change in prevalence, and is biased when labels are delayed or selected by the model. We extend an algebraic framework that obtains pooled (micro) and per-class (macro) measures from one confusion matrix to streams, weighting the counts in time by exponential fading or a sliding window; the update costs O(m^2) per step, whatever the stream length, and each pooled measure stays a weighted combination of the per-class ones. Delayed and selective feedback enter as an observation mask: until its label arrives, an instance is counted with the model's calibrated class probabilities, and the bias of this interim estimate is driven by their calibration error. Audited conventions report no value for a class without verified instances. We separate prior shift from concept drift by monitoring prior-free coordinates (recall and specificity) and attribute a change in a measure to the two sources with a first-order Jacobian decomposition. On synthetic streams and four real datasets, a detector on these coordinates with a memory per class raises no excess false alarms under label shift, unlike the Jacobian alarm and error-rate detectors. The correction for delayed labels recovers recall in some real settings and is only partial for precision, specificity and F1. Twenty-five statements are checked in Lean 4, and the code is in the aggmetrics package.

cs.LG↗

A Unified Algebraic Framework for Classification Performance Evaluation

We propose a unified algebraic framework for classification performance evaluation covering binary, multiclass, multilabel, ordinal, hierarchical, cost-sensitive, and soft-label settings. Actual and predicted labels are represented as binary indicator matrices, where three aggregation operators (global, column-wise, row-wise) correspond directly to micro, macro/weighted, and exemplar averaging. Any binary measure expressed in terms of the four confusion-matrix counts extends to all these settings by substituting an operator, with no measure-specific derivation. We show that structural properties governing an extension are derivable from the binary formula. Micro-averaging equals denominator-weighted macro-averaging precisely for aggregation-decomposable (linear-fractional) measures, a strict class characterised algebraically. For soft ground truth, we prove from t-norm axioms alone that the product t-norm is the unique choice whose confusion counts preserve marginal memberships. In multiclass settings, micro-precision, micro-recall, and micro-F1 collapse identically onto accuracy. Furthermore, binary skew-invariance transfers unconditionally to multilabel aggregation, but only partially to multiclass problems. For measures with a linear numerator and prediction-independent denominator, the optimal decision threshold is the share of the numerator weight favouring a negative prediction, revealing when standard training targets the measure. Under one-hot multiclass encoding, a measure fails to attain its theoretical minimum whenever its zero-true-positive value still depends on true negatives, establishing non-trivial performance floors even for completely incorrect classifiers (e.g., zero correct predictions on 10 classes yields a label accuracy of 0.8).

cs.LG↗

Optimizing IoT Threat Detection with Kolmogorov-Arnold Networks (KANs)

The exponential growth of the Internet of Things (IoT) has led to the emergence of substantial security concerns, with IoT networks becoming the primary target for cyberattacks. This study examines the potential of Kolmogorov-Arnold Networks (KANs) as an alternative to conventional machine learning models for intrusion detection in IoT networks. The study demonstrates that KANs, which employ learnable activation functions, outperform traditional MLPs and achieve competitive accuracy compared to state-of-the-art models such as Random Forest and XGBoost, while offering superior interpretability for intrusion detection in IoT networks.

cs.LG↗

Retrieval-Enhanced Named Entity Recognition

When combined with In-Context Learning, a technique that enables models to adapt to new tasks by incorporating task-specific examples or demonstrations directly within the input prompt, autoregressive language models have achieved good performance in a wide range of tasks and applications. However, this combination has not been properly explored in the context of named entity recognition, where the structure of this task poses unique challenges. We propose RENER (Retrieval-Enhanced Named Entity Recognition), a technique for named entity recognition using autoregressive language models based on In-Context Learning and information retrieval techniques. When presented with an input text, RENER fetches similar examples from a dataset of training examples that are used to enhance a language model to recognize named entities from this input text. RENER is modular and independent of the underlying language model and information retrieval algorithms. Experimental results show that in the CrossNER collection we achieve state-of-the-art performance with the proposed technique and that information retrieval can increase the F-score by up to 11 percentage points.

cs.CL↗