Samuel Hildebrand | Searcharxiv

arXiv2026

Scalpel3: A High-Performance Data Carving Architecture for Recovery of Fragmented Files

File carving recovers files from raw storage without filesystem metadata, a key capability in digital forensics, data recovery, and digital exploration. Existing tools recover contiguous files effectively, but, to our knowledge, no publicly available, format-agnostic, high-performance framework exists in which researchers can develop and deploy new fragmented recovery strategies. Scalpel3 fills this gap with a massively threaded architecture for contiguous and fragmented recovery. Researchers need only write single-threaded validation and reassembly code for a new file type; Scalpel3 supplies worker scheduling, synchronization, checkpointing, and I/O. This separation allows new recovery methods to be added without modifying the backend infrastructure. The architecture also integrates the ONNX Runtime, allowing learned models to be used within validators and recovery strategies. Operational features include interactive human-in-the-loop control, block deduplication, persistent restart checkpoints, incremental output, and a FUSE filesystem for hybrid workflows. We evaluate Scalpel3 on a mixed corpus of more than 80,000 files under contiguous recovery and three controlled fragmentation scenarios: gaps, out-of-order block placement, and both together. Results show fast and accurate contiguous recovery and demonstrate that Scalpel3's massively threaded architecture makes validated fragmented results available substantially earlier than single-threaded execution. Furthermore, strategies tailored to individual file types maintain high overall accuracy across increasingly difficult layouts. Together, these results demonstrate that Scalpel3 provides a practical foundation for developing and deploying fragmented recovery strategies at scale.

cs.DC↗

arXiv2025

FATHOMS-RAG: A Framework for the Assessment of Thinking and Observation in Multimodal Systems that use Retrieval Augmented Generation

Retrieval-augmented generation (RAG) has emerged as a promising paradigm for improving factual accuracy in large language models (LLMs). We introduce a benchmark designed to evaluate RAG pipelines as a whole, evaluating a pipeline's ability to ingest, retrieve, and reason about several modalities of information, differentiating it from existing benchmarks that focus on particular aspects such as retrieval. We present (1) a small, human-created dataset of 93 questions designed to evaluate a pipeline's ability to ingest textual data, tables, images, and data spread across these modalities in one or more documents; (2) a phrase-level recall metric for correctness; (3) a nearest-neighbor embedding classifier to identify potential pipeline hallucinations; (4) a comparative evaluation of 2 pipelines built with open-source retrieval mechanisms and 4 closed-source foundation models; and (5) a third-party human evaluation of the alignment of our correctness and hallucination metrics. We find that closed-source pipelines significantly outperform open-source pipelines in both correctness and hallucination metrics, with wider performance gaps in questions relying on multimodal and cross-document information. Human evaluation of our metrics showed average agreement of 4.62 for correctness and 4.53 for hallucination detection on a 1-5 Likert scale (5 indicating "strongly agree").

cs.AI↗