arXiv subjects

Jash Shah

Publications and source records attributed to Jash Shah.

6 recordsLinked to original sources

arXiv2026

Beyond Naturalness: Probing Automated Text-To-Speech Evaluators on Linguistically Grounded Dimensions

Automated Text-to-Speech (TTS) evaluation methods (Mean Opinion Score (MOS) predictors and Audio Large Language Models (Audio-LLM) judges) are expected to reflect human perception, yet it is unclear how well they capture the distinct aspects of speech that listeners actually perceive. We deconstruct "naturalness" into a linguistically grounded annotation schema spanning 10 distinct perceptual dimensions, and use it to construct the first dimension-level meta-evaluation benchmark for TTS, comprising 860 utterances annotated by trained linguist raters. Results from benchmarking four MOS predictors and four Audio-LLM judges reveal that MOS predictors collapse onto acoustic signal quality, while Audio-LLM judges show selective, prompt-dependent detection that does not generalise across all dimensions. Neither class reliably captures a breadth of linguistically structured speech errors. Our dataset, annotation schema, and evaluation code are publicly released to support more targeted and interpretable TTS evaluation.

cs.SD↗

arXiv2026

Students' Perceptions of Peer Grading

Peer grading is widely used in education, yet it elicits mixed reactions from educators and students. Although many studies have examined students' views of peer grading, their findings are scattered, and no clear overall picture has emerged. To address this gap, we conducted a mixed-source thematic analysis of literature and student discussions on Reddit. To scale our analysis of the Reddit data, we fine-tuned a Gemini 2.5 text-classification model to classify an initial dataset of 659 posts and 6,607 comments by relevance. Manual review of the items classified as relevant by the model yielded a final dataset of 114 posts and 300 comments. Drawing on evidence from 107 papers and the Reddit dataset, we found that students perceive peer grading as both beneficial and problematic. Positive perceptions included learning and understanding benefits, skill development, engagement, and collaboration, while negative perceptions centered on unreliable grading, unfairness, weak feedback quality, emotional stress, and workload. Reddit discussions also suggested an emerging concern that remains underexplored in the literature: AI use in peer grading may weaken students' trust in the accuracy and authenticity of the process. We further identified eight mitigation strategies and mapped them to the negative perceptions they help address. Among these, instructor oversight and training played the most central role.

cs.CY↗

arXiv2025

AU-Harness: An Open-Source Toolkit for Holistic Evaluation of Audio LLMs

Large Audio Language Models (LALMs) are rapidly advancing, but evaluating them remains challenging due to inefficient and non-standardized toolkits that limit fair comparison and systematic assessment. Existing evaluation frameworks exhibit three critical limitations: (1) slow and inefficient processing pipeline that bottlenecks large-scale studies, (2) inadequate multi-turn dialogue support, leaving fundamental questions about cross-turn context integration and performance dynamics over extended conversations in LALMs unanswered; and (3) the absence of unified and scalable evaluation framework capable of keeping pace with the rapid growth of both LALMs and audio benchmarks. To address these issues, we introduce AU-Harness, an efficient and comprehensive evaluation framework for LALMs. Our system achieves a speedup of up to 151% over existing evaluation toolkits through optimized batch processing and parallel execution, enabling large-scale evaluations previously considered impractical. We provide standardized prompting protocols and flexible configurations for fair model comparison across diverse scenarios. AU-Harness unlocks a range of in-depth analyses difficult to conduct without a unified foundation, including multi-turn dialogue dynamics, enabling the study of true audio reasoning capabilities in existing LALMs. AU-Harness provides both practical evaluation tools and insights into model limitations, advancing systematic LALM development.

cs.SD↗

arXiv2025

Scaling Success: A Systematic Review of Peer Grading Strategies for Accuracy, Efficiency, and Learning in Contemporary Education

Peer grading has emerged as a scalable solution for assessment in large and online classrooms, offering both logistical efficiency and pedagogical value. However, designing effective peer-grading systems remains challenging due to persistent concerns around accuracy, fairness, reliability, and student engagement. This paper presents a systematic review of 122 peer-reviewed studies on peer grading spanning over four decades. Drawing from this literature, we propose a comprehensive taxonomy that organizes peer grading systems along two key dimensions: (1) evaluation approaches and (2) reviewer weighting strategies. We analyze how different design choices impact grading accuracy, fairness, student workload, and learning outcomes. Our findings highlight the strengths and limitations of each method. Notably, we found that formative feedback -- often regarded as the most valuable aspect of peer assessment -- is seldom incorporated as a quality-based weighting factor in summative grade synthesis techniques. Furthermore, no single reviewer weighting strategy proves universally optimal; each has its trade-offs. Hybrid strategies that combine multiple techniques could show the greatest promise. Our taxonomy offers a practical framework for educators and researchers aiming to design peer grading systems that are accurate, equitable, and pedagogically meaningful.

cs.CY↗

arXiv2025

From Papers to Panoramas: Building Hierarchies of Scientific Literature at Scale

Scientific knowledge is growing rapidly, making it difficult to track progress and high-level conceptual links across broad disciplines. While tools like citation networks and search engines help retrieve related papers, they lack the abstraction needed to capture the density and structure of activity across subfields. We motivate the goal of organizing broad swaths of scientific literature into a high-quality hierarchical structure that spans multiple levels of abstraction---from broad domains to specific studies. Such a representation can provide insights into which fields are well-explored and which are under-explored. To achieve this goal, we develop a hybrid approach that combines efficient embedding-based clustering with LLM-based prompting, striking a balance between scalability and semantic precision. Compared to LLM-heavy methods like iterative tree construction, our approach achieves superior quality-speed trade-offs. Our hierarchies capture different dimensions of research contributions, reflecting the interdisciplinary and multifaceted nature of modern science. We evaluate its utility by measuring how effectively an LLM-based agent can navigate the hierarchy to locate target papers. Results show that our method improves interpretability and offers an alternative pathway for exploring scientific literature beyond traditional search methods. Code, data and demo are available: https://github.com/JHU-CLSP/science-hierarchography

cs.CL↗

arXiv2022

A Method for Waste Segregation using Convolutional Neural Networks

Segregation of garbage is a primary concern in many nations across the world. Even though we are in the modern era, many people still do not know how to distinguish between organic and recyclable waste. It is because of this that the world is facing a major crisis of waste disposal. In this paper, we try to use deep learning algorithms to help solve this problem of waste classification. The waste is classified into two categories like organic and recyclable. Our proposed model achieves an accuracy of 94.9%. Although the other two models also show promising results, the Proposed Model stands out with the greatest accuracy. With the help of deep learning, one of the greatest obstacles to efficient waste management can finally be removed.

cs.CV↗