arXiv subjects

Rahul Pandey

Publications and source records attributed to Rahul Pandey.

16 recordsLinked to original sources

arXiv2026

Can We Read the Mind of an Audio LLM? A Verbalizable, Multilingual Middle-Layer Workspace

An audio language model is a black box in a specific way: we see what it says, never what it works out on the way there, and chain-of-thought monitoring helps only if the model writes its reasoning down. Reading a base Qwen3-Omni with a logit lens at the audio-token positions, we find that the answer to a spoken question becomes legible - in words - in the model's middle layers, before it emits any token. Five findings follow. (1) The readout carries concepts in neither the question, the options, nor the model's own transcription: on a clip whose verbatim transcription is empty garbling, it reconstructs Watergate and scandal, passes through the role president, and resolves to Nixon - a hidden multi-hop chain, read with no chain-of-thought. (2) The content is language-agnostic: one audio-inferred concept surfaces in several scripts at once, and 38% of top-1 readouts are Chinese on English inputs. (3) It is paralinguistic: given the same clip as audio and as the model's own emotion-free caption, the audio mind forms the sound source, speaker role, or affect that the caption discards, and answers correctly more often. (4) The audio-driven signal is absent at the input, turns on about a tenth of the way into the network, separates most cleanly from the text prior in the middle band (35-80% of depth), and activation patching shows it is causally used and committed before the last fifth of the layers. (5) Deleting single layers maps the pipeline: reading the sound in is localized to the entry layers and answer delivery to the output layer, while retrieval is distributed across the interior. Throughout, a waveform-swap control - identical text, only the sound changed - isolates the audio-driven signal from a prior over the printed options. This is a qualitative account of what an audio model works out before it speaks: the quantities are controls, not benchmark scores.

cs.SD↗

arXiv2026

Graph Query Generation with Constraint-guided Large Language Agents

Knowledge Graph Question Answering (KGQA) has advanced through structured query generation, yet most efforts target RDF/SPARQL, leaving Cypher and property graphs underexplored, despite increasing demand for unified KGQA in industry settings. We propose UniQGen, a novel constraint-based framework that employs LLM agents to dynamically extract and refine representative graph query clauses into executable, intent-aligned graph queries across query languages. The foundation of our method is a variant of Chase & Backchase, a family of algorithms for query optimization and reformulation. We extend Chase & Backchase with a dynamic reasoning process over query constraints that also interact with LLMs for query quality estimation. With a Cypher-supported Freebase graph deployed on Amazon Neptune, we extensively evaluate our approach on popular KGQA benchmarks (GraphQ, GrailQA, and WebQSP). We demonstrate that UniQGen outperforms state-of-the-art graph query generation techniques in both accuracy and efficiency, with F1 gains of 31.6% on GraphQ and 4.9% on GrailQA. Unlike prior methods, our framework does not require fine-tuning for schema matching, making it more extensible to schema-less graphs and semantics in query workloads, and is more suitable for enterprise-grade KGQA. We release Cypher outputs and a Neptune-ready Freebase snapshot to support reproducible, cross-language KGQA research.

cs.DB↗

arXiv2025

Incentivizing Consistent, Effective and Scalable Reasoning Capability in Audio LLMs via Reasoning Process Rewards

The role of reasoning in Audio Large Language Models remains widely underexplored, as introducing a reasoning process often degrades rather than improves performance during inference, a phenomenon we term test-time inverse scaling, where longer reasoning chains yield progressively worse results. We demonstrate that this stems not from fundamental limitations of reasoning itself, but from inadequate training: models without proper guidance for the reasoning process produce hallucinatory, inconsistent reasoning that accumulates errors over longer chains. To address these challenges, we introduce CESAR (Consistent, Effective, and Scalable Audio Reasoners), shifting from outcome verification to rewarding the reasoning process. Our online reinforcement learning framework employs Group Relative Policy Optimization with a multi-faceted reward suite that incentivizes not only correctness and format but also consistency, structured analytical patterns, causal reasoning, domain-knowledge integration, and calibrated reasoning depth. CESAR resolves test-time inverse scaling, transforming reasoning from detriments into gains while revealing model-specific ``reasoning sweet spots", where performance peaks during test-time scaling. We achieve state-of-the-art results on MMAU Test-mini, substantially outperforming Gemini 2.5 Pro and GPT-4o Audio, and near-human-level performance on MMSU reasoning tasks. Through AI-as-judge evaluations and qualitative comparisons, we provide both quantitative and qualitative validation of our improved reasoning quality. Importantly, enhanced reasoning creates synergistic effects, simultaneously improving multimodal reasoning and perception capabilities. Overall, CESAR establishes a principled method for developing robust and scalable reasoning in Audio LLMs.

cs.LG↗

arXiv2024

ORIS: Online Active Learning Using Reinforcement Learning-based Inclusive Sampling for Robust Streaming Analytics System

Effective labeled data collection plays a critical role in developing and fine-tuning robust streaming analytics systems. However, continuously labeling documents to filter relevant information poses significant challenges like limited labeling budget or lack of high-quality labels. There is a need for efficient human-in-the-loop machine learning (HITL-ML) design to improve streaming analytics systems. One particular HITL- ML approach is online active learning, which involves iteratively selecting a small set of the most informative documents for labeling to enhance the ML model performance. The performance of such algorithms can get affected due to human errors in labeling. To address these challenges, we propose ORIS, a method to perform Online active learning using Reinforcement learning-based Inclusive Sampling of documents for labeling. ORIS aims to create a novel Deep Q-Network-based strategy to sample incoming documents that minimize human errors in labeling and enhance the ML model performance. We evaluate the ORIS method on emotion recognition tasks, and it outperforms traditional baselines in terms of both human labeling performance and the ML model performance.

cs.LG↗

arXiv2023

PROCTER: PROnunciation-aware ConTextual adaptER for personalized speech recognition in neural transducers

End-to-End (E2E) automatic speech recognition (ASR) systems used in voice assistants often have difficulties recognizing infrequent words personalized to the user, such as names and places. Rare words often have non-trivial pronunciations, and in such cases, human knowledge in the form of a pronunciation lexicon can be useful. We propose a PROnunCiation-aware conTextual adaptER (PROCTER) that dynamically injects lexicon knowledge into an RNN-T model by adding a phonemic embedding along with a textual embedding. The experimental results show that the proposed PROCTER architecture outperforms the baseline RNN-T model by improving the word error rate (WER) by 44% and 57% when measured on personalized entities and personalized rare entities, respectively, while increasing the model size (number of trainable parameters) by only 1%. Furthermore, when evaluated in a zero-shot setting to recognize personalized device names, we observe 7% WER improvement with PROCTER, as compared to only 1% WER improvement with text-only contextual attention

eess.AS↗

arXiv2022

Combined DFT, SCAPS-1D, and wxAMPS frameworks for design optimization of efficient Cs2BiAgI6-based perovskite solar cells with different charge transport layers

In this study, combined DFT, SCAPS-1D, and wxAMPS frameworks are used to investigate the optimized designs of Cs2BiAgI6 double perovskite-based solar cells. The first-principle calculation is employed to investigate the structural stability, optical responses, and electronic contribution of the constituent elements in Cs2BiAgI6 absorber material, where SCAPS-1D and wxAMPS simulators are used to scrutinize different configurations of Cs2BiAgI6 solar cells. Here, PCBM, ZnO, TiO2, C60, IGZO, SnO2, WS2, and CeO2 are used as ETL, and Cu2O, CuSCN, CuSbS2, NiO, P3HT, PEDOT: PSS, Spiro-MeOTAD, CuI, CuO, V2O5, CBTS, CFTS are used as HTL, and Au is used as a back contact. About ninety-six combinations of Cs2BiAgI6-based solar cell structures are investigated, in which eight sets of solar cell structures are identified as the most efficient structures. Besides, holistic investigation on the effect of different factors such as the thickness of different layers, series and shunt resistances, temperature, capacitance, Mott-Schottky and generation-recombination rates, and J-V (current-voltage density) and QE (quantum efficiency) characteristics is performed. The results show CBTS as the best HTL for Cs2BiAgI6 with all eight ETLs used in this work, resulting in a power conversion efficiency (PCE) of 19.99%, 21.55%, 21.59%, 17.47%, 20.42%, 21.52%, 14.44%, 21.43% with PCBM, TiO2, ZnO, C60, IGZO, SnO2, CeO2, WS2, respectively. The proposed strategy may pave the way for further design optimization of lead-free double perovskite solar cells.

cond-mat.mtrl-sci↗

arXiv2020

Contracting Boundary of a Cusped Space

Let $G$ be a finitely generated group. Cashen and Mackay proved that if the contracting boundary of $G$ with the topology of fellow travelling quasi-geodesics is compact then $G$ is a hyperbolic group. Let $\mathcal{H}$ be a finite collection of finitely generated infinite index subgroups of $G$. Let $G^h$ be the cusped space obtained by attaching combinatorial horoballs to each left cosets of elements of $\mathcal {H}$. In this article, we prove that if the combinatorial horoballs are contracting and $G^h$ has compact contracting boundary then $G$ is hyperbolic relative to $\mathcal{H}$.

math.GR↗

arXiv2020

Modeling and mitigating human annotation errors to design efficient stream processing systems with human-in-the-loop machine learning

High-quality human annotations are necessary for creating effective machine learning-driven stream processing systems. We study hybrid stream processing systems based on a Human-In-The-Loop Machine Learning (HITL-ML) paradigm, in which one or many human annotators and an automatic classifier (trained at least partially by the human annotators) label an incoming stream of instances. This is typical of many near-real-time social media analytics and web applications, including annotating social media posts during emergencies by digital volunteer groups. From a practical perspective, low-quality human annotations result in wrong labels for retraining automated classifiers and indirectly contribute to the creation of inaccurate classifiers. Considering human annotation as a psychological process allows us to address these limitations. We show that human annotation quality is dependent on the ordering of instances shown to annotators and can be improved by local changes in the instance sequence/order provided to the annotators, yielding a more accurate annotation of the stream. We adapt a theoretically-motivated human error framework of mistakes and slips for the human annotation task to study the effect of ordering instances (i.e., an "annotation schedule"). Further, we propose an error-avoidance approach to the active learning paradigm for stream processing applications robust to these likely human errors (in the form of slips) when deciding a human annotation schedule. We support the human error framework using crowdsourcing experiments and evaluate the proposed algorithm against standard baselines for active learning via extensive experimentation on classification tasks of filtering relevant social media posts during natural disasters.

cs.HC↗

arXiv2019

Classifying Relevant Social Media Posts During Disasters Using Ensemble of Domain-agnostic and Domain-specific Word Embeddings

The use of social media as a means of communication has significantly increased over recent years. There is a plethora of information flow over the different topics of discussion, which is widespread across different domains. The ease of information sharing has increased noisy data being induced along with the relevant data stream. Finding such relevant data is important, especially when we are dealing with a time-critical domain like disasters. It is also more important to filter the relevant data in a real-time setting to timely process and leverage the information for decision support. However, the short text and sometimes ungrammatical nature of social media data challenge the extraction of contextual information cues, which could help differentiate relevant vs. non-relevant information. This paper presents a novel method to classify relevant social media posts during disaster events by ensembling the features of both domain-specific word embeddings as well as more generic domain-agnostic word embeddings. Therefore, we develop and evaluate a hybrid feature engineering framework for integrating diverse semantic representations using a combination of word embeddings to efficiently classify a relevant social media post. The application of the proposed classification framework could help in filtering public posts at large scale, given the growing usage of social media posts in recent years.

cs.SI↗

arXiv2019

Relevancy Classification of Multimodal Social Media Streams for Emergency Services

Social media has become an integral part of our daily lives. During time-critical events, the public shares a variety of posts on social media including reports for resource needs, damages, and help offerings for the affected community. Such posts can be relevant and may contain valuable situational awareness information. However, the information overload of social media challenges the timely processing and extraction of relevant information by the emergency services. Furthermore, the growing usage of multimedia content in the social media posts in recent years further adds to the challenge in timely mining relevant information from social media. In this paper, we present a novel method for multimodal relevancy classification of social media posts, where relevancy is defined with respect to the information needs of emergency management agencies. Specifically, we experiment with the combination of semantic textual features with the image features to efficiently classify a relevant multimodal social media post. We validate our method using an evaluation of classifying the data from three real-world crisis events. Our experiments demonstrate that features based on the proposed hybrid framework of exploiting both textual and image content improve the performance of identifying relevant posts. In the light of these experiments, the application of the proposed classification method could reduce cognitive load on emergency services, in filtering multimodal public posts at large scale.

cs.SI↗

arXiv2019

Modeling Human Annotation Errors to Design Bias-Aware Systems for Social Stream Processing

High-quality human annotations are necessary to create effective machine learning systems for social media. Low-quality human annotations indirectly contribute to the creation of inaccurate or biased learning systems. We show that human annotation quality is dependent on the ordering of instances shown to annotators (referred as 'annotation schedule'), and can be improved by local changes in the instance ordering provided to the annotators, yielding a more accurate annotation of the data stream for efficient real-time social media analytics. We propose an error-mitigating active learning algorithm that is robust with respect to some cases of human errors when deciding an annotation schedule. We validate the human error model and evaluate the proposed algorithm against strong baselines by experimenting on classification tasks of relevant social media posts during crises. According to these experiments, considering the order in which data instances are presented to human annotators leads to both an increase in accuracy for machine learning and awareness toward some potential biases in human learning that may affect the automated classifier.

cs.SI↗

arXiv2019

Acylindrical Hyperbolicity of Subgroups

Suppose $G$ is a finitely generated group and $H$ is a subgroup of $G$. Let $\partial_{c}^{\mathcal{F}\mathcal{Q}}G$ denote the contracting boundary of $G$ with the topology of fellow travelling quasi-geodesics defined by Cashen-Mackay \cite{cashen2017}. In this article, we show that if the limit set $\Lambda(H)$ of $H$ in $\partial_{c}^{\mathcal{F}\mathcal{Q}}G$ is compact and contains at least three points then the action of the subgroup $H$ on the space of distinct triples $\Theta_{3}(\Lambda(H))$ is properly discontinuous. By applying a result of B. Sun \cite{BinSun}, if the limit set $\Lambda(H)$ is compact and the action of $H$ on $\partial_{c}^{\mathcal{F}\mathcal{Q}}G$ is non-elementary then $H$ becomes an acylindrically hyperbolic group

math.GT↗

arXiv2018

Distributional Semantics Approach to Detect Intent in Twitter Conversations on Sexual Assaults

The recent surge in women reporting sexual assault and harassment (e.g., #metoo campaign) has highlighted a longstanding societal crisis. This injustice is partly due to a culture of discrediting women who report such crimes and also, rape myths (e.g., 'women lie about rape'). Social web can facilitate the further proliferation of deceptive beliefs and culture of rape myths through intentional messaging by malicious actors. This multidisciplinary study investigates Twitter posts related to sexual assaults and rape myths for characterizing the types of malicious intent, which leads to the beliefs on discrediting women and rape myths. Specifically, we first propose a novel malicious intent typology for social media using the guidance of social construction theory from policy literature that includes Accusational, Validational, or Sensational intent categories. We then present and evaluate a malicious intent classification model for a Twitter post using semantic features of the intent senses learned with the help of convolutional neural networks. Lastly, we analyze a Twitter dataset of four months using the intent classification model to study narrative contexts in which malicious intents are expressed and discuss their implications for gender violence policy design.

cs.SI↗

arXiv2018

AI for Trustworthiness! Credible User Identification on Social Web for Disaster Response Agencies

Although social media provides a vibrant platform to discuss real-world events, the quantity of information generated can overwhelm decision making based on that information. By better understanding who is participating in information sharing, we can more effectively filter information as the event unfolds. Fine-grained understanding of credible sources can even help develop a trusted network of users for specific events or situations. Given the culture of relying on trusted actors for work practices in the humanitarian and disaster response domain, we propose to identify potential credible users as organizational and organizational-affiliated user accounts on social media in realtime for effective information collection and dissemination. Therefore, we examine social media using AI and Machine Learning methods during three types of humanitarian or disaster events and identify key actors responding to social media conversations as organization (business, group, or institution), organization-affiliated (individual with an organizational affiliation), and non-affiliated (individual without organizational affiliation) identities. We propose a credible user classification approach using a diverse set of social, activity, and descriptive representation features extracted from user profile metadata. Our extensive experiments showed a contrasting participation behavior of the user identities by their content practices, such as the use of higher authoritative content sharing by organization and organization-affiliated users. This study provides a direction for designing realtime credible content analytics systems for humanitarian and disaster response agencies.

cs.SI↗

arXiv2018

Ranking of Social Media Alerts with Workload Bounds in Emergency Operation Centers

Extensive research on social media usage during emergencies has shown its value to provide life-saving information, if a mechanism is in place to filter and prioritize messages. Existing ranking systems can provide a baseline for selecting which updates or alerts to push to emergency responders. However, prior research has not investigated in depth how many and how often should these updates be generated, considering a given bound on the workload for a user due to the limited budget of attention in this stressful work environment. This paper presents a novel problem and a model to quantify the relationship between the performance metrics of ranking systems (e.g., recall, NDCG) and the bounds on the user workload. We then synthesize an alert-based ranking system that enforces these bounds to avoid overwhelming end-users. We propose a Pareto optimal algorithm for ranking selection that adaptively determines the preference of top-k ranking and user workload over time. We demonstrate the applicability of this approach for Emergency Operation Centers (EOCs) by performing an evaluation based on real world data from six crisis events. We analyze the trade-off between recall and workload recommendation across periodic and realtime settings. Our experiments demonstrate that the proposed ranking selection approach can improve the efficiency of monitoring social media requests while optimizing the need for user attention.

cs.SI↗

arXiv2015

Inferring Networks of Substitutable and Complementary Products

In a modern recommender system, it is important to understand how products relate to each other. For example, while a user is looking for mobile phones, it might make sense to recommend other phones, but once they buy a phone, we might instead want to recommend batteries, cases, or chargers. These two types of recommendations are referred to as substitutes and complements: substitutes are products that can be purchased instead of each other, while complements are products that can be purchased in addition to each other. Here we develop a method to infer networks of substitutable and complementary products. We formulate this as a supervised link prediction task, where we learn the semantics of substitutes and complements from data associated with products. The primary source of data we use is the text of product reviews, though our method also makes use of features such as ratings, specifications, prices, and brands. Methodologically, we build topic models that are trained to automatically discover topics from text that are successful at predicting and explaining such relationships. Experimentally, we evaluate our system on the Amazon product catalog, a large dataset consisting of 9 million products, 237 million links, and 144 million reviews.

cs.SI↗