arXiv subjects

Ramayya Krishnan

Publications and source records attributed to Ramayya Krishnan.

25 records · Page 2Linked to original sources

arXiv2025

Firm or Fickle? Evaluating Large Language Models Consistency in Sequential Interactions

Large Language Models (LLMs) have shown remarkable capabilities across various tasks, but their deployment in high-stake domains requires consistent and coherent behavior across multiple rounds of user interaction. This paper introduces a comprehensive framework for evaluating and improving LLM response consistency, making three key contributions. Code and data are available at: https://github.com/yubol-bobo/MT-Consistency. First, we introduce Position-Weighted Consistency (PWC), a metric designed to capture both the importance of early-stage stability and recovery patterns in multi-turn interactions. Second, we present MT-Consistency, a carefully curated benchmark dataset spanning diverse domains and difficulty levels, specifically designed to evaluate LLM consistency under various challenging follow-up scenarios. Third, we introduce Confidence-Aware Response Generation (CARG), a framework that significantly improves response stability by explicitly integrating internal model confidence scores during the generation process. Experimental results demonstrate that CARG significantly improves response stability without sacrificing accuracy, offering a practical path toward more dependable LLM behavior in critical, real-world deployments.

cs.CL↗

arXiv2024

Reliability, Resilience and Human Factors Engineering for Trustworthy AI Systems

As AI systems become integral to critical operations across industries and services, ensuring their reliability and safety is essential. We offer a framework that integrates established reliability and resilience engineering principles into AI systems. By applying traditional metrics such as failure rate and Mean Time Between Failures (MTBF) along with resilience engineering and human reliability analysis, we propose an integrate framework to manage AI system performance, and prevent or efficiently recover from failures. Our work adapts classical engineering methods to AI systems and outlines a research agenda for future technical studies. We apply our framework to a real-world AI system, using system status data from platforms such as openAI, to demonstrate its practical applicability. This framework aligns with emerging global standards and regulatory frameworks, providing a methodology to enhance the trustworthiness of AI systems. Our aim is to guide policy, regulation, and the development of reliable, safe, and adaptable AI technologies capable of consistent performance in real-world environments.

cs.AI↗

arXiv2023

Artificial Intelligence/Operations Research Workshop 2 Report Out

This workshop Report Out focuses on the foundational elements of trustworthy AI and OR technology, and how to ensure all AI and OR systems implement these elements in their system designs. Four sessions on various topics within Trustworthy AI were held, these being Fairness, Explainable AI/Causality, Robustness/Privacy, and Human Alignment and Human-Computer Interaction. Following discussions of each of these topics, workshop participants also brainstormed challenge problems which require the collaboration of AI and OR researchers and will result in the integration of basic techniques from both fields to eventually benefit societal needs.

cs.AI↗

arXiv2023

Changes in Commuter Behavior from COVID-19 Lockdowns in the Atlanta Metropolitan Area

This paper analyzes the impact of COVID-19 related lockdowns in the Atlanta, Georgia metropolitan area by examining commuter patterns in three periods: prior to, during, and after the pandemic lockdown. A cellular phone location dataset is utilized in a novel pipeline to infer the home and work locations of thousands of users from the Density-based Spatial Clustering of Applications with Noise (DBSCAN) algorithm. The coordinates derived from the clustering are put through a reverse geocoding process from which word embeddings are extracted in order to categorize the industry of each work place based on the workplace name and Point of Interest (POI) mapping. Frequencies of commute from home locations to work locations are analyzed in and across all three time periods. Public health and economic factors are discussed to explain potential reasons for the observed changes in commuter patterns.

cs.LG↗

arXiv2016

Strong Attractors in Stochastic Adaptive Networks: Emergence and Characterization

We propose a family of models to study the evolution of ties in a network of interacting agents by reinforcement and penalization of their connections according to certain local laws of interaction. The family of stochastic dynamical systems, on the edges of a graph, exhibits \emph{good} convergence properties, in particular, we prove a strong-stability result: a subset of binary matrices or graphs -- characterized by certain compatibility properties -- is a global almost sure attractor of the family of stochastic dynamical systems. To illustrate finer properties of the corresponding strong attractor, we present some simulation results that capture, e.g., the conspicuous phenomenon of emergence and downfall of leaders in social networks.

physics.soc-ph↗

arXiv2012

Contrasting Multiple Social Network Autocorrelations for Binary Outcomes, With Applications To Technology Adoption

The rise of socially targeted marketing suggests that decisions made by consumers can be predicted not only from their personal tastes and characteristics, but also from the decisions of people who are close to them in their networks. One obstacle to consider is that there may be several different measures for "closeness" that are appropriate, either through different types of friendships, or different functions of distance on one kind of friendship, where only a subset of these networks may actually be relevant. Another is that these decisions are often binary and more difficult to model with conventional approaches, both conceptually and computationally. To address these issues, we present a hierarchical model for individual binary outcomes that uses and extends the machinery of the auto-probit method for binary data. We demonstrate the behavior of the parameters estimated by the multiple network-regime auto-probit model (m-NAP) under various sensitivity conditions, such as the impact of the prior distribution and the nature of the structure of the network, and demonstrate on several examples of correlated binary data in networks of interest to Information Systems, including the adoption of Caller Ring-Back Tones, whose use is governed by direct connection but explained by additional network topologies.

cs.SI↗

arXiv2001

Bigger May Not Be Better: An Empirical Analysis of Optimal Membership Rules in Peer-To-Peer Networks

Peer to peer networks will become an increasingly important distribution channel for consumer information goods and may play a role in the distribution of information within corporations. Our research analyzes optimal membership rules for these networks in light of positive and negative externalities additional users impose on the network. Using a dataset gathered from the six largest OpenNap-based networks, we find that users impose a positive network externality based on the desirability of the content they provide and a negative network externality based on demands they place on the network. Further we find that the marginal value of additional users is declining and the marginal cost is increasing in the number of current users. This suggests that multiple small networks may serve user communities more efficiently than single monolithic networks and that network operators may wish to specialize in their content and restrict membership based on capacity constraints and user content desirability.

cs.CY↗