arXiv subjects

Nithin Surendran

Publications and source records attributed to Nithin Surendran.

3 recordsLinked to original sources

arXiv2026

Operationalising Multi-Dimensional Evaluation for Conversational Agents: A Scalable, Governed Pipeline with Selective Re-evaluation and Model Benchmarking

Evaluating retail conversational agents requires methods beyond lexical-overlap metrics to assess intent alignment, factuality, helpfulness, clarity, tone, and overall response quality. Although LLM-as-a-judge methods provide scalable alternatives to human evaluation, production deployment introduces challenges in governance, reproducibility, cost, schema consistency, traceability, and reliability. We present GenAI Evaluation, a governed, configuration-driven pipeline for large-scale evaluation of retail conversational systems. It processes production chatbot logs through normalization, sharding, asynchronous execution, and schema-constrained LLM scoring. The framework evaluates helpfulness, truthfulness, clarity, tone alignment, and translation-specific dimensions. Selective re-evaluation processes only incomplete, malformed, or schema-invalid records, while schema locking, versioned configurations, validation logs, and record-level provenance support auditability. The framework processes approximately 50,000 records daily and has evaluated more than two million interactions. Validation used 12,980 stratified-random human-labeled records from four trained annotators. Classification covered 14 intents, 156 sub-intents, 18 major domains, and 129 sub-domains. The pipeline achieved a macro F1 score of 0.93 and 89% human-acceptability accuracy for translation.

cs.AI↗

arXiv2026

OPTIMUS: Optimization Productivity Tool for Intelligent Management of Utilizable Space

We study department-level retail space optimization, where limited bay capacity must be allocated among planograms (POGs) under business and operational constraints. The problem is formulated as a linear binary knapsack model, with potential SKUs treated as items characterized by space requirements and weighted value contributions from sales, margin, units, and assortment similarity. Dynamic Programming (DP) is employed to obtain exact and reproducible assortment decisions in O(nc) time, avoiding the variance inherent in heuristic approaches. These decisions are integrated with a second-stage bay optimization model formulated as a mixed-integer program. Evaluated end-to-end across ten optimization runs spanning multiple departments and store clusters, the OPTIMUS framework achieves an average sales lift of 11.8% and an average margin lift of 9.5%. Overall, OPTIMUS provides a scalable, interpretable, and profit-driven solution for enterprise-scale retail space management.

math.OC↗

arXiv2025

Causal inference and model explainability tools for retail

Most major retailers today have multiple divisions focused on various aspects, such as marketing, supply chain, online customer experience, store customer experience, employee productivity, and vendor fulfillment. They also regularly collect data corresponding to all these aspects as dashboards and weekly/monthly/quarterly reports. Although several machine learning and statistical techniques have been in place to analyze and predict key metrics, such models typically lack interpretability. Moreover, such techniques also do not allow the validation or discovery of causal links. In this paper, we aim to provide a recipe for applying model interpretability and causal inference for deriving sales insights. In this paper, we review the existing literature on causal inference and interpretability in the context of problems in e-commerce and retail, and apply them to a real-world dataset. We find that an inherently explainable model has a lower variance of SHAP values, and show that including multiple confounders through a double machine learning approach allows us to get the correct sign of causal effect.

cs.LG↗