arXiv subjects

Arnab Nandi

Publications and source records attributed to Arnab Nandi.

15 recordsLinked to original sources

arXiv2026

Vega-Video: Integrating Video into the Grammar of Graphics

Video data is increasingly used alongside conventional data for interactive data exploration, necessitating interfaces for exploring and presenting mixed-modality data. However, integrating video into visualizations remains difficult due to its distinct paradigms and inherent performance challenges. We identify three classes of video data visualization - synchronization, annotation, and transformation - and integrate them into the Vega declarative grammar. We show that these abstractions enable high-performance implementation. To reconcile Vega's instantaneous dataflow with video player state, we introduce a split-signal architecture that preserves declarative semantics while masking video update delays. We detect continuous scrubbing interactions at compile time to apply encoding-aware optimizations that improve responsiveness by up to 4x. We also repurpose VOD protocols to transform videos in real time, delivering sub-200ms updates even on multi-hour-long compilations. These contributions enable seamless integration of conventional and video data visualization.

cs.HC↗

arXiv2026

Human-Data Interaction, Exploration, and Visualization in the AI Era: Challenges and Opportunities

The rapid advancement of AI is transforming human-centered systems, with profound implications for human-AI interaction, human-data interaction, and visual analytics. In the AI era, data analysis increasingly involves large-scale, heterogeneous, and multimodal data that is predominantly unstructured, as well as foundation models such as LLMs and VLMs, which introduce additional uncertainty into analytical processes. These shifts expose persistent challenges for human-data interactive systems, including perceptually misaligned latency, scalability constraints, limitations of existing interaction and exploration paradigms, and growing uncertainty regarding the reliability and interpretability of AI-generated insights. Responding to these challenges requires moving beyond conventional efficiency and scalability metrics, redefining the roles of humans and machines in analytical workflows, and incorporating cognitive, perceptual, and design principles into every level of the human-data interaction stack. This paper investigates the challenges introduced by recent advances in AI and examines how these developments are reshaping the ways users engage with data, while outlining limitations and open research directions for building human-centered AI systems for interactive data analysis in the AI era.

cs.DB↗

arXiv2026

Vidformer: Drop-in Declarative Optimization for Rendering Video-Native Query Results

When interactively exploring video data, video-native querying involves consuming query results as videos, including steps such as compilation of extracted video clips or data overlays. These video-native queries are bottlenecked by rendering, not the execution of the underlying queries. This rendering is currently performed using post-processing scripts that are often slow. This step poses a critical point of friction in interactive video data workloads: even short clips contain thousands of high-definition frames; conventional OpenCV/Python scripts must decode -> transform -> encode the entire data stream before a single pixel appears, leaving users waiting for many seconds, minutes, or hours. To address these issues, we present Vidformer, a drop-in rendering accelerator for video-native querying which, (i) transparently lifts existing visualization code into a declarative representation, (ii) transparently optimizes and parallelizes rendering, and (iii) instantly serves videos through a Video on Demand protocol with just-in-time segment rendering. We demonstrate that Vidformer cuts full-render time by 2-3x across diverse annotation workloads, and, more critically, drops time-to-playback to 0.25-0.5s. This represents a 400x improvement that decouples clip length from first-frame playback latency, and unlocks the ability to perform interactive video-native querying with sub-second latencies. Furthermore, we show how our approach enables interactive video-native LLM-based conversational querying as well.

cs.DB↗

arXiv2024

Cross-Modal Entity Matching for Visually Rich Documents

Visually rich documents (e.g. leaflets, banners, magazine articles) are physical or digital documents that utilize visual cues to augment their semantics. Information contained in these documents are ad-hoc and often incomplete. Existing works that enable structured querying on these documents do not take this into account. This makes it difficult to contextualize the information retrieved from querying these documents and gather actionable insights from them. We propose Juno -- a cross-modal entity matching framework to address this limitation. It augments heterogeneous documents with supplementary information by matching a text span in the document with semantically similar tuples from an external database. Our main contribution in this is a deep neural network with attention that goes beyond traditional keyword-based matching and finds matching tuples by aligning text spans and relational tuples on a multimodal encoding space without any prior knowledge about the document type or the underlying schema. Exhaustive experiments on multiple real-world datasets show that Juno generalizes to heterogeneous documents with diverse layouts and formats. It outperforms state-of-the-art baselines by more than 6 F1 points with up to 60% less human-labeled samples. Our experiments further show that Juno is a computationally robust framework. We can train it only once, and then adapt it dynamically for multiple resource-constrained environments without sacrificing its downstream performance. This makes it suitable for on-device deployment in various edge-devices. To the best of our knowledge, ours is the first work that investigates the information incompleteness of visually rich documents and proposes a generalizable, performant and computationally robust framework to address it in an end-to-end way.

cs.LG↗

arXiv2024

Noise-Aware Training of Layout-Aware Language Models

A visually rich document (VRD) utilizes visual features along with linguistic cues to disseminate information. Training a custom extractor that identifies named entities from a document requires a large number of instances of the target document type annotated at textual and visual modalities. This is an expensive bottleneck in enterprise scenarios, where we want to train custom extractors for thousands of different document types in a scalable way. Pre-training an extractor model on unlabeled instances of the target document type, followed by a fine-tuning step on human-labeled instances does not work in these scenarios, as it surpasses the maximum allowable training time allocated for the extractor. We address this scenario by proposing a Noise-Aware Training method or NAT in this paper. Instead of acquiring expensive human-labeled documents, NAT utilizes weakly labeled documents to train an extractor in a scalable way. To avoid degradation in the model's quality due to noisy, weakly labeled samples, NAT estimates the confidence of each training sample and incorporates it as uncertainty measure during training. We train multiple state-of-the-art extractor models using NAT. Experiments on a number of publicly available and in-house datasets show that NAT-trained models are not only robust in performance -- it outperforms a transfer-learning baseline by up to 6% in terms of macro-F1 score, but it is also more label-efficient -- it reduces the amount of human-effort required to obtain comparable performance by up to 73%.

cs.CL↗

arXiv2020

Interpretable Multi-Headed Attention for Abstractive Summarization at Controllable Lengths

Abstractive summarization at controllable lengths is a challenging task in natural language processing. It is even more challenging for domains where limited training data is available or scenarios in which the length of the summary is not known beforehand. At the same time, when it comes to trusting machine-generated summaries, explaining how a summary was constructed in human-understandable terms may be critical. We propose Multi-level Summarizer (MLS), a supervised method to construct abstractive summaries of a text document at controllable lengths. The key enabler of our method is an interpretable multi-headed attention mechanism that computes attention distribution over an input document using an array of timestep independent semantic kernels. Each kernel optimizes a human-interpretable syntactic or semantic property. Exhaustive experiments on two low-resource datasets in the English language show that MLS outperforms strong baselines by up to 14.70% in the METEOR score. Human evaluation of the summaries also suggests that they capture the key concepts of the document at various length-budgets.

cs.CL↗

arXiv2020

Discovery of Driving Patterns by Trajectory Segmentation

Telematics data is becoming increasingly available due to the ubiquity of devices that collect data during drives, for different purposes, such as usage based insurance (UBI), fleet management, navigation of connected vehicles, etc. Consequently, a variety of data-analytic applications have become feasible that extract valuable insights from the data. In this paper, we address the especially challenging problem of discovering behavior-based driving patterns from only externally observable phenomena (e.g. vehicle's speed). We present a trajectory segmentation approach capable of discovering driving patterns as separate segments, based on the behavior of drivers. This segmentation approach includes a novel transformation of trajectories along with a dynamic programming approach for segmentation. We apply the segmentation approach on a real-word, rich dataset of personal car trajectories provided by a major insurance company based in Columbus, Ohio. Analysis and preliminary results show the applicability of approach for finding significant driving patterns.

cs.AI↗

arXiv2019

Short and Long-term Pattern Discovery Over Large-Scale Geo-Spatiotemporal Data

Pattern discovery in geo-spatiotemporal data (such as traffic and weather data) is about finding patterns of collocation, co-occurrence, cascading, or cause and effect between geospatial entities. Using simplistic definitions of spatiotemporal neighborhood (a common characteristic of the existing general-purpose frameworks) is not semantically representative of geo-spatiotemporal data. We therefore introduce a new geo-spatiotemporal pattern discovery framework which defines a semantically correct definition of neighborhood; and then provides two capabilities, one to explore propagation patterns and the other to explore influential patterns. Propagation patterns reveal common cascading forms of geospatial entities in a region. Influential patterns demonstrate the impact of temporally long-term geospatial entities on their neighborhood. We apply this framework on a large dataset of traffic and weather data at countrywide scale, collected for the contiguous United States over two years. Our important findings include the identification of 90 common propagation patterns of traffic and weather entities (e.g., rain --> accident --> congestion), which results in identification of four categories of states within the US; and interesting influential patterns with respect to the "location", "duration", and "type" of long-term entities (e.g., a major construction --> more traffic incidents). These patterns and the categorization of the states provide useful insights on the driving habits and infrastructure characteristics of different regions in the US, and could be of significant value for applications such as urban planning and personalized insurance.

cs.DB↗

arXiv2018

To Ship or Not to (Function) Ship (Extended version)

Sampling is often used to reduce query latency for interactive big data analytics. The established parallel data processing paradigm relies on function shipping, where a coordinator dispatches queries to worker nodes and then collects the results. The commoditization of high-performance networking makes data shipping possible, where the coordinator directly reads data in the workers' memory using RDMA while workers process other queries. In this work, we explore when to use function shipping or data shipping for interactive query processing with sampling. Whether function shipping or data shipping should be preferred depends on the amount of data transferred, the current CPU utilization, the sampling method and the number of queries executed over the data set. The results show that data shipping is up to 6.5x faster when performing clustered sampling with heavily-utilized workers.

cs.DB↗

arXiv2017

Characterizing Driving Context from Driver Behavior

Because of the increasing availability of spatiotemporal data, a variety of data-analytic applications have become possible. Characterizing driving context, where context may be thought of as a combination of location and time, is a new challenging application. An example of such a characterization is finding the correlation between driving behavior and traffic conditions. This contextual information enables analysts to validate observation-based hypotheses about the driving of an individual. In this paper, we present DriveContext, a novel framework to find the characteristics of a context, by extracting significant driving patterns (e.g., a slow-down), and then identifying the set of potential causes behind patterns (e.g., traffic congestion). Our experimental results confirm the feasibility of the framework in identifying meaningful driving patterns, with improvements in comparison with the state-of-the-art. We also demonstrate how the framework derives interesting characteristics for different contexts, through real-world examples.

cs.AI↗

arXiv2017

InfiniViz: Interactive Visual Exploration using Progressive Bin Refinement

Interactive visualizations can accelerate the data analysis loop through near-instantaneous feedback. To achieve interactivity, techniques such as data cubes and sampling are typically employed. While data cubes can speedup querying for moderate-sized datasets, they are ineffective at doing so at a larger scales due to the size of the materialized data cubes. On the other hand, while sampling can help scale to large datasets, it adds sampling error and the associated issues into the process. While increasing accuracy by looking at more data may sometimes be valuable, providing result minutiae might not be necessary if they do not impart additional significant information. Indeed, such details not only incur a higher \emph{computational} cost, but also tax the \emph{cognitive} load of the analyst with worthless trivia. To reduce both the computational and cognitive expenses, we introduce \emph{InfiniViz}. Through a novel result refinement-based querying paradigm, \emph{InfiniViz} provides error-free results for large datasets by increasing bin resolutions progressively over time. Through real and simulated workloads over real and benchmark datasets, we evaluate and demonstrate \emph{InfiniViz}'s utility at reducing both cognitive and computational costs, while minimizing information loss.

cs.DB↗

arXiv2017

Perfect and Maximum Randomness in Stratified Sampling over Joins

Supporting sampling in the presence of joins is an important problem in data analysis, but is inherently challenging due to the need to avoid correlation between output tuples. Current solutions provide either correlated or non-correlated samples. Sampling might not always be feasible in the non-correlated sampling-based approaches -- the sample size or intermediate data size might be exceedingly large. On the other hand, a correlated sample may not be representative of the join. This paper presents a \emph{unified} strategy towards join sampling, while considering sample correlation every step of the way. We provide two key contributions. First, in the case where a \emph{correlated} sample is \emph{acceptable}, we provide techniques, for all join types, to sample base relations so that their join is \emph{as random as possible}. Second, in the case where a correlated sample is \emph{not acceptable}, we provide enhancements to the state-of-the-art algorithms to reduce their execution time and intermediate data size.

cs.DB↗

arXiv2016

A Closer Look at Variance Implementations in Modern Database Systems

Variance is a popular and often necessary component of sampled aggregation queries. It is typically used as a secondary measure to ascertain statistical properties of the result such as its error. Yet, it is more expensive to compute than simple, primary measures such as \texttt{SUM}, \texttt{MEAN}, and \texttt{COUNT}. There exist numerous techniques to compute variance. While the definition of variance is considered to require multiple passes on the data, other mathematical representations can compute the value in a single pass. Some single-pass representations, however, can suffer from severe precision loss, especially for large number of data points. In this paper, we study variance implementations in various real-world systems and find that major database systems such as PostgreSQL 9.4 and most likely System X, a major commercially used closed-source database, use a representation that is efficient, but suffers from floating point precision loss resulting from catastrophic cancellation. We note deficiencies in another popular representation, used by databases such as MySQL and Impala, that suffers from not being distributive and therefore cannot take advantage of modern parallel computational resources. We review literature over the past five decades on variance calculation in both the statistics and database communities, and summarize recommendations on implementing variance functions in various settings, such as approximate query processing and large-scale distributed aggregation.

cs.DB↗

arXiv2016

Graphical Perception in Animated Bar Charts

Interactive visual applications create animations that encode changes in the data. For example, cross-filtering dynamically updates linked visualizations based on the user's continuous brushing actions. The animated effects resulting from these interactions depends both on how interaction (e.g., brushing speed) controls properties of the animation such as frame rate, as well as how the data that is being explored dictates the data encoded in the animation. Past work has found that frame rate matters to general perception, however a critical question is which of these animation and data properties affects the perceptual accuracy of judgement tasks, and to what extent. Although graphical perception has been well studied for static data visualizations, it is ripe for exploration in the animated setting. We designed two animated judgment tasks of a target bar in an animated bar chart and empirically evaluate the effects of 2 animations properties - highlighting of the target bar and frame rate - as well as 3 data properties that affect the target bar's value throughout the animation. In short, we find that the rate and timing of animation changes is easier detected in larger values; that encodings such as color are easier to detect than shapes; and that timing is important - earlier changes were harder to perceive as compared to later changes in the animation. Our results are an initial step to understanding perceptual accuracy for animated data visualizations, both for presentations and ultimately as part of interactive applications.

cs.HC↗

arXiv2009

Qunits: queried units in database search

Keyword search against structured databases has become a popular topic of investigation, since many users find structured queries too hard to express, and enjoy the freedom of a ``Google-like'' query box into which search terms can be entered. Attempts to address this problem face a fundamental dilemma. Database querying is based on the logic of predicate evaluation, with a precisely defined answer set for a given query. On the other hand, in an information retrieval approach, ranked query results have long been accepted as far superior to results based on boolean query evaluation. As a consequence, when keyword queries are attempted against databases, relatively ad-hoc ranking mechanisms are invented (if ranking is used at all), and there is little leverage from the large body of IR literature regarding how to rank query results. Our proposal is to create a clear separation between ranking and database querying. This divides the problem into two parts, and allows us to address these separately. The first task is to represent the database, conceptually, as a collection of independent ``queried units'', or ``qunits'', each of which represents the desired result for some query against the database. The second task is to evaluate keyword queries against a collection of qunits, which can be treated as independent documents for query purposes, thereby permitting the use of standard IR techniques. We provide insights that encourage the use of this query paradigm, and discuss preliminary investigations into the efficacy of a qunits-based framework based on a prototype implementation.

cs.DB↗