arXiv subjects

Lisa Liu

Publications and source records attributed to Lisa Liu.

3 recordsLinked to original sources

arXiv2024

Automated Root Cause Analysis System for Complex Data Products

We present ARCAS (Automated Root Cause Analysis System), a diagnostic platform based on a Domain Specific Language (DSL) built for fast diagnostic implementation and low learning curve. Arcas is composed of a constellation of automated troubleshooting guides (Auto-TSGs) that can execute in parallel to detect issues using product telemetry and apply mitigation in near-real-time. The DSL is tailored specifically to ensure that subject matter experts can deliver highly curated and relevant Auto-TSGs in a short time without having to understand how they will interact with the rest of the diagnostic platform, thus reducing time-to-mitigate and saving crucial engineering cycles when they matter most. This contrasts with platforms like Datadog and New Relic, which primarily focus on monitoring and require manual intervention for mitigation. ARCAS uses a Large Language Model (LLM) to prioritize Auto-TSGs outputs and take appropriate actions, thus suppressing the costly requirement of understanding the general behavior of the system. We explain the key concepts behind ARCAS and demonstrate how it has been successfully used for multiple products across Azure Synapse Analytics and Microsoft Fabric Synapse Data Warehouse.

cs.SE↗

arXiv2022

Automatic vehicle trajectory data reconstruction at scale

In this paper we propose an automatic trajectory data reconciliation to correct common errors in vision-based vehicle trajectory data. Given "raw" vehicle detection and tracking information from automatic video processing algorithms, we propose a pipeline including (a) an online data association algorithm to match fragments that describe the same object (vehicle), which is formulated as a min-cost network circulation problem of a graph, and (b) a one-step trajectory rectification procedure formulated as a quadratic program to enhance raw detection data. The pipeline leverages vehicle dynamics and physical constraints to associate tracked objects when they become fragmented, remove measurement noises and outliers and impute missing data due to fragmentations. We assess the capability of the proposed two-step pipeline to reconstruct three benchmarking datasets: (1) a microsimulation dataset that is artificially downgraded to replicate upstream errors, (2) a 15-min NGSIM data that is manually perturbed, and (3) tracking data consists of 3 scenes from collections of video data recorded from 16-17 cameras on a section of the I-24 MOTION system, and compare with the corresponding manually-labeled ground truth vehicle bounding boxes. All of the experiments show that the reconciled trajectories improve the accuracy on all the tested input data for a wide range of measures. Lastly, we show the design of a software architecture that is currently deployed on the full-scale I-24 MOTION system consisting of 276 cameras that covers 4.2 miles of I-24. We demonstrate the scalability of the proposed reconciliation pipeline to process high-volume data on a daily basis.

cs.DS↗

arXiv2022

Sums of Reciprocals of Recurrence Relations

There is a growing literature on sums of reciprocals of polynomial functions of recurrence relations with constant coefficients and fixed depth, such as Fibonacci and Tribonacci numbers, products of such numbers, and balancing numbers (numbers $n$ such that the sum of the integers less than $n$ equals the sum of the $r$ integers immediately after, for some $r$ which is called the balancer of $n$; If $n$ is included in the summation, we have the cobalancing numbers, and $r$ is called the cobalancer of $n$). We generalize previous work to reciprocal sums of depth two recurrence sequences with arbitrary coefficients and the Tribonacci numbers, and show our method provides an alternative proof of some existing results. We define $(a,b)$ balancing and cobalancing numbers, where $a$ and $b$ are constants that multiply the left-hand side and right-hand side respectively, and derive recurrence relations describing these sequences. We show that for balancing numbers, the coefficients $(3,1)$ is unique such that every integer is a $(3,1)$ balancing number, and proved there does not exist an analogous set of coefficients for cobalancing numbers. We also found patterns for certain coefficients that have no balancing or cobalancing numbers.

math.NT↗