SearcharxivSearch

arXiv subjects

Shivkumar Kalyanaraman

Publications and source records attributed to Shivkumar Kalyanaraman.

At least 19 recordsLinked to original sources

SPIRIT: Short-term Prediction of solar IRradIance for zero-shot Transfer learning using Foundation Models

Traditional solar forecasting models are based on several years of site-specific historical irradiance data, often spanning five or more years, which are unavailable for newer photovoltaic farms. As renewable energy is highly intermittent, building accurate solar irradiance forecasting systems is essential for efficient grid management and enabling the ongoing proliferation of solar energy, which is crucial to achieve the United Nations' net zero goals. In this work, we propose SPIRIT, a novel approach leveraging foundation models for solar irradiance forecasting, making it applicable to newer solar installations. Our approach outperforms state-of-the-art models in zero-shot transfer learning by about 70%, enabling effective performance at new locations without relying on any historical data. Further improvements in performance are achieved through fine-tuning, as more location-specific data becomes available. These findings are supported by statistical significance, further validating our approach. SPIRIT represents a pivotal step towards rapid, scalable, and adaptable solar forecasting solutions, advancing the integration of renewable energy into global power systems.

cs.LG

LABELING COPILOT: A Deep Research Agent for Automated Data Curation in Computer Vision

Curating high-quality, domain-specific datasets is a major bottleneck for deploying robust vision systems, requiring complex trade-offs between data quality, diversity, and cost when researching vast, unlabeled data lakes. We introduce Labeling Copilot, the first data curation deep research agent for computer vision. A central orchestrator agent, powered by a large multimodal language model, uses multi-step reasoning to execute specialized tools across three core capabilities: (1) Calibrated Discovery sources relevant, in-distribution data from large repositories; (2) Controllable Synthesis generates novel data for rare scenarios with robust filtering; and (3) Consensus Annotation produces accurate labels by orchestrating multiple foundation models via a novel consensus mechanism incorporating non-maximum suppression and voting. Our large-scale validation proves the effectiveness of Labeling Copilot's components. The Consensus Annotation module excels at object discovery: on the dense COCO dataset, it averages 14.2 candidate proposals per image-nearly double the 7.4 ground-truth objects-achieving a final annotation mAP of 37.1%. On the web-scale Open Images dataset, it navigated extreme class imbalance to discover 903 new bounding box categories, expanding its capability to over 1500 total. Concurrently, our Calibrated Discovery tool, tested at a 10-million sample scale, features an active learning strategy that is up to 40x more computationally efficient than alternatives with equivalent sample efficiency. These experiments validate that an agentic workflow with optimized, scalable tools provides a robust foundation for curating industrial-scale datasets.

cs.CV

Grammars of Formal Uncertainty: When to Trust LLMs in Automated Reasoning Tasks

Large language models (LLMs) show remarkable promise for democratizing automated reasoning by generating formal specifications. However, a fundamental tension exists: LLMs are probabilistic, while formal verification demands deterministic guarantees. This paper addresses this epistemological gap by comprehensively investigating failure modes and uncertainty quantification (UQ) in LLM-generated formal artifacts. Our systematic evaluation of five frontier LLMs reveals Satisfiability Modulo Theories (SMT) based autoformalization's domain-specific impact on accuracy (from +34.8% on logical tasks to -44.5% on factual ones), with known UQ techniques like the entropy of token probabilities failing to identify these errors. We introduce a probabilistic context-free grammar (PCFG) framework to model LLM outputs, yielding a refined uncertainty taxonomy. We find uncertainty signals are task-dependent (e.g., grammar entropy for logic, AUROC>0.93). Finally, a lightweight fusion of these signals enables selective verification, drastically reducing errors (14-100%) with minimal abstention, transforming LLM-driven formalization into a reliable engineering discipline.

cs.CL

AI Greenferencing: Routing AI Inferencing to Green Modular Data Centers with Heron

AI power demand is growing unprecedentedly thanks to the high power density of AI compute and the emerging inferencing workload. On the supply side, abundant wind power is waiting for grid access in interconnection queues. In this light, this paper argues bringing AI workload to modular compute clusters co-located in wind farms. Our deployment right-sizing strategy makes it economically viable to deploy more than 6 million high-end GPUs today that could consume cheap, green power at its source. We built Heron, a cross-site software router, that could efficiently leverage the complementarity of power generation across wind farms by routing AI inferencing workload around power drops. Using 1-week ofcoding and conversation production traces from Azure and (real) variable wind power traces, we show how Heron improves aggregate goodput of AI compute by up to 80% compared to the state-of-the-art.

cs.DC

EnCortex: A General, Extensible and Scalable Framework for Decision Management in New-age Energy Systems

With increased global warming, there has been a significant emphasis to replace fossil fuel-dependent energy sources with clean, renewable sources. These new-age energy systems are becoming more complex with an increasing proportion of renewable energy sources (like solar and wind), energy storage systems (like batteries), and demand side control in the mix. Most new-age sources being highly dependent on weather and climate conditions bring about high variability and uncertainty. Energy operators rely on such uncertain data to make different planning and operations decisions periodically, and sometimes in real-time, to maintain the grid stability and optimize their objectives (cost savings, carbon footprint, etc.). Hitherto, operators mostly rely on domain knowledge, heuristics, or solve point problems to take decisions. These approaches fall short because of their specific assumptions and limitations. Further, there is a lack of a unified framework for both research and production environments at scale. In this paper, we propose EnCortex to address these challenges. EnCortex provides a general, easy-to-use, extensible, and scalable energy decision framework that enables operators to plan, build and execute their real-world scenarios efficiently. We show that using EnCortex, we can define and compose complex new-age scenarios, owing to industry-standard abstractions of energy entities and the modularity of the framework. EnCortex provides a foundational structure to support several state-of-the-art optimizers with minimal effort. EnCortex supports both quick developments for research prototypes and scaling the solutions to production environments. We demonstrate the utility of EnCortex with three complex new-age real-world scenarios and show that significant cost and carbon footprint savings can be achieved.

eess.SY

An Agentic Approach to Automatic Creation of P&ID Diagrams from Natural Language Descriptions

The Piping and Instrumentation Diagrams (P&IDs) are foundational to the design, construction, and operation of workflows in the engineering and process industries. However, their manual creation is often labor-intensive, error-prone, and lacks robust mechanisms for error detection and correction. While recent advancements in Generative AI, particularly Large Language Models (LLMs) and Vision-Language Models (VLMs), have demonstrated significant potential across various domains, their application in automating generation of engineering workflows remains underexplored. In this work, we introduce a novel copilot for automating the generation of P&IDs from natural language descriptions. Leveraging a multi-step agentic workflow, our copilot provides a structured and iterative approach to diagram creation directly from Natural Language prompts. We demonstrate the feasibility of the generation process by evaluating the soundness and completeness of the workflow, and show improved results compared to vanilla zero-shot and few-shot generation approaches.

cs.LG

Proof of Thought : Neurosymbolic Program Synthesis allows Robust and Interpretable Reasoning

Large Language Models (LLMs) have revolutionized natural language processing, yet they struggle with inconsistent reasoning, particularly in novel domains and complex logical sequences. This research introduces Proof of Thought, a framework that enhances the reliability and transparency of LLM outputs. Our approach bridges LLM-generated ideas with formal logic verification, employing a custom interpreter to convert LLM outputs into First Order Logic constructs for theorem prover scrutiny. Central to our method is an intermediary JSON-based Domain-Specific Language, which by design balances precise logical structures with intuitive human concepts. This hybrid representation enables both rigorous validation and accessible human comprehension of LLM reasoning processes. Key contributions include a robust type system with sort management for enhanced logical integrity, explicit representation of rules for clear distinction between factual and inferential knowledge, and a flexible architecture that allows for easy extension to various domain-specific applications. We demonstrate Proof of Thought's effectiveness through benchmarking on StrategyQA and a novel multimodal reasoning task, showing improved performance in open-ended scenarios. By providing verifiable and interpretable results, our technique addresses critical needs for AI system accountability and sets a foundation for human-in-the-loop oversight in high-stakes domains.

cs.AI

A deep learning approach to solar-irradiance forecasting in sky-videos

Ahead-of-time forecasting of incident solar-irradiance on a panel is indicative of expected energy yield and is essential for efficient grid distribution and planning. Traditionally, these forecasts are based on meteorological physics models whose parameters are tuned by coarse-grained radiometric tiles sensed from geo-satellites. This research presents a novel application of deep neural network approach to observe and estimate short-term weather effects from videos. Specifically, we use time-lapsed videos (sky-videos) obtained from upward facing wide-lensed cameras (sky-cameras) to directly estimate and forecast solar irradiance. We introduce and present results on two large publicly available datasets obtained from weather stations in two regions of North America using relatively inexpensive optical hardware. These datasets contain over a million images that span for 1 and 12 years respectively, the largest such collection to our knowledge. Compared to satellite based approaches, the proposed deep learning approach significantly reduces the normalized mean-absolute-percentage error for both nowcasting, i.e. prediction of the solar irradiance at the instance the frame is captured, as well as forecasting, ahead-of-time irradiance prediction for a duration for upto 4 hours.

cs.CV

DeepSolarEye: Power Loss Prediction and Weakly Supervised Soiling Localization via Fully Convolutional Networks for Solar Panels

The impact of soiling on solar panels is an important and well-studied problem in renewable energy sector. In this paper, we present the first convolutional neural network (CNN) based approach for solar panel soiling and defect analysis. Our approach takes an RGB image of solar panel and environmental factors as inputs to predict power loss, soiling localization, and soiling type. In computer vision, localization is a complex task which typically requires manually labeled training data such as bounding boxes or segmentation masks. Our proposed approach consists of specialized four stages which completely avoids localization ground truth and only needs panel images with power loss labels for training. The region of impact area obtained from the predicted localization masks are classified into soiling types using the webly supervised learning. For improving localization capabilities of CNNs, we introduce a novel bi-directional input-aware fusion (BiDIAF) block that reinforces the input at different levels of CNN to learn input-specific feature maps. Our empirical study shows that BiDIAF improves the power loss prediction accuracy by about 3% and localization accuracy by about 4%. Our end-to-end model yields further improvement of about 24% on localization when learned in a weakly supervised manner. Our approach is generalizable and showed promising results on web crawled solar panel images. Our system has a frame rate of 22 fps (including all steps) on a NVIDIA TitanX GPU. Additionally, we collected first of it's kind dataset for solar panel image analysis consisting 45,000+ images.

cs.CV

On Determining the Fair Bandwidth Share for ABR Connections in ATM Networks

In a multi-service network such as ATM, adaptive data services(such as ABR) share the bandwidth unused by higher priority services. The network indicates to the ABR sources the fair and efficient rates at which they should transmit to minimize their cell loss. In this paper, we propose a new method for determining the "effective" number of active connections, and the fair bandwidth share for each connection.

cs.NI

TCP Selective Acknowledgments and UBR Drop Policies to Improve ATM-UBR Performance over Terrestrial and Satellite Networks

We study the performance of Selective Acknowledgments with TCP over the ATM-UBR service category. We examine various UBR drop policies, TCP mechanisms and network configurations to recommend optimal parameters for TCP over UBR. We discuss various TCP congestion control mechanisms compare their performance for LAN and WAN networks. We describe the effect of satellite delays on TCP performance over UBR and present simulation results for LAN, WAN and satellite networks. SACK TCP improves the performance of TCP over UBR, especially for large delay networks. Intelligent drop policies at the switches are an important factor for good performance in local area networks.

cs.NI

A Survey of Protocols and Open Issues in ATM Multipoint Communication

Asynchronous transfer mode (ATM) networks must define multicast capabilities in order to efficiently support numerous applications, such as video conferencing and distributed applications, in addition to LAN emulation (LANE) and Internet protocol (IP) multicasting. Several problems and issues arise in ATM multicasting, such as signaling, routing, connection admission control, and traffic management problems. IP integrated services over ATM poses further challenges to ATM multicasting. Scalability and simplicity are the two main concerns for ATM multicasting. This paper provides a survey of the current work on multicasting problems in general, and ATM multicasting in particular. A number of proposed schemes is examined, such as the schemes MARS, MCS, SEAM, SMART, RSVP, and various multipoint traffic management and transport-layer schemes. The paper also indicates a number of key open issues that remain unresolved.

cs.NI

Use-it or Lose-it Policies for the Available Bit Rate (ABR) Service in ATM Networks

The Available Bit Rate (ABR) service has been developed to support 21st century data applications over Asynchronous Transfer Mode (ATM). The ABR service uses a closed-loop rate-based traffic management framework where the network divides left-over bandwidth among contending sources. The ATM Forum traffic management group also incorporated open-loop control capabilities to make the ABR service robust to temporary network failures and source inactivity. An important problem addressed was whether rate allocations of sources should be taken away if sources do not use them. The proposed solutions, popularly known as the Use-It-or-Lose-It (UILI) policies, have had significant impact on the ABR service capabilities. In this paper we discuss the design, development, and the final shape of these policies and their impact on the ABR service. We compare the various alternatives through a performance evaluation.

cs.NI

On Source Rules for ABR Service on ATM Networks with Satellite Links

During the design of ABR traffic management at the ATM Forum, we performed several analyses to ensure that the ABR service will operate efficiently over satellite links. In the cases where the performance was unacceptable, we suggested modifications to the traffic management specifications. This paper describes one such issue related to the count of missing resource management cells (Crm) parameter of the ABR source behavior. The analysis presented here led to the changes which are now part of the ATM traffic management (TM 4.0) specification. In particular, the size of the transient buffer exposure (TBE) parameter was set to 24 bits, and no size was enforced for the Crm parameter. This simple change improved the throughput over OC-3 satellite links from 45 Mbps to 140 Mbps.

cs.NI

A Survey of Congestion Control Techniques and Data Link Protocols in Satellite Networks

Satellite communication systems are the means of realizing a global broadband integrated services digital network. Due to the statistical nature of the integrated services traffic, the resulting rate fluctuations and burstiness render congestion control a complicated, yet indispensable function. The long propagation delay of the earth-satellite link further imposes severe demands and constraints on the congestion control schemes, as well as the media access control techniques and retransmission protocols that can be employed in a satellite network. The problems in designing satellite network protocols, as well as some of the solutions proposed to tackle these problems, will be the primary focus of this survey.

cs.NI

Performance of TCP over ABR with Long-Range Dependent VBR Background Traffic over Terrestrial and Satellite ATM networks

Compressed video is well known to be self-similar in nature. We model VBR carrying Long-Range Dependent (LRD), multiplexed MPEG-2 video sources. The actual traffic for the model is generated using fast-fourier transform of generate the fractional gaussian noise (FGN) sequence. Our model of compressed video sources bears similarity to an MPEG-2 Transport Stream carrying video, i.e., it is long-range dependent and generates traffic in a piecewise-CBR fashion. We study the effect of such VBR traffic on ABR carrying TCP traffic. The effect of such VBR traffic is that the ABR capacity is highly variant. We find that a switch algorithm like ERICA+ can tolerate this variance in ABR capacity while maintaining high throughput and low delay. We present simulation results for terrestrial and satellite configurations.

cs.NI

Design Issues for providing Minimum Rate Guarantees to the ATM Unspecified Bit Rate Service

Recent enhancements have been proposed to the ATM Unspecified Bit Rate (UBR) service that guarantee a minimum rate at the frame level to the UBR VCs. These enhancements have been called Guaranteed Frame Rate (GFR). In this paper, we discuss the motivation, design and implementation issues for GFR. We present the design of buffer management and policing mechanisms to implement GFR. We study the effects of policing, per-VC buffer allocation, and per-VC queuing on providing GFR to TCP/IP traffic. We conclude that per-VC scheduling is necessary to provide minimum rate guarantees to TCP traffic. We examine the role of frame tagging in the presence of scheduling and buffer management for providing minumum rate guarantees. The use of GFR to support the Internet Controlled Load Service is also discussed.

cs.NI

The ERICA Switch Algorithm for ABR Traffic Management in ATM Networks

We propose an explicit rate indication scheme for congestion avoidance in ATM networks. In this scheme, the network switches monitor their load on each link, determining a load factor, the available capacity, and the number of currently active virtual channels. This information is used to advise the sources about the rates at which they should transmit. The algorithm is designed to achieve efficiency, fairness, controlled queueing delays, and fast transient response. The algorithm is also robust to measurement errors caused due to variation in ABR demand and capacity. We present performance analysis of the scheme using both analytical arguments and simulation results. The scheme is being implemented by several ATM switch manufacturers.

cs.NI