arXiv subjects

Anirban Roy

Publications and source records attributed to Anirban Roy.

At least 37 records · Page 2Linked to original sources

arXiv2025

The Amazon Nova Family of Models: Technical Report and Model Card

We present Amazon Nova, a new generation of state-of-the-art foundation models that deliver frontier intelligence and industry-leading price performance. Amazon Nova Pro is a highly-capable multimodal model with the best combination of accuracy, speed, and cost for a wide range of tasks. Amazon Nova Lite is a low-cost multimodal model that is lightning fast for processing images, video, documents and text. Amazon Nova Micro is a text-only model that delivers our lowest-latency responses at very low cost. Amazon Nova Canvas is an image generation model that creates professional grade images with rich customization controls. Amazon Nova Reel is a video generation model offering high-quality outputs, customization, and motion control. Our models were built responsibly and with a commitment to customer trust, security, and reliability. We report benchmarking results for core capabilities, agentic performance, long context, functional adaptation, runtime performance, and human evaluation.

cs.AI↗

arXiv2024

Shrinking POMCP: A Framework for Real-Time UAV Search and Rescue

Efficient path optimization for drones in search and rescue operations faces challenges, including limited visibility, time constraints, and complex information gathering in urban environments. We present a comprehensive approach to optimize UAV-based search and rescue operations in neighborhood areas, utilizing both a 3D AirSim-ROS2 simulator and a 2D simulator. The path planning problem is formulated as a partially observable Markov decision process (POMDP), and we propose a novel ``Shrinking POMCP'' approach to address time constraints. In the AirSim environment, we integrate our approach with a probabilistic world model for belief maintenance and a neurosymbolic navigator for obstacle avoidance. The 2D simulator employs surrogate ROS2 nodes with equivalent functionality. We compare trajectories generated by different approaches in the 2D simulator and evaluate performance across various belief types in the 3D AirSim-ROS simulator. Experimental results from both simulators demonstrate that our proposed shrinking POMCP solution achieves significant improvements in search times compared to alternative methods, showcasing its potential for enhancing the efficiency of UAV-assisted search and rescue operations.

cs.RO↗

arXiv2024

Addressing Uncertainty in LLMs to Enhance Reliability in Generative AI

In this paper, we present a dynamic semantic clustering approach inspired by the Chinese Restaurant Process, aimed at addressing uncertainty in the inference of Large Language Models (LLMs). We quantify uncertainty of an LLM on a given query by calculating entropy of the generated semantic clusters. Further, we propose leveraging the (negative) likelihood of these clusters as the (non)conformity score within Conformal Prediction framework, allowing the model to predict a set of responses instead of a single output, thereby accounting for uncertainty in its predictions. We demonstrate the effectiveness of our uncertainty quantification (UQ) technique on two well known question answering benchmarks, COQA and TriviaQA, utilizing two LLMs, Llama2 and Mistral. Our approach achieves SOTA performance in UQ, as assessed by metrics such as AUROC, AUARC, and AURAC. The proposed conformal predictor is also shown to produce smaller prediction sets while maintaining the same probabilistic guarantee of including the correct response, in comparison to existing SOTA conformal prediction baseline.

cs.AI↗

arXiv2024

Semi-Empirical Approach to [CII] Line Intensity Mapping

The line intensity mapping technique involves measuring the cumulative emission from specific spectral lines emitted by galaxies and intergalactic gas. This method provides a way to study the matter distribution and the evolution of large-scale structures throughout the history of the Universe. However, modeling intensity mapping from ab-initio approaches can be challenging due to significant astrophysical uncertainties and noticeable degeneracies among astrophysical and cosmological parameters. To address these challenges, we develop a semi-empirical, data-driven framework for galaxy evolution, which features a minimal set of assumptions and parameters gauged on observations. By integrating this with stellar evolution and radiative transfer prescriptions for line emissions, we derive the cosmic [CII] intensity over an extended redshift range $0 \lesssim z \lesssim 10$. Our approach is quite general and can be easily applied to other key lines used in intensity mapping studies, such as [OIII] and the CO ladder. We then evaluate the detectability of the [CII] power spectra using current and forthcoming observational facilities. Our findings offer critical insights into the feasibility and potential contributions of intensity mapping for probing the large-scale structure of the Universe and understanding galaxy evolution.

astro-ph.GA↗

arXiv2024

Cross-correlation Techniques to Mitigate the Interloper Contamination for Line Intensity Mapping Experiments

Line intensity mapping (LIM) serves as a potent probe in astrophysics, relying on the statistical analysis of integrated spectral line emissions originating from distant star-forming galaxies. While LIM observations hold the promise of achieving a broad spectrum of scientific objectives, a significant hurdle for future experiments lies in distinguishing the targeted spectral line emitted at a specific redshift from undesired line emissions originating at different redshifts. The presence of these interloping lines poses a challenge to the accuracy of cosmological analyses. In this study, we introduce a novel approach to quantify line-line cross-correlations (LIM-LLX), enabling us to investigate the true signal amidst instrumental noise and interloping emissions. For example, at a redshift of approximately $z\sim3.7$, we observed that the measured auto-power spectrum of [CII] exhibited substantial bias, from interloping line emission. However, cross-correlating [CII] with CO(6-5) lines using a FYST-like experiment yielded a promising result, with a Signal-to-noise ratio (SNR) of $\sim 10$. This measurement is notably unbiased. Additionally, we explore the extensive capabilities of cross-correlation by leveraging various CO transitions to probe the tomographic Universe at lower redshifts through LIM-LLX. We further demonstrate that incorporating low-frequency channels, such as 90 GHz and 150 GHz, into FYST's EoR-Spec-like experiment can maximize the potential for cross-correlation studies, effectively reducing the bias introduced by instrumental noise and interlopers.

astro-ph.CO↗

arXiv2024

A Measurement of CO(3-2) Line Emission from eBOSS Galaxies at $z\sim 0.5$ using Planck Data

Line intensity mapping (LIM) is a novel observational technique in astrophysics that utilizes the integrated emission from multiple atomic and molecular transition lines from galaxies to probe the complex physics of galaxy formation and evolution, as well as the large-scale structure of the universe. Modeling multiple line luminosities of galaxies with varying masses or their host halo masses poses significant uncertainty due to the lack of observational data across a wide redshift range and the intricate nature of astrophysical processes, making them challenging to model analytically or in simulations. While future experiments aim to measure multiple line intensities up to $z\sim 8$ across a wide volume using tomographic methods, we leverage publicly available datasets from the CMB experiment Planck and the galaxy survey eBOSS to constrain the CO(3-2) emission from galaxies. We correlate galaxies from eBOSS data onto the full-sky CO(2-1) map produced by Planck and report the first measurement of the average CO(3-2) intensity, $I_{CO} = 45.7 \pm 14.2\, \mathrm{Jy/sr}$ at $z\sim 0.5$ with $3.2σ$ confidence. Our findings demonstrate that stacking methods are already viable with existing observations from CMB experiments and galaxy surveys, and are complementary to traditional LIM experiments.

astro-ph.GA↗

arXiv2024

Concept-based Analysis of Neural Networks via Vision-Language Models

The analysis of vision-based deep neural networks (DNNs) is highly desirable but it is very challenging due to the difficulty of expressing formal specifications for vision tasks and the lack of efficient verification procedures. In this paper, we propose to leverage emerging multimodal, vision-language, foundation models (VLMs) as a lens through which we can reason about vision models. VLMs have been trained on a large body of images accompanied by their textual description, and are thus implicitly aware of high-level, human-understandable concepts describing the images. We describe a logical specification language $\texttt{Con}_{\texttt{spec}}$ designed to facilitate writing specifications in terms of these concepts. To define and formally check $\texttt{Con}_{\texttt{spec}}$ specifications, we build a map between the internal representations of a given vision model and a VLM, leading to an efficient verification procedure of natural-language properties for vision models. We demonstrate our techniques on a ResNet-based classifier trained on the RIVAL-10 dataset using CLIP as the multimodal model.

cs.LG↗

arXiv2024

BRI3L: A Brightness Illusion Image Dataset for Identification and Localization of Regions of Illusory Perception

Visual illusions play a significant role in understanding visual perception. Current methods in understanding and evaluating visual illusions are mostly deterministic filtering based approach and they evaluate on a handful of visual illusions, and the conclusions therefore, are not generic. To this end, we generate a large-scale dataset of 22,366 images (BRI3L: BRightness Illusion Image dataset for Identification and Localization of illusory perception) of the five types of brightness illusions and benchmark the dataset using data-driven neural network based approaches. The dataset contains label information - (1) whether a particular image is illusory/nonillusory, (2) the segmentation mask of the illusory region of the image. Hence, both the classification and segmentation task can be evaluated using this dataset. We follow the standard psychophysical experiments involving human subjects to validate the dataset. To the best of our knowledge, this is the first attempt to develop a dataset of visual illusions and benchmark using data-driven approach for illusion classification and localization. We consider five well-studied types of brightness illusions: 1) Hermann grid, 2) Simultaneous Brightness Contrast, 3) White illusion, 4) Grid illusion, and 5) Induced Grating illusion. Benchmarking on the dataset achieves 99.56% accuracy in illusion identification and 84.37% pixel accuracy in illusion localization. The application of deep learning model, it is shown, also generalizes over unseen brightness illusions like brightness assimilation to contrast transitions. We also test the ability of state-of-theart diffusion models to generate brightness illusions. We have provided all the code, dataset, instructions etc in the github repo: https://github.com/aniket004/BRI3L

cs.CV↗

arXiv2023

Direct Amortized Likelihood Ratio Estimation

We introduce a new amortized likelihood ratio estimator for likelihood-free simulation-based inference (SBI). Our estimator is simple to train and estimates the likelihood ratio using a single forward pass of the neural estimator. Our approach directly computes the likelihood ratio between two competing parameter sets which is different from the previous approach of comparing two neural network output values. We refer to our model as the direct neural ratio estimator (DNRE). As part of introducing the DNRE, we derive a corresponding Monte Carlo estimate of the posterior. We benchmark our new ratio estimator and compare to previous ratio estimators in the literature. We show that our new ratio estimator often outperforms these previous approaches. As a further contribution, we introduce a new derivative estimator for likelihood ratio estimators that enables us to compare likelihood-free Hamiltonian Monte Carlo (HMC) with random-walk Metropolis-Hastings (MH). We show that HMC is equally competitive, which has not been previously shown. Finally, we include a novel real-world application of SBI by using our neural ratio estimator to design a quadcopter. Code is available at https://github.com/SRI-CSL/dnre.

stat.ML↗

arXiv2023

Cap2Aug: Caption guided Image to Image data Augmentation

Visual recognition in a low-data regime is challenging and often prone to overfitting. To mitigate this issue, several data augmentation strategies have been proposed. However, standard transformations, e.g., rotation, cropping, and flipping provide limited semantic variations. To this end, we propose Cap2Aug, an image-to-image diffusion model-based data augmentation strategy using image captions as text prompts. We generate captions from the limited training images and using these captions edit the training images using an image-to-image stable diffusion model to generate semantically meaningful augmentations. This strategy generates augmented versions of images similar to the training images yet provides semantic diversity across the samples. We show that the variations within the class can be captured by the captions and then translated to generate diverse samples using the image-to-image diffusion model guided by the captions. However, naive learning on synthetic images is not adequate due to the domain gap between real and synthetic images. Thus, we employ a maximum mean discrepancy (MMD) loss to align the synthetic images to the real images for minimizing the domain gap. We evaluate our method on few-shot and long-tail classification tasks and obtain performance improvements over state-of-the-art, especially in the low-data regimes.

cs.CV↗

arXiv2023

Multiple Testing Framework for Out-of-Distribution Detection

We study the problem of Out-of-Distribution (OOD) detection, that is, detecting whether a learning algorithm's output can be trusted at inference time. While a number of tests for OOD detection have been proposed in prior work, a formal framework for studying this problem is lacking. We propose a definition for the notion of OOD that includes both the input distribution and the learning algorithm, which provides insights for the construction of powerful tests for OOD detection. We propose a multiple hypothesis testing inspired procedure to systematically combine any number of different statistics from the learning algorithm using conformal p-values. We further provide strong guarantees on the probability of incorrectly classifying an in-distribution sample as OOD. In our experiments, we find that threshold-based tests proposed in prior work perform well in specific settings, but not uniformly well across different types of OOD instances. In contrast, our proposed method that combines multiple statistics performs uniformly well across different datasets and neural networks.

stat.ML↗

arXiv2023

TIJO: Trigger Inversion with Joint Optimization for Defending Multimodal Backdoored Models

We present a Multimodal Backdoor Defense technique TIJO (Trigger Inversion using Joint Optimization). Recent work arXiv:2112.07668 has demonstrated successful backdoor attacks on multimodal models for the Visual Question Answering task. Their dual-key backdoor trigger is split across two modalities (image and text), such that the backdoor is activated if and only if the trigger is present in both modalities. We propose TIJO that defends against dual-key attacks through a joint optimization that reverse-engineers the trigger in both the image and text modalities. This joint optimization is challenging in multimodal models due to the disconnected nature of the visual pipeline which consists of an offline feature extractor, whose output is then fused with the text using a fusion module. The key insight enabling the joint optimization in TIJO is that the trigger inversion needs to be carried out in the object detection box feature space as opposed to the pixel space. We demonstrate the effectiveness of our method on the TrojVQA benchmark, where TIJO improves upon the state-of-the-art unimodal methods from an AUC of 0.6 to 0.92 on multimodal dual-key backdoors. Furthermore, our method also improves upon the unimodal baselines on unimodal backdoors. We present ablation studies and qualitative results to provide insights into our algorithm such as the critical importance of overlaying the inverted feature triggers on all visual features during trigger inversion. The prototype implementation of TIJO is available at https://github.com/SRI-CSL/TIJO.

cs.CV↗

arXiv2023

AircraftVerse: A Large-Scale Multimodal Dataset of Aerial Vehicle Designs

We present AircraftVerse, a publicly available aerial vehicle design dataset. Aircraft design encompasses different physics domains and, hence, multiple modalities of representation. The evaluation of these cyber-physical system (CPS) designs requires the use of scientific analytical and simulation models ranging from computer-aided design tools for structural and manufacturing analysis, computational fluid dynamics tools for drag and lift computation, battery models for energy estimation, and simulation models for flight control and dynamics. AircraftVerse contains 27,714 diverse air vehicle designs - the largest corpus of engineering designs with this level of complexity. Each design comprises the following artifacts: a symbolic design tree describing topology, propulsion subsystem, battery subsystem, and other design details; a STandard for the Exchange of Product (STEP) model data; a 3D CAD design using a stereolithography (STL) file format; a 3D point cloud for the shape of the design; and evaluation results from high fidelity state-of-the-art physics models that characterize performance metrics such as maximum flight distance and hover-time. We also present baseline surrogate models that use different modalities of design representation to predict design performance metrics, which we provide as part of our dataset release. Finally, we discuss the potential impact of this dataset on the use of learning in aircraft design and, more generally, in CPS. AircraftVerse is accompanied by a data card, and it is released under Creative Commons Attribution-ShareAlike (CC BY-SA) license. The dataset is hosted at https://zenodo.org/record/6525446, baseline models and code at https://github.com/SRI-CSL/AircraftVerse, and the dataset description at https://aircraftverse.onrender.com/.

cs.RO↗

arXiv2023

$\texttt{LIMpy}$: A Semi-analytic Approach to Simulating Multi-line Intensity Maps at Millimetre Wavelengths

Mapping of multiple lines such as the fine-structure emission from [CII] (157.7 $μ\text{m}$), [OIII] (52 \& 88.4 $μ\text{m}$), and rotational emission lines from CO are of particular interest for upcoming line intensity mapping (LIM) experiments at millimetre wavelengths, due to their brightness features. Several upcoming experiments aim to cover a broad range of scientific goals, from detecting signatures of the epoch of reionization to the physics of star formation and its role in galaxy evolution. In this paper, we develop a semi-analytic approach to modelling line strengths as functions of the star formation rate (SFR) or infrared (IR) luminosity based on observations of local and high-z galaxies. This package, $\texttt{LIMpy}$ (Line Intensity Mapping in Python), estimates the intensity and power spectra of [CII], [OIII], and CO rotational transition lines up to the $J$-levels (1-0) to (13-12) based both on analytic formalism and on simulations. We develop a relation among halo mass, SFR, and multi-line intensities that permits us to construct a generic formula for the evolution of several line strengths up to $z \sim 10$. We implement a variety of star formation models and multi-line luminosity relations to estimate the astrophysical uncertainties on the intensity power spectrum of these lines. As a demonstration, we predict the signal-to-noise ratio of [CII] detection for an EoR-Spec-like instrument on the Fred Young Submillimeter Telescope (FYST). Furthermore, the ability to use any halo catalogue allows the $\texttt{LIMpy}$ code to be easily integrated into existing simulation pipelines, providing a flexible tool to study intensity mapping in the context of complex galaxy formation physics.

astro-ph.GA↗

arXiv2023

Automatic Measures for Evaluating Generative Design Methods for Architects

The recent explosion of high-quality image-to-image methods has prompted interest in applying image-to-image methods towards artistic and design tasks. Of interest for architects is to use these methods to generate design proposals from conceptual sketches, usually hand-drawn sketches that are quickly developed and can embody a design intent. More specifically, instantiating a sketch into a visual that can be used to elicit client feedback is typically a time consuming task, and being able to speed up this iteration time is important. While the body of work in generative methods has been impressive, there has been a mismatch between the quality measures used to evaluate the outputs of these systems and the actual expectations of architects. In particular, most recent image-based works place an emphasis on realism of generated images. While important, this is one of several criteria architects look for. In this work, we describe the expectations architects have for design proposals from conceptual sketches, and identify corresponding automated metrics from the literature. We then evaluate several image-to-image generative methods that may address these criteria and examine their performance across these metrics. From these results, we identify certain challenges with hand-drawn conceptual sketches and describe possible future avenues of investigation to address them.

cs.CV↗

arXiv2022

Design of Unmanned Air Vehicles Using Transformer Surrogate Models

Computer-aided design (CAD) is a promising new area for the application of artificial intelligence (AI) and machine learning (ML). The current practice of design of cyber-physical systems uses the digital twin methodology, wherein the actual physical design is preceded by building detailed models that can be evaluated by physics simulation models. These physics models are often slow and the manual design process often relies on exploring near-by variations of existing designs. AI holds the promise of breaking these design silos and increasing the diversity and performance of designs by accelerating the exploration of the design space. In this paper, we focus on the design of electrical unmanned aerial vehicles (UAVs). The high-density batteries and purely electrical propulsion systems have disrupted the space of UAV design, making this domain an ideal target for AI-based design. In this paper, we develop an AI Designer that synthesizes novel UAV designs. Our approach uses a deep transformer model with a novel domain-specific encoding such that we can evaluate the performance of new proposed designs without running expensive flight dynamics models and CAD tools. We demonstrate that our approach significantly reduces the overall compute requirements for the design process and accelerates the design space exploration. Finally, we identify future research directions to achieve full-scale deployment of AI-assisted CAD for UAVs.

cs.LG↗

arXiv2022

Assessing Effectiveness of Pulsed Input on Mixing Characteristics of Non-Newtonian fluids in T-shaped Channels

Mixing of reagents in microfluidics is necessary for various applications however, due to the laminar nature of flows, efficient mixing in a small span of length and time becomes difficult. The analysis of mixing of non-Newtonian fluids is critical as they are commonly encountered in practical applications. Towards this, we investigated an effective way for mixing of non-Newtonian fluids using pulsatile velocity inlet conditions. In the present study, the non-Newtonian fluid is modelled using the power law model with varying fluid rheology from shear thickening to shear thinning. For enhancing the mixing, pulsed velocity inlet condition is applied with varying phase angle and frequency and compared with constant velocity inlet condition. We demonstrated enhanced mixing using pulsing velocity inlet condition and achieved a maximum mixing of 97.6% using pulsed input velocity with a phase difference of 180° and considering a frequency of 5 Hz for the case of shear-thinning fluid (n=0.6). For the same condition, the mixing index is 89.1% and 85.2% for Newtonian and shear thickening fluid (n=1.4), respectively. The present study will be helpful in designing micromixers for mixing non-Newtonian fluid effectively in a small span of length and time.

physics.flu-dyn↗

arXiv2022

CODiT: Conformal Out-of-Distribution Detection in Time-Series Data

Machine learning models are prone to making incorrect predictions on inputs that are far from the training distribution. This hinders their deployment in safety-critical applications such as autonomous vehicles and healthcare. The detection of a shift from the training distribution of individual datapoints has gained attention. A number of techniques have been proposed for such out-of-distribution (OOD) detection. But in many applications, the inputs to a machine learning model form a temporal sequence. Existing techniques for OOD detection in time-series data either do not exploit temporal relationships in the sequence or do not provide any guarantees on detection. We propose using deviation from the in-distribution temporal equivariance as the non-conformity measure in conformal anomaly detection framework for OOD detection in time-series data.Computing independent predictions from multiple conformal detectors based on the proposed measure and combining these predictions by Fisher's method leads to the proposed detector CODiT with guarantees on false detection in time-series data. We illustrate the efficacy of CODiT by achieving state-of-the-art results on computer vision datasets in autonomous driving. We also show that CODiT can be used for OOD detection in non-vision datasets by performing experiments on the physiological GAIT sensory dataset. Code, data, and trained models are available at https://github.com/kaustubhsridhar/time-series-OOD.

cs.LG↗