arXiv subjects

Jose Alvarez

Publications and source records attributed to Jose Alvarez.

12 recordsLinked to original sources

arXiv2025

Cosmos-Transfer1: Conditional World Generation with Adaptive Multimodal Control

We introduce Cosmos-Transfer, a conditional world generation model that can generate world simulations based on multiple spatial control inputs of various modalities such as segmentation, depth, and edge. In the design, the spatial conditional scheme is adaptive and customizable. It allows weighting different conditional inputs differently at different spatial locations. This enables highly controllable world generation and finds use in various world-to-world transfer use cases, including Sim2Real. We conduct extensive evaluations to analyze the proposed model and demonstrate its applications for Physical AI, including robotics Sim2Real and autonomous vehicle data enrichment. We further demonstrate an inference scaling strategy to achieve real-time world generation with an NVIDIA GB200 NVL72 rack. To help accelerate research development in the field, we open-source our models and code at https://github.com/nvidia-cosmos/cosmos-transfer1.

cs.CV↗

arXiv2024

Multi-Dimensional Pruning: Joint Channel, Layer and Block Pruning with Latency Constraint

As we push the boundaries of performance in various vision tasks, the models grow in size correspondingly. To keep up with this growth, we need very aggressive pruning techniques for efficient inference and deployment on edge devices. Existing pruning approaches are limited to channel pruning and struggle with aggressive parameter reductions. In this paper, we propose a novel multi-dimensional pruning framework that jointly optimizes pruning across channels, layers, and blocks while adhering to latency constraints. We develop a latency modeling technique that accurately captures model-wide latency variations during pruning, which is crucial for achieving an optimal latency-accuracy trade-offs at high pruning ratio. We reformulate pruning as a Mixed-Integer Nonlinear Program (MINLP) to efficiently determine the optimal pruned structure with only a single pass. Our extensive results demonstrate substantial improvements over previous methods, particularly at large pruning ratios. In classification, our method significantly outperforms prior art HALP with a Top-1 accuracy of 70.0(v.s. 68.6) and an FPS of 5262 im/s(v.s. 4101 im/s). In 3D object detection, we establish a new state-of-the-art by pruning StreamPETR at a 45% pruning ratio, achieving higher FPS (37.3 vs. 31.7) and mAP (0.451 vs. 0.449) than the dense baseline.

cs.CV↗

arXiv2023

FocalFormer3D : Focusing on Hard Instance for 3D Object Detection

False negatives (FN) in 3D object detection, {\em e.g.}, missing predictions of pedestrians, vehicles, or other obstacles, can lead to potentially dangerous situations in autonomous driving. While being fatal, this issue is understudied in many current 3D detection methods. In this work, we propose Hard Instance Probing (HIP), a general pipeline that identifies \textit{FN} in a multi-stage manner and guides the models to focus on excavating difficult instances. For 3D object detection, we instantiate this method as FocalFormer3D, a simple yet effective detector that excels at excavating difficult objects and improving prediction recall. FocalFormer3D features a multi-stage query generation to discover hard objects and a box-level transformer decoder to efficiently distinguish objects from massive object candidates. Experimental results on the nuScenes and Waymo datasets validate the superior performance of FocalFormer3D. The advantage leads to strong performance on both detection and tracking, in both LiDAR and multi-modal settings. Notably, FocalFormer3D achieves a 70.5 mAP and 73.9 NDS on nuScenes detection benchmark, while the nuScenes tracking benchmark shows 72.1 AMOTA, both ranking 1st place on the nuScenes LiDAR leaderboard. Our code is available at \url{https://github.com/NVlabs/FocalFormer3D}.

cs.CV↗

arXiv2023

Adaptive Sharpness-Aware Pruning for Robust Sparse Networks

Robustness and compactness are two essential attributes of deep learning models that are deployed in the real world. The goals of robustness and compactness may seem to be at odds, since robustness requires generalization across domains, while the process of compression exploits specificity in one domain. We introduce Adaptive Sharpness-Aware Pruning (AdaSAP), which unifies these goals through the lens of network sharpness. The AdaSAP method produces sparse networks that are robust to input variations which are unseen at training time. We achieve this by strategically incorporating weight perturbations in order to optimize the loss landscape. This allows the model to be both primed for pruning and regularized for improved robustness. AdaSAP improves the robust accuracy of pruned models on image classification by up to +6% on ImageNet C and +4% on ImageNet V2, and on object detection by +4% on a corrupted Pascal VOC dataset, over a wide range of compression ratios, pruning criteria, and network architectures, outperforming recent pruning art by large margins.

cs.LG↗

arXiv2021

Effective dose fractionation schemes of radiotherapy for prostate cancer

Radiation therapy has remained as one of the main cancer treatment modalities and a highly cost-effective single modality treatment of cancer care. Typical regimens for fractionated external beam radiotherapy comprise a constant dose administered on weekdays, and no radiation on weekends. However, every patient has a tumor with distinct properties depending on intra-tumor heterogeneity, aggressiveness, and interactive properties with other cells that may make it more resistant or sensitive to radiation treatment. Accordingly, the concept of personalized cancer treatment is emerging to specialize each patient treatment case to the unique properties of the tumor. In this paper, we examine adaptive radiation treatment strategies for heterogeneous tumors using a dynamical system model that consists of radiation-resistant and parental cell populations with unique interactive properties. We study different adaptive dosage strategies for PC3 and DU145 prostate cancer cell lines. We show that stronger doses of radiation given in longer time intervals, while keeping the overall dosage the same, reduce final tumor volume by more than half in PC3 cell lines, but by only five percent in DU145 cell lines. In addition, we tested an adaptive dosing schedule by administering a stronger dosage on Friday to compensate for the treatment-off period during the weekend, which was effective in decreasing the final tumor volume of both cell lines. This result creates interesting possibilities for new radiotherapy strategies at clinics that cannot provide treatment on weekends. Finally, we propose a dosage plan incorporating our findings.

physics.med-ph↗

arXiv2021

AdaViT: Adaptive Tokens for Efficient Vision Transformer

We introduce A-ViT, a method that adaptively adjusts the inference cost of vision transformer (ViT) for images of different complexity. A-ViT achieves this by automatically reducing the number of tokens in vision transformers that are processed in the network as inference proceeds. We reformulate Adaptive Computation Time (ACT) for this task, extending halting to discard redundant spatial tokens. The appealing architectural properties of vision transformers enables our adaptive token reduction mechanism to speed up inference without modifying the network architecture or inference hardware. We demonstrate that A-ViT requires no extra parameters or sub-network for halting, as we base the learning of adaptive halting on the original network parameters. We further introduce distributional prior regularization that stabilizes training compared to prior ACT approaches. On the image classification task (ImageNet1K), we show that our proposed A-ViT yields high efficacy in filtering informative spatial features and cutting down on the overall compute. The proposed method improves the throughput of DeiT-Tiny by 62% and DeiT-Small by 38% with only 0.3% accuracy drop, outperforming prior art by a large margin. Project page at https://a-vit.github.io/

cs.CV↗

arXiv2018

Monitoring Dynamic Mobile Ad-Hoc Networks: A Fully Distributed Hybrid Architecture

The mobile ad-hoc networks (MANETs) represent a broad area of study and market interest. They provide a wide set of applications in multiple domains. In that context, the functional and non-functional monitoring of these networks is crucial. For that purpose, monitoring techniques have been deeply studied in wired networks using gossip-based or hierarchical-based approaches. However, when applied to a MANET, several problematics arise mainly due to the absence of a centralized administration, the inherent MANETs constraints and the nodes mobility. In this paper, we present a hybrid distributed monitoring architecture for mobile adhoc networks in context of mobility pattern. We get inspired of gossip-based and hierarchical-based algorithms for query dissemination and data aggregation. We define gossip-based mechanisms that help our virtual hierarchical topology to complete the data aggregation, and then ensure the stability and robustness of our approach in dynamic environments. Further, we propose a fully distributed monitoring protocol that ease the nodes communications. We evaluate our approach through a simulated testbed by using NS3 and Docker, and illustrate the efficiency of our mechanisms.

cs.NI↗

arXiv2018

MANETs monitoring with a distributed hybrid architecture

Monitoring techniques have been deeply studied in wired networks using gossip and hierarchical approaches. However, when applied to a MANET, several problematics arise. We present a hybrid distributed monitoring architecture for MANETs. We get inspired of gossip-based and hierarchical-based algorithms for query dissemination and data aggregation. We define gossip-based mechanisms that help our virtual hierarchical topology to complete the data aggregation, and then ensure the stability and robustness of our approach in dynamic environments. We propose a fully distributed monitoring protocol that ease the nodes communications. We evaluate our approach by using NS3 and Docker.

cs.NI↗

arXiv2017

DHYMON: a Continuous Decentralized Hybrid Monitoring Architecture for MANETs

We introduce a novel decentralized monitoring algorithm for mobile ad-hoc networks. This algorithm is a combination of gossip-based and tree-based approaches. Its main feature is on multi root nodes selection which provides an opportunity to obtain more accurate results as it maximizes the coverage of the network. Our proposal relies on a thorough defined algorithm and its efficient and effective implementation. The algorithm is divided in two main parts that are its query procedure and its disseminate procedure. To assess different parameters such as accuracy and convergence time, we conducted experiments with an in-house emulated test bed based on Docker and NS3. To show the effectiveness and scalability of our proposal, we conduct intensive emulations that demonstrate very promising results regarding the gain of accuracy.

cs.NI↗

arXiv2016

DecomposeMe: Simplifying ConvNets for End-to-End Learning

Deep learning and convolutional neural networks (ConvNets) have been successfully applied to most relevant tasks in the computer vision community. However, these networks are computationally demanding and not suitable for embedded devices where memory and time consumption are relevant. In this paper, we propose DecomposeMe, a simple but effective technique to learn features using 1D convolutions. The proposed architecture enables both simplicity and filter sharing leading to increased learning capacity. A comprehensive set of large-scale experiments on ImageNet and Places2 demonstrates the ability of our method to improve performance while significantly reducing the number of parameters required. Notably, on Places2, we obtain an improvement in relative top-1 classification accuracy of 7.7\% with an architecture that requires 92% fewer parameters compared to VGG-B. The proposed network is also demonstrated to generalize to other tasks by converting existing networks.

cs.CV↗

arXiv2014

Motion Estimation via Robust Decomposition with Constrained Rank

In this work, we address the problem of outlier detection for robust motion estimation by using modern sparse-low-rank decompositions, i.e., Robust PCA-like methods, to impose global rank constraints. Robust decompositions have shown to be good at splitting a corrupted matrix into an uncorrupted low-rank matrix and a sparse matrix, containing outliers. However, this process only works when matrices have relatively low rank with respect to their ambient space, a property not met in motion estimation problems. As a solution, we propose to exploit the partial information present in the decomposition to decide which matches are outliers. We provide evidences showing that even when it is not possible to recover an uncorrupted low-rank matrix, the resulting information can be exploited for outlier detection. To this end we propose the Robust Decomposition with Constrained Rank (RD-CR), a proximal gradient based method that enforces the rank constraints inherent to motion estimation. We also present a general framework to perform robust estimation for stereo Visual Odometry, based on our RD-CR and a simple but effective compressed optimization method that achieves high performance. Our evaluation on synthetic data and on the KITTI dataset demonstrates the applicability of our approach in complex scenarios and it yields state-of-the-art performance.

cs.CV↗

arXiv2014

Direct observation of the leakage current in epitaxial diamond Schottky barrier devices by conductive-probe atomic force microscopy and Raman imaging

The origin of the high leakage current measured in several vertical-type diamond Schottky devices is conjointly investigated by conducting probe atomic force microscopy (CP-AFM) and confocal micro-Raman/Photoluminescence (PL) imaging analysis. Local areas characterized by a strong decrease of the local resistance (5-6 orders of magnitude drop) with respect to their close surrounding have been identified in several different regions of the sample surface. The same local areas, also referenced as electrical hot-spots, reveal a slightly constrained diamond lattice and three dominant Raman bands in the low-wavenumber region (590, 914 and 1040 cm-1). These latter bands are usually assigned to the vibrational modes involving boron impurities and its possible complexes that can electrically act as traps for charge carriers. Local current-voltage measurements performed at the hot-spots point out a trap-filled-limited (TFL) current as the main conduction mechanism favoring the leakage current in the Schottky devices.

cond-mat.mtrl-sci↗