arXiv subjects

Jae Lee

Publications and source records attributed to Jae Lee.

3 recordsLinked to original sources

arXiv2024

Pegasus-v1 Technical Report

This technical report introduces Pegasus-1, a multimodal language model specialized in video content understanding and interaction through natural language. Pegasus-1 is designed to address the unique challenges posed by video data, such as interpreting spatiotemporal information, to offer nuanced video content comprehension across various lengths. This technical report overviews Pegasus-1's architecture, training strategies, and its performance in benchmarks on video conversation, zero-shot video question answering, and video summarization. We also explore qualitative characteristics of Pegasus-1 , demonstrating its capabilities as well as its limitations, in order to provide readers a balanced view of its current state and its future direction.

cs.MM↗

arXiv2021

Retrospective Evaluation of an Always-on Cherenkov Imaging System for Radiotherapy Quality Improvement

Purpose: Cherenkov imaging is now clinically available to track the course of radiation therapy as a treatment verification tool. The aim of this work was to discover the benefits of always-on Cherenkov images as a novel incident detection and quality improvement system through retrospective review of imaging in our center. Methods: Continuous imaging of all patients was attempted during a 12-month period by automating the acquisition of Cherenkov imaging using an always-on commercial system. Multi-camera systems were installed in two treatment bunkers in the radiation oncology clinic at our center and one bunker in an affiliated satellite clinic. Images were acquired as part of normal treatment procedure and reviewed retrospectively with potential incidents flagged for evaluation by the physician and medical physics teams. Results: In total, 622 patients were imaged as part of this study. In this summary, 9 patients were identified with incidents occurring during their course of treatment that were detected only with Cherenkov imaging. Incidents were found relating to issues during simulation, planning, pre-treatment review, and treatment delivery, however none of the incidents were detected prior to treatment delivery. Primary areas of improvement identified in this study are dose to unintended areas in planning, dose to unintended areas due to positioning, and non-ideal bolus placement during setup. Case studies are presented highlighting the detection of these issues using Cherenkov imaging. Conclusions: All detected events were deemed below the threshold for reporting, but their observation could lead to quality improvement in practice. Perhaps most importantly, the imaging was seamless with no effort required by the radiotherapy team and provided both real-time and permanent records of what was delivered in each fraction.

physics.med-ph↗

arXiv2020

Exploring Extended Reality with ILLIXR: A New Playground for Architecture Research

As we enter the era of domain-specific architectures, systems researchers must understand the requirements of emerging application domains. Augmented and virtual reality (AR/VR) or extended reality (XR) is one such important domain. This paper presents ILLIXR, the first open source end-to-end XR system (1) with state-of-the-art components, (2) integrated with a modular and extensible multithreaded runtime, (3) providing an OpenXR compliant interface to XR applications (e.g., game engines), and (4) with the ability to report (and trade off) several quality of experience (QoE) metrics. We analyze performance, power, and QoE metrics for the complete ILLIXR system and for its individual components. Our analysis reveals several properties with implications for architecture and systems research. These include demanding performance, power, and QoE requirements, a large diversity of critical tasks, inter-dependent execution pipelines with challenges in scheduling and resource management, and a large tradeoff space between performance/power and human perception related QoE metrics. ILLIXR and our analysis have the potential to propel new directions in architecture and systems research in general, and impact XR in particular. ILLIXR is open-source and available at https://illixr.github.io

cs.DC↗