arXiv subjects

Yeonkyung Lee

Publications and source records attributed to Yeonkyung Lee.

4 recordsLinked to original sources

arXiv2026

ViKey: Enhancing Temporal Understanding in Videos via Visual Prompting

Recent advancements in Video Large Language Models (VideoLLMs) have enabled strong performance across diverse multimodal video tasks. To reduce the high computational cost of processing dense video frames, efficiency-oriented methods such as frame selection have been widely adopted. While effective at minimizing redundancy, these methods often cause notable performance drops on tasks requiring temporal reasoning. Unlike humans, who can infer event progression from sparse visual cues, VideoLLMs frequently misinterpret temporal relations when intermediate frames are omitted. To address this limitation, we explore visual prompting (VP) as a lightweight yet effective way to enhance temporal understanding in VideoLLMs. Our analysis reveals that simply annotating each frame with explicit ordinal information helps the model perceive temporal continuity. This visual cue also supports frame-level referencing and mitigates positional ambiguity within a sparsely sampled sequence. Building on these insights, we introduce ViKey, a training-free framework that combines VP with a lightweight Keyword-Frame Mapping (KFM) module. KFM leverages frame indices as dictionary-like keys to link textual cues to the most relevant frames, providing explicit temporal anchors during inference. Despite its simplicity, our approach substantially improves temporal reasoning and, on some datasets, preserves dense-frame baseline performance with as few as 20% of frames.

cs.CV↗

arXiv2026

Convolutional Neural Networks for classifying galaxy mergers: Can faint tidal features aid in classifying mergers?

Identifying mergers from observational data has been a crucial aspect of studying galaxy evolution and formation. Tidal features, typically fainter than 26 ${\rm mag\,arcsec^{-2}}$, exhibit a diverse range of appearances depending on the merger characteristics and are expected to be investigated in greater detail with the Rubin Observatory Large Synoptic Survey Telescope (LSST), which will reveal the low surface brightness universe with unprecedented precision. Our goal is to assess the feasibility of developing a convolutional neural network (CNN) that can distinguish between mergers and non-mergers based on LSST-like deep images. To this end, we used Illustris TNG50, one of the highest-resolution cosmological hydrodynamic simulations to date, allowing us to generate LSST-like mock images with a depth $\sim$ 29 ${\rm mag\,arcsec^{-2}}$ for low-redshift ($z=0.16$) galaxies, with labeling based on their merger status as ground truth. We focused on 151 Milky Way-like galaxies in field environments, comprising 81 non-mergers and 70 mergers. After applying data augmentation and hyperparameter tuning, a CNN model was developed with an accuracy of 65--67\%. Through additional image processing, the model was further optimized, achieving an accuracy of 67--70\% when trained on images containing only faint features. This represents an improvement of $\sim$ 5\% compared to training on images with bright features only. This suggests that faint tidal features can serve as effective indicators for distinguishing between mergers and non-mergers. The future direction for further improvement based on this study is also discussed.

astro-ph.GA↗

arXiv2025

PRETI: Patient-Aware Retinal Foundation Model via Metadata-Guided Representation Learning

Retinal foundation models have significantly advanced retinal image analysis by leveraging self-supervised learning to reduce dependence on labeled data while achieving strong generalization. Many recent approaches enhance retinal image understanding using report supervision, but obtaining clinical reports is often costly and challenging. In contrast, metadata (e.g., age, gender) is widely available and serves as a valuable resource for analyzing disease progression. To effectively incorporate patient-specific information, we propose PRETI, a retinal foundation model that integrates metadata-aware learning with robust self-supervised representation learning. We introduce Learnable Metadata Embedding (LME), which dynamically refines metadata representations. Additionally, we construct patient-level data pairs, associating images from the same individual to improve robustness against non-clinical variations. To further optimize retinal image representation, we propose Retina-Aware Adaptive Masking (RAAM), a strategy that selectively applies masking within the retinal region and dynamically adjusts the masking ratio during training. PRETI captures both global structures and fine-grained pathological details, resulting in superior diagnostic performance. Extensive experiments demonstrate that PRETI achieves state-of-the-art results across diverse diseases and biomarker predictions using in-house and public data, indicating the importance of metadata-guided foundation models in retinal disease analysis. Our code and pretrained model are available at https://github.com/MICV-yonsei/PRETI

eess.IV↗

arXiv2025

Spatial Transport Optimization by Repositioning Attention Map for Training-Free Text-to-Image Synthesis

Diffusion-based text-to-image (T2I) models have recently excelled in high-quality image generation, particularly in a training-free manner, enabling cost-effective adaptability and generalization across diverse tasks. However, while the existing methods have been continuously focusing on several challenges, such as "missing objects" and "mismatched attributes," another critical issue of "mislocated objects" remains where generated spatial positions fail to align with text prompts. Surprisingly, ensuring such seemingly basic functionality remains challenging in popular T2I models due to the inherent difficulty of imposing explicit spatial guidance via text forms. To address this, we propose STORM (Spatial Transport Optimization by Repositioning Attention Map), a novel training-free approach for spatially coherent T2I synthesis. STORM employs Spatial Transport Optimization (STO), rooted in optimal transport theory, to dynamically adjust object attention maps for precise spatial adherence, supported by a Spatial Transport (ST) Cost function that enhances spatial understanding. Our analysis shows that integrating spatial awareness is most effective in the early denoising stages, while later phases refine details. Extensive experiments demonstrate that STORM surpasses existing methods, effectively mitigating mislocated objects while improving missing and mismatched attributes, setting a new benchmark for spatial alignment in T2I synthesis.

cs.CV↗