arXiv subjects

Chul Lee

Publications and source records attributed to Chul Lee.

25 records · Page 2Linked to original sources

arXiv2020

BMBC:Bilateral Motion Estimation with Bilateral Cost Volume for Video Interpolation

Video interpolation increases the temporal resolution of a video sequence by synthesizing intermediate frames between two consecutive frames. We propose a novel deep-learning-based video interpolation algorithm based on bilateral motion estimation. First, we develop the bilateral motion network with the bilateral cost volume to estimate bilateral motions accurately. Then, we approximate bi-directional motions to predict a different kind of bilateral motions. We then warp the two input frames using the estimated bilateral motions. Next, we develop the dynamic filter generation network to yield dynamic blending filters. Finally, we combine the warped frames using the dynamic blending filters to generate intermediate frames. Experimental results show that the proposed algorithm outperforms the state-of-the-art video interpolation algorithms on several benchmark datasets.

cs.CV↗

arXiv2020

Exploiting Temporal Coherence for Multi-modal Video Categorization

Multimodal ML models can process data in multiple modalities (e.g., video, images, audio, text) and are useful for video content analysis in a variety of problems (e.g., object detection, scene understanding). In this paper, we focus on the problem of video categorization by using a multimodal approach. We have developed a novel temporal coherence-based regularization approach, which applies to different types of models (e.g., RNN, NetVLAD, Transformer). We demonstrate through experiments how our proposed multimodal video categorization models with temporal coherence out-perform strong state-of-the-art baseline models.

cs.CV↗

arXiv2020

Cross-modal Learning for Multi-modal Video Categorization

Multi-modal machine learning (ML) models can process data in multiple modalities (e.g., video, audio, text) and are useful for video content analysis in a variety of problems (e.g., object detection, scene understanding, activity recognition). In this paper, we focus on the problem of video categorization using a multi-modal ML technique. In particular, we have developed a novel multi-modal ML approach that we call "cross-modal learning", where one modality influences another but only when there is correlation between the modalities -- for that, we first train a correlation tower that guides the main multi-modal video categorization tower in the model. We show how this cross-modal principle can be applied to different types of models (e.g., RNN, Transformer, NetVLAD), and demonstrate through experiments how our proposed multi-modal video categorization models with cross-modal learning out-perform strong state-of-the-art baseline models.

cs.CV↗

arXiv2020

NTIRE 2020 Challenge on Image Demoireing: Methods and Results

This paper reviews the Challenge on Image Demoireing that was part of the New Trends in Image Restoration and Enhancement (NTIRE) workshop, held in conjunction with CVPR 2020. Demoireing is a difficult task of removing moire patterns from an image to reveal an underlying clean image. The challenge was divided into two tracks. Track 1 targeted the single image demoireing problem, which seeks to remove moire patterns from a single image. Track 2 focused on the burst demoireing problem, where a set of degraded moire images of the same scene were provided as input, with the goal of producing a single demoired image as output. The methods were ranked in terms of their fidelity, measured using the peak signal-to-noise ratio (PSNR) between the ground truth clean images and the restored images produced by the participants' methods. The tracks had 142 and 99 registered participants, respectively, with a total of 14 and 6 submissions in the final testing stage. The entries span the current state-of-the-art in image and burst image demoireing problems.

cs.CV↗

arXiv2019

Infrared study of Large scale h-BN film and Graphene/h-BN heterostructure

We synthesize a series of CVD h-BN films and perform critical infrared spectroscopic characterization. For high-temperature (HT, Temp = 1400 degrees) grown h-BN thin film only E1u-mode infrared phonon is activated demonstrating highly aligned 2D h-BN planes over large area, whereas low-temperature (LT, Temp = 1000 degrees) grown film shows two phonon peaks, E1u and A2u, due to stacking of h-BN plane at tilted angle. For CVD graphene transferred on HT h-BN/SiO2/Si substrate, interband transition spectrum s1 shifts strongly to lower energy compared with that on LT h-BN/SiO2/Si and on bare SiO2/Si substrate, revealing that residual carrier density n in graphene is suppressed by use of HT h-BN layer. Also the interband transition width of s1 defined by effective temperature is reduced from 400 K for G/SiO2/Si to 300 K for HT h-BN/SiO2/Si. The behaviors of n and effective temperature show that HT h-BN film can decouple CVD graphene from the impurity and defect of SiO2 leading to large scale free-standing like graphene.

cond-mat.mtrl-sci↗

arXiv2016

A Maximum A Posteriori Estimation Framework for Robust High Dynamic Range Video Synthesis

High dynamic range (HDR) image synthesis from multiple low dynamic range (LDR) exposures continues to be actively researched. The extension to HDR video synthesis is a topic of significant current interest due to potential cost benefits. For HDR video, a stiff practical challenge presents itself in the form of accurate correspondence estimation of objects between video frames. In particular, loss of data resulting from poor exposures and varying intensity make conventional optical flow methods highly inaccurate. We avoid exact correspondence estimation by proposing a statistical approach via maximum a posterior (MAP) estimation, and under appropriate statistical assumptions and choice of priors and models, we reduce it to an optimization problem of solving for the foreground and background of the target frame. We obtain the background through rank minimization and estimate the foreground via a novel multiscale adaptive kernel regression technique, which implicitly captures local structure and temporal motion by solving an unconstrained optimization problem. Extensive experimental results on both real and synthetic datasets demonstrate that our algorithm is more capable of delivering high-quality HDR videos than current state-of-the-art methods, under both subjective and objective assessments. Furthermore, a thorough complexity analysis reveals that our algorithm achieves better complexity-performance trade-off than conventional methods.

cs.CV↗

arXiv2015

Persistent Sharing of Fitness App Status on Twitter

As the world becomes more digitized and interconnected, information that was once considered to be private such as one's health status is now being shared publicly. To understand this new phenomenon better, it is crucial to study what types of health information are being shared on social media and why, as well as by whom. In this paper, we study the traits of users who share their personal health and fitness related information on social media by analyzing fitness status updates that MyFitnessPal users have shared via Twitter. We investigate how certain features like user profile, fitness activity, and fitness network in social media can potentially impact the long-term engagement of fitness app users. We also discuss implications of our findings to achieve a better retention of these users and to promote more sharing of their status updates.

cs.SI↗