arXiv subjects

Sungho Lee

Publications and source records attributed to Sungho Lee.

At least 37 records · Page 2Linked to original sources

arXiv2023

Exploiting Time-Frequency Conformers for Music Audio Enhancement

With the proliferation of video platforms on the internet, recording musical performances by mobile devices has become commonplace. However, these recordings often suffer from degradation such as noise and reverberation, which negatively impact the listening experience. Consequently, the necessity for music audio enhancement (referred to as music enhancement from this point onward), involving the transformation of degraded audio recordings into pristine high-quality music, has surged to augment the auditory experience. To address this issue, we propose a music enhancement system based on the Conformer architecture that has demonstrated outstanding performance in speech enhancement tasks. Our approach explores the attention mechanisms of the Conformer and examines their performance to discover the best approach for the music enhancement task. Our experimental results show that our proposed model achieves state-of-the-art performance on single-stem music enhancement. Furthermore, our system can perform general music enhancement with multi-track mixtures, which has not been examined in previous work.

cs.SD↗

arXiv2023

Blind Estimation of Audio Processing Graph

Musicians and audio engineers sculpt and transform their sounds by connecting multiple processors, forming an audio processing graph. However, most deep-learning methods overlook this real-world practice and assume fixed graph settings. To bridge this gap, we develop a system that reconstructs the entire graph from a given reference audio. We first generate a realistic graph-reference pair dataset and train a simple blind estimation system composed of a convolutional reference encoder and a transformer-based graph decoder. We apply our model to singing voice effects and drum mixing estimation tasks. Evaluation results show that our method can reconstruct complex signal routings, including multi-band processing and sidechaining.

cs.SD↗

arXiv2022

Global HRTF Interpolation via Learned Affine Transformation of Hyper-conditioned Features

Estimating Head-Related Transfer Functions (HRTFs) of arbitrary source points is essential in immersive binaural audio rendering. Computing each individual's HRTFs is challenging, as traditional approaches require expensive time and computational resources, while modern data-driven approaches are data-hungry. Especially for the data-driven approaches, existing HRTF datasets differ in spatial sampling distributions of source positions, posing a major problem when generalizing the method across multiple datasets. To alleviate this, we propose a deep learning method based on a novel conditioning architecture. The proposed method can predict an HRTF of any position by interpolating the HRTFs of known distributions. Experimental results show that the proposed architecture improves the model's generalizability across datasets with various coordinate systems. Additional demonstrations show that the model robustly reconstructs the target HRTFs from the spatially downsampled HRTFs in both quantitative and perceptual measures.

eess.AS↗

arXiv2022

Differentiable Artificial Reverberation

Artificial reverberation (AR) models play a central role in various audio applications. Therefore, estimating the AR model parameters (ARPs) of a reference reverberation is a crucial task. Although a few recent deep-learning-based approaches have shown promising performance, their non-end-to-end training scheme prevents them from fully exploiting the potential of deep neural networks. This motivates the introduction of differentiable artificial reverberation (DAR) models, allowing loss gradients to be back-propagated end-to-end. However, implementing the AR models with their difference equations "as is" in the deep learning framework severely bottlenecks the training speed when executed with a parallel processor like GPU due to their infinite impulse response (IIR) components. We tackle this problem by replacing the IIR filters with finite impulse response (FIR) approximations with the frequency-sampling method. Using this technique, we implement three DAR models -- differentiable Filtered Velvet Noise (FVN), Advanced Filtered Velvet Noise (AFVN), and Delay Network (DN). For each AR model, we train its ARP estimation networks for analysis-synthesis (RIR-to-ARP) and blind estimation (reverberant-speech-to-ARP) task in an end-to-end manner with its DAR model counterpart. Experiment results show that the proposed method achieves consistent performance improvement over the non-end-to-end approaches in both objective metrics and subjective listening test results.

cs.SD↗

arXiv2022

Facing to Latency of Hyperledger Fabric for Blockchain-enabled IoT: Modeling and Analysis

Hyperledger Fabric (HLF), one of the most popular private blockchains, has recently received attention for blockchain-enabled Internet of Things (IoT). However, for IoT applications to handle time-sensitive data, the processing latency in HLF has emerged as a new challenge. In this article, therefore, we establish a practical HLF latency model for HLF-enabled IoT. We first discuss the structure and transaction flow of HLF-enabled IoT. After implementing real HLF, we capture the latencies that each transaction experiences and show that the total latency of HLF can be modeled as a Gamma distribution, which is validated by conducting a goodness-of-fit test (i.e., the Kolmogorov-Smirnov (KS) test). We also provide the parameter values of the modeled latency distribution for various HLF environments. Furthermore, we explore the impacts of three important HLF parameters including transaction generation rate, block size, and block-generation timeout on the HLF latency. As a result, this article provides design insights on minimizing the latency for HLF-enabled IoT.

cs.DC↗

arXiv2021

Enhanced Correlation Matching based Video Frame Interpolation

We propose a novel DNN based framework called the Enhanced Correlation Matching based Video Frame Interpolation Network to support high resolution like 4K, which has a large scale of motion and occlusion. Considering the extensibility of the network model according to resolution, the proposed scheme employs the recurrent pyramid architecture that shares the parameters among each pyramid layer for optical flow estimation. In the proposed flow estimation, the optical flows are recursively refined by tracing the location with maximum correlation. The forward warping based correlation matching enables to improve the accuracy of flow update by excluding incorrectly warped features around the occlusion area. Based on the final bi-directional flows, the intermediate frame at arbitrary temporal position is synthesized using the warping and blending network and it is further improved by refinement network. Experiment results demonstrate that the proposed scheme outperforms the previous works at 4K video data and low-resolution benchmark datasets as well in terms of objective and subjective quality with the smallest number of model parameters.

cs.CV↗

arXiv2021

Ensuring Data Freshness for Blockchain-enabled Monitoring Networks

The age of information (AoI) is a recently proposed metric for quantifying data freshness in real-time status monitoring systems where timeliness is of importance. In this paper, the problem of characterizing and controlling the AoI is studied in the context of blockchain-enabled monitoring networks (BeMN). In BeMN, status updates from sources are transmitted and recorded in a blockchain. To investigate the statistical characteristics of the AoI in BeMN, the transmission latency and the consensus latency are first rigorously modeled. Then, the average AoI, the AoI violation probability, and the peak AoI violation probability are derived in a closed form so as to quantify the performance of BeMN. Furthermore, a simplified form is derived for the AoI violation probability, and it is shown that this quantity can capture the upper or lower bound of the actual AoI violation probability. Simulation results show that each BeMN parameters (i.e., target successful transmission probability, block size, and timeout) can have conflicting effects on the AoI-related performance. Subsequently, design insights are provided to maintain the freshness of the status data in BeMN. Then, experimental results with a real Hyperledger Fabric platform further validate the accuracy of our modeling and analysis.

eess.SP↗

arXiv2020

Age of Information Analysis in Hyperledger Fabric Blockchain-enabled Monitoring Networks

Age of information (AoI) is a recently proposed metric for quantifying data freshness in real-time status monitoring systems where timeliness is of importance. In this paper, we explore the data freshness in the Hyperledger Fabric Blockchain-enabled monitoring network (HeMN) by leveraging the AoI metric. In HeMN, status updates from sources are transmitted through an uplink and recorded in a Hyperledger Fabric (HLF) network. To provide a stochastic guarantee of data freshness, we derive a closed-form of AoI violation probability by considering the transmission latency and the consensus latency. Then, we validate our analytic results through the implemented HLF platform. We also investigate the effect of the target successful transmission probability (STP) on the AoI violation probability.

eess.SP↗

arXiv2020

Is Blockchain Suitable for Data Freshness? -- Age-of-Information Perspective

Recent advances in blockchain have led to a significant interest in developing blockchain-based applications. While data can be retained in blockchains, the stored values can be deleted or updated. From a user viewpoint that searches for the data, it is unclear whether the discovered data from the blockchain storage is relevant for real-time decision-making process for blockchain-based application. The data freshness issue serves as a critical factor especially in dynamic networks handling real-time information. In general, transactions to renew the data require additional processing time inside the blockchain network, which is called ledger-commitment latency. Due to this problem, some users may receive outdated data. As a result, it is important to investigate if blockchain is suitable for providing real-time data services. In this article, we first describe blockchain-enabled (BCE) networks with Hyperledger Fabric (HLF). Then, we define age of information (AoI) of BCE networks and investigate the influential factors in this AoI. Analysis and experiments are conducted to support our proposed framework. Lastly, we conclude by discussing some future challenges.

cs.DC↗

arXiv2020

IGRINS Slit-Viewing Camera Software

We have developed observation control software for the Immersion GRating INfrared Spectrometer (IGRINS) slit-viewing camera module, which maintains the position of an astronomical target on the spectroscopic slit. It is composed of several packages that monitor and control the system, acquire the images, and compensate for the tracking error by sending tracking feedback information to the telescope control system. For efficient development and maintenance of each software package, we have applied software engineering methods, i.e., a spiral software development with model-based design. It is not trivial to define the shape and center of astronomical object point spread functions (PSFs), which do not have symmetric Gaussian profiles in short exposure (<4 s) guiding images. Efforts to determine the PSF centroid are additionally complicated by the core saturation of bright guide stars. We have applied both a two-dimensional Gaussian fitting algorithm (2DGA) and center balancing algorithm (CBA) to identify an appropriate method for IGRINS in the near-infrared K-band. The CBA derives the expected center position along the slit-width by referencing the spillover flux ratio of the PSF wings on both sides of the slit. In this research, we have compared the accuracy and reliability of the CBA to the 2DGA by using data from IGRINS commissioning observations at McDonald Observatory. We find that the performance of each algorithm depends on the brightness of the targets and the seeing conditions, with the CBA performing better in typical observing scenarios. The algorithms and test results we present can be utilized with future spectroscopic slit observations in various observing conditions and for a variety of spectrograph designs.

astro-ph.IM↗

arXiv2019

Copy-and-Paste Networks for Deep Video Inpainting

We present a novel deep learning based algorithm for video inpainting. Video inpainting is a process of completing corrupted or missing regions in videos. Video inpainting has additional challenges compared to image inpainting due to the extra temporal information as well as the need for maintaining the temporal coherency. We propose a novel DNN-based framework called the Copy-and-Paste Networks for video inpainting that takes advantage of additional information in other frames of the video. The network is trained to copy corresponding contents in reference frames and paste them to fill the holes in the target frame. Our network also includes an alignment network that computes affine matrices between frames for the alignment, enabling the network to take information from more distant frames for robustness. Our method produces visually pleasing and temporally coherent results while running faster than the state-of-the-art optimization-based method. In addition, we extend our framework for enhancing over/under exposed frames in videos. Using this enhancement technique, we were able to significantly improve the lane detection accuracy on road videos.

cs.CV↗

arXiv2019

Onion-Peel Networks for Deep Video Completion

We propose the onion-peel networks for video completion. Given a set of reference images and a target image with holes, our network fills the hole by referring the contents in the reference images. Our onion-peel network progressively fills the hole from the hole boundary enabling it to exploit richer contextual information for the missing regions every step. Given a sufficient number of recurrences, even a large hole can be inpainted successfully. To attend to the missing information visible in the reference images, we propose an asymmetric attention block that computes similarities between the hole boundary pixels in the target and the non-hole pixels in the references in a non-local manner. With our attention block, our network can have an unlimited spatial-temporal window size and fill the holes with globally coherent contents. In addition, our framework is applicable to the image completion guided by the reference images without any modification, which is difficult to do with the previous methods. We validate that our method produces visually pleasing image and video inpainting results in realistic test cases.

cs.CV↗

arXiv2018

IGRINS Spectral Library

We present a library of high-resolution (R $\equiv$ $λ$/$Δ$$λ$ $\sim$ 45,000) and high signal-to-noise ratio (S/N $\geq$ 200) near-infrared spectra for stars of a wide range of spectral types and luminosity classes. The spectra were obtained with the Immersion GRating INfrared Spectrograph (IGRINS) covering the full range of the H (1.496-1.780 $μ$m) and K (2.080-2.460 $μ$m) atmospheric windows. The targets were primarily selected for being MK standard stars covering a wide range of effective temperatures and surface gravities with metallicities close to the Solar value. Currently, the library includes flux-calibrated and telluric-absorption-corrected spectra of 84 stars, with prospects for expansion to provide denser coverage of the parametric space. Throughout the H and K atmospheric windows, we identified spectral lines that are sensitive to $T_\mathrm{eff}$ or $\log g$ and defined corresponding spectral indices. We also provide their equivalent widths. For those indices, we derive empirical relations between the measured equivalent widths and the stellar atmospheric parameters. Therefore, the derived empirical equations can be used to calculate $T_\mathrm{eff}$ and $\log g$ of a star without requiring stellar atmospheric models.

astro-ph.SR↗

arXiv2017

Inner warm disk of ESO H$α$ 279A revealed by NA I and CO overtone emission lines

We present analysis of near-infrared, high-resolution spectroscopy towards the Flat-spectrum YSO (Young Stellar Object) ESO H$α$ 279a (1.5 solar mass) in the Serpens star forming region, at the distance of 429 pc. Using the Immersion GRating INfrared Spectrometer (IGRINS, R=45,000), we detect emission lines originating from the accretion channel flow, jet, and inner disk. Specifically, we identify hydrogen Brackett series recombination, [Fe II], [Fe III], [Fe IV], Ca I, Na I, H2, H2O and CO overtone emission lines. By modeling five bands of CO overtone emission lines, and the symmetric double-peaked line profile for Na I emission lines, we find that ESO H$α$ 279a has an actively accreting Keplerian disk. From our Keplerian disk model, we find that Na I emission lines originate between 0.04 AU and 1.00 AU, while CO overtone emission lines are from the outer part of disk, in the range between 0.22 AU and 3.00 AU. It reveals that the neutral atomic Na gas is a good tracer of the innermost region of the actively accreting disk. We derive a mass accretion rate of 2-10x10^{-7} M_solar/yr from the measured Br_gamma emission luminosity of 1.78(+-0.31)x10^{31} erg/s.

astro-ph.SR↗

arXiv2017

Excitation of Molecular Hydrogen in the Orion Bar Photodissociation Region From a Deep Near-Infrared IGRINS Spectrum

We present a deep near-infrared spectrum of the Orion Bar Photodissociation Region (PDR) taken with the Immersion Grating INfrared Spectrometer (IGRINS) on the 2.7 m telescope at the McDonald Observatory. IGRINS has high spectral resolution (R~45000) and instantaneous broad wavelength coverage (1.45-2.45 microns), enabling us to detect 87 emission lines from rovibrationally excited molecular hydrogen (H_2) that arise from transitions out of 69 upper rovibration levels of the electronic ground state. These levels cover a large range of rotational and vibrational quantum numbers and excitation energies, making them an excellent probe of the excitation mechanisms of H_2 and physical conditions within the PDR. The Orion Bar PDR is thought to consist of cooler high density clumps or filaments (T=50-250 K, n_H = 10^5 - 10^7 cm^-3) embedded in a warmer lower density medium (T=250-1000 K, n_H=10^4 - 10^5 cm^-3). We fit a grid of simple constant-temperature and constant-density Cloudy models, which recreate the observed H_2 level populations well, to constrain the temperature to a range of 600 to 650 K and the density to n_H = 2.5 x 10^3 to 10^4 cm^-3. The best fit model gives T = 625 K and n_H = 5x10^3 cm^-3. This well constrained warm temperature is consistent with kinetic temperatures found by other studies for the Orion Bar's lower density medium. However, the range of densities well fit by the model grid is marginally lower than those reported by other studies. We could be observing lower density gas than the surrounding medium, or perhaps a density-sensitive parameter in our models is not properly estimated.

astro-ph.GA↗

arXiv2017

Fluorescent H_2 Emission Lines from the Reflection Nebula NGC 7023 Observed with IGRINS

We have analyzed the temperature, velocity and density of H2 gas in NGC 7023 with a high-resolution near-infrared spectrum of the northwestern filament of the reflection nebula. By observing NGC 7023 in the H and K bands at R ~ 45,000 with the Immersion GRating INfrared Spectrograph (IGRINS), we detected 68 H2 emission lines within the 1" x 15" slit. The diagnostic ratios of 2-1 S(1)/1-0 S(1) is 0.41-0.56. In addition, the estimated ortho-to-para ratios (OPR) is 1.63-1.82, indicating that the H2 emission transitions in the observed region arises mostly from gas excited by UV fluorescence. Gradients in the temperature, velocity, and OPR within the observed area imply motion of the photodissociation region (PDR) relative to the molecular cloud. In addition, we derive the column density of H2 from the observed emission lines and compare these results with PDR models in the literature covering a range of densities and incident UV field intensities. The notable difference between PDR model predictions and the observed data, in high rotational J levels of v = 1, is that the predicted formation temperature for newly-formed H2 should be lower than that of the model predictions. To investigate the density distribution, we combine pixels in 1" x 1" areas and derive the density distribution at the 0.002 pc scale. The derived gradient of density suggests that NGC 7023 has a clumpy structure, including a high clump density of ~10^5 cm^-3 with a size smaller than ~5 x 10^-3 pc embedded in lower density regions of 10^3-10^4 cm^-3.

astro-ph.GA↗

arXiv2016

Three-dimensional Shock Structure of Orion KL Outflow with IGRINS

We report a study of the three-dimensional (3D) outflow structure of a 15$\arcsec$ $\times$ 13$\arcsec$ area around H$_{2}$ peak 1 in Orion KL with slit-scan observations (13 slits) using the Immersion Grating Infrared Spectrograph. The datacubes, with high velocity-resolution ($\sim$ 7.5 {\kms}) provide high contrast imaging within ultra-narrow bands, and enable the detection of the main stream of the previously reported H$_{2}$ outflow fingers. We identified 31 distinct fingers in H$_{2}$ 1$-$0 S(1) $λ$2.122 $\micron$ emission. The line profile at each finger shows multiple-velocity peaks with a strong low-velocity component around the systemic velocity at ${\VLSR}$ = $+$8 {\kms} and high velocity emission ($|$${\VLSR}$$|$ = 45$-$135 {\kms}) indicating a typical bow-shock. The observed radial velocity gradients of $\sim$ 4 {\kms} arcsec$^{-1}$ agree well with the velocities inferred from large-scale proper motions, where the projected motion is proportional to distance from a common origin. We construct a conceptual 3D map of the fingers with the estimated inclination angles of 57$\degree$$-$74$\degree$. The extinction difference ($Δ$$A_{\rm v}$ $>$ 10 mag) between blueshifted and redshifted fingers indicates high internal extinction. The extinction, the overall angular spread and scale of the flow argue for an ambient medium with very high density (10$^{5}$$-$10$^{6}$ cm$^{-3}$), consistent with molecular line observations of the OMC core. The radial velocity gradients and the 3D distributions of the fingers together support the hypothesis of simultaneous, radial explosion of the Orion KL outflow.

astro-ph.GA↗

arXiv2016

IGRINS Near-IR High-Resolution Spectroscopy of Multiple Jets around LkH$α$ 234

We present the results of high-resolution near-IR spectroscopy toward the multiple outflows around the Herbig Be star Lk{\Ha} 234 using the Immersion Grating Infrared Spectrograph (IGRINS). Previous studies indicate that the region around Lk{\Ha} 234 is complex, with several embedded YSOs and the outflows associated with them. In simultaneous H$-$ and K$-$band spectra from HH 167, we detected 5 {\FeII} and 14 H$_{2}$ emission lines. We revealed a new {\FeII} jet driven by radio continuum source VLA 3B. Position-velocity diagrams of H$_{2}$ 1$-$0 S(1) $λ$2.122 $\micron$ line show multiple velocity peaks. The kinematics may be explained by a geometrical bow shock model. We detected a component of H$_{2}$ emission at the systemic velocity (V$_{LSR}$ $=$ $-$10.2 {\kms}) along the whole slit in all slit positions, which may arise from the ambient photodissociation region. Low-velocity gas dominates the molecular hydrogen emission from knots A and B in HH 167, which is close to the systemic velocity, {\FeII} emission lines are detected at farther from the systemic velocity, at V$_{LSR}$ $=$ $-$100 $-$ $-$130 {\kms}. We infer that the H$_{2}$ emission arises from shocked gas entrained by a high-velocity outflow. Population diagrams of H$_{2}$ lines imply that the gas is thermalized at a temperature of 2,500 $-$ 3,000 K and the emission results from shock excitation.

astro-ph.SR↗