arXiv subjects

Jungwon Lee

Publications and source records attributed to Jungwon Lee.

At least 19 recordsLinked to original sources

arXiv2025

Asymptotic statistics for finite continued fractions with restricted digits

Zaremba's conjecture concerns a formation of continued fraction expansions for rational numbers with partial quotient bounded by an absolute constant. We present asymptotic estimates for the size of $\epsilon$-thickening of certain fractal sets of bounded-type, which in turn provide a remark on Zaremba's conjecture in an averaging sense. We also discuss a generalisation for complex continued fractions over imaginary quadratic fields.

math.NT↗

arXiv2025

Mahler measure, motivic regulators and Dirichlet $L$-values

Inspired by the work of Deninger, we present a formula that relates the Mahler measure of a two-variable variant of cyclotomic polynomial to regulator of class in motivic cohomology associated to cyclotomic fields and linear combination of special values of the derivative of Dirichlet $L$-functions. The formula is derived by studying the Beilinson regulator map applied to systematically constructed elements in the motivic cohomology group. Under linear independence hypothesis on the derivative of partial Dirichlet $L$-values at $s=0$ and $-1$, we study a Galois module structure of the relevant motivic cohomology and obtain the refined identity for a single $L$-value.

math.NT↗

arXiv2025

Multidimensional statistics for finite orbits of generalised continued fractions

We statistically compare the relationships between frequencies of digits in continued fraction expansions of typical rational points in the unit interval and higher dimensional generalisations. This takes the form of a Large Deviation and Central Limit Theorem, including multidimensional results for random vectors. These results apply to classical multidimensional continued fraction transformations including Brun's algorithm and the Jacobi--Perron algorithm, and more generally for maps satisfying mild contraction hypothesis on the inverse branches. We prove in particular that the finite trajectories capture the generic ergodic behaviour of infinite trajectories.

math.DS↗

arXiv2024

Euclidean algorithms are Gaussian over imaginary quadratic fields

The distributional analysis of Euclidean algorithms was carried out by Baladi and Vall\'{e}e. They showed the asymptotic normality of the number of division steps and associated costs in the Euclidean algorithm as a random variable on the set of rational numbers with bounded denominator based on the transfer operator methods. We extend their result to the Euclidean algorithm over appropriate imaginary quadratic fields by studying dynamics of the nearest integer complex continued fraction map, which is piecewise analytic and expanding but not a full branch map. By observing a finite Markov partition with a regular CW-structure, which enables us to associate the transfer operator acting on a direct sum of spaces of $C^1$-functions, we obtain the limit Gaussian distribution as well as residual equidistribution.

math.DS↗

arXiv2023

FRDiff : Feature Reuse for Universal Training-free Acceleration of Diffusion Models

The substantial computational costs of diffusion models, especially due to the repeated denoising steps necessary for high-quality image generation, present a major obstacle to their widespread adoption. While several studies have attempted to address this issue by reducing the number of score function evaluations (NFE) using advanced ODE solvers without fine-tuning, the decreased number of denoising iterations misses the opportunity to update fine details, resulting in noticeable quality degradation. In our work, we introduce an advanced acceleration technique that leverages the temporal redundancy inherent in diffusion models. Reusing feature maps with high temporal similarity opens up a new opportunity to save computation resources without compromising output quality. To realize the practical benefits of this intuition, we conduct an extensive analysis and propose a novel method, FRDiff. FRDiff is designed to harness the advantages of both reduced NFE and feature reuse, achieving a Pareto frontier that balances fidelity and latency trade-offs in various generative tasks.

cs.CV↗

arXiv2023

Dynamics of Ostrowski skew-product: I. Limit laws and Hausdorff dimensions

We present a dynamical and spectral study of Ostrowski's map based on the use of transfer operators. The Ostroswki dynamical system is obtained as a skew-product of the Gauss map (it has the Gauss map as a base and interval fibers) and produces expansions of real numbers with respect to an irrational base given by continued fractions. By studying spectral properties of the associated transfer operator, we show that the Ostroswki dynamical system admits an absolutely continuous invariant measure with piecewise holomorphic density and exponential mixing properties. We deduce a central limit theorem for associated random variables of an arithmetic nature and motivated by applications in inhomogeneous Diophantine approximation, we get Bowen--Ruelle type implicit estimates in terms of spectral elements for the Hausdorff dimension of a certain bounded digit set.

math.DS↗

arXiv2023

Temporal Dynamic Quantization for Diffusion Models

The diffusion model has gained popularity in vision applications due to its remarkable generative performance and versatility. However, high storage and computation demands, resulting from the model size and iterative generation, hinder its use on mobile devices. Existing quantization techniques struggle to maintain performance even in 8-bit precision due to the diffusion model's unique property of temporal variation in activation. We introduce a novel quantization method that dynamically adjusts the quantization interval based on time step information, significantly improving output quality. Unlike conventional dynamic quantization techniques, our approach has no computational overhead during inference and is compatible with both post-training quantization (PTQ) and quantization-aware training (QAT). Our extensive experiments demonstrate substantial improvements in output quality with the quantized diffusion model across various datasets.

cs.CV↗

arXiv2023

End-to-End Multi-Task Denoising for joint SDR and PESQ Optimization

Supervised learning based on a deep neural network recently has achieved substantial improvement on speech enhancement. Denoising networks learn mapping from noisy speech to clean one directly, or to a spectrum mask which is the ratio between clean and noisy spectra. In either case, the network is optimized by minimizing mean square error (MSE) between ground-truth labels and time-domain or spectrum output. However, existing schemes have either of two critical issues: spectrum and metric mismatches. The spectrum mismatch is a well known issue that any spectrum modification after short-time Fourier transform (STFT), in general, cannot be fully recovered after inverse short-time Fourier transform (ISTFT). The metric mismatch is that a conventional MSE metric is sub-optimal to maximize our target metrics, signal-to-distortion ratio (SDR) and perceptual evaluation of speech quality (PESQ). This paper presents a new end-to-end denoising framework with the goal of joint SDR and PESQ optimization. First, the network optimization is performed on the time-domain signals after ISTFT to avoid spectrum mismatch. Second, two loss functions which have improved correlations with SDR and PESQ metrics are proposed to minimize metric mismatch. The experimental result showed that the proposed denoising scheme significantly improved both SDR and PESQ performance over the existing methods.

cs.SD↗

arXiv2022

Zero-Shot Learning of a Conditional Generative Adversarial Network for Data-Free Network Quantization

We propose a novel method for training a conditional generative adversarial network (CGAN) without the use of training data, called zero-shot learning of a CGAN (ZS-CGAN). Zero-shot learning of a conditional generator only needs a pre-trained discriminative (classification) model and does not need any training data. In particular, the conditional generator is trained to produce labeled synthetic samples whose characteristics mimic the original training data by using the statistics stored in the batch normalization layers of the pre-trained model. We show the usefulness of ZS-CGAN in data-free quantization of deep neural networks. We achieved the state-of-the-art data-free network quantization of the ResNet and MobileNet classification models trained on the ImageNet dataset. Data-free quantization using ZS-CGAN showed a minimal loss in accuracy compared to that obtained by conventional data-dependent quantization.

cs.CV↗

arXiv2022

Learning with Succinct Common Representation Based on Wyner's Common Information

A new bimodal generative model is proposed for generating conditional and joint samples, accompanied with a training method with learning a succinct bottleneck representation. The proposed model, dubbed as the variational Wyner model, is designed based on two classical problems in network information theory -- distributed simulation and channel synthesis -- in which Wyner's common information arises as the fundamental limit on the succinctness of the common representation. The model is trained by minimizing the symmetric Kullback--Leibler divergence between variational and model distributions with regularization terms for common information, reconstruction consistency, and latent space matching terms, which is carried out via an adversarial density ratio estimation technique. The utility of the proposed approach is demonstrated through experiments for joint and conditional generation with synthetic and real-world datasets, as well as a challenging zero-shot image retrieval task.

cs.LG↗

arXiv2022

An ergodic approach towards an equidistribution result of Ferrero--Washington

An important ingredient in the Ferrero--Washington proof of the vanishing of cyclotomic $\mu$-invariant for Kubota--Leopoldt $p$-adic $L$-functions is an equidistribution result which they established using the Weyl criterion. The purpose of our manuscript is to provide an alternative proof by adopting a dynamical approach. A key ingredient to our methods is studying an ergodic skew-product map on $\mathbb{Z}_p \times [0,1]$, which is then suitably identified as a factor of the $2$-sided Bernoulli shift on the sample space $\{0,1,2,\cdots,p-1\}^{\mathbb{Z}}$.

math.NT↗

arXiv2021

Dual-Teacher Class-Incremental Learning With Data-Free Generative Replay

This paper proposes two novel knowledge transfer techniques for class-incremental learning (CIL). First, we propose data-free generative replay (DF-GR) to mitigate catastrophic forgetting in CIL by using synthetic samples from a generative model. In the conventional generative replay, the generative model is pre-trained for old data and shared in extra memory for later incremental learning. In our proposed DF-GR, we train a generative model from scratch without using any training data, based on the pre-trained classification model from the past, so we curtail the cost of sharing pre-trained generative models. Second, we introduce dual-teacher information distillation (DT-ID) for knowledge distillation from two teachers to one student. In CIL, we use DT-ID to learn new classes incrementally based on the pre-trained model for old classes and another model (pre-)trained on the new data for new classes. We implemented the proposed schemes on top of one of the state-of-the-art CIL methods and showed the performance improvement on CIFAR-100 and ImageNet datasets.

cs.CV↗

arXiv2021

Towards Fair Federated Learning with Zero-Shot Data Augmentation

Federated learning has emerged as an important distributed learning paradigm, where a server aggregates a global model from many client-trained models while having no access to the client data. Although it is recognized that statistical heterogeneity of the client local data yields slower global model convergence, it is less commonly recognized that it also yields a biased federated global model with a high variance of accuracy across clients. In this work, we aim to provide federated learning schemes with improved fairness. To tackle this challenge, we propose a novel federated learning system that employs zero-shot data augmentation on under-represented data to mitigate statistical heterogeneity and encourage more uniform accuracy performance across clients in federated networks. We study two variants of this scheme, Fed-ZDAC (federated learning with zero-shot data augmentation at the clients) and Fed-ZDAS (federated learning with zero-shot data augmentation at the server). Empirical results on a suite of datasets demonstrate the effectiveness of our methods on simultaneously improving the test accuracy and fairness.

cs.CV↗

arXiv2020

HyperCon: Image-To-Video Model Transfer for Video-To-Video Translation Tasks

Video-to-video translation is more difficult than image-to-image translation due to the temporal consistency problem that, if unaddressed, leads to distracting flickering effects. Although video models designed from scratch produce temporally consistent results, training them to match the vast visual knowledge captured by image models requires an intractable number of videos. To combine the benefits of image and video models, we propose an image-to-video model transfer method called Hyperconsistency (HyperCon) that transforms any well-trained image model into a temporally consistent video model without fine-tuning. HyperCon works by translating a temporally interpolated video frame-wise and then aggregating over temporally localized windows on the interpolated video. It handles both masked and unmasked inputs, enabling support for even more video-to-video translation tasks than prior image-to-video model transfer techniques. We demonstrate HyperCon on video style transfer and inpainting, where it performs favorably compared to prior state-of-the-art methods without training on a single stylized or incomplete video. Our project website is available at https://ryanszeto.com/projects/hypercon .

cs.CV↗

arXiv2020

Learning Sparse Low-Precision Neural Networks With Learnable Regularization

We consider learning deep neural networks (DNNs) that consist of low-precision weights and activations for efficient inference of fixed-point operations. In training low-precision networks, gradient descent in the backward pass is performed with high-precision weights while quantized low-precision weights and activations are used in the forward pass to calculate the loss function for training. Thus, the gradient descent becomes suboptimal, and accuracy loss follows. In order to reduce the mismatch in the forward and backward passes, we utilize mean squared quantization error (MSQE) regularization. In particular, we propose using a learnable regularization coefficient with the MSQE regularizer to reinforce the convergence of high-precision weights to their quantized values. We also investigate how partial L2 regularization can be employed for weight pruning in a similar manner. Finally, combining weight pruning, quantization, and entropy coding, we establish a low-precision DNN compression pipeline. In our experiments, the proposed method yields low-precision MobileNet and ShuffleNet models on ImageNet classification with the state-of-the-art compression ratios of 7.13 and 6.79, respectively. Moreover, we examine our method for image super resolution networks to produce 8-bit low-precision models at negligible performance loss.

cs.CV↗

arXiv2020

NTIRE 2020 Challenge on Real Image Denoising: Dataset, Methods and Results

This paper reviews the NTIRE 2020 challenge on real image denoising with focus on the newly introduced dataset, the proposed methods and their results. The challenge is a new version of the previous NTIRE 2019 challenge on real image denoising that was based on the SIDD benchmark. This challenge is based on a newly collected validation and testing image datasets, and hence, named SIDD+. This challenge has two tracks for quantitatively evaluating image denoising performance in (1) the Bayer-pattern rawRGB and (2) the standard RGB (sRGB) color spaces. Each track ~250 registered participants. A total of 22 teams, proposing 24 methods, competed in the final phase of the challenge. The proposed methods by the participating teams represent the current state-of-the-art performance in image denoising targeting real noisy images. The newly collected SIDD+ datasets are publicly available at: https://bit.ly/siddplus_data.

cs.CV↗

arXiv2020

Data-Free Network Quantization With Adversarial Knowledge Distillation

Network quantization is an essential procedure in deep learning for development of efficient fixed-point inference models on mobile or edge platforms. However, as datasets grow larger and privacy regulations become stricter, data sharing for model compression gets more difficult and restricted. In this paper, we consider data-free network quantization with synthetic data. The synthetic data are generated from a generator, while no data are used in training the generator and in quantization. To this end, we propose data-free adversarial knowledge distillation, which minimizes the maximum distance between the outputs of the teacher and the (quantized) student for any adversarial samples from a generator. To generate adversarial samples similar to the original data, we additionally propose matching statistics from the batch normalization layers for generated data and the original data in the teacher. Furthermore, we show the gain of producing diverse adversarial samples by using multiple generators and multiple students. Our experiments show the state-of-the-art data-free model compression and quantization results for (wide) residual networks and MobileNet on SVHN, CIFAR-10, CIFAR-100, and Tiny-ImageNet datasets. The accuracy losses compared to using the original datasets are shown to be very minimal.

cs.CV↗

arXiv2020

End-to-End Multi-Task Denoising for the Joint Optimization of Perceptual Speech Metrics

Although supervised learning based on a deep neural network has recently achieved substantial improvement on speech enhancement, the existing schemes have either of two critical issues: spectrum or metric mismatches. The spectrum mismatch is a well known issue that any spectrum modification after short-time Fourier transform (STFT), in general, cannot be fully recovered after inverse short-time Fourier transform (ISTFT). The metric mismatch is that a conventional mean square error (MSE) loss function is typically sub-optimal to maximize perceptual speech measure such as signal-to-distortion ratio (SDR), perceptual evaluation of speech quality (PESQ) and short-time objective intelligibility (STOI). This paper presents a new end-to-end denoising framework. First, the network optimization is performed on the time-domain signals after ISTFT to avoid the spectrum mismatch. Second, three loss functions based on SDR, PESQ and STOI are proposed to minimize the metric mismatch. The experimental result showed the proposed denoising scheme significantly improved SDR, PESQ and STOI performance over the existing methods. Moreover, the proposed scheme also provided good generalization performance over generative denoising models on the perceptual speech metrics not used as a loss function during training.

cs.SD↗