Searcharxiv⌕ Search

arXiv subjects

Seonghak Kim

Publications and source records attributed to Seonghak Kim.

At least 19 recordsLinked to original sources
arXiv2026

Irregular Diffusions and Loss of Regularity in Polyconvex Gradient Flows

We investigate diffusion-type partial differential equations that are irregular in the sense that they admit weak solutions which are nowhere smooth, even for prescribed smooth data. By reformulating these equations as first-order partial differential relations and adapting the method of convex integration, we develop a construction scheme based on new geometric structures, referred to as $\mathcal{T}_N$-configurations, together with a simplified structural hypothesis on the diffusion functions, termed Condition $O_N$. Under this condition, we show that the associated initial and boundary value problems with certain smooth initial-boundary data admit infinitely many Lipschitz weak solutions that are nowhere $C^1$. We further analyze specific $\mathcal{T}_N$-configurations and establish nondegeneracy conditions that are essential for verifying Condition $O_N$. As an application, we construct examples of strongly polyconvex energy functionals whose gradient flows generate irregular diffusion equations, thereby revealing a failure of regularity and uniqueness even within the class of polyconvex gradient flows.

math.AP↗
arXiv2025

Distilling Balanced Knowledge from a Biased Teacher

Conventional knowledge distillation, designed for model compression, fails on long-tailed distributions because the teacher model tends to be biased toward head classes and provides limited supervision for tail classes. We propose Long-Tailed Knowledge Distillation (LTKD), a novel framework that reformulates the conventional objective into two components: a cross-group loss, capturing mismatches in prediction distributions across class groups (head, medium, and tail), and a within-group loss, capturing discrepancies within each group's distribution. This decomposition reveals the specific sources of the teacher's bias. To mitigate the inherited bias, LTKD introduces (1) a rebalanced cross-group loss that calibrates the teacher's group-level predictions and (2) a reweighted within-group loss that ensures equal contribution from all groups. Extensive experiments on CIFAR-100-LT, TinyImageNet-LT, and ImageNet-LT demonstrate that LTKD significantly outperforms existing methods in both overall and tail-class accuracy, thereby showing its ability to distill balanced knowledge from a biased teacher for real-world applications.

cs.CV↗
arXiv2025

FiGKD: Fine-Grained Knowledge Distillation via High-Frequency Detail Transfer

Knowledge distillation (KD) is a widely adopted technique for transferring knowledge from a high-capacity teacher model to a smaller student model by aligning their output distributions. However, existing methods often underperform in fine-grained visual recognition tasks, where distinguishing subtle differences between visually similar classes is essential. This performance gap stems from the fact that conventional approaches treat the teacher's output logits as a single, undifferentiated signal-assuming all contained information is equally beneficial to the student. Consequently, student models may become overloaded with redundant signals and fail to capture the teacher's nuanced decision boundaries. To address this issue, we propose Fine-Grained Knowledge Distillation (FiGKD), a novel frequency-aware framework that decomposes a model's logits into low-frequency (content) and high-frequency (detail) components using the discrete wavelet transform (DWT). FiGKD selectively transfers only the high-frequency components, which encode the teacher's semantic decision patterns, while discarding redundant low-frequency content already conveyed through ground-truth supervision. Our approach is simple, architecture-agnostic, and requires no access to intermediate feature maps. Extensive experiments on CIFAR-100, TinyImageNet, and multiple fine-grained recognition benchmarks show that FiGKD consistently outperforms state-of-the-art logit-based and feature-based distillation methods across a variety of teacher-student configurations. These findings confirm that frequency-aware logit decomposition enables more efficient and effective knowledge transfer, particularly in resource-constrained settings.

cs.CV↗
arXiv2024

Convergence in the incompressible limit of the corner singularities

In this paper, we treat the corner singularity expansion and its convergence result regarding the penalized system obtained by eliminating the pressure variable in the Stokes problem of incompressible flow. The penalized problem is a kind of the Lam\'{e} system, so we first discuss the corner singularity theory of the Lam\'{e} system with inhomogeneous Dirichlet boundary condition on a non-convex polygon. Considering the inhomogeneous condition, we show the decomposition of its solution, composed of singular parts and a smoother remainder near a re-entrant corner, and furthermore, we provide the explicit formulae of coefficients in singular parts. In particular, these formulae can be used in the development of highly accurate numerical scheme. In addition, we formulate coefficients in singular parts regarding the Stokes equations with inhomogeneous boundary condition and non-divergence-free property of velocity field, and thus we show the convergence results of coefficients in singular parts and remainder regarding the concerned penalized problem.

math.AP↗
arXiv2024

Adhesion and volume filling in one-dimensional population dynamics under no-flux boundary condition

We study the (generalized) one-dimensional population model developed by Anguige \& Schmeiser [1], which reflects cell-cell adhesion and volume filling under no-flux boundary condition. In this generalized model, depending on the adhesion and volume filling parameters $\alpha,\beta\in[0,1],$ the resulting equation is classified into six types. Among these, we focus on the type exhibiting strong effects of both adhesion and volume filling, which results in a class of advection-diffusion equations of the forward-backward-forward type. For five distinct cases of initial maximum, minimum and average population densities, we derive the corresponding patterns for the global behavior of weak solutions to the initial and no-flux boundary value problem. Due to the presence of a negative diffusion regime, we indeed prove that the problem is ill-posed and admits infinitely many global-in-time weak solutions, with the exception of one specific case of the initial datum. This nonuniqueness is inherent in the method of convex integration that we use to solve the Dirichlet problem of a partial differential inclusion arising from the ill-posed problem.

math.AP↗
arXiv2024

Adhesion and volume filling in one-dimensional population dynamics under Dirichlet boundary condition

We generalize the one-dimensional population model of Anguige \& Schmeiser [1] reflecting the cell-to-cell adhesion and volume filling and classify the resulting equation into the six types. Among these types, we fix one that yields a class of advection-diffusion equations of forward-backward-forward type and prove the existence of infinitely many global-in-time weak solutions to the initial-Dirichlet boundary value problem when the maximum value of an initial population density exceeds a certain threshold. Such solutions are extracted from the method of convex integration by M\"uller \& \v Sver\'ak [12]; they exhibit fine-scale density mixtures over a finite time interval, then become smooth and identical, and decay exponentially and uniformly to zero as time approaches infinity.

math.AP↗
arXiv2023

On integral convexity, variational solutions and nonlinear semigroups

In this paper we provide a different approach for existence of the variational solutions of the gradient flows associated to functionals on Sobolev spaces studied in \cite{BDDMS20}. The crucial condition is the convexity of the functional under which we show that the variational solutions coincide with the solutions generated by the nonlinear semigroup associated to the functional. For integral functionals of the form $\mathbf F(u)=\int_\Omega f(x,Du(x)) dx,$ where $f(x,\xi)$ is $C^1$ in $\xi$, we also make some remarks on the connections between convexity of $\mathbf F$ (called the integral convexity of $f$) and certain monotonicity conditions of the gradient map $D_\xi f.$ In particular, we provide an example to show that even for functions of the simple form $f=f(\xi)$, the usual quasimonotonicity of $D_\xi f$ is not sufficient for the integral convexity of $f.$

math.AP↗
arXiv2023

Cosine Similarity Knowledge Distillation for Individual Class Information Transfer

Previous logits-based Knowledge Distillation (KD) have utilized predictions about multiple categories within each sample (i.e., class predictions) and have employed Kullback-Leibler (KL) divergence to reduce the discrepancy between the student and teacher predictions. Despite the proliferation of KD techniques, the student model continues to fall short of achieving a similar level as teachers. In response, we introduce a novel and effective KD method capable of achieving results on par with or superior to the teacher models performance. We utilize teacher and student predictions about multiple samples for each category (i.e., batch predictions) and apply cosine similarity, a commonly used technique in Natural Language Processing (NLP) for measuring the resemblance between text embeddings. This metric's inherent scale-invariance property, which relies solely on vector direction and not magnitude, allows the student to dynamically learn from the teacher's knowledge, rather than being bound by a fixed distribution of the teacher's knowledge. Furthermore, we propose a method called cosine similarity weighted temperature (CSWT) to improve the performance. CSWT reduces the temperature scaling in KD when the cosine similarity between the student and teacher models is high, and conversely, it increases the temperature scaling when the cosine similarity is low. This adjustment optimizes the transfer of information from the teacher to the student model. Extensive experimental results show that our proposed method serves as a viable alternative to existing methods. We anticipate that this approach will offer valuable insights for future research on model compression.

cs.CV↗
arXiv2023

Maximizing Discrimination Capability of Knowledge Distillation with Energy Function

To apply the latest computer vision techniques that require a large computational cost in real industrial applications, knowledge distillation methods (KDs) are essential. Existing logit-based KDs apply the constant temperature scaling to all samples in dataset, limiting the utilization of knowledge inherent in each sample individually. In our approach, we classify the dataset into two categories (i.e., low energy and high energy samples) based on their energy score. Through experiments, we have confirmed that low energy samples exhibit high confidence scores, indicating certain predictions, while high energy samples yield low confidence scores, meaning uncertain predictions. To distill optimal knowledge by adjusting non-target class predictions, we apply a higher temperature to low energy samples to create smoother distributions and a lower temperature to high energy samples to achieve sharper distributions. When compared to previous logit-based and feature-based methods, our energy-based KD (Energy KD) achieves better performance on various datasets. Especially, Energy KD shows significant improvements on CIFAR-100-LT and ImageNet datasets, which contain many challenging samples. Furthermore, we propose high energy-based data augmentation (HE-DA) for further improving the performance. We demonstrate that higher performance improvement could be achieved by augmenting only a portion of the dataset rather than the entire dataset, suggesting that it can be employed on resource-limited devices. To the best of our knowledge, this paper represents the first attempt to make use of energy function in knowledge distillation and data augmentation, and we believe it will greatly contribute to future research.

cs.CV↗
arXiv2023

Robustness-Reinforced Knowledge Distillation with Correlation Distance and Network Pruning

The improvement in the performance of efficient and lightweight models (i.e., the student model) is achieved through knowledge distillation (KD), which involves transferring knowledge from more complex models (i.e., the teacher model). However, most existing KD techniques rely on Kullback-Leibler (KL) divergence, which has certain limitations. First, if the teacher distribution has high entropy, the KL divergence's mode-averaging nature hinders the transfer of sufficient target information. Second, when the teacher distribution has low entropy, the KL divergence tends to excessively focus on specific modes, which fails to convey an abundant amount of valuable knowledge to the student. Consequently, when dealing with datasets that contain numerous confounding or challenging samples, student models may struggle to acquire sufficient knowledge, resulting in subpar performance. Furthermore, in previous KD approaches, we observed that data augmentation, a technique aimed at enhancing a model's generalization, can have an adverse impact. Therefore, we propose a Robustness-Reinforced Knowledge Distillation (R2KD) that leverages correlation distance and network pruning. This approach enables KD to effectively incorporate data augmentation for performance improvement. Extensive experiments on various datasets, including CIFAR-100, FGVR, TinyImagenet, and ImageNet, demonstrate our method's superiority over current state-of-the-art methods.

cs.CV↗
arXiv2023

Time splitting method for nonlinear Schr\"odinger equation with rough initial data in $L^2$

We establish convergence results related to the operator splitting scheme on the Cauchy problem for the nonlinear Schr\"odinger equation with rough initial data in $L^2$, $$ \left\{ \begin{array}{ll} i\partial_t u +\Delta u = \lambda |u|^{p} u, & (x,t) \in \mathbb{R}^d \times \mathbb{R}_+, u (x,0) =\phi (x), & x\in\mathbb{R}^d, \end{array} \right. $$ where $\lambda \in \{-1,1\}$ and $p >0$. While the Lie approximation $Z_L$ is known to converge to the solution $u$ when the initial datum $\phi$ is sufficiently smooth, the convergence result for rough initial data is open to question. In this paper, for rough initial data $\phi\in L^2 (\mathbb{R}^d)$, we prove the $L^2$ convergence of the filtered Lie approximation $Z_{flt}$ to the solution $u$ in the mass-subcritical range, $0< p < \frac{4}{d}$. Furthermore, we provide a precise convergence result for radial initial data $\phi\in L^2 (\mathbb{R}^d)$.

math.NA↗
arXiv2020

A$^3$: Accelerating Attention Mechanisms in Neural Networks with Approximation

With the increasing computational demands of neural networks, many hardware accelerators for the neural networks have been proposed. Such existing neural network accelerators often focus on popular neural network types such as convolutional neural networks (CNNs) and recurrent neural networks (RNNs); however, not much attention has been paid to attention mechanisms, an emerging neural network primitive that enables neural networks to retrieve most relevant information from a knowledge-base, external memory, or past states. The attention mechanism is widely adopted by many state-of-the-art neural networks for computer vision, natural language processing, and machine translation, and accounts for a large portion of total execution time. We observe today's practice of implementing this mechanism using matrix-vector multiplication is suboptimal as the attention mechanism is semantically a content-based search where a large portion of computations ends up not being used. Based on this observation, we design and architect A3, which accelerates attention mechanisms in neural networks with algorithmic approximation and hardware specialization. Our proposed accelerator achieves multiple orders of magnitude improvement in energy efficiency (performance/watt) as well as substantial speedup over the state-of-the-art conventional hardware.

cs.DC↗
arXiv2017

Strichartz estimates for the magnetic Schrödinger equation with potentials $V$ of critical decay

We study the Strichartz estimates for the magnetic Schrödinger equation in dimension $n\geq3$. More specifically, for all Schrödinger admissible pairs $(r,q)$, we establish the estimate $$ \|e^{itH}f\|_{L^{q}_{t}(\mathbb{R}; L^{r}_{x}(\mathbb{R}^n))} \leq C_{n,r,q,H} \|f\|_{L^2(\mathbb{R}^n)} $$ when the operator $H= -Δ_A +V$ satisfies suitable conditions. In the purely electric case $A\equiv0$, we extend the class of potentials $V$ to the Fefferman-Phong class. In doing so, we apply a weighted estimate for the Schrödinger equation developed by Ruiz and Vega. Moreover, for the endpoint estimate of the magnetic case in $\mathbb{R}^3$, we investigate an equivalence $$ \| H^{\frac{1}{4}} f \|_{L^r(\mathbb{R}^3)} \approx C_{H,r} \big\| (-Δ)^{\frac{1}{4}} f \big\|_{L^r(\mathbb{R}^3)} $$ and find sufficient conditions on $H$ and $r$ for which the equivalence holds.

math.AP↗
arXiv2016

Non-parabolic diffusion problems in one space dimension

We study some non-parabolic diffusion problems in one-space dimension, where the diffusion flux exhibits forward and backward nature of the Perona-Malik, Höllig or non-Fourier type. Classical weak solutions to such problems are constructed in a way to capture some expected and unexpected properties, including anomalous asymptotic behaviors and energy dissipation or allocation. Specific properties of solutions will depend on the type of the diffusion flux, but the primary method of our study relies on reformulating diffusion equations involved as an inhomogeneous partial differential inclusion and on constructing solutions from the differential inclusion by a combination of the convex integration and Baire's category methods. In doing so, we introduce the appropriate notion of subsolutions of the partial differential inclusion and their transition gauge, which plays a pivotal role in dealing with some specific features of the constructed weak solutions.

math.AP↗
arXiv2016

Two-phase forward solutions for one-dimensional forward-backward parabolic equations with linear convection and reaction

We study the existence and properties of Lipschitz continuous weak solutions to the Neumann boundary value problem for a class of one-dimensional quasilinear forward-backward diffusion equations with linear convection and reaction. The diffusion flux function is assumed to be of a forward-backward type that contains two forward-diffusion phases. We prove that, for all smooth initial data, there exists at least one weak solution whose spatial derivative stays in the two forward phases. Also, for all smooth initial data that have a derivative value lying in a certain phase transition range, we show that there exist infinitely many such solutions that exhibit instantaneous phase transitions between the two forward phases. Moreover, we introduce the notion of transition gauge for such forward solutions and prove that the gauge of all constructed two-phase forward solutions can be arbitrarily close to a certain fixed constant.

math.AP↗
arXiv2016

Convex integration with linear constraints and its applications

We study solutions of the first order partial differential inclusions of the form $\nabla u\in K$, where $u:Ω\subset\mathbb{R}^n\to\mathbb{R}^m$ and $K$ is a set of $m\times n$ real matrices, and derive a companion version to the result of {Müller and Šverák} [20], concerning a general linear constraint on the components of $\nabla u$. We then consider two applications: the vectorial eikonal equation and a $T_4$-configuration both under linear constraints.

math.AP↗