arXiv subjects

Sanghyun Park

Publications and source records attributed to Sanghyun Park.

At least 37 records · Page 2Linked to original sources

arXiv2025

Advanced Knowledge Transfer: Refined Feature Distillation for Zero-Shot Quantization in Edge Computing

We introduce AKT (Advanced Knowledge Transfer), a novel method to enhance the training ability of low-bit quantized (Q) models in the field of zero-shot quantization (ZSQ). Existing research in ZSQ has focused on generating high-quality data from full-precision (FP) models. However, these approaches struggle with reduced learning ability in low-bit quantization due to its limited information capacity. To overcome this limitation, we propose effective training strategy compared to data generation. Particularly, we analyzed that refining feature maps in the feature distillation process is an effective way to transfer knowledge to the Q model. Based on this analysis, AKT efficiently transfer core information from the FP model to the Q model. AKT is the first approach to utilize both spatial and channel attention information in feature distillation in ZSQ. Our method addresses the fundamental gradient exploding problem in low-bit Q models. Experiments on CIFAR-10 and CIFAR-100 datasets demonstrated the effectiveness of the AKT. Our method led to significant performance enhancement in existing generative models. Notably, AKT achieved significant accuracy improvements in low-bit Q models, achieving state-of-the-art in the 3,5bit scenarios on CIFAR-10. The code is available at https://github.com/Inpyo-Hong/AKT-Advanced-knowledge-Transfer.

cs.CV↗

arXiv2025

Real-Time Sleepiness Detection for Driver State Monitoring System

A driver face monitoring system can detect driver fatigue, which is a significant factor in many accidents, using computer vision techniques. In this paper, we present a real-time technique for driver eye state detection. First, the face is detected, and the eyes are located within the face region for tracking. A normalized cross-correlation-based online dynamic template matching technique, combined with Kalman filter tracking, is proposed to track the detected eye positions in subsequent image frames. A support vector machine with histogram of oriented gradients (HOG) features is used to classify the state of the eyes as open or closed. If the eyes remain closed for a specified period, the driver is considered to be asleep, and an alarm is triggered.

cs.CV↗

arXiv2025

Style Extraction on Text Embeddings Using VAE and Parallel Dataset

This study investigates the stylistic differences among various Bible translations using a Variational Autoencoder (VAE) model. By embedding textual data into high-dimensional vectors, the study aims to detect and analyze stylistic variations between translations, with a specific focus on distinguishing the American Standard Version (ASV) from other translations. The results demonstrate that each translation exhibits a unique stylistic distribution, which can be effectively identified using the VAE model. These findings suggest that the VAE model is proficient in capturing and differentiating textual styles, although it is primarily optimized for distinguishing a single style. The study highlights the model's potential for broader applications in AI-based text generation and stylistic analysis, while also acknowledging the need for further model refinement to address the complexity of multi-dimensional stylistic relationships. Future research could extend this methodology to other text domains, offering deeper insights into the stylistic features embedded within various types of textual data.

cs.CL↗

arXiv2025

TinyThinker: Distilling Reasoning through Coarse-to-Fine Knowledge Internalization with Self-Reflection

Large Language Models exhibit impressive reasoning capabilities across diverse tasks, motivating efforts to distill these capabilities into smaller models through generated reasoning data. However, direct training on such synthesized reasoning data may lead to superficial imitation of reasoning process, rather than fostering a genuine integration of reasoning capabilities with underlying knowledge. To address this, we propose TinyThinker, a framework introducing two novel approaches. First, we introduce a three-stage process that incrementally guides the student model through the reasoning process, progressively refining knowledge from coarse to fine granularity. Second, we develop a two-phase training framework comprising an initial reasoning acquisition phase followed by a self-reflection phase utilizing self-generated data. Experiments on commonsense reasoning benchmarks demonstrate that TinyThinker achieves superior performance compared to baselines. Ablation studies further validate the effectiveness of each component in our framework. We expect that TinyThinker can be extended to other knowledge-intensive reasoning tasks, offering an alternative strategy for developing effective reasoning capabilities in smaller language models. Codes are available at https://github.com/shengminp/TinyThinker

cs.CL↗

arXiv2024

Mitigating Adversarial Attacks in LLMs through Defensive Suffix Generation

Large language models (LLMs) have exhibited outstanding performance in natural language processing tasks. However, these models remain susceptible to adversarial attacks in which slight input perturbations can lead to harmful or misleading outputs. A gradient-based defensive suffix generation algorithm is designed to bolster the robustness of LLMs. By appending carefully optimized defensive suffixes to input prompts, the algorithm mitigates adversarial influences while preserving the models' utility. To enhance adversarial understanding, a novel total loss function ($L_{\text{total}}$) combining defensive loss ($L_{\text{def}}$) and adversarial loss ($L_{\text{adv}}$) generates defensive suffixes more effectively. Experimental evaluations conducted on open-source LLMs such as Gemma-7B, mistral-7B, Llama2-7B, and Llama2-13B show that the proposed method reduces attack success rates (ASR) by an average of 11\% compared to models without defensive suffixes. Additionally, the perplexity score of Gemma-7B decreased from 6.57 to 3.93 when applying the defensive suffix generated by openELM-270M. Furthermore, TruthfulQA evaluations demonstrate consistent improvements with Truthfulness scores increasing by up to 10\% across tested configurations. This approach significantly enhances the security of LLMs in critical applications without requiring extensive retraining.

cs.CV↗

arXiv2024

Multi-Response Preference Optimization with Augmented Ranking Dataset

Recent advancements in Large Language Models (LLMs) have been remarkable, with new models consistently surpassing their predecessors. These advancements are underpinned by extensive research on various training mechanisms. Among these, Preference Optimization has played a significant role in improving the performance of LLMs by incorporating human preferences into the training process. However, constructing preference optimization datasets is challenging and the optimization process is highly sensitive to the dataset quality. In this study, we propose a novel approach to augment Preference Optimization datasets. Additionally, we introduce a Multi-response-based Preference Optimization training method that enables the simultaneous learning of multiple responses.

cs.CL↗

arXiv2024

MRNet: Multifaceted Resilient Networks for Medical Image-to-Image Translation

We propose a Multifaceted Resilient Network(MRNet), a novel architecture developed for medical image-to-image translation that outperforms state-of-the-art methods in MRI-to-CT and MRI-to-MRI conversion. MRNet leverages the Segment Anything Model (SAM) to exploit frequency-based features to build a powerful method for advanced medical image transformation. The architecture extracts comprehensive multiscale features from diverse datasets using a powerful SAM image encoder and performs resolution-aware feature fusion that consistently integrates U-Net encoder outputs with SAM-derived features. This fusion optimizes the traditional U-Net skip connection while leveraging transformer-based contextual analysis. The translation is complemented by an innovative dual-mask configuration incorporating dynamic attention patterns and a specialized loss function designed to address regional mapping mismatches, preserving both the gross anatomy and tissue details. Extensive validation studies have shown that MRNet outperforms state-of-the-art architectures, particularly in maintaining anatomical fidelity and minimizing translation artifacts.

eess.IV↗

arXiv2024

VideoPatchCore: An Effective Method to Memorize Normality for Video Anomaly Detection

Video anomaly detection (VAD) is a crucial task in video analysis and surveillance within computer vision. Currently, VAD is gaining attention with memory techniques that store the features of normal frames. The stored features are utilized for frame reconstruction, identifying an abnormality when a significant difference exists between the reconstructed and input frames. However, this approach faces several challenges due to the simultaneous optimization required for both the memory and encoder-decoder model. These challenges include increased optimization difficulty, complexity of implementation, and performance variability depending on the memory size. To address these challenges,we propose an effective memory method for VAD, called VideoPatchCore. Inspired by PatchCore, our approach introduces a structure that prioritizes memory optimization and configures three types of memory tailored to the characteristics of video data. This method effectively addresses the limitations of existing memory-based methods, achieving good performance comparable to state-of-the-art methods. Furthermore, our method requires no training and is straightforward to implement, making VAD tasks more accessible. Our code is available online at github.com/SkiddieAhn/Paper-VideoPatchCore.

cs.CV↗

arXiv2024

SPIN: SE(3)-Invariant Physics Informed Network for Binding Affinity Prediction

Accurate prediction of protein-ligand binding affinity is crucial for rapid and efficient drug development. Recently, the importance of predicting binding affinity has led to increased attention on research that models the three-dimensional structure of protein-ligand complexes using graph neural networks to predict binding affinity. However, traditional methods often fail to accurately model the complex's spatial information or rely solely on geometric features, neglecting the principles of protein-ligand binding. This can lead to overfitting, resulting in models that perform poorly on independent datasets and ultimately reducing their usefulness in real drug development. To address this issue, we propose SPIN, a model designed to achieve superior generalization by incorporating various inductive biases applicable to this task, beyond merely training on empirical data from datasets. For prediction, we defined two types of inductive biases: a geometric perspective that maintains consistent binding affinity predictions regardless of the complexs rotations and translations, and a physicochemical perspective that necessitates minimal binding free energy along their reaction coordinate for effective protein-ligand binding. These prior knowledge inputs enable the SPIN to outperform comparative models in benchmark sets such as CASF-2016 and CSAR HiQ. Furthermore, we demonstrated the practicality of our model through virtual screening experiments and validated the reliability and potential of our proposed model based on experiments assessing its interpretability.

cs.LG↗

arXiv2024

NOTE: Notable generation Of patient Text summaries through Efficient approach based on direct preference optimization

The discharge summary is a one of critical documents in the patient journey, encompassing all events experienced during hospitalization, including multiple visits, medications, tests, surgery/procedures, and admissions/discharge. Providing a summary of the patient's progress is crucial, as it significantly influences future care and planning. Consequently, clinicians face the laborious and resource-intensive task of manually collecting, organizing, and combining all the necessary data for a discharge summary. Therefore, we propose "NOTE", which stands for "Notable generation Of patient Text summaries through an Efficient approach based on direct preference optimization". NOTE is based on Medical Information Mart for Intensive Care- III dataset and summarizes a single hospitalization of a patient. Patient events are sequentially combined and used to generate a discharge summary for each hospitalization. In the present circumstances, large language models' application programming interfaces (LLMs' APIs) are widely available, but importing and exporting medical data presents significant challenges due to privacy protection policies in healthcare institutions. Moreover, to ensure optimal performance, it is essential to implement a lightweight model for internal server or program within the hospital. Therefore, we utilized DPO and parameter efficient fine tuning (PEFT) techniques to apply a fine-tuning method that guarantees superior performance. To demonstrate the practical application of the developed NOTE, we provide a webpage-based demonstration software. In the future, we will aim to deploy the software available for actual use by clinicians in hospital. NOTE can be utilized to generate various summaries not only discharge summaries but also throughout a patient's journey, thereby alleviating the labor-intensive workload of clinicians and aiming for increased efficiency.

cs.CV↗

arXiv2023

Semiclassical Boltzmann magnetotransport theory in anisotropic systems with a nonvanishing Berry curvature

Understanding the transport behavior of an electronic system under the influence of a magnetic field remains a key subject in condensed matter physics. Particularly in topological materials, their nonvanishing Berry curvature can lead to many interesting phenomena in magnetotransport owing to the coupling between the magnetic field and Berry curvature. By fully incorporating both the field-driven anisotropy and inherent anisotropy in the band dispersion, we study the semiclassical Boltzmann magnetotransport theory in topological materials with a nonvanishing Berry curvature. We show that as a solution to the Boltzmann transport equation the effective mean-free-path vector is given by the integral equation, including the effective velocity arising from the coupling between the magnetic field, Berry curvature and mobility. We also calculate the conductivity of Weyl semimetals with an isotropic energy dispersion, and find that the coupling between the magnetic field and Berry curvature induces anisotropy in the relaxation time, showing a substantial deviation from the result obtained assuming a constant relaxation time.

cond-mat.mes-hall↗

arXiv2020

Learning what they think vs. learning what they do: The micro-foundations of vicarious learning

Vicarious learning is a vital component of organizational learning. We theorize and model two fundamental processes underlying vicarious learning: observation of actions (learning what they do) vs. belief sharing (learning what they think). The analysis of our model points to three key insights. First, vicarious learning through either process is beneficial even when no agent in a system of vicarious learners begins with a knowledge advantage. Second, vicarious learning through belief sharing is not universally better than mutual observation of actions and outcomes. Specifically, enabling mutual observability of actions and outcomes is superior to sharing of beliefs when the task environment features few alternatives with large differences in their value and there are no time pressures. Third, symmetry in vicarious learning in fact adversely affects belief sharing but improves observational learning. All three results are shown to be the consequence of how vicarious learning affects self-confirming biased beliefs.

econ.TH↗

arXiv2019

Semiclassical Boltzmann transport theory of few-layer black phosphorus in various phases

Black phosphorus (BP), a two-dimensional (2D) van der Waals layered material composed of phosphorus atoms, has been one of the most actively studied 2D materials in recent years due to its tunable energy band gap (tunable even to a negative value) and its highly anisotropic electronic structure. Depending on the sign of the band gap tuning parameter, few-layer BP can be in a gapped insulator phase, gapless Dirac semimetal phase, or gapless semi-Dirac transition point between the two phases. Using the fully anisotropic multiband Boltzmann transport theory, we systematically study the dc conductivity of few-layer BP as a function of the carrier density and temperature by varying the band gap tuning parameter, and determine the characteristic density and temperature dependence corresponding to each phase.

cond-mat.mes-hall↗

arXiv2018

Diluted magnetic Dirac-Weyl materials: Susceptibility and ferromagnetism in three-dimensional chiral gapless semimetals

We theoretically investigate the temperature-dependent static susceptibility and long-range magnetic coupling of three-dimensional (3D) chiral gapless electron-hole systems (semimetals) with arbitrary band dispersion [i.e., $\varepsilon(k) \sim k^N$, where $k$ is the wave vector and $N$ is a positive integer]. We study the magnetic properties of these systems in the presence of dilute random magnetic impurities. Assuming carrier-mediated Ruderman-Kittel-Kasuya-Yosida indirect exchange interaction, we find that the magnetic ordering of intrinsic 3D chiral semimetals in the presence of dilute magnetic impurities is ferromagnetic for all values of $N$. Using finite-temperature self-consistent field approximation, we calculate the ferromagnetic transition temperature ($T_{\rm c}$). We find that $T_{\rm c}$ increases with increasing $N$ due to the enhanced density of states, and the calculated $T_{\rm c}$ is experimentally accessible assuming reasonable coupling between the magnetic impurities and itinerant carriers.

cond-mat.mes-hall↗

arXiv2017

Semiclassical Boltzmann transport theory for multi-Weyl semimetals

Multi-Weyl semimetals (m-WSMs) are a new type of Weyl semimetal that have linear dispersion along one symmetry direction but anisotropic non-linear dispersion along the two transverse directions with a topological charge larger than one. Using the Boltzmann transport theory and fully incorporating the anisotropy of the system, we study the DC conductivity as a function of carrier density and temperature. We find that the characteristic density and temperature dependence of the transport coefficients at the level of Boltzmann theory are controlled by the topological charge of the multi-Weyl point and distinguish m-WSMs from their linear Weyl counterparts.

cond-mat.mes-hall↗

arXiv2011

B+-tree Index Optimization by Exploiting Internal Parallelism of Flash-based Solid State Drives

Previous research addressed the potential problems of the hard-disk oriented design of DBMSs of flashSSDs. In this paper, we focus on exploiting potential benefits of flashSSDs. First, we examine the internal parallelism issues of flashSSDs by conducting benchmarks to various flashSSDs. Then, we suggest algorithm-design principles in order to best benefit from the internal parallelism. We present a new I/O request concept, called psync I/O that can exploit the internal parallelism of flashSSDs in a single process. Based on these ideas, we introduce B+-tree optimization methods in order to utilize internal parallelism. By integrating the results of these methods, we present a B+-tree variant, PIO B-tree. We confirmed that each optimization method substantially enhances the index performance. Consequently, PIO B-tree enhanced B+-tree's insert performance by a factor of up to 16.3, while improving point-search performance by a factor of 1.2. The range search of PIO B-tree was up to 5 times faster than that of the B+-tree. Moreover, PIO B-tree outperformed other flash-aware indexes in various synthetic workloads. We also confirmed that PIO B-tree outperforms B+-tree in index traces collected inside the Postgresql DBMS with TPC-C benchmark.

cs.DB↗

arXiv2002

Reaction Pathways Based on the Gradient of the Mean First-Passage Time

Finding representative reaction pathways is necessary for understanding mechanisms of molecular processes, but is considered to be extremely challenging. We propose a new method to construct reaction paths based on mean first-passage times. This approach incorporates information of all possible reaction events as well as the effect of temperature. The method is applied to exemplary reactions in a continuous and in a discrete setting. The suggested approach holds great promise for large reaction networks that are completely characterized by the method through a pathway graph.

physics.chem-ph↗

arXiv2002

Representative Pathways of Excitation Migration in Photosystem I

Photosystem I is a protein-pigment complex that performs photosynthesis in plants, green algae, and cyanobacteria. It contains an aggregate of chlorophylls that absorbs light and delivers the resulting electronic excitation to the special pair of chlorophylls where the excitation energy is used for producing charge separation across the cell membrane. The seemingly random arrangement of chlorophylls in photosystem I poses the question which pathways excitation migration follows towards the special pair after absorption of light at any of its chlorophylls. We employ a master equation to describe the process of excitation migration in photosystem I, and find representative paths of excitation migration based on the mean first-passage time from each chlorophyll to the special pair. The resulting paths, beginning at each of the chlorophylls and ending at the special pair, provide a complete yet distilled picture of the excitation migration towards the special pair.

physics.bio-ph↗