Searcharxiv⌕ Search

arXiv subjects

Dongwoo Kim

Publications and source records attributed to Dongwoo Kim.

At least 19 recordsLinked to original sources

Analyzing Learning-Dynamics Across Loss and Prediction Choices for Diffusion Models

Denoising generative models predict clean data ($x$), noise ($ε$), or velocity ($v$), while training losses can be measured in any of these spaces. Although the resulting nine combinations are algebraically related, they can yield substantially different denoising accuracy and sample quality. We disentangle the effects of prediction and loss spaces. For a fixed prediction target, we show that changing the loss space is exactly equivalent to timestep reweighting and provide a heuristic analysis of how this reweighting affects generation quality. We then show that a rank-$m$ linear bottleneck compressing a $D$-dimensional noisy input imposes an irreducible squared-error floor of $D-m$ on noise and velocity prediction, whereas clean-data prediction can exploit low-dimensional data structure. Combining this bound with prediction-dependent error conversion yields distinct necessary conditions on bottleneck capacity. Experiments with U-Net and JiT across CIFAR-10, Imagenette, and ImageNet-256 support these theoretical predictions. Under compression, prediction choice strongly affects both denoising accuracy and sample quality, with noise-prediction errors closely matching the theoretical lower bound. Our results provide a quantitative understanding of how architectural bottlenecks interact with prediction and loss choices in denoising generative models.

cs.LG↗

Transductive Generalization for GNNs via Optimal Transport

Analyzing the generalization of graph neural networks (GNNs) on node classification is challenging: message passing induces dependencies that break the i.i.d. assumption underlying standard inductive generalization theory. While distribution-free transductive theory resolves this dependency issue, existing classical bounds are either intractable or correlate poorly with empirical generalization. To fill this gap, we propose a transductive generalization bound based on optimal transport, connecting generalization to intra-class concentration and inter-class separation in the feature space. Empirically, our bound aligns strongly with the generalization gap. Building on this representation-based approach, we analyze how feature geometry evolves under repeated message passing. This explains the non-monotonic relationship between GNN depth and generalization, providing new insights into oversmoothing research. Practically, our bound serves as a geometry-aware regularizer that corrects the harmful behavior of message passing, yielding consistent performance improvements. The code is available at https://github.com/ml-postech/Transductive-OT-Gen-Bound.

cs.LG↗

In-Place Feedback: Reliable Refinement for Multi-Turn Expert-LLM Collaboration

LLM-generated drafts often contain subtle factual or logical errors, yet prior work shows that models struggle to reliably integrate multi-turn feedback aimed at fixing them. We propose in-place feedback, an interaction paradigm in which the user directly edits the model's previous response and the model continues generation from the edited context. In-place feedback consistently outperforms standard multi-turn feedback across five reasoning-intensive benchmarks while requiring fewer tokens, and our fine-grained analysis shows that it applies corrections more reliably and propagates them to subsequent reasoning. A user study with domain experts refining LLM-generated summaries corroborates these findings: participants report higher final-output satisfaction and substantially lower fatigue with in-place feedback, and a mixed strategy combining in-place and multi-turn feedback improves every dimension we measure. These results suggest that editing errors in place is a more reliable way to incorporate expert corrections than multi-turn feedback, while combining the two acts complementarily to form a more effective collaboration than either mechanism alone.

cs.LG↗

New Model, Old Habits: Behavior Transfer under Model Swapping in Multi-Session LLM Agents

Multi-session agents can switch between LLMs to balance performance and inference cost. Such switches require transferring a history for continuity, yet their effects on the recipient remain underexplored. Through controlled experiments with multi-session tasks, we uncover behavior transfer: recipients tend to adopt reasoning and action patterns from the models that generated their histories. This effect is most evident with raw-history transfer but is substantially reduced by summarization. Based on these findings, we propose Direction- and Authorship-aware Summarization of History (DASH) that preserves stronger-model behavior while minimizing weak-to-strong transfer via summarization. Experiments with repeated swaps show that DASH outperforms raw-history transfer and naive summarization baselines. These results suggest that effective multi-model orchestration requires considering not only which model to use, but also which behavioral patterns to carry forward.

cs.LG↗

Does the Gender Wage Gap Originate at Labor Market Entry? Evidence from South Korea

When in the lifecycle does a large gender wage gap emerge? South Korea has the largest aggregate gender pay gap (29%) in the OECD. Among recent college graduates the conditional gap is only 3.3% over 2008-2019, falling from 3.4% to 2.5%. Correcting for differential selection into employment with semiparametric and machine learning methods leaves it unchanged. Nonparametric and semiparametric bounds contain the point estimates. Selection-corrected quantile regression shows that the remaining entry-level gap is concentrated at the top of the distribution. In Korean panel data the corrected gap widens severalfold across the prime-age workforce, where the selection correction becomes first order. Korea's gender wage disparity is mostly generated after labor market entry.

econ.GN↗

Pseudo-Panel Event Studies: Econometric Foundations for Credible Applications

Event studies with panel data are widely used to estimate treatment effects. With repeated cross-sections, future treatment dates are unobserved for untreated individuals. Pseudo-panel event studies impute pre-treatment outcomes of the treated using matched untreated individuals. This paper establishes identification in such designs and characterizes bias when identification conditions fail. I show that the pseudo-panel regression is equivalent to weighted least squares with group means. This representation simplifies computation and yields analytical inference. In simulations, analytical confidence intervals attain near-nominal coverage, whereas conventional intervals substantially undercover. I also develop a doubly robust estimator using richer covariates, a symmetric matching design with synthetic type fixed effects, and a fully nonparametric estimator that accommodates heterogeneous treatment effects. In a Korean panel survey, the pseudo-panel regression yields an employment child penalty of 42.1%, close to 46.9% in the actual panel. Applications to repeated cross-sections yield employment penalties of about one-third.

econ.EM↗

Groups Don't Move as One: A Bregman Clustering View of Linearized Group Influence

Influence functions provide an efficient alternative to retraining for assessing the effects of removing or reweighting training data. For groups, however, the standard first-order estimate reduces individual contributions to a sum that can conceal substantial differences among members, including opposing effects that cancel. We establish a Bregman centroid formulation of the linearized group influence objective, interpreting this aggregate as a single representative and using Bregman information (BI) to quantify the variation it omits. Extending this formulation to multiple clusters reveals distinct contributions whose sum recovers the original estimate. We further propose an estimator that jointly optimizes the weights of these cluster contributions, rather than simply summing them. Experiments uncover cluster-specific influence patterns and a positive association between target-specific BI and approximation error. The proposed estimator achieves higher $R^2$ than standard group influence in predicting target changes under the nonlinear proximal Bregman response function. Simplified clustering and correction variants reduce computational cost. The centroid view thus explains what additive group influence leaves out and provides a basis for using that information to improve estimation.

cs.LG↗

Concept Unlearning via Cross-Attention Activation Projection for Diffusion Models

Existing closed-form methods for concept unlearning in text-to-image diffusion models typically derive editing directions from fixed text embeddings, which may not fully capture how concepts are expressed across latent states, timesteps, and layers. To capture this variation, we investigate cross-attention activations collected during denoising. In controlled probing experiments using the same anchor prompts, activation-derived bases achieve approximately five times the recall of text-derived bases on held-out prompts expressing the target concepts. Based on this finding, we propose Cross-Attention Subspace Erasure (CASE), a closed-form method that constructs layer-specific forget and retain subspaces from cross-attention activations. These subspaces define a retain-constrained linear operator incorporated directly into cross-attention weights, requiring no gradient-based fine-tuning or additional inference-time computation. Across ten concepts spanning four categories, CASE achieves the highest harmonic-mean score among evaluated baselines in all four categories, balancing suppression, retention, adversarial robustness, and generation quality. Further experiments demonstrate robustness to recovery attacks and a favorable suppression-retention trade-off when jointly unlearning up to 100 artistic styles. The benefits of activation-derived editing also extend to larger diffusion models, including SDXL and FLUX.

cs.CV↗

Scaling Influence Functions in LLMs through Eigenbasis-Corrected One-Bit Gradient Projection

Influence functions estimate how individual training examples affect the behavior of large language models (LLMs). Analyzing how training data influence different behaviors of an LLM involves repeated influence computation. Reusing stored training gradients reduces the computational cost, but storing full gradients is prohibitively expensive at LLM scale. We study how to compress these gradients while preserving influence estimates for future queries that are unknown at storage time. Through a worst-case analysis, we characterize the optimal fixed-dimensional linear representation and propose eigenbasis-corrected one-bit gradient projection (EOGP) to approximate it at scale. Specifically, EOGP uses EK-FAC to reduce gradient dimensionality, then applies PCA within the retained subspace to learn compression directions from the training gradients. We then apply one-bit quantization to the resulting coordinates, allowing more coordinates to be retained within a fixed storage budget. On GPT-2, EOGP predicts retraining outcomes more accurately than the evaluated compression baselines while using one-sixteenth of their per-example storage. On OLMo 2 SFT models from 1B to 32B parameters, EOGP remains competitive with the baselines allocated over 100 times as much storage per example.

cs.LG↗

The Interplay of Harness Design and Post-Training in LLM Agents

Tool-integrated LLM agents are often wrapped within a harness: the scaffolding that determines which tools are exposed, how they are described, and what auxiliary information accompanies each per-step observation. While agents are routinely post-trained, this scaffolding is typically treated as a fixed engineering detail, with design effort limited to the training-free regime. Moreover, existing post-training algorithms assume a static environment, even though tool environments and tasks often shift upon deployment. To address this gap, we extend $\texttt{ALFWorld}$ (i) to treat the harness as a controllable design dimension and (ii) to support evaluation under task and tool environment shifts. Building on this, we systematically analyze how the harness design influences post-training in both in-distribution and out-of-distribution (OOD) settings. We empirically show that harness-aware post-training not only improves in-distribution performance but also enables agents to robustly adapt to OOD settings. Under a harness with minimal design effort, post-training suffers a drastic performance drop under stronger tool environment shifts, further highlighting the importance of harness-aware post-training under such shifts.

cs.LG↗

Long Live the Librarian! A Persistent Search Sub-Agent for Energy-Efficient Multi-Agent Software Engineering Systems

Multi-agent systems (MAS) have substantially advanced autonomous software engineering (SWE), but their growing inference energy demands raise sustainability concerns. In this paper, we demonstrate that this cost is concentrated in an overlooked source: redundant output tokens generated across agents. Two empirical findings ground this claim. First, our per-token energy attribution for MAS reveals a sharp asymmetry: an output token consumes 30 to 1,000 times more energy than an input or cached token. Second, MAS inflate per-episode output because agents repeatedly re-explore overlapping repository regions. To address this inefficiency, we propose Librarian, a persistent search sub-agent that tracks repository-search history and suppresses redundant exploration actions across agents. By returning short references to file regions instead of full file excerpts, Librarian further reduces output-token volume. On SWE-Bench Verified and SWE-Bench-Live, Librarian reduces per-episode GPU energy consumption of existing multi-agent SWE systems by 11 to 30% while preserving task performance. Our code is available at https://github.com/ml-postech/Librarian.

cs.MA↗

The Race for Elite Destinations: Education Competition and Low Fertility in Korea

South Korea has the world's lowest fertility and an intense education race. Families devote nine percent of lifetime income to education, mostly to private tutoring with near-zero measured returns. I show that competition for coveted careers generates an assignment externality: when all families spend more, the admission bar rises and children become costly. In a quantitative model calibrated to this mechanism, replacing score-based assignment with a capacity-preserving lottery raises completed fertility by 0.24 children per couple. Pronatal transfers deliver one eighth of that increase. Education taxes deliver almost none. Preference shifts, not a fiercer race, explain the decline across cohorts.

econ.GN↗

Point-Identifying Semiparametric Sample Selection Models with No Excluded Variable

Sample selection is pervasive in applied economic studies. This paper proposes semiparametric selection models that achieve point identification without relying on exclusion restrictions. Our identification conditions require at least one continuously distributed covariate and certain nonlinearity in the selection process. We propose a two-step sieve plug-in estimator that is $\sqrt{n}$-consistent, asymptotically normal, and computationally straightforward, allowing for heteroskedasticity. We further derive the semiparametric efficiency bound for the model and propose a weighted variant of the estimator that attains the bound. Our approach provides a middle ground between Lee (2009)'s nonparametric bounds and Honoré and Hu (2020)'s linear selection bounds, while ensuring point identification. Simulation evidence confirms its excellent finite-sample performance. We apply our method to estimate the racial and gender wage disparities using data from the US Current Population Survey. Our estimates often lie outside the Honoré and Hu bounds.

econ.EM↗

Routing by Reaching: Composition of Pre-trained GFlowNets for Multi-Objective Generation

Generative Flow Networks (GFlowNets) learn to sample diverse candidates in proportion to a reward function, making them well-suited for scientific discovery, where exploring multiple promising solutions is crucial. Further extending GFlowNets to multi-objective settings has attracted growing interest as real-world applications often involve multiple, conflicting objectives. However, existing approaches require joint training for each combination of objectives, meaning that any change in the objective set necessitates retraining from scratch. We propose a framework that composes pre-trained GFlowNets at inference time, enabling rapid adaptation without fine-tuning or retraining. Importantly, our framework is flexible, capable of handling diverse reward combinations ranging from linear scalarization to complex nonlinear operators, which are often handled separately in previous literature. We prove that our method exactly recovers the target distribution for linear scalarization, and quantify the approximation quality for nonlinear operators through a distortion factor. Experiments on a synthetic 2D grid and real-world molecule generation tasks demonstrate that our approach achieves performance comparable to baselines.

cs.LG↗

Structure-Aware Variational State Preparation for Quantum Basket Option Pricing

Basket option pricing often relies on Monte Carlo estimation, for which quantum amplitude estimation (QAE) provides a quadratic speed-up. However, the practical benefit of QAE can be limited by the depth of the state-preparation circuit. We propose a structure-aware quantum state-preparation framework for QAE-based basket option pricing. The framework uses tensor-train (TT) rank information to design shallow variational state-preparation circuits. In the independent regime, TT ranks remove unnecessary entangling links from a hardware-efficient ansatz. In correlated basket settings, we instead prepare asset-wise marginals locally and train a compact latent block to match the basket cumulative distribution function. The Basket-CDF objective targets the basket pushforward distribution rather than the full joint state, directly aligning state preparation with basket-dependent payoffs. Numerical experiments show that the proposed circuits replace the exponential state-preparation depth scaling of exact amplitude loading with linear scaling, while maintaining low-percent basket-pricing errors. Additional sampling-based training experiments and an end-to-end QAE integration study support compatibility with sample-estimated training and standard QAE-based pricing workflows.

quant-ph↗

Representation-Centric Survey of Supervised Skeletal Action Recognition and the New Benchmark

3D skeletal action recognition has emerged as a powerful alternative to traditional RGB and depth-based approaches, offering robustness to environmental variations, computational efficiency, and enhanced privacy. Despite remarkable progress, current research remains fragmented across diverse input representations and lacks evaluation under scenarios that reflect real-world challenges. This paper presents a representation-centric review of supervised skeletal action recognition, systematically categorizing state-of-the-art methods by their input feature types: joint coordinates, bone vectors, motion flows, and extended representations, and analyzing how these choices influence spatiotemporal modeling strategies. Building on the insights from this review, we introduce ANUBIS, a large-scale, challenging dataset designed to address critical gaps in existing benchmarks. ANUBIS incorporates multi-view recordings with back-view perspectives, complex multi-person interactions, fine-grained and violent actions, and contemporary social behaviors. We benchmark a diverse set of state-of-the-art models on ANUBIS and conduct an in-depth analysis of how different feature types affect recognition performance across 102 action categories. Our results show strong action-feature dependencies, highlight the limitations of naive multi-representational fusion, and point toward the need for task-aware, semantically aligned integration strategies. This work offers both a comprehensive foundation and a practical benchmarking resource, aiming to guide the next generation of robust, generalizable skeleton-based action recognition systems for complex real-world scenarios. The dataset, benchmarking framework, and code are available at https://yliu1082.github.io/ANUBIS/.

cs.CV↗

Identification and Estimation of Semiparametric Multilayered Sample Selection Models

Many selection problems are multilayered: agents first decide whether to participate and then sort among ordered or unordered categories. This paper shows that the sorting layer changes the geometry of identification. Unlike binary selection, in which selection bias can be summarized by a scalar control function, ordered and multinomial sorting generally produce multi-index control functions whose dimension determines the continuous covariate variation needed for identification. I establish matched non-identification and point-identification results for both architectures, showing how nonlinearity in the selection structure can substitute for excluded variables. I also show how additional structural restrictions reduce the control-function dimension and make estimation practical. I propose root-n-consistent two-step sieve plug-in estimators and apply the framework to gender wage gaps among Korean college graduates. Accounting for sorting reshapes the entry-level gap along the firm-size margin, where the corrected female coefficient turns positive for large-firm employment.

econ.EM↗

Semi-nonparametric models of multidimensional matching: an optimal transport approach

This paper develops a set of empirically tractable and flexible sieve estimators for semi-nonparametric multidimensional matching models with transferable utility, focusing on worker-job matching. We generalize the parametric quadratic-Gaussian framework employed by Bojilov and Galichon (2016) and Lindenlaub (2017), which relies on joint normality of observed characteristics. We allow unrestricted distributions of characteristics and show identification of the production technology and the equilibrium wage and matching functions using optimal transport theory. Given identification, we propose efficient, consistent, and asymptotically normal sieve estimators. We revisit Lindenlaub's empirical application and show that, between 1990 and 2010, the U.S. economy experienced much larger technological progress favoring cognitive abilities than the original findings suggest. Furthermore, our flexible model specifications provide a significantly better fit for patterns in the evolution of wage inequality.

econ.EM↗