arXiv subjects

Gyubin Lee

Publications and source records attributed to Gyubin Lee.

9 recordsLinked to original sources

arXiv2026

CounterFlow: A Two-Phase Inference-Time Sampling for Counterfactual Video Foley Generation

We investigate Counterfactual Video Foley Generation, which aims to adopt a sound-source identity that contradicts the visual evidence while remaining temporally synchronized to a silent video. Existing Video&Text-to-Audio (VT2A) models struggle with this, often remaining anchored to the visually implied sound source when video and text contents disagree. We present ConterFlow, an inference-time dual-phase sampling scheme for pretrained flow-matching VT2A models. Phase 1 builds a video-derived temporal structure while suppressing the visually implied source; Phase 2 drops video conditioning to focus entirely on shaping audio timbre toward the target prompt. ConterFlow substantially improves counterfactual Video Foley generation compared to naive negative prompting and state-of-the-art baselines. To evaluate replacement quality, we propose a metric leveraging a text-audio co-embedding space to measure both target-prompt evidence and residual visually implied source leakage. Video demonstrations and code are available at https://gyubin-lee.github.io/counterflow-demo/

cs.MM↗

arXiv2026

PiAnnotate: A Web Annotation Tool for Piano Fingering, with a Diagnostic Probe

Piano fingering shapes how a passage can be played, yet it is difficult to label after a performance. An annotator must decide which finger produced each note while reconciling the score, timing, video, and hand motion. We present PiAnnotate, a web-based pipeline for adding expert fingering annotations to the FurElise performance dataset. The tool brings together a piano-roll view, performance video, and a 3D MANO hand mesh so that reviewers can inspect each assignment in musical and physical context. Rather than storing only the final answer, PiAnnotate keeps paired rule-based and human-edited fingering tracks. These paired tracks make the annotation history auditable by showing where a geometric rule was sufficient, where experts intervened, and how labels changed across review passes. As a final diagnostic, we train a small Transformer probe on the paired tracks. The probe improves on the rule baseline on held-out pieces while remaining conservative about changing labels that were already correct, suggesting that the edited labels contain learnable structure rather than only isolated fixes.

cs.SD↗

arXiv2026

Learning to Theorize the World from Observation

What does it mean to understand the world? Contemporary world models often operationalize understanding as accurate future prediction in latent or observation space. Developmental cognitive science, however, suggests a different view: human understanding emerges through the construction of internal theories of how the world works, even before mature language is acquired. Inspired by this theory-building view of cognition, we introduce Learning-to-Theorize, a learning paradigm for inferring explicit explanatory theories of the world from raw, non-textual observations. We instantiate this paradigm with the Neural Theorizer (NEO), a probabilistic neural model that induces latent programs as a learned Language of Thought and executes them through a shared transition model. In NEO, a theory is represented as an executable, compositional program whose learned primitives can be systematically recombined to explain novel phenomena. Experiments show that this formulation enables explanation-driven generalization, allowing observations to be understood in terms of the programs that generate them.

cs.LG↗

arXiv2026

Tipiano: Cascaded Piano Hand Motion Synthesis via Fingertip Priors

Synthesizing realistic piano hand motions requires both precision and naturalness. Physics-based methods achieve precision but produce stiff motions; data-driven models learn natural dynamics but struggle with positional accuracy. Piano motion exhibits a natural hierarchy: fingertip positions are nearly deterministic given piano geometry and fingering, while wrist and intermediate joints offer stylistic freedom. We present [OURS], a four-stage framework exploiting this hierarchy: (1) statistics-based fingertip positioning, (2) FiLM-conditioned trajectory refinement, (3) wrist estimation, and (4) STGCN-based pose synthesis. We contribute expert-annotated fingerings for the F\"urElise dataset (153 pieces, ~10 hours). Experiments demonstrate F1 = 0.910, substantially outperforming diffusion baselines (F1 = 0.121), with user study (N=41) confirming quality approaching motion capture. Expert evaluation by professional pianists (N=5) identified anticipatory motion as the key remaining gap, providing concrete directions for future improvement.

cs.AI↗

arXiv2025

Adaptive Inference-Time Scaling via Cyclic Diffusion Search

Diffusion models have demonstrated strong generative capabilities across domains ranging from image synthesis to complex reasoning tasks. However, most inference-time scaling methods rely on fixed denoising schedules, limiting their ability to allocate computation based on instance difficulty or task-specific demands adaptively. We introduce the challenge of adaptive inference-time scaling-dynamically adjusting computational effort during inference-and propose Adaptive Bi-directional Cyclic Diffusion (ABCD), a flexible, search-based inference framework. ABCD refines outputs through bi-directional diffusion cycles while adaptively controlling exploration depth and termination. It comprises three components: Cyclic Diffusion Search, Automatic Exploration-Exploitation Balancing, and Adaptive Thinking Time. Experiments show that ABCD improves performance across diverse tasks while maintaining computational efficiency.

cs.LG↗

arXiv2024

Topological Fermi-arc surface state covered by floating electrons on a two-dimensional electride

Two-dimensional electrides can acquire topologically non-trivial phases due to intriguing interplay between the cationic atomic layers and anionic electron layers. However, experimental evidence of topological surface states has yet to be verified. Here, via angle-resolved photoemission spectroscopy (ARPES) and scanning tunnelling microscopy (STM), we probe the magnetic Weyl states of the ferromagnetic electride $[Gd_{2}$C]^{2+}\cdot2e^{-}$. In particular, the presence of Weyl cones and Fermi-arc states is demonstrated through photon energy-dependent ARPES measurements, agreeing with theoretical band structure calculations. Notably, the STM measurements reveal that the Fermi-arc states exist underneath a floating quantum electron liquid on the top Gd layer, forming double-stacked surface states in a heterostructure. Our work thus not only unveils the non-trivial topology of the $[Gd_{2}$C]^{2+}\cdot2e^{-}$ electride but also realizes a surface heterostructure that can host phenomena distinct from the bulk.

cond-mat.mtrl-sci↗

arXiv2024

CONMOD: Controllable Neural Frame-based Modulation Effects

Deep learning models have seen widespread use in modelling LFO-driven audio effects, such as phaser and flanger. Although existing neural architectures exhibit high-quality emulation of individual effects, they do not possess the capability to manipulate the output via control parameters. To address this issue, we introduce Controllable Neural Frame-based Modulation Effects (CONMOD), a single black-box model which emulates various LFO-driven effects in a frame-wise manner, offering control over LFO frequency and feedback parameters. Additionally, the model is capable of learning the continuous embedding space of two distinct phaser effects, enabling us to steer between effects and achieve creative outputs. Our model outperforms previous work while possessing both controllability and universality, presenting opportunities to enhance creativity in modern LFO-driven audio effects.

eess.AS↗

arXiv2022

Quantum electron liquid and its possible phase transition

Purely quantum electron systems exhibit intriguing correlated electronic phases by virtue of quantum fluctuations in addition to electron-electron interactions. To realize such quantum electron systems, a key ingredient is dense electrons decoupled from other degrees of freedom. Here, we report the discovery of a pure quantum electron liquid, which spreads up to ~ 3 Å in the vacuum on the surface of electride crystal. An extremely high electron density and its weak hybridisation with buried atomic orbitals evidence the quantum and pure nature of electrons, that exhibit a polarized liquid phase as demonstrated by our spin-dependent measurement. Further, upon enhancing the electron correlation strength, the dynamics of quantum electrons changes to that of non-Fermi liquid along with an anomalous band deformation, suggestive of a transition to a hexatic liquid crystal phase. Our findings cultivate the frontier of quantum electron systems, and serve as a platform for exploring correlated electronic phases in a pure fashion.

cond-mat.str-el↗

arXiv2020

A weak topological insulator state in quasi-one-dimensional superconductor TaSe$_3$

A well-established way to find novel Majorana particles in a solid-state system is to have superconductivity arising from the topological electronic structure. To this end, the heterostructure systems that consist of normal superconductor and topological material have been actively explored in the past decade. However, a search for the single material system that simultaneously exhibits intrinsic superconductivity and topological phase has been largely limited, although such a system is far more favorable especially for the quantum device applications. Here, we report the electronic structure study of a quasi-one-dimensional (q1D) superconductor TaSe$_3$. Our results of angle-resolved photoemission spectroscopy (ARPES) and first-principles calculation clearly show that TaSe$_3$ is a topological superconductor. The characteristic bulk inversion gap, in-gap state and its shape of non-Dirac dispersion concurrently point to the topologically nontrivial nature of this material. The further investigations of the Z$_2$ indices and the topologically distinctive surface band crossings disclose that it belongs to the weak topological insulator (WTI) class. Hereby, TaSe$_3$ becomes the first verified example of an intrinsic 1D topological superconductor. It hopefully provides a promising platform for future applications utilizing Majorana bound states localized at the end of 1D intrinsic topological superconductors.

cond-mat.supr-con↗