arXiv subjects

Chutian Wang

Publications and source records attributed to Chutian Wang.

15 recordsLinked to original sources

arXiv2026

Hyperball May Not Be a Free Lunch

For scale-invariant deep networks, Hyperball-style optimizers have shown strong performance in large-scale training by fixing the norms of matrix-valued parameters and normalizing updates. However, the source of their advantage remains unclear. Starting from the angular displacement between consecutive parameter states, we derive an angular effective learning rate that accounts for the parameter-update angle, parameter norm, and update norm. We also show that the conventional norm-based measure is a special case under parameter-update orthogonality. We then decompose optimizer updates into radial and tangential components and analyze how radial updates affect one-step angular displacement. Under the training configurations considered, numerical results show that the radial component has only a limited direct effect on the angular effective learning rate. It therefore cannot explain why MuonH converges more slowly than MuonWD early in training but overtakes it later. To further isolate the underlying mechanism, we devise a heuristic experiment that modifies only the learning-rate schedule so that the dynamics of each optimizer reproduce those of the other. The results suggest that their main difference stems from the evolution of the effective step size rather than an intrinsically superior update direction induced by Hyperball. Our pretraining experiments further show that more aggressive learning-rate decay can accelerate MuonH early in training but may impair its later performance. Thus, maintaining a constant angular velocity does not eliminate the learning-rate-scheduling problem; careful scheduling remains essential to realizing the potential of Hyperball-style optimizers. Our code is publicly available at https://github.com/mangocrazz/hyperball-may-not-be-a-free-lunch.

cs.LG↗

arXiv2026

Deep Speckle Holography Redefines Label-free Nanoparticle Phenotyping

Nanoparticle metrology has long been constrained by the assumption that, in mixed and unprocessed fluids, particle size, morphology, composition, and species-specific abundance cannot be resolved simultaneously from a single label-free measurement. Here, we revisit this long-standing limitation by showing that complex forward speckle-holographic fields define an information-rich optical space for multidimensional particle signatures. We report deep speckle holography, a physics-informed generative framework that profiles particle identity, size, morphology, and species-resolved abundance from a single non-contact optical measurement. Across purified suspensions, mixed particle populations, environmental waters, human urine, and other unprocessed native fluids, the method enables direct nanoparticle inference without purification, labeling, or destructive preprocessing, delivering concurrent multidimensional readouts in 0.9 s over a dynamic range spanning 10 orders of magnitude. Deep speckle holography establishes a route toward direct label-free nanoparticle phenotyping in real-world fluids, moving nanoscale measurement beyond isolated-particle characterization toward multidimensional inference in complex mixtures, and expanding the scope of questions nanoscale measurement can address, from real-time tracking of nanoparticle transformations in living and environmental systems to non-invasive quality control of nanomedicine formulations, and beyond.

eess.IV↗

arXiv2025

Inverse Design of Metasurface for Spectral Imaging

Inverse design of metasurfaces for the joint optimization of optical modulation and algorithmic decoding in computational optics presents significant challenges, especially in applications such as hyperspectral imaging. We introduce a physics-data co-driven framework for designing reconfigurable metasurfaces fabricated from the phase-change material Ge2Sb2Se4Te1 to achieve compact, compressive spectral imaging in the shortwave infrared region. Central to our approach is a differentiable neural simulator, trained on over 320,000 simulated geometries, that accurately predicts spectral responses across 11 crystallization states. This differentiability enables end-to-end joint optimization of the metasurface geometry, its spectral encoding function, and a deep reconstruction network. We also propose a soft shape regularization technique that preserves manufacturability during gradient-based updates. Experiments show that our optimized system improves reconstruction fidelity by up to 7.6 dB in the peak-signal-to-noise ratio, with enhanced noise resilience and improved measurement matrix conditioning, underscoring the potential of our approach for high-performance hyperspectral imaging.

eess.IV↗

arXiv2025

Search Self-play: Pushing the Frontier of Agent Capability without Supervision

Reinforcement learning with verifiable rewards (RLVR) has become the mainstream technique for training LLM agents. However, RLVR highly depends on well-crafted task queries and corresponding ground-truth answers to provide accurate rewards, which requires significant human effort and hinders the scaling of RL processes, especially in agentic scenarios. Although a few recent works explore task synthesis methods, the difficulty of generated agentic tasks can hardly be controlled to provide effective RL training advantages. To achieve agentic RLVR with higher scalability, we explore self-play training for deep search agents, in which the learning LLM utilizes multi-turn search engine calling and acts simultaneously as both a task proposer and a problem solver. The task proposer aims to generate deep search queries with well-defined ground-truth answers and increasing task difficulty. The problem solver tries to handle the generated search queries and output the correct answer predictions. To ensure that each generated search query has accurate ground truth, we collect all the searching results from the proposer's trajectory as external knowledge, then conduct retrieval-augmentation generation (RAG) to test whether the proposed query can be correctly answered with all necessary search documents provided. In this search self-play (SSP) game, the proposer and the solver co-evolve their agent capabilities through both competition and cooperation. With substantial experimental results, we find that SSP can significantly improve search agents' performance uniformly on various benchmarks without any supervision under both from-scratch and continuous RL training setups. The code is at https://github.com/Qwen-Applications/SSP.

cs.LG↗

arXiv2024

SweepEvGS: Event-Based 3D Gaussian Splatting for Macro and Micro Radiance Field Rendering from a Single Sweep

Recent advancements in 3D Gaussian Splatting (3D-GS) have demonstrated the potential of using 3D Gaussian primitives for high-speed, high-fidelity, and cost-efficient novel view synthesis from continuously calibrated input views. However, conventional methods require high-frame-rate dense and high-quality sharp images, which are time-consuming and inefficient to capture, especially in dynamic environments. Event cameras, with their high temporal resolution and ability to capture asynchronous brightness changes, offer a promising alternative for more reliable scene reconstruction without motion blur. In this paper, we propose SweepEvGS, a novel hardware-integrated method that leverages event cameras for robust and accurate novel view synthesis across various imaging settings from a single sweep. SweepEvGS utilizes the initial static frame with dense event streams captured during a single camera sweep to effectively reconstruct detailed scene views. We also introduce different real-world hardware imaging systems for real-world data collection and evaluation for future research. We validate the robustness and efficiency of SweepEvGS through experiments in three different imaging settings: synthetic objects, real-world macro-level, and real-world micro-level view synthesis. Our results demonstrate that SweepEvGS surpasses existing methods in visual rendering quality, rendering speed, and computational efficiency, highlighting its potential for dynamic practical applications.

cs.CV↗

arXiv2024

Neuromorphic Imaging with Super-Resolution

Neuromorphic imaging is an emerging technique that imitates the human retina to sense variations in dynamic scenes. It responds to pixel-level brightness changes by asynchronous streaming events and boasts microsecond temporal precision over a high dynamic range, yielding blur-free recordings under extreme illumination. Nevertheless, this modality falls short in spatial resolution and leads to a low level of visual richness and clarity. Pursuing hardware upgrades is expensive and might cause compromised performance due to more burdens on computational requirements. Another option is to harness offline, plug-in-play super-resolution solutions. However, existing ones, which demand substantial sample volumes for lengthy training on massive computing resources, are largely restricted by real data availability owing to the current imperfect high-resolution devices, as well as the randomness and variability of motion. To tackle these challenges, we introduce the first self-supervised neuromorphic super-resolution prototype. It can be self-adaptive to per input source from any low-resolution camera to estimate an optimal, high-resolution counterpart of any scale, without the need of side knowledge and prior training. Evaluated on downstream tasks, such a simple yet effective method can obtain competitive results against the state-of-the-arts, significantly promoting flexibility but not sacrificing accuracy. It also delivers enhancements for inferior natural images and optical micrographs acquired under non-ideal imaging conditions, breaking through the limitations that are challenging to overcome with frame-based techniques. In the current landscape where the use of high-resolution cameras for event-based sensing remains an open debate, our solution is a cost-efficient and practical alternative, paving the way for more intelligent imaging systems.

eess.IV↗

arXiv2024

Cap2Sum: Learning to Summarize Videos by Generating Captions

With the rapid growth of video data on the internet, video summarization is becoming a very important AI technology. However, due to the high labelling cost of video summarization, existing studies have to be conducted on small-scale datasets, leading to limited performance and generalization capacity. In this work, we introduce the use of dense video captions as a supervision signal to train video summarization models. Motivated by this, we propose Cap2Sum, a model that learns to summarize videos by generating captions, to exploit dense video caption annotations. This weakly-supervised approach allows us to train the models on large-scale dense video caption datasets to achieve better performance and generalization capacity. To further improve the generalization capacity, we introduce a CLIP (a strong vision-language model) Prior mechanism to enhance the learning of important objects that captions may ignore in the videos. In practice, Cap2Sum can perform zero-shot video summarization or be fine-tuned by the ground-truth summary or video caption of the target dataset. To examine the performance of Cap2Sum after weakly-supervised fine-tuning by the video captions, we propose two new datasets, TVSum-Caption and SumMe-Caption, which are derived from two common video summarization datasets and will be publicly released. We conduct extensive experiments and the results demonstrate that our method achieves significant improvements in performance and generalization capacity compared with previous methods.

cs.MM↗

arXiv2024

Ev-GS: Event-based Gaussian splatting for Efficient and Accurate Radiance Field Rendering

Computational neuromorphic imaging (CNI) with event cameras offers advantages such as minimal motion blur and enhanced dynamic range, compared to conventional frame-based methods. Existing event-based radiance field rendering methods are built on neural radiance field, which is computationally heavy and slow in reconstruction speed. Motivated by the two aspects, we introduce Ev-GS, the first CNI-informed scheme to infer 3D Gaussian splatting from a monocular event camera, enabling efficient novel view synthesis. Leveraging 3D Gaussians with pure event-based supervision, Ev-GS overcomes challenges such as the detection of fast-moving objects and insufficient lighting. Experimental results show that Ev-GS outperforms the method that takes frame-based signals as input by rendering realistic views with reduced blurring and improved visual quality. Moreover, it demonstrates competitive reconstruction quality and reduced computing occupancy compared to existing methods, which paves the way to a highly efficient CNI approach for signal processing.

cs.CV↗

arXiv2024

Neuromorphic Shack-Hartmann wave normal sensing

The Shack-Hartmann wavefront sensor is widely employed in adaptive optics systems to measure optical aberrations. However, simultaneously achieving high sensitivity and large dynamic range is still challenging, limiting the performance of diagnosing fast-changing turbulence. To overcome this limitation, we propose neuromorphic Shack-Hartmann wave normal sensing (NeuroSH). NeuroSH is a unifying framework that harnesses the computational neuromorphic imaging paradigm to extract the high-dimensional wave normal from temporal diversity measurements. Both numerical analysis and experimental verification demonstrate the feasibility of NeuroSH. To the best of our knowledge, the proposed NeuroSH is the first scheme to surpass the optical dynamic range limitation under challenging dynamic scenarios, thereby advancing ultra-fast turbulence mitigation technology for cutting-edge imagers.

physics.optics↗

arXiv2024

Neuromorphic Imaging and Classification with Graph Learning

Bio-inspired neuromorphic cameras asynchronously record pixel brightness changes and generate sparse event streams. They can capture dynamic scenes with little motion blur and more details in extreme illumination conditions. Due to the multidimensional address-event structure, most existing vision algorithms cannot properly handle asynchronous event streams. While several event representations and processing methods have been developed to address such an issue, they are typically driven by a large number of events, leading to substantial overheads in runtime and memory. In this paper, we propose a new graph representation of the event data and couple it with a Graph Transformer to perform accurate neuromorphic classification. Extensive experiments show that our approach leads to better results and excels at the challenging realistic situations where only a small number of events and limited computational resources are available, paving the way for neuromorphic applications embedded into mobile facilities.

cs.CV↗

arXiv2024

Neuromorphic Imaging with Joint Image Deblurring and Event Denoising

Neuromorphic imaging reacts to per-pixel brightness changes of a dynamic scene with high temporal precision and responds with asynchronous streaming events as a result. It also often supports a simultaneous output of an intensity image. Nevertheless, the raw events typically involve a large amount of noise due to the high sensitivity of the sensor, while capturing fast-moving objects at low frame rates results in blurry images. These deficiencies significantly degrade human observation and machine processing. Fortunately, the two information sources are inherently complementary -- events with microsecond-level temporal resolution, which are triggered by the edges of objects recorded in a latent sharp image, can supply rich motion details missing from the blurry one. In this work, we bring the two types of data together and introduce a simple yet effective unifying algorithm to jointly reconstruct blur-free images and noise-robust events in an iterative coarse-to-fine fashion. Specifically, an event-regularized prior offers precise high-frequency structures and dynamic features for blind deblurring, while image gradients serve as a kind of faithful supervision in regulating neuromorphic noise removal. Comprehensively evaluated on real and synthetic samples, such a synergy delivers superior reconstruction quality for both images with severe motion blur and raw event streams with a storm of noise, and also exhibits greater robustness to challenging realistic scenarios such as varying levels of illumination, contrast and motion magnitude. Meanwhile, it can be driven by much fewer events and holds a competitive edge at computational time overhead, rendering itself preferable as available computing resources are limited. Our solution gives impetus to the improvement of both sensing data and paves the way for highly accurate neuromorphic reasoning and analysis.

eess.IV↗

arXiv2023

On the use of deep learning for phase recovery

Phase recovery (PR) refers to calculating the phase of the light field from its intensity measurements. As exemplified from quantitative phase imaging and coherent diffraction imaging to adaptive optics, PR is essential for reconstructing the refractive index distribution or topography of an object and correcting the aberration of an imaging system. In recent years, deep learning (DL), often implemented through deep neural networks, has provided unprecedented support for computational imaging, leading to more efficient solutions for various PR problems. In this review, we first briefly introduce conventional methods for PR. Then, we review how DL provides support for PR from the following three stages, namely, pre-processing, in-processing, and post-processing. We also review how DL is used in phase image processing. Finally, we summarize the work in DL for PR and outlook on how to better use DL to improve the reliability and efficiency in PR. Furthermore, we present a live-updating resource (https://github.com/kqwang/phase-recovery) for readers to learn more about PR.

physics.optics↗

arXiv2023

Extracting unconventional spin texture in two dimensional topological crystalline insulator bismuthene via tuning bulk-edge interactions

Tuning the interaction between the bulk and edge states of topological materials is a powerful tool for manipulating edge transport behavior, opening up exciting opportunities for novel electronic and spintronic applications. This approach is particularly suited to topological crystalline insulators (TCI), a class of topologically nontrivial compounds that are endowed with multiple degrees of topological protection. In this study, we investigate how bulk-edge interactions can influence the edge transport in planar bismuthene, a TCI with metallic edge states protected by in-plane mirror symmetry, using first principles calculations and symmetrized Wannier tight-binding models. By exploring the impact of various perturbation effects, such as device size, substrate potentials, and applied transverse electric field, we examine the evolution of the electronic structure and edge transport in planar bismuthene. Our findings demonstrate that the TCI states of planar bismuthene can be engineered to exhibit either a gapped or conducting unconventional helical spin texture via a combination of substrate and electric field effects. Furthermore, under strong electric fields, the edge states can be stabilized through a delicate control of the bulk-edge interactions. These results open up new directions for discovering novel spin transport patterns in topological materials and provide critical insights for the fabrication of topological spintronic devices.

cond-mat.mes-hall↗

arXiv2021

Localized Wannier function based tight-binding models for two-dimensional allotropes of bismuth

With its monoelemental composition, various crystalline forms and an inherently strong spin-orbit coupling, bismuth has been regarded as an ideal prototype material to expand our understanding of topological electronic structures. In particular, two-dimensional bismuth thin films have attracted a growing interest due to potential applications in topological transistors and spintronics. This calls for an effective physical model to give an accurate interpretation of the novel topological phenomena shown by two-dimensional bismuth. However, the conventional semi-empirical approach of adapting bulk bismuth hoppings fails to capture the topological features of two-dimensional bismuth allotropes because the electronic band topology is heavily influenced by crystalline symmetries as well as atom spacings. Here we provide a new parameterization using localized Wannier functions derived from the Bloch states in first-principles calculations. We construct new tight-binding models for three types of two-dimensional bismuth allotropes: a Bi (111) bilayer, bismuthene and a Bi(110) bilayer. We demonstrate that our tight-binding models can successfully reproduce the band structures, symmetries and topological features of these two-dimensional allotropes. We anticipate that these models can be extended to other similar two-dimensional topological structures such as antimonene and arsenene. Moreover, these models can serve as a starting point for investigating the electron/spin transport and electromagnetic response in low-dimensional topological devices.

cond-mat.mtrl-sci↗

arXiv2020

Electronic bandstructure of in-plane ferroelectric van der Waals $β'-In_{2}Se_{3}$

Layered indium selenides ($In_{2}Se_{3}$) have recently been discovered to host robust out-of-plane and in-plane ferroelectricity in the $α$ and $β$' phases, respectively. In this work, we utilise angle-resolved photoelectron spectroscopy to directly measure the electronic bandstructure of $β'-In_{2}Se_{3}$, and compare to hybrid density functional theory (DFT) calculations. In agreement with DFT, we find the band structure is highly two-dimensional, with negligible dispersion along the c-axis. Due to n-type doping we are able to observe the conduction band minima, and directly measure the minimum indirect (0.97 eV) and direct (1.46 eV) bandgaps. We find the Fermi surface in the conduction band is characterized by anisotropic electron pockets with sharp in-plane dispersion about the $\overline{M}$ points, yielding effective masses of 0.21 $m_{0}$ along $\overline{KM}$ and 0.33 $m_{0}$ along $\overline{ΓM}$. The measured band structure is well supported by hybrid density functional theory calculations. The highly two-dimensional (2D) bandstructure with moderate bandgap and small effective mass suggest that $β'-In_{2}Se_{3}$ is a potentially useful new van der Waals semiconductor. This together with its ferroelectricity makes it a viable material for high-mobility ferroelectric-photovoltaic devices, with applications in non-volatile memory switching and renewable energy technologies.

cond-mat.mtrl-sci↗