arXiv subjects

Edmund Y. Lam

Publications and source records attributed to Edmund Y. Lam.

At least 19 recordsLinked to original sources

arXiv2026

Geometry-Aligned Semantic Matching for Cross-Modal Planar Image Registration

Cross-modal image matching establishes stable and accurate geometric correspondences across modalities for planar registration. Existing semantic representations provide cross-modal consistency, but semantic similarity does not necessarily imply geometric correspondence. Meanwhile, fine-grained CNN features provide accurate local details but lack global cross-modal semantic guidance for stable refinement. To address these issues, we propose CDPM, which first establishes geometrically consistent semantic representations and then preserves their dominant role in correspondence estimation during fine-grained localization. Specifically, we progressively adapt DINOv3 using geometrically consistent cross-modal patch pairs, enabling feature similarity to better reflect true cross-modal spatial correspondences. We then construct a DINO-Centric Feature Pyramid, where multi-scale DINO representations maintain stable cross-modal correspondences, while a lightweight CNN branch provides auxiliary structural details for precise local refinement. Extensive experiments on three cross-modal datasets demonstrate the superior performance of CDPM. On VIS-IR, compared with the dense matcher RoMa, CDPM improves AUC@3/5/10/20 by 7.36, 13.40, 13.75, and 10.42 percentage points, respectively, and reduces mACE from 5.83 to 2.78 pixels. It also outperforms RoMa v2 across all metrics while requiring 45.6% fewer FLOPs. The online demo and dataset are available, and the code will be released on our project page at https://warren-wzw.github.io/CDPM/.

cs.CV↗

arXiv2026

Physical Fidelity of Wave-Based Inverse Problems under Broken Time-Reversal Symmetry

Modern coherent imaging commonly treats defocus interference as an undesirable artifact to be suppressed. Here, we show that this interference is instead a deterministic projection shadow associated with broken time-reversal symmetry (TRS) under finite-aperture observation. The resulting non-unitary observation operator imposes irretrievable information loss, making this shadow an unavoidable coherent residual. We demonstrate that suppressing this residual beyond the measurement support can yield physically inconsistent reconstructions by assigning unconstrained components within the null space, thereby disrupting the propagation of the reconstructed complex field. We term this failure mode physical overfitting and introduce the complex-field TRS residual metric, R_TRS(z). By using the measured axial evolution of the projection shadow as a z-resolved physical reference, the metric extends fidelity assessment from two-dimensional (2D) detector-plane agreement to propagation-resolved three-dimensional (3D) complex-field consistency. These results establish a physical constraint on phase-space retrieval and wave-based inverse reconstruction.

physics.optics↗

arXiv2026

EPOFusion: Exposure aware Progressive Optimization Method for Infrared and Visible Image Fusion

Overexposure caused by strong daylight and oncoming headlights frequently overwhelms visible sensors, resulting in critical information loss in visual perception. Infrared and visible image fusion can compensate for such degradation via multimodal complementarity. However, most fusion methods lack region-aware optimization for overexposed areas and cannot effectively exploit infrared cues in saturated regions, resulting in insufficient infrared detail preservation or redundant information in the fused results. To address this, we propose EPOFusion, an exposure-aware fusion framework. It employs a spatial guidance module to identify regions requiring infrared compensation, together with a region-aware fusion loss to strengthen informative infrared structures. In addition, an iterative feature refinement head equipped with a multiscale context fusion module progressively refines fused representations, enabling effective integration of complementary infrared information while maintaining visual consistency in normally exposed regions. The infrared and visible overexposure (IVOE) dataset consists of a synthetic training subset providing infrared-compensation supervision and a real-world subset for fusion and downstream perception evaluation under authentic overexposure. EPOFusion demonstrates superior VIF and $Q^{AB/F}$ performance with favorable visual quality, improving $Q^{AB/F}$ by 10.7% over the existing overexposure-oriented fusion baseline, while further improving downstream mIoU and mAP50 by 5.6% and 6.5%, respectively. Code, results, and the IVOE dataset will be made available at https://warren-wzw.github.io/EPOFusion/.

cs.CV↗

arXiv2026

Large-Scale Bayesian Tensor Reconstruction via Approximate Message Passing

While CANDECOMP/PARAFAC (CP) decomposition (CPD) is fundamental for tensor reconstruction, Bayesian CPD often scales poorly because variational updates require repeated matrix inversions. We develop CP generalized approximate message passing (CP-GAMP) for incomplete noisy Bayesian CPD. The algorithm uses Gaussian message approximations to avoid high-dimensional inversions, and it combines a Bernoulli-Gaussian prior with expectation-maximization updates to estimate effective CP rank and noise variance. We also give a formal state evolution (SE) recursion and relate its fixed points to replica-symmetric saddle points, so CP-GAMP's SE-predicted error can be compared with the formal replica-symmetric minimum mean-squared error (MMSE) benchmark in the matched limit. Synthetic and image-inpainting experiments show that CP-GAMP substantially reduces runtime relative to variational Bayesian CPD while maintaining competitive reconstruction accuracy.

cs.LG↗

arXiv2026

ERF-GS: Reconstructing Fast Motion from Disjoint Event-RGB Viewpoints

Deep learning-driven representations such as neural radiance fields (NeRFs) and 3D Gaussian splatting (3DGS) have revolutionized the field of dynamic 3D scene reconstruction with improved visual precision and scalability. However, the reconstruction of fast-moving objects remains a challenge; existing methods based on conventional frame-based videos often struggle in scenarios such as sports events and animal videography. We propose an event-RGB fusion Gaussian splatting (ERF-GS) framework that integrates event information into both optimization and densification stages of the Gaussian splatting pipeline, taking advantage of novel event sensors with high frame-rate. Unlike many other event-assisted scene reconstruction methods, ERF-GS was developed using realistic simulation settings and realizes event-based learning detached from RGB inputs. This design enables its application beyond straightforward synthetic data into the realm of natural video with complex layout, low frame rates and severe motion blur. Our experiments show that ERF-GS outperforms both the 4DGS baseline and the concurrent E-D3DGS on different variants of the Neu3D and Nvidia datasets which include blurry RGB frames and disjoint RGB-event viewpoints. Our code is available at https://github.com/andrewbxy/ERF-GS.

cs.CV↗

arXiv2026

Computation-Aware Event-to-Frame Reconstruction via Selective Attention

Event-to-frame (E2F) reconstruction bridges asynchronous event streams with frame-based vision pipelines, but existing methods often face a trade-off between reconstruction quality and computational efficiency. In this work, we propose an efficient E2F framework that emphasizes causal temporal modeling and computation-aware design. The architecture adopts a recurrent encoder-decoder to incrementally aggregate event information with compact hidden states. To improve robustness under fast motion and illumination variations, a selective context fusion strategy is introduced to integrate event-driven features with prior intensity cues. Within this fusion process, a lightweight hybrid attention mechanism enhances feature selectivity without relying on heavy attention operations. Experimental results on standard benchmarks demonstrate that the proposed approach achieves competitive reconstruction performance while maintaining a favorable balance between accuracy and model complexity.

cs.CV↗

arXiv2026

Deep Speckle Holography Redefines Label-free Nanoparticle Phenotyping

Nanoparticle metrology has long been constrained by the assumption that, in mixed and unprocessed fluids, particle size, morphology, composition, and species-specific abundance cannot be resolved simultaneously from a single label-free measurement. Here, we revisit this long-standing limitation by showing that complex forward speckle-holographic fields define an information-rich optical space for multidimensional particle signatures. We report deep speckle holography, a physics-informed generative framework that profiles particle identity, size, morphology, and species-resolved abundance from a single non-contact optical measurement. Across purified suspensions, mixed particle populations, environmental waters, human urine, and other unprocessed native fluids, the method enables direct nanoparticle inference without purification, labeling, or destructive preprocessing, delivering concurrent multidimensional readouts in 0.9 s over a dynamic range spanning 10 orders of magnitude. Deep speckle holography establishes a route toward direct label-free nanoparticle phenotyping in real-world fluids, moving nanoscale measurement beyond isolated-particle characterization toward multidimensional inference in complex mixtures, and expanding the scope of questions nanoscale measurement can address, from real-time tracking of nanoparticle transformations in living and environmental systems to non-invasive quality control of nanomedicine formulations, and beyond.

eess.IV↗

arXiv2026

Rapid tracking through strongly scattering media with physics-informed neuromorphic speckle analysis

This work addresses the critical problem of tracking fast-moving objects through strongly scattering media in a low-light environment. Different from existing approaches that use frame-based cameras with fixed exposure times, which trade off signal-to-noise ratio for temporal resolution, we introduce computational neuromorphic tracking (CNT), a physics-informed framework that combines asynchronous event sensing with task-driven speckle analysis for robust motion estimation. We formulate the neuromorphic speckle aggregation as a spatiotemporal speckle representation, jointly optimizing the temporal and spatial parameters to maximize tracking stability under extreme conditions. Extensive experiments demonstrate that our method enables robust motion tracking of 10x faster motion and under 10x dimmer illumination compared to conventional systems. These improvements significantly broaden the operational regime for tracking through scattering media, providing an efficient and scalable solution for demanding scenarios involving rapid motion and low-light conditions.

cs.CV↗

arXiv2026

Dark-EvGS: Event Camera as an Eye for Radiance Field in the Dark

In low-light environments, conventional cameras often struggle to capture clear multi-view images of objects due to dynamic range limitations and motion blur caused by long exposure. Event cameras, with their high-dynamic range and high-speed properties, have the potential to mitigate these issues. Additionally, 3D Gaussian Splatting (GS) enables radiance field reconstruction, facilitating bright frame synthesis from multiple viewpoints in low-light conditions. However, naively using an event-assisted 3D GS approach still faced challenges because, in low light, events are noisy, frames lack quality, and the color tone may be inconsistent. To address these issues, we propose Dark-EvGS, the first event-assisted 3D GS framework that enables the reconstruction of bright frames from arbitrary viewpoints along the camera trajectory. Triplet-level supervision is proposed to gain holistic knowledge, granular details, and sharp scene rendering. The color tone matching block is proposed to guarantee the color consistency of the rendered frames. Furthermore, we introduce the first real-captured dataset for the event-guided bright frame synthesis task via 3D GS-based radiance field reconstruction. Experiments demonstrate that our method achieves better results than existing methods, conquering radiance field reconstruction under challenging low-light conditions. The code and sample data are included in the supplementary material.

cs.CV↗

arXiv2026

Joint Segmentation and Grading with Iterative Optimization for Multimodal Glaucoma Diagnosis

Accurate diagnosis of glaucoma is challenging, as early-stage changes are subtle and often lack clear structural or appearance cues. Most existing approaches rely on a single modality, such as fundus or optical coherence tomography (OCT), capturing only partial pathological information and often missing early disease progression. In this paper, we propose an iterative multimodal optimization model (IMO) for joint segmentation and grading. IMO integrates fundus and OCT features through a mid-level fusion strategy, enhanced by a cross-modal feature alignment (CMFA) module to reduce modality discrepancies. An iterative refinement decoder progressively optimizes the multimodal features through a denoising diffusion mechanism, enabling fine-grained segmentation of the optic disc and cup while supporting accurate glaucoma grading. Extensive experiments show that our method effectively integrates multimodal features, providing a comprehensive and clinically significant approach to glaucoma assessment. Source codes are available at https://github.com/warren-wzw/IMO.git.

cs.CV↗

arXiv2026

Personalized Federated Learning with Residual Fisher Information for Medical Image Segmentation

Federated learning enables multiple clients (institutions) to collaboratively train machine learning models without sharing their private data. To address the challenge of data heterogeneity across clients, personalized federated learning (pFL) aims to learn customized models for each client. In this work, we propose pFL-ResFIM, a novel pFL framework that achieves client-adaptive personalization at the parameter level. Specifically, we introduce a new metric, Residual Fisher Information Matrix (ResFIM), to quantify the sensitivity of model parameters to domain discrepancies. To estimate ResFIM for each client model under privacy constraints, we employ a spectral transfer strategy that generates simulated data reflecting the domain styles of different clients. Based on the estimated ResFIM, we partition model parameters into domain-sensitive and domain-invariant components. A personalized model for each client is then constructed by aggregating only the domain-invariant parameters on the server. Extensive experiments on public datasets demonstrate that pFL-ResFIM consistently outperforms state-of-the-art methods, validating its effectiveness.

cs.CV↗

arXiv2026

FL-MedSegBench: A Comprehensive Benchmark for Federated Learning on Medical Image Segmentation

Federated learning (FL) offers a privacy-preserving paradigm for collaborative medical image analysis without sharing raw data. However, the absence of standardized benchmarks for medical image segmentation hinders fair and comprehensive evaluation of FL methods. To address this gap, we introduce FL-MedSegBench, the first comprehensive benchmark for federated learning on medical image segmentation. Our benchmark encompasses nine segmentation tasks across ten imaging modalities, covering both 2D and 3D formats with realistic clinical heterogeneity. We systematically evaluate eight generic FL (gFL) and five personalized FL (pFL) methods across multiple dimensions: segmentation accuracy, fairness, communication efficiency, convergence behavior, and generalization to unseen domains. Extensive experiments reveal several key insights: (i) pFL methods, particularly those with client-specific batch normalization (\textit{e.g.}, FedBN), consistently outperform generic approaches; (ii) No single method universally dominates, with performance being dataset-dependent; (iii) Communication frequency analysis shows normalization-based personalization methods exhibit remarkable robustness to reduced communication frequency; (iv) Fairness evaluation identifies methods like Ditto and FedRDN that protect underperforming clients; (v) A method's generalization to unseen domains is strongly tied to its ability to perform well across participating clients. We will release an open-source toolkit to foster reproducible research and accelerate clinically applicable FL solutions, providing empirically grounded guidelines for real-world clinical deployment. The source code is available at https://github.com/meiluzhu/FL-MedSegBench.

cs.CV↗

arXiv2026

Gradually Excavating External Knowledge for Implicit Complex Question Answering

Recently, large language models (LLMs) have gained much attention for the emergence of human-comparable capabilities and huge potential. However, for open-domain implicit question-answering problems, LLMs may not be the ultimate solution due to the reasons of: 1) uncovered or out-of-date domain knowledge, 2) one-shot generation and hence restricted comprehensiveness. To this end, this work proposes a gradual knowledge excavation framework for open-domain complex question answering, where LLMs iteratively and actively acquire external information, and then reason based on acquired historical knowledge. Specifically, during each step of the solving process, the model selects an action to execute, such as querying external knowledge or performing a single logical reasoning step, to gradually progress toward a final answer. Our method can effectively leverage plug-and-play external knowledge and dynamically adjust the strategy for solving complex questions. Evaluated on the StrategyQA dataset, our method achieves 78.17% accuracy with less than 6% parameters of its competitors, setting new SOTA for ~10B-scale LLMs.

cs.CL↗

arXiv2026

Semantic-E2VID: a Semantic-Enriched Paradigm for Event-to-Video Reconstruction

Event cameras provide a promising sensing modality for high-speed and high-dynamic-range vision by asynchronously capturing brightness changes. A fundamental task in event-based vision is event-to-video (E2V) reconstruction, which aims to recover intensity videos from event streams. Most existing E2V approaches formulate reconstruction as a temporal--spatial signal recovery problem, relying on temporal aggregation and spatial feature learning to infer intensity frames. While effective to some extent, this formulation overlooks a critical limitation of event data: due to the change-driven sensing mechanism, event streams are inherently semantically under-determined, lacking object-level structure and contextual information that are essential for faithful reconstruction. In this work, we revisit E2V from a semantic perspective and argue that effective reconstruction requires going beyond temporal and spatial modeling to explicitly account for missing semantic information. Based on this insight, we propose \textit{Semantic-E2VID}, a semantic-enriched end-to-end E2V framework that reformulates reconstruction as a process of semantic learning, fusing and decoding. Our approach first performs semantic abstraction by bridging event representations with semantics extracted from a pretrained Segment Anything Model (SAM), while avoiding modality-induced feature drift. The learned semantics are then fused into the event latent space in a representation-compatible manner, enabling event features to capture object-level structure and contextual cues. Furthermore, semantic-aware supervision is introduced to explicitly guide the reconstruction process toward semantically meaningful regions, complementing conventional pixel-level and temporal objectives. Extensive experiments on six public benchmarks demonstrate that Semantic-E2VID consistently outperforms state-of-the-art E2V methods.

cs.CV↗

arXiv2025

Large-field-of-view lensless imaging with miniaturized sensors

Lensless cameras replace bulky optics with thin modulation masks, enabling compact imaging systems. However, existing methods rely on an idealized model that assumes a globally shift-invariant point spread function (PSF) and sufficiently large sensors. In reality, the PSF varies spatially across the field of view (FOV), and finite sensor boundaries truncate modulated light--effects that intensify as sensors shrink, degrading peripheral reconstruction quality and limiting the effective FOV. We address these limitations through a local-to-global hierarchical framework grounded in a locally shift-invariant convolution model that explicitly accounts for PSF variation and sensor truncation. Patch-wise learned deconvolution first adaptively estimates local PSFs and reconstructs regions independently. A hierarchical enhancement network then progressively expands its receptive field--from small patches through intermediate blocks to the full image--integrating fine local details with global contextual information. Experiments on public datasets show that our method achieves superior reconstruction quality over a larger effective FOV with significantly reduced sensor sizes. Under extreme miniaturization--sensors reduced to 8% of the original area--we achieve improvements of 2 dB (PSNR) and 5% (SSIM), with particularly notable gains in structural fidelity. Code is available at https://github.com/KB504-public/l2g_lensless_imaging .

eess.IV↗

arXiv2025

Inverse Design of Metasurface for Spectral Imaging

Inverse design of metasurfaces for the joint optimization of optical modulation and algorithmic decoding in computational optics presents significant challenges, especially in applications such as hyperspectral imaging. We introduce a physics-data co-driven framework for designing reconfigurable metasurfaces fabricated from the phase-change material Ge2Sb2Se4Te1 to achieve compact, compressive spectral imaging in the shortwave infrared region. Central to our approach is a differentiable neural simulator, trained on over 320,000 simulated geometries, that accurately predicts spectral responses across 11 crystallization states. This differentiability enables end-to-end joint optimization of the metasurface geometry, its spectral encoding function, and a deep reconstruction network. We also propose a soft shape regularization technique that preserves manufacturability during gradient-based updates. Experiments show that our optimized system improves reconstruction fidelity by up to 7.6 dB in the peak-signal-to-noise ratio, with enhanced noise resilience and improved measurement matrix conditioning, underscoring the potential of our approach for high-performance hyperspectral imaging.

eess.IV↗

arXiv2025

EventTracer: Fast Path Tracing-based Event Stream Rendering

Simulating event streams from 3D scenes has become a common practice in event-based vision research, as it meets the demand for large-scale, high temporal frequency data without setting up expensive hardware devices or undertaking extensive data collections. Yet existing methods in this direction typically work with noiseless RGB frames that are costly to render, and therefore they can only achieve a temporal resolution equivalent to 100-300 FPS, far lower than that of real-world event data. In this work, we propose EventTracer, a path tracing-based rendering pipeline that simulates high-fidelity event sequences from complex 3D scenes in an efficient and physics-aware manner. Specifically, we speed up the rendering process via low sample-per-pixel (SPP) path tracing, and train a lightweight event spiking network to denoise the resulting RGB videos into realistic event sequences. To capture the physical properties of event streams, the network is equipped with a bipolar leaky integrate-and-fired (BiLIF) spiking unit and trained with a bidirectional earth mover distance (EMD) loss. Our EventTracer pipeline runs at a speed of about 4 minutes per second of 720p video, and it inherits the merit of accurate spatiotemporal modeling from its path tracing backbone. We show in two downstream tasks that EventTracer captures better scene details and demonstrates a greater similarity to real-world event data than other event simulators, which establishes it as a promising tool for creating large-scale event-RGB datasets at a low cost, narrowing the sim-to-real gap in event-based vision, and boosting various application scenarios such as robotics, autonomous driving, and VRAR.

cs.CV↗

arXiv2025

Learned Off-aperture Encoding for Wide Field-of-view RGBD Imaging

End-to-end (E2E) designed imaging systems integrate coded optical designs with decoding algorithms to enhance imaging fidelity for diverse visual tasks. However, existing E2E designs encounter significant challenges in maintaining high image fidelity at wide fields of view, due to high computational complexity, as well as difficulties in modeling off-axis wave propagation while accounting for off-axis aberrations. In particular, the common approach of placing the encoding element into the aperture or pupil plane results in only a global control of the wavefront. To overcome these limitations, this work explores an additional design choice by positioning a DOE off-aperture, enabling a spatial unmixing of the degrees of freedom and providing local control over the wavefront over the image plane. Our approach further leverages hybrid refractive-diffractive optical systems by linking differentiable ray and wave optics modeling, thereby optimizing depth imaging quality and demonstrating system versatility. Experimental results reveal that the off-aperture DOE enhances the imaging quality by over 5 dB in PSNR at a FoV of approximately $45^\circ$ when paired with a simple thin lens, outperforming traditional on-aperture systems. Furthermore, we successfully recover color and depth information at nearly $28^\circ$ FoV using off-aperture DOE configurations with compound optics. Physical prototypes for both applications validate the effectiveness and versatility of the proposed method.

eess.IV↗