arXiv subjects

Dongxu Zhao

Publications and source records attributed to Dongxu Zhao.

6 recordsLinked to original sources

arXiv2026

Fisheye3R: Adapting Unified 3D Feed-Forward Foundation Models to Fisheye Lenses

Feed-forward foundation models for multi-view 3-dimensional (3D) reconstruction have been trained on large-scale datasets of perspective images; when tested on wide field-of-view images, e.g., from a fisheye camera, their performance degrades. This degradation arises from changes in spatial arrangements of pixels induced by the non-linear projection model that maps 3D points onto the 2D image plane. While one may surmise that training on fisheye images would resolve this problem, there are far fewer fisheye images with ground truth than perspective images, which limits generalization. To enable inference on imagery exhibiting high radial distortion, we propose \textit{Fisheye3R}, a novel adaptation framework that extends these multi-view 3D reconstruction foundation models to natively accommodate fisheye inputs without performance regression on perspective images. To address the scarcity of fisheye images and ground truth, we introduce flexible learning schemes that support self-supervised adaptation using only unlabeled perspective images and supervised adaptation without any fisheye training data. Extensive experiments across three foundation models, including VGGT, $π^3$, and MapAnything, demonstrate that our approach consistently improves camera pose, depth, point map, and field-of-view estimation on fisheye images. Code is available at https://github.com/android-xr/fisheye3r.

cs.CV↗

arXiv2026

Microparticles manipulation with a three-dimensional closed optical trap

Gaussian optical tweezers have strong phototoxicity to bioactive substances and it is difficult to achieve capture and manipulation of multiple particles. Moreover, vortex optical tweezers face several challenges, such as weak axial confinement and dependence on complex optical elements like a spatial light modulator (SLM). Therefore, we design and construct a novel three-dimensional (3D) "optical spindle trap" (OST) system that does not require a SLM. By employing intracavity mode modulation and a simple extra-cavity lens modulation, we generate a fully enclosed 3D dark potential well with zero central intensity. Experimental results demonstrate that the system achieves stable trapping of single or multiple micrometer-sized particles with excellent 3D confinement and enables the trapping of mouse hepatocytes under low-power conditions. The system exhibits high physical stability. Its closed dark-field structure can also reduce phototoxic damage to biological samples. Furthermore, the dimensions of the optical trap can be flexibly tuned to suit various application scenarios. This study provides a novel, highly efficient, gentle, and low-cost optical trap platform for biomedicine and micro/nanoscale manipulation.

physics.optics↗

arXiv2026

Reconfigurable all-optical inference via tunable second-harmonic generation and spin-orbit coupling cascade

Spin-orbit coupling (SOC) is widely exploited as a fundamental mechanism for generating orbital angular momentum (OAM); however, conventional approaches typically lack flexibility and tunability. Here, we introduce a continuously tunable second-harmonic generation (SHG)-SOC cascade mechanism modulated by a spatially movable nonlinear crystal. Under linearly polarized excitation, the SHG-SOC cascade engages synchronously with both degenerate and nondegenerate SHG processes, thereby expanding the OAM spectrum and significantly enhancing the information density and feature-mapping capacity of the optical field. Moreover, the OAM spectral distribution can be continuously reconfigured simply by translating the nonlinear crystal. This deterministic physical evolution, which maps simple OAM modes onto a tunable high-dimensional OAM space, is mathematically analogous to the high-dimensional feature expansion performed by a kernel function of a support vector machine (SVM) in machine learning. Such dimensional expansion can project linearly inseparable input data into a high-dimensional space where they become linearly separable. Exploiting this physics-algorithm analogy, we develop a reconfigurable all-optical inference platform. As a proof of concept, we successfully perform classification tasks, including the recognition of Iris flowers and Palmer penguins. This work establishes a scalable, physically reconfigurable architecture for high-dimensional all-optical computing and neuromorphic photonics.

physics.optics↗

arXiv2025

VIN-NBV: A View Introspection Network for Next-Best-View Selection

Next Best View (NBV) algorithms aim to maximize 3D scene acquisition quality using minimal resources, e.g. number of acquisitions, time taken, or distance traversed. Prior methods often rely on coverage maximization as a proxy for reconstruction quality, but for complex scenes with occlusions and finer details, this is not always sufficient and leads to poor reconstructions. Our key insight is to train an acquisition policy that directly optimizes for reconstruction quality rather than just coverage. To achieve this, we introduce the View Introspection Network (VIN): a lightweight neural network that predicts the Relative Reconstruction Improvement (RRI) of a potential next viewpoint without making any new acquisitions. We use this network to power a simple, yet effective, sequential samplingbased greedy NBV policy. Our approach, VIN-NBV, generalizes to unseen object categories, operates without prior scene knowledge, is adaptable to resource constraints, and can handle occlusions. We show that our RRI fitness criterion leads to a ~30% gain in reconstruction quality over a coverage-based criterion using the same greedy strategy. Furthermore, VIN-NBV also outperforms deep reinforcement learning methods, Scan-RL and GenNBV, by ~40%.

cs.CV↗

arXiv2023

Joint Depth Prediction and Semantic Segmentation with Multi-View SAM

Multi-task approaches to joint depth and segmentation prediction are well-studied for monocular images. Yet, predictions from a single-view are inherently limited, while multiple views are available in many robotics applications. On the other end of the spectrum, video-based and full 3D methods require numerous frames to perform reconstruction and segmentation. With this work we propose a Multi-View Stereo (MVS) technique for depth prediction that benefits from rich semantic features of the Segment Anything Model (SAM). This enhanced depth prediction, in turn, serves as a prompt to our Transformer-based semantic segmentation decoder. We report the mutual benefit that both tasks enjoy in our quantitative and qualitative studies on the ScanNet dataset. Our approach consistently outperforms single-task MVS and segmentation models, along with multi-task monocular methods.

cs.CV↗

arXiv2023

MVPSNet: Fast Generalizable Multi-view Photometric Stereo

We propose a fast and generalizable solution to Multi-view Photometric Stereo (MVPS), called MVPSNet. The key to our approach is a feature extraction network that effectively combines images from the same view captured under multiple lighting conditions to extract geometric features from shading cues for stereo matching. We demonstrate these features, termed `Light Aggregated Feature Maps' (LAFM), are effective for feature matching even in textureless regions, where traditional multi-view stereo methods fail. Our method produces similar reconstruction results to PS-NeRF, a state-of-the-art MVPS method that optimizes a neural network per-scene, while being 411$\times$ faster (105 seconds vs. 12 hours) in inference. Additionally, we introduce a new synthetic dataset for MVPS, sMVPS, which is shown to be effective to train a generalizable MVPS method.

cs.CV↗