arXiv subjects

Bin Ren

Publications and source records attributed to Bin Ren.

At least 55 records · Page 3Linked to original sources

arXiv2026

SceneSplat++: A Large Dataset and Comprehensive Benchmark for Language Gaussian Splatting

3D Gaussian Splatting (3DGS) serves as a highly performant and efficient encoding of scene geometry, appearance, and semantics. Moreover, grounding language in 3D scenes has proven to be an effective strategy for 3D scene understanding. Current Language Gaussian Splatting line of work fall into three main groups: (i) per-scene optimization-based, (ii) per-scene optimization-free, and (iii) generalizable approach. However, most of them are evaluated only on rendered 2D views of a handful of scenes and viewpoints close to the training views, limiting ability and insight into holistic 3D understanding. To address this gap, we propose the first large-scale benchmark that systematically assesses these three groups of methods directly in 3D space, evaluating on 1060 scenes across three indoor datasets and one outdoor dataset. Benchmark results demonstrate a clear advantage of the generalizable paradigm, particularly in relaxing the scene-specific limitation, enabling fast feed-forward inference on novel scenes, and achieving superior segmentation performance. We further introduce GaussianWorld-49K a carefully curated 3DGS dataset comprising around 49K diverse indoor and outdoor scenes obtained from multiple sources, with which we demonstrate the generalizable approach could harness strong data priors. Our codes, benchmark, and datasets are released at https://scenesplatpp.gaussianworld.ai/.

cs.CV↗

arXiv2026

Vib2Mol: from vibrational spectra to molecular structures-a unified deep learning framework

There will be a paradigm shift in chemical and biological research, to be enabled by autonomous, closed-loop, real-time self-directed decision-making experimentation. Spectrum-to-structure correlation, which is to elucidate molecular structures with spectral information, is the core step in understanding the experimental results and to close the loop. However, current approaches usually divide the task into either database-dependent retrieval and database-independent generation and neglect the inherent complementarity between them. In this study, we proposed Vib2Mol, a unified deep learning framework designed to flexibly handle diverse spectrum-to-structure tasks according to the available prior knowledge by bridging the retrieval and generation. Empowered by our coarse-to-fine retrieval and generate-then-rerank strategies, Vib2Mol not only achieves state-of-the-art performance in analyzing theoretical Infrared and Raman spectra, but also outperform previous models on experimental data. Moreover, our model demonstrates promising capabilities in predicting reaction products and sequencing peptides, enabling vibrational spectroscopy a potential guide for autonomous scientific discovery workflows.

physics.chem-ph↗

arXiv2026

A Unified Masked Jigsaw Puzzle Framework for Vision and Language Models

In federated learning, Transformer, as a popular architecture, faces critical challenges in defending against gradient attacks and improving model performance in both Computer Vision (CV) and Natural Language Processing (NLP) tasks. It has been revealed that the gradient of Position Embeddings (PEs) in Transformer contains sufficient information, which can be used to reconstruct the input data. To mitigate this issue, we introduce a Masked Jigsaw Puzzle (MJP) framework. MJP starts with random token shuffling to break the token order, and then a learnable \textit{unknown (unk)} position embedding is used to mask out the PEs of the shuffled tokens. In this manner, the local spatial information which is encoded in the position embeddings is disrupted, and the models are forced to learn feature representations that are less reliant on the local spatial information. Notably, with the careful use of MJP, we can not only improve models' robustness against gradient attacks, but also boost their performance in both vision and text application scenarios, such as classification for images (\textit{e.g.,} ImageNet-1K) and sentiment analysis for text (\textit{e.g.,} Yelp and Amazon). Experimental results suggest that MJP is a unified framework for different Transformer-based models in both vision and language tasks. Code is publicly available via https://github.com/ywxsuperstar/transformerattack

cs.CV↗

arXiv2026

Residual Cross-Modal Fusion Networks for Audio-Visual Navigation

Audio-visual embodied navigation aims to enable an agent to autonomously localize and reach a sound source in unseen 3D environments by leveraging auditory cues. The key challenge of this task lies in effectively modeling the interaction between heterogeneous features during multimodal fusion, so as to avoid single-modality dominance or information degradation, particularly in cross-domain scenarios. To address this, we propose a Cross-Modal Residual Fusion Network, which introduces bidirectional residual interactions between audio and visual streams to achieve complementary modeling and fine-grained alignment, while maintaining the independence of their representations. Unlike conventional methods that rely on simple concatenation or attention gating, CRFN explicitly models cross-modal interactions via residual connections and incorporates stabilization techniques to improve convergence and robustness. Experiments on the Replica and Matterport3D datasets demonstrate that CRFN significantly outperforms state-of-the-art fusion baselines and achieves stronger cross-domain generalization. Notably, our experiments also reveal that agents exhibit differentiated modality dependence across different datasets. The discovery of this phenomenon provides a new perspective for understanding the cross-modal collaboration mechanism of embodied agents.

cs.CV↗

arXiv2025

HD 143811 AB b: A Directly Imaged Planet Orbiting a Spectroscopic Binary in Sco-Cen

We present confirmation of HD 143811 AB b, a substellar companion to spectroscopic binary HD 143811 AB through direct imaging with the Gemini Planet Imager (GPI) and Keck NIRC2. HD 143811 AB was observed as a part of the Gemini Planet Imager Exoplanet Survey (GPIES) in 2016 and 2019 and is a member of the Sco-Cen star formation region. The exoplanet is detected $\sim 430$ mas from the host star by GPI. With two GPI epochs and one from Keck/NIRC2 in 2022, we confirm through common proper motion analysis that the object is bound to its host star. We derive an orbit with a semi-major axis of $64 ^{+32}_{-14}$ au and eccentricity ${0.23 ^{+0.24}_{-0.16}}$. Spectral analysis of the GPI $H$-band spectrum and NIRC2 \textit{L'} photometry provides additional proof that this object is a substellar companion. We compare the spectrum of HD 143811 AB b to PHOENIX stellar models and Exo-REM exoplanet atmosphere models and find that Exo-REM models provide the best fits to the data. From the Exo-REM models, we derive an effective temperature of $1042^{+178}_{-132}$ K for the planet and translate the derived luminosity of the planet to a mass of $5.6 \pm 1.1~M_\textrm{Jup}$ assuming hot-start evolutionary models. HD 143811 AB b is the first directly imaged planet around a binary that is not on an ultra-wide orbit. Future characterization of this object will shed light on the formation of planets around binary star systems.

astro-ph.EP↗

arXiv2025

Loomis Painter: Reconstructing the Painting Process

Step-by-step painting tutorials are vital for learning artistic techniques, but existing video resources (e.g., YouTube) lack interactivity and personalization. While recent generative models have advanced artistic image synthesis, they struggle to generalize across media and often show temporal or structural inconsistencies, hindering faithful reproduction of human creative workflows. To address this, we propose a unified framework for multi-media painting process generation with a semantics-driven style control mechanism that embeds multiple media into a diffusion models conditional space and uses cross-medium style augmentation. This enables consistent texture evolution and process transfer across styles. A reverse-painting training strategy further ensures smooth, human-aligned generation. We also build a large-scale dataset of real painting processes and evaluate cross-media consistency, temporal coherence, and final-image fidelity, achieving strong results on LPIPS, DINO, and CLIP metrics. Finally, our Perceptual Distance Profile (PDP) curve quantitatively models the creative sequence, i.e., composition, color blocking, and detail refinement, mirroring human artistic progression.

cs.CV↗

arXiv2025

Optimizing 3D Gaussian Splattering for Mobile GPUs

Image-based 3D scene reconstruction, which transforms multi-view images into a structured 3D representation of the surrounding environment, is a common task across many modern applications. 3D Gaussian Splatting (3DGS) is a new paradigm to address this problem and offers considerable efficiency as compared to the previous methods. Motivated by this, and considering various benefits of mobile device deployment (data privacy, operating without internet connectivity, and potentially faster responses), this paper develops Texture3dgs, an optimized mapping of 3DGS for a mobile GPU. A critical challenge in this area turns out to be optimizing for the two-dimensional (2D) texture cache, which needs to be exploited for faster executions on mobile GPUs. As a sorting method dominates the computations in 3DGS on mobile platforms, the core of Texture3dgs is a novel sorting algorithm where the processing, data movement, and placement are highly optimized for 2D memory. The properties of this algorithm are analyzed in view of a cost model for the texture cache. In addition, we accelerate other steps of the 3DGS algorithm through improved variable layout design and other optimizations. End-to-end evaluation shows that Texture3dgs delivers up to 4.1$\times$ and 1.7$\times$ speedup for the sorting and overall 3D scene reconstruction, respectively -- while also reducing memory usage by up to 1.6$\times$ -- demonstrating the effectiveness of our design for efficient mobile 3D scene reconstruction.

cs.CV↗

arXiv2025

Characterization of the Host Binary of the Directly Imaged Exoplanet HD 143811 AB b

HD~143811~AB is the host star to the directly imaged planet HD~143811~AB~b, which was recently discovered using data from the Gemini Planet Imager and Keck NIRC2. A member of the Sco-Cen star-forming region with an age of $13 \pm 4$ Myr, HD~143811~AB is somewhat rare among hosts of directly imaged planets as it is a close stellar binary, with an $\sim$18 day period. Accurate values for the orbital and stellar parameters of this binary are needed to understand the formation and evolutionary history of the planet in orbit. We utilize archival high-resolution spectroscopy from FEROS on the MPG/ESO 2.2-meter telescope to fit the orbit of the binary, and combine with unresolved photometry to derive the basic stellar properties of the system. From the orbit, we derive precise values of orbital period of $18.59098 \pm 0.00007$ days, and mass ratio of $0.885 \pm 0.003$. When combined with stellar evolutionary models, we find masses of both components of $M_A = 1.30^{+0.03}_{-0.05}$ M$_\odot$ and $M_B = 1.15^{+0.03}_{-0.04}$ M$_\odot$. While the current data are consistent with the planet and stellar orbits being coplanar, the 3D orientations of both systems are currently poorly constrained, with additional observations required to more rigorously test for coplanarity.

astro-ph.EP↗

arXiv2025

Multimodal Spatial Reasoning in the Large Model Era: A Survey and Benchmarks

Humans possess spatial reasoning abilities that enable them to understand spaces through multimodal observations, such as vision and sound. Large multimodal reasoning models extend these abilities by learning to perceive and reason, showing promising performance across diverse spatial tasks. However, systematic reviews and publicly available benchmarks for these models remain limited. In this survey, we provide a comprehensive review of multimodal spatial reasoning tasks with large models, categorizing recent progress in multimodal large language models (MLLMs) and introducing open benchmarks for evaluation. We begin by outlining general spatial reasoning, focusing on post-training techniques, explainability, and architecture. Beyond classical 2D tasks, we examine spatial relationship reasoning, scene and layout understanding, as well as visual question answering and grounding in 3D space. We also review advances in embodied AI, including vision-language navigation and action models. Additionally, we consider emerging modalities such as audio and egocentric video, which contribute to novel spatial understanding through new sensors. We believe this survey establishes a solid foundation and offers insights into the growing field of multimodal spatial reasoning. Updated information about this survey, codes and implementation of the open benchmarks can be found at https://github.com/zhengxuJosh/Awesome-Spatial-Reasoning.

cs.CV↗

arXiv2025

NTIRE 2025 Challenge on Low Light Image Enhancement: Methods and Results

This paper presents a comprehensive review of the NTIRE 2025 Low-Light Image Enhancement (LLIE) Challenge, highlighting the proposed solutions and final outcomes. The objective of the challenge is to identify effective networks capable of producing brighter, clearer, and visually compelling images under diverse and challenging conditions. A remarkable total of 762 participants registered for the competition, with 28 teams ultimately submitting valid entries. This paper thoroughly evaluates the state-of-the-art advancements in LLIE, showcasing the significant progress.

cs.CV↗

arXiv2025

Habitable World Discovery and Characterization: Coronagraph Concept of Operations and Data Post-Processing

The discovery and characterization of habitable worlds was the top scientific recommendation of the Astro2020 decadal survey and is a key objective of the Habitable Worlds Observatory. Biosignature identification drives exceedingly challenging observations, which require raw contrasts of roughly 10$^{-10}$ contrast and ultimately, 1$σ$ photometric precision of roughly 3$\times 10^{-12}$ contrast. Despite significant advances for the Nancy Grace Roman Space Telescope's Coronagraph Instrument, technological gaps still exist in a wide range of technologies such as starlight suppression, deformable mirrors, wavefront control, low noise detectors, and high-contrast spectroscopy. Even with these new technologies matured, the Habitable Worlds Observatory must carefully obtain the observations and rely on post-processing of the data to achieve its science objectives. During the START and TAG efforts, a working group was convened to explore the Coronagraph Concept of Operations and Post Processing (COPP) in the context of the Habitable Worlds Observatory. This COPP working group evaluated coronagraphic concept of operations to enable different post processing approaches, such as reference differential imaging and angular differential imaging, polarization differential imaging, orbital differential imaging, coherent differential imaging, spectral processing, and point-spread function subtraction algorithms that incorporate ancillary telemetry and data. Future integrated modeling simulations and testbed demonstrations are needed to determine the achievable post processing gains for each approach. We report a summary of this working group's activities and findings, as well as an outlook for maturation of these techniques and infusion into the Habitable Worlds Observatory technology portfolio.

astro-ph.IM↗

arXiv2025

Debris disks and their properties with the Habitable Worlds Observatory

The study of the last stages of planet formation, also known as debris disks, is fundamental to place constrains on the formation of planetary sized bodies. Debris disks are composed of dust and occasionally small amounts of gas, both released through dynamical interactions of small rocky bodies and dust particles, such as collisions and evaporation. The distribution of the dust can reveal the presence of forming planets and its composition can directly trace that of comets, asteroids and even planets. While we have been observing debris disks for 40 years now, most observations so far have been restricted to the cold outer regions of the system, and therefore information of the terrestrial zone is still missing. The improved spatial resolution, inner working angle and sensitivity that the Habitable Worlds Observatory will provide will enable a much closer look into the structure and composition of debris disks (particularly of its inner region) and enable the search for the forming rocky planets within the disk.

astro-ph.IM↗

arXiv2025

EarthMind: Leveraging Cross-Sensor Data for Advanced Earth Observation Interpretation with a Unified Multimodal LLM

Earth Observation (EO) data analysis is vital for monitoring environmental and human dynamics. Recent Multimodal Large Language Models (MLLMs) show potential in EO understanding but remain restricted to single-sensor inputs, overlooking the complementarity across heterogeneous modalities. We propose EarthMind, a unified vision-language framework that handles both single- and cross-sensor inputs via an innovative hierarchical cross-modal attention (ie, HCA) design. Specifically, HCA hierarchically captures visual relationships across sensors and aligns them with language queries, enabling adaptive fusion of optical and Synthetic Aperture Radar (SAR) features. To support cross-sensor learning, we curate FusionEO, a 30K-pair dataset with diverse annotations, and establish EarthMind-Bench, a 2,841-pair benchmark with expert annotations for perception and reasoning tasks. Extensive experiments show that EarthMind achieves state-of-the-art results on EarthMind-Bench and surpasses existing MLLMs on multiple EO benchmarks.

cs.CV↗

arXiv2025

Socially Pertinent Robots in Gerontological Healthcare

Despite the many recent achievements in developing and deploying social robotics, there are still many underexplored environments and applications for which systematic evaluation of such systems by end-users is necessary. While several robotic platforms have been used in gerontological healthcare, the question of whether or not a social interactive robot with multi-modal conversational capabilities will be useful and accepted in real-life facilities is yet to be answered. This paper is an attempt to partially answer this question, via two waves of experiments with patients and companions in a day-care gerontological facility in Paris with a full-sized humanoid robot endowed with social and conversational interaction capabilities. The software architecture, developed during the H2020 SPRING project, together with the experimental protocol, allowed us to evaluate the acceptability (AES) and usability (SUS) with more than 60 end-users. Overall, the users are receptive to this technology, especially when the robot perception and action skills are robust to environmental clutter and flexible to handle a plethora of different interactions.

cs.RO↗

arXiv2025

AIM 2025 challenge on Inverse Tone Mapping Report: Methods and Results

This paper presents a comprehensive review of the AIM 2025 Challenge on Inverse Tone Mapping (ITM). The challenge aimed to push forward the development of effective ITM algorithms for HDR image reconstruction from single LDR inputs, focusing on perceptual fidelity and numerical consistency. A total of \textbf{67} participants submitted \textbf{319} valid results, from which the best five teams were selected for detailed analysis. This report consolidates their methodologies and performance, with the lowest PU21-PSNR among the top entries reaching 29.22 dB. The analysis highlights innovative strategies for enhancing HDR reconstruction quality and establishes strong benchmarks to guide future research in inverse tone mapping.

cs.CV↗

arXiv2025

ShapeSplat: A Large-scale Dataset of Gaussian Splats and Their Self-Supervised Pretraining

3D Gaussian Splatting (3DGS) has become the de facto method of 3D representation in many vision tasks. This calls for the 3D understanding directly in this representation space. To facilitate the research in this direction, we first build ShapeSplat, a large-scale dataset of 3DGS using the commonly used ShapeNet, ModelNet and Objaverse datasets. Our dataset ShapeSplat consists of 206K objects spanning over 87 unique categories, whose labels are in accordance with the respective datasets. The creation of this dataset utilized the compute equivalent of 3.8 GPU years on a TITAN XP GPU. We utilize our dataset for unsupervised pretraining and supervised finetuning for classification and segmentation tasks. To this end, we introduce Gaussian-MAE, which highlights the unique benefits of representation learning from Gaussian parameters. Through exhaustive experiments, we provide several valuable insights. In particular, we show that (1) the distribution of the optimized GS centroids significantly differs from the uniformly sampled point cloud (used for initialization) counterpart; (2) this change in distribution results in degradation in classification but improvement in segmentation tasks when using only the centroids; (3) to leverage additional Gaussian parameters, we propose Gaussian feature grouping in a normalized feature space, along with splats pooling layer, offering a tailored solution to effectively group and embed similar Gaussians, which leads to notable improvement in finetuning tasks.

cs.CV↗

arXiv2025

DSGC-Net: A Dual-Stream Graph Convolutional Network for Crowd Counting via Feature Correlation Mining

Deep learning-based crowd counting methods have achieved remarkable progress in recent years. However, in complex crowd scenarios, existing models still face challenges when adapting to significant density distribution differences between regions. Additionally, the inconsistency of individual representations caused by viewpoint changes and body posture differences further limits the counting accuracy of the models. To address these challenges, we propose DSGC-Net, a Dual-Stream Graph Convolutional Network based on feature correlation mining. DSGC-Net introduces a Density Approximation (DA) branch and a Representation Approximation (RA) branch. By modeling two semantic graphs, it captures the potential feature correlations in density variations and representation distributions. The DA branch incorporates a density prediction module that generates the density distribution map, and constructs a density-driven semantic graph based on density similarity. The RA branch establishes a representation-driven semantic graph by computing global representation similarity. Then, graph convolutional networks are applied to the two semantic graphs separately to model the latent semantic relationships, which enhance the model's ability to adapt to density variations and improve counting accuracy in multi-view and multi-pose scenarios. Extensive experiments on three widely used datasets demonstrate that DSGC-Net outperforms current state-of-the-art methods. In particular, we achieve MAE of 48.9 and 5.9 in ShanghaiTech Part A and Part B datasets, respectively. The released code is available at: https://github.com/Wu-eon/CrowdCounting-DSGCNet.

cs.CV↗

arXiv2025

Masked Clustering Prediction for Unsupervised Point Cloud Pre-training

Vision transformers (ViTs) have recently been widely applied to 3D point cloud understanding, with masked autoencoding as the predominant pre-training paradigm. However, the challenge of learning dense and informative semantic features from point clouds via standard ViTs remains underexplored. We propose MaskClu, a novel unsupervised pre-training method for ViTs on 3D point clouds that integrates masked point modeling with clustering-based learning. MaskClu is designed to reconstruct both cluster assignments and cluster centers from masked point clouds, thus encouraging the model to capture dense semantic information. Additionally, we introduce a global contrastive learning mechanism that enhances instance-level feature learning by contrasting different masked views of the same point cloud. By jointly optimizing these complementary objectives, i.e., dense semantic reconstruction, and instance-level contrastive learning. MaskClu enables ViTs to learn richer and more semantically meaningful representations from 3D point clouds. We validate the effectiveness of our method via multiple 3D tasks, including part segmentation, semantic segmentation, object detection, and classification, where MaskClu sets new competitive results. The code and models will be released at:https://github.com/Amazingren/maskclu.

cs.CV↗