arXiv subjects

Lang Zhang

Publications and source records attributed to Lang Zhang.

14 recordsLinked to original sources

arXiv2026

DA-WAM: Decision-Aligned Future Latents for Driving World Models

Anticipating how scenes evolve under ego actions is fundamental to safe autonomous driving, yet the full potential of world models for decision-making remains unrealized. The critical challenge lies in ensuring that future modeling is not merely predictive, but decision-informative: the predicted future must directly shape which trajectory is selected. Existing approaches decouple future representation learning from planning optimization, or share predicted states across trajectory candidates, thereby diluting the action-specific consequences that ought to guide selection. To bridge this gap, we propose DA-WAM, a framework that unifies predictive representation learning, action-conditioned future modeling, and trajectory scoring under a single decision-making objective. DA-WAM maintains predictive supervision throughout planner optimization via an online encoder and a stable momentum target, allowing future representations to co-evolve with the driving task. An action-conditioned predictor generates a distinct future latent state per trajectory candidate, which is then evaluated by a future-latent-conditioned factorized scorer. For the expert-matched trajectory, the predicted future latent is supervised by the observed future representation, while safety-critical hard negatives provide additional supervision near planning boundaries. Extensive experiments on NAVSIM-v1 and NAVSIM-v2 demonstrate state-of-the-art performance, while ablations and diagnostic analyses validate the key components.

cs.RO↗

arXiv2026

LeapBot-WA: World-Anchor Action Models via Predictive Latent Alignments

World Action Models (WAMs) have emerged as a powerful paradigm for embodied intelligence, yet the prevailing reliance on pixel-level video generation creates a fundamental bottleneck. Forcing models to reconstruct task-irrelevant visual details dissipates representational capacity and renders policies vulnerable to visual distractors. In this paper, we propose LeapBot-WA, which establishes a novel Predictive-Latent paradigm for WAMs by operationalizing the Joint-Embedding Predictive Architecture (JEPA) as a World-Anchor. Departing from the traditional reliance on visual synthesis, LeapBot-WA shifts the core of world modeling to Predictive Semantic Alignment, extracting abstract physical dynamics directly within a latent foundation space. To bridge the modality gap between non-Gaussian predictive features and diffusion priors, we introduce the Isotropic Semantic Autoencoder (ISAE), which reshapes the anchor's latent space into a diffusion-friendly manifold to prevent off-manifold drift. Furthermore, we design an Asymmetric Mixture-of-Transformers (MoT) architecture. During training, an Anchor Diffusion Transformer acts as a privileged dynamics expert to guide the Action Diffusion Transformer; at inference, this heavy dynamics branch is pruned, enabling zero-overhead execution. LeapBot-WA achieves state-of-the-art performance among predictive models on LIBERO and matches top-tier generative WAMs on RoboTwin 2.0 without requiring large-scale trajectory pre-training. It further demonstrates superior zero-shot robustness to unseen environments and successful real-world transfer, establishing a highly efficient and robust latent-centric paradigm for scalable robotic control. Code: https://github.com/LeapWM/leapbot-wa.

cs.RO↗

arXiv2026

EyeCue: Driver Cognitive Distraction Detection via Gaze-Empowered Egocentric Video Understanding

Driver cognitive distraction is a major cause of road collisions and remains difficult to detect. Unlike manual or visual distraction, cognitive distraction is diverted by thoughts unrelated to driving, even when the driver appears visually attentive and exhibits no explicit physical movements. In this work, we propose EyeCue, a gaze-empowered egocentric video understanding framework, to detect driver cognitive distraction. A key insight is that cognitive distraction manifests in the interaction between eye gaze and visual context. To capture this interaction, EyeCue integrates eye gaze with egocentric video to enable context-aware modeling of the driver's attention over time. Furthermore, to tackle the limited scale and diversity of existing datasets, we introduce CogDrive, a comprehensive multi-scenario dataset that augments four existing driving datasets with cognitive distraction annotations. Through extensive evaluations on CogDrive, we show that EyeCue achieves the highest accuracy of 74.38%, outperforming 11 baselines from 6 model families by over 7%. Notably, EyeCue can achieve an accuracy of over 70% across various driving scenarios (different road types, times of day, and weather conditions) with strong generalizability. These results highlight the importance of modeling gaze-context interactions and the effectiveness of cross-modal interaction modeling for multimodal cognitive distraction detection. Our codes and CogDrive dataset resources are available at https://github.com/langzhang2000/EyeCue.

cs.CV↗

arXiv2025

LiSTAR: Ray-Centric World Models for 4D LiDAR Sequences in Autonomous Driving

Synthesizing high-fidelity and controllable 4D LiDAR data is crucial for creating scalable simulation environments for autonomous driving. This task is inherently challenging due to the sensor's unique spherical geometry, the temporal sparsity of point clouds, and the complexity of dynamic scenes. To address these challenges, we present LiSTAR, a novel generative world model that operates directly on the sensor's native geometry. LiSTAR introduces a Hybrid-Cylindrical-Spherical (HCS) representation to preserve data fidelity by mitigating quantization artifacts common in Cartesian grids. To capture complex dynamics from sparse temporal data, it utilizes a Spatio-Temporal Attention with Ray-Centric Transformer (START) that explicitly models feature evolution along individual sensor rays for robust temporal coherence. Furthermore, for controllable synthesis, we propose a novel 4D point cloud-aligned voxel layout for conditioning and a corresponding discrete Masked Generative START (MaskSTART) framework, which learns a compact, tokenized representation of the scene, enabling efficient, high-resolution, and layout-guided compositional generation. Comprehensive experiments validate LiSTAR's state-of-the-art performance across 4D LiDAR reconstruction, prediction, and conditional generation, with substantial quantitative gains: reducing generation MMD by a massive 76%, improving reconstruction IoU by 32%, and lowering prediction L1 Med by 50%. This level of performance provides a powerful new foundation for creating realistic and controllable autonomous systems simulations. Project link: https://ocean-luna.github.io/LiSTAR.gitub.io.

cs.CV↗

arXiv2025

MagicRoad: Semantic-Aware 3D Road Surface Reconstruction via Obstacle Inpainting

Road surface reconstruction is essential for autonomous driving, supporting centimeter-accurate lane perception and high-definition mapping in complex urban environments.While recent methods based on mesh rendering or 3D Gaussian splatting (3DGS) achieve promising results under clean and static conditions, they remain vulnerable to occlusions from dynamic agents, visual clutter from static obstacles, and appearance degradation caused by lighting and weather changes. We present a robust reconstruction framework that integrates occlusion-aware 2D Gaussian surfels with semantic-guided color enhancement to recover clean, consistent road surfaces. Our method leverages a planar-adapted Gaussian representation for efficient large-scale modeling, employs segmentation-guided video inpainting to remove both dynamic and static foreground objects, and enhances color coherence via semantic-aware correction in HSV space. Extensive experiments on urban-scale datasets demonstrate that our framework produces visually coherent and geometrically faithful reconstructions, significantly outperforming prior methods under real-world conditions.

cs.CV↗

arXiv2025

Other Vehicle Trajectories Are Also Needed: A Driving World Model Unifies Ego-Other Vehicle Trajectories in Video Latent Space

Advanced end-to-end autonomous driving systems predict other vehicles' motions and plan ego vehicle's trajectory. The world model that can foresee the outcome of the trajectory has been used to evaluate the autonomous driving system. However, existing world models predominantly emphasize the trajectory of the ego vehicle and leave other vehicles uncontrollable. This limitation hinders their ability to realistically simulate the interaction between the ego vehicle and the driving scenario. In this paper, we propose a driving World Model named EOT-WM, unifying Ego-Other vehicle Trajectories in videos for driving simulation. Specifically, it remains a challenge to match multiple trajectories in the BEV space with each vehicle in the video to control the video generation. We first project ego-other vehicle trajectories in the BEV space into the image coordinate for vehicle-trajectory match via pixel positions. Then, trajectory videos are encoded by the Spatial-Temporal Variational Auto Encoder to align with driving video latents spatially and temporally in the unified visual space. A trajectory-injected diffusion Transformer is further designed to denoise the noisy video latents for video generation with the guidance of ego-other vehicle trajectories. In addition, we propose a metric based on control latent similarity to evaluate the controllability of trajectories. Extensive experiments are conducted on the nuScenes dataset, and the proposed model outperforms the state-of-the-art method by 30% in FID and 55% in FVD. The model can also predict unseen driving scenes with self-produced trajectories.

cs.CV↗

arXiv2025

PAINT: Paying Attention to INformed Tokens to Mitigate Hallucination in Large Vision-Language Model

Large Vision Language Models (LVLMs) have demonstrated remarkable capabilities in understanding and describing visual content, achieving state-of-the-art performance across various vision-language tasks. However, these models often generate descriptions containing objects or details that are absent in the input image, a phenomenon commonly known as hallucination. Our work investigates the key reasons behind this issue by analyzing the pattern of self-attention in transformer layers. We find that hallucinations often arise from the progressive weakening of attention weight to visual tokens in the deeper layers of the LLM. Some previous works naively boost the attention of all visual tokens to mitigate this issue, resulting in suboptimal hallucination reduction. To address this, we identify two critical sets of visual tokens that facilitate the transfer of visual information from the vision encoder to the LLM. Local tokens encode grounded information about objects present in an image, while summary tokens capture the overall aggregated representation of the image. Importantly, these two sets of tokens require different levels of weight enhancement. To this end, we propose \textbf{PAINT} (\textbf{P}aying \textbf{A}ttention to \textbf{IN}formed \textbf{T}okens), a plug-and-play framework that intervenes in the self-attention mechanism of the LLM, selectively boosting the attention weights of local and summary tokens with experimentally learned margins. Evaluation on the MSCOCO image captioning dataset demonstrate that our approach reduces hallucination rates by up to 62.3\% compared to baseline models while maintaining accuracy. Code is available at \href{https://github.com/hasanar1f/PAINT}{https://github.com/hasanar1f/PAINT}

cs.CV↗

arXiv2025

Myocardial T1 mapping at 5T using multi-inversion recovery real-time spoiled GRE

Objective: To develop an accurate myocardial T1 mapping technique at 5T using Look-Locker-based multiple inversion-recovery with the real-time spoiled gradient echo (GRE) acquisition. Approach: The proposed T1 mapping technique (mIR-rt) samples the recovery of inverted magnetization using the real-time GRE and the images captured during diastole are selected for T1 fitting. Multiple-inversion recoveries are employed to increase the sample size for accurate fitting. The T1 mapping method was validated using Bloch simulation, phantom studies, and in 16 healthy volunteers at 5T. Main results: In both simulation and phantom studies, the T1 values measured by mIR-rt closely approximate the reference T1 values, with errors less than 3%, while the conventional MOLLI sequence underestimates T1 values. The myocardial T1 values at 5T are 1553 +/- 52 ms, 1531 +/- 53 ms, and 1526 +/- 60 ms (mean +/- standard deviation) at the apex, middle, and base, respectively. The T1 values measured by MOLLI (1350 +/- 48 ms, 1349 +/- 47 ms, and 1354 +/- 45 ms at the apex, middle, and base) were significantly lower than those of mIR-rt with p<0.001 for all three layers. The mIR-rt sequence method used in our study provides high reproducibility, particularly in the middle slices, supporting its practical relevance for myocardial T1 mapping. Significance: The proposed method is feasible for myocardial T1 mapping at 5T and provides better accuracy than the conventional MOLLI sequence.

physics.med-ph↗

arXiv2024

Diff-DTI: Fast Diffusion Tensor Imaging Using A Feature-Enhanced Joint Diffusion Model

Magnetic resonance diffusion tensor imaging (DTI) is a critical tool for neural disease diagnosis. However, long scan time greatly hinders the widespread clinical use of DTI. To accelerate image acquisition, a feature-enhanced joint diffusion model (Diff-DTI) is proposed to obtain accurate DTI parameter maps from a limited number of diffusion-weighted images (DWIs). Diff-DTI introduces a joint diffusion model that directly learns the joint probability distribution of DWIs with DTI parametric maps for conditional generation. Additionally, a feature enhancement fusion mechanism (FEFM) is designed and incorporated into the generative process of Diff-DTI to preserve fine structures in the generated DTI maps. A comprehensive evaluation of the performance of Diff-DTI was conducted on the Human Connectome Project dataset. The results demonstrate that Diff-DTI outperforms existing state-of-the-art fast DTI imaging methods in terms of visual quality and quantitative metrics. Furthermore, Diff-DTI has shown the ability to produce high-fidelity DTI maps with only three DWIs, thus overcoming the requirement of a minimum of six DWIs for DTI.

eess.IV↗

arXiv2022

All electromagnetic scattering bodies are matrix-valued oscillators

In this article, we introduce a new viewpoint on electromagnetic scattering. Tailoring spectral electromagnetic response underpins important applications ranging from sensing to energy conversion, and is flourishing with new ideas from non-Hermitian physics. There exist excellent theoretical tools for modeling such responses, particularly coupled-mode theories and quasinormal-mode expansions. Yet these approaches offer little insight into the outer limits of what is possible when broadband light interacts with any designable nanophotonic pattern. We show that a special scattering matrix, the "$\mathbb{T}$" matrix, can always be decomposed into a set of fictitious Drude--Lorentz oscillators with matrix-valued (spatially nonlocal) coefficients. For any application and any scatterer, the only designable degrees of freedom are these matrix coefficients, implying strong constraints on lineshapes and response functions that had previously been "hidden." To demonstrate the power of this approach, we apply it to near-field radiative heat transfer, where there has been a long-standing gap between the best known designs and theoretical limits to maximum energy exchange. Our new framework identifies upper bounds that come quite close to the current state-of-the-art, and explains why unconventional plasmonic materials should be superior to conventional plasmonic materials. More generally, this approach can be seamlessly applied to high-interest applications across nanophotonics -- including for metasurfaces, imaging, and photovoltaics -- and may be generalizable to unique challenges that arise in acoustic and/or quantum scattering theory.

physics.optics↗

arXiv2020

Maximal single-frequency electromagnetic response

Modern nanophotonic and meta-optical devices utilize a tremendous number of structural degrees of freedom to enhance light--matter interactions. A fundamental question is how large such enhancements can be. We develop an analytical framework to derive upper bounds to single-frequency electromagnetic response, across near- and far-field regimes, for any materials, naturally incorporating the tandem effects of material- and radiation-induced losses. Our framework relies on a power-conservation law for the polarization fields induced in any scatterer. It unifies previous theories on optical scattering bounds and reveals new insight for optimal nanophotonic design, with applications including far-field scattering, near-field local-density-of-states engineering, optimal wavefront shaping, and the design of perfect absorbers. Our bounds predict strikingly large minimal thicknesses for arbitrarily patterned perfect absorbers, ranging from 50--100 nm for typical materials at visible wavelengths to micrometer-scale thicknesses for polar dielectrics at infrared wavelengths. We use inverse design to discover metasurface structures approaching the minimum-thickness perfect-absorber bounds.

physics.optics↗

arXiv2020

Minimum Dielectric-Resonator Mode Volumes

We show that global lower bounds to the mode volume of a dielectric resonator can be computed via Lagrangian duality. State-of-the-art designs rely on sharp tips, but such structures appear to be highly sub-optimal at nanometer-scale feature sizes, and we demonstrate that computational inverse design offers orders-of-magnitude possible improvements. Our bound can be applied for geometries that are simultaneously resonant at multiple frequencies, for high-efficiency nonlinear-optics applications, and we identify the unavoidable penalties that must accompany such multiresonant structures.

physics.optics↗

arXiv2020

Optimal materials for maximum near-field radiative heat transfer

We consider the space of all causal bulk materials, 2D materials, and metamaterials for maximum near-field radiative heat transfer (RHT). Causality constrains the bandwidth over which plasmonic response can occur, explaining two key traits in ideal materials: small background permittivities (minimal high-energy transitions in 2D materials), and Drude-like free-carrier response, which together optimally yield 10X enhancements beyond the theoretical state-of-the-art. We identify transparent conducting oxides, III-Nitrides, and graphene as materials that should offer nearly ideal near-field RHT rates, if doped to exhibit plasmonic resonances at what we term "near-field Wien frequencies." Deep-subwavelength patterning can provide marginal further gains, at the expense of extremely small feature sizes. Optimal materials have moderate loss rates and plasmonic response at 19 μm for 300K temperature, suggesting a new opportunity for plasmonics at mid- to far-infrared wavelengths, with low carrier concentrations and no requirement to minimize loss.

physics.optics↗

arXiv2019

Dirac-vortex topological cavity

Cavity design is crucial for single-mode semiconductor lasers such as the distributed feedback (DFB) and vertical-cavity surface-emitting lasers (VCSEL). By recognizing that both optical resonators feature a single mid-gap mode localized at the topological defect in a one-dimensional (1D) lattice, we generalize the topological cavity design into 2D using a honeycomb photonic crystal with a vortex Dirac mass -- the analog of Jackiw-Rossi zero modes. We theoretically predict and experimentally demonstrate that such a Dirac-vortex cavity can have a tunable mode area across a few orders of magnitudes, arbitrary mode degeneracy, robustly large free-spectral-range, vector-beam output of low divergence, and compatibility with high-index substrates. This topological cavity could enable photonic crystal surface-emitting lasers (PCSEL) with stabler single-mode operation.

physics.optics↗