arXiv subjects

Zhiwei Li

Publications and source records attributed to Zhiwei Li.

At least 37 records · Page 2Linked to original sources

arXiv2026

AGC: Adaptive Geodesic Correction for Adversarial Robustness on Vision-Language Models

Vision-language models like CLIP have demonstrated remarkable zero-shot transfer capabilities. However, their susceptibility to imperceptible adversarial perturbations remains a critical security concern. While test-time defenses offer a pragmatic solution for deployed models, existing approaches typically rely on gradient-based optimization during inference, incurring significant computational overhead. In this paper, we revisit the role of data augmentation in CLIP robustness and observe that augmentations are not equally effective: specific augmentations consistently provide robust geometric cues that align with correct class semantics in the hyperspherical feature space. Based on this, we propose Adaptive Geodesic Correction (AGC), a training-free defense mechanism that requires no parameter updates. AGC identifies a reliable augmentation as a geometric anchor and corrects the input feature towards it, utilizing an adaptive step size to balance robustness against clean accuracy preservation. AGC achieves superior performance across eight fine-grained datasets and three CLIP backbones, improving average robust accuracy by 44.4\% over state-of-the-art baseline while delivering a 10$\times$ reduction in inference latency. Our findings reveal a fundamental geometric property of CLIP features, offering a highly efficient and effective paradigm for robust multimodal deployment.

cs.CV↗

arXiv2026

DGPO: Beyond Pairwise Preferences with Directional Consistent Groupwise Optimization

Although Large Language Models (LLMs) have made remarkable progress, current preference optimization methods still struggle to align directional consistency while preserving reasoning diversity. To address this limitation, we propose Directional-Groupwise Preference Optimization (DGPO), a lightweight framework that aggregates supervision signals at the group level and explicitly models direction-aware alignment through multi-candidate comparisons. DGPO organizes forward and reverse question-answer instances into structured sets and optimizes a margin-based likelihood objective that separates coherent reasoning paths from inconsistent alternatives. This group-wise formulation captures richer relative information than pairwise objectives and reinforces consistency across diverse reasoning pathways. Empirical results show that our constructed reverse data yields a 3.2% average improvement across five benchmarks, while DGPO further delivers consistent gains across multiple datasets and model families, achieving average accuracy improvements of up to 3.6%.

cs.CL↗

arXiv2026

A Survey of Personalized Federated Foundation Models for Privacy-Preserving Recommendation

Integrating Foundation Models (FMs) into recommendation systems is an emerging and promising research direction. However, centralized paradigms face growing pressure from privacy concerns and strict regulatory requirements. Federated learning offers a viable solution that enables collaborative model refinement while keeping raw user data on local devices or organizational silos. Yet, applying FMs in this setting creates a fundamental tension, where the system must balance the leverage of global knowledge with the necessity of capturing user personality. This survey provides a comprehensive overview of Personalized Federated Foundation Models for privacy-preserving recommendation, and reviews recent progress in this emerging field. We first analyze personalization techniques that function effectively under federated settings. Furthermore, we discuss the adaptation of foundation models to such federated architectures to balance generalization with user-specific needs for achieving privacy-preserving recommendation. In contrast to existing reviews, our work specifically emphasizes the architectural intersection of federation, personalization, and foundation models. \looseness=-1

cs.LG↗

arXiv2026

Reproducibility Beyond Artifacts: Interactional Support for Collaborative Machine Learning

Machine learning (ML) reproducibility is often framed as a problem of incomplete artifact recording. This framing leads to systems that prioritize capturing datasets, code, configurations, and execution environments. However, in collaborative and interdisciplinary ML projects, reproducibility failures often arise not only from missing artifacts but from difficulties in interpreting prior work, aligning evolving components, and reconstructing experimental intent over time. Drawing on a 19-month deployment of a data-centric ML management system in a clinical research project, we identify recurring interactional breakdowns that persist despite comprehensive structural traceability. Based on these findings, we propose a two-layer socio-technical ML management system combining lifecycle-aware artifact infrastructure with an interactional layer designed to mediate coordination, explanation, and shared understanding. We discuss how an AI-mediated semantic interface reframes reproducibility as an ongoing socio-technical accomplishment rather than a static property of recorded traces, and outline implications for human-centered ML infrastructure design.

cs.HC↗

arXiv2026

TTP: Test-Time Padding for Adversarial Detection and Robust Adaptation on Vision-Language Models

Vision-Language Models (VLMs), such as CLIP, have achieved impressive zero-shot recognition performance but remain highly susceptible to adversarial perturbations, posing significant risks in safety-critical scenarios. Previous training-time defenses rely on adversarial fine-tuning, which requires labeled data and costly retraining, while existing test-time strategies fail to reliably distinguish between clean and adversarial inputs, thereby preventing both adversarial robustness and clean accuracy from reaching their optimum. To address these limitations, we propose Test-Time Padding (TTP), a lightweight defense framework that performs adversarial detection followed by targeted adaptation at inference. TTP identifies adversarial inputs via the cosine similarity shift between CLIP feature embeddings computed before and after spatial padding, yielding a universal threshold for reliable detection across architectures and datasets. For detected adversarial cases, TTP employs trainable padding to restore disrupted attention patterns, coupled with a similarity-aware ensemble strategy for a more robust final prediction. For clean inputs, TTP leaves them unchanged by default or optionally integrates existing test-time adaptation techniques for further accuracy gains. Comprehensive experiments on diverse CLIP backbones and fine-grained benchmarks show that TTP consistently surpasses state-of-the-art test-time defenses, delivering substantial improvements in adversarial robustness without compromising clean accuracy. The code for this paper will be released soon.

cs.CV↗

arXiv2026

Learning to Collaborate via Structures: Cluster-Guided Item Alignment for Federated Recommendation

Federated recommendation facilitates collaborative model training across distributed clients while keeping sensitive user interaction data local. Conventional approaches typically rely on synchronizing high-dimensional item representations between the server and clients. This paradigm implicitly assumes that precise geometric alignment of embedding coordinates is necessary for collaboration across clients. We posit that establishing relative semantic relationships among items is more effective than enforcing shared representations. Specifically, global semantic relations serve as structural constraints for items. Within these constraints, the framework allows item representations to vary locally on each client, which flexibility enables the model to capture fine-grained user personalization while maintaining global consistency. To this end, we propose Cluster-Guided FedRec framework (CGFedRec), a framework that transforms uploaded embeddings into compact cluster labels. In this framework, the server functions as a global structure discoverer to learn item clusters and distributes only the resulting labels. This mechanism explicitly cuts off the downstream transmission of item embeddings, relieving clients from maintaining global shared item embeddings. Consequently, CGFedRec achieves the effective injection of global collaborative signals into local item representations without transmitting full embeddings. Extensive experiments demonstrate that our approach significantly improves communication efficiency while maintaining superior recommendation accuracy across multiple datasets.

cs.IR↗

arXiv2026

DiffPlace: Street View Generation via Place-Controllable Diffusion Model Enhancing Place Recognition

Generative models have advanced significantly in realistic image synthesis, with diffusion models excelling in quality and stability. Recent multi-view diffusion models improve 3D-aware street view generation, but they struggle to produce place-aware and background-consistent urban scenes from text, BEV maps, and object bounding boxes. This limits their effectiveness in generating realistic samples for place recognition tasks. To address these challenges, we propose DiffPlace, a novel framework that introduces a place-ID controller to enable place-controllable multi-view image generation. The place-ID controller employs linear projection, perceiver transformer, and contrastive learning to map place-ID embeddings into a fixed CLIP space, allowing the model to synthesize images with consistent background buildings while flexibly modifying foreground objects and weather conditions. Extensive experiments, including quantitative comparisons and augmented training evaluations, demonstrate that DiffPlace outperforms existing methods in both generation quality and training support for visual place recognition. Our results highlight the potential of generative models in enhancing scene-level and place-aware synthesis, providing a valuable approach for improving place recognition in autonomous driving

cs.CV↗

arXiv2026

UV-M3TL: A Unified and Versatile Multimodal Multi-Task Learning Framework for Assistive Driving Perception

Advanced Driver Assistance Systems (ADAS) need to understand human driver behavior while perceiving their navigation context, but jointly learning these heterogeneous tasks would cause inter-task negative transfer and impair system performance. Here, we propose a Unified and Versatile Multimodal Multi-Task Learning (UV-M3TL) framework to simultaneously recognize driver behavior, driver emotion, vehicle behavior, and traffic context, while mitigating inter-task negative transfer. Our framework incorporates two core components: dual-branch spatial channel multimodal embedding (DB-SCME) and adaptive feature-decoupled multi-task loss (AFD-Loss). DB-SCME enhances cross-task knowledge transfer while mitigating task conflicts by employing a dual-branch structure to explicitly model salient task-shared and task-specific features. AFD-Loss improves the stability of joint optimization while guiding the model to learn diverse multi-task representations by introducing an adaptive weighting mechanism based on learning dynamics and feature decoupling constraints. We evaluate our method on the AIDE dataset, and the experimental results demonstrate that UV-M3TL achieves state-of-the-art performance across all four tasks. To further prove the versatility, we evaluate UV-M3TL on additional public multi-task perception benchmarks (BDD100K, CityScapes, NYUD-v2, and PASCAL-Context), where it consistently delivers strong performance across diverse task combinations, attaining state-of-the-art results on most tasks.

cs.CV↗

arXiv2026

TransFR: Transferable Federated Recommendation with Adapter Tuning on Pre-trained Language Models

Federated recommendations (FRs), facilitating multiple local clients to collectively learn a global model without disclosing user private data, have emerged as a prevalent on-device service. In conventional FRs, a dominant paradigm is to utilize discrete identities to represent clients and items, which are then mapped to domain-specific embeddings to participate in model training. Despite considerable performance, we reveal three inherent limitations that can not be ignored in federated settings, i.e., non-transferability across domains, ineffectiveness in cold-start settings, and potential privacy violations during federated training. To this end, we propose a transferable federated recommendation model, TransFR, which delicately incorporates the general capabilities empowered by pre-trained models and the personalized abilities by fine-tuning local private data. Specifically, it first learns domain-agnostic representations of items by exploiting pre-trained models with public textual corpora. To tailor for FR tasks, we further introduce efficient federated adapter-tuning and test-time adaptation mechanisms, which facilitate personalized local adapters for each client by fitting their private data distributions. We theoretically prove the advantages of incorporating adapter tuning in FRs regarding both effectiveness and privacy. Through extensive experiments, we show that our TransFR model surpasses several state-of-the-art FRs on transferability.

cs.IR↗

arXiv2026

Bifurcation Analysis Framework of Spiking Neuron Models

Neuromorphic computing targets energy-efficient event-driven information processing by placing artificial spiking-neurons at its core. Artificial neuron devices and circuits have multiple operating modes and produce region-dependent nonlinear dynamics that are not captured by system analysis methods for linear systems, such as transfer functions, Fourier/Laplace transform, Bode diagram, etc. Thus, new tools are needed to evaluate the nonlinear behavior of neurons and to guide the design and optimization of artificial neuron implementations. Here we present a generalized bifurcation analysis framework based on nonlinear dynamical systems theory. A CMOS axon-hillock neuron, memristor neuron, and the FitzHugh-Nagumo biological neuron model are selected for demonstration. We evaluate Hopf bifurcation conditions to define the rest and firing domains in parameter space of the system, and predict the near-onset firing rate. The results are further compared with numerical simulations. The framework standardizes the analysis for various neuron models and physical realizations. It yields practical design and optimization guidelines for artificial neurons for neuromorphic computing systems, including parameter combinations to make a neuron fire/rest and to control the corresponding firing properties, e.g., firing rate and amplitude.

physics.app-ph↗

arXiv2025

3SGen: Unified Subject, Style, and Structure-Driven Image Generation with Adaptive Task-specific Memory

Recent image generation approaches often address subject, style, and structure-driven conditioning in isolation, leading to feature entanglement and limited task transferability. In this paper, we introduce 3SGen, a task-aware unified framework that performs all three conditioning modes within a single model. 3SGen employs an MLLM equipped with learnable semantic queries to align text-image semantics, complemented by a VAE branch that preserves fine-grained visual details. At its core, an Adaptive Task-specific Memory (ATM) module dynamically disentangles, stores, and retrieves condition-specific priors, such as identity for subjects, textures for styles, and spatial layouts for structures, via a lightweight gating mechanism along with several scalable memory items. This design mitigates inter-task interference and naturally scales to compositional inputs. In addition, we propose 3SGen-Bench, a unified image-driven generation benchmark with standardized metrics for evaluating cross-task fidelity and controllability. Extensive experiments on our proposed 3SGen-Bench and other public benchmarks demonstrate our superior performance across diverse image-driven generation tasks.

cs.CV↗

arXiv2025

Room-Temperature Electrical Readout of Spin Defects in van der Waals Materials

Negatively charged boron vacancy ($\mathrm{V_B^-}$) in hexagonal boron nitride (hBN) is the most extensively studied room-temperature quantum spin system in two-dimensional (2D) materials. Nevertheless, the current effective readout of $\mathrm{V_B^-}$ spin states is carried out by systematically optical methods. This limits their exploitation in compact and miniaturized quantum devices, which would otherwise hold substantial promise to address quantum sensing and quantum information tasks. In this study, we demonstrated a photoelectric spin readout technique for $\mathrm{V_B^-}$ spins in hBN. The observed photocurrent signals stem from the spin-dependent ionization dynamics of boron vacancies, mediated by spin-dependent non-radiative transitions to a metastable state. We further extend this electrical detection technique to enable the readout of dynamical decoupling sequences, including the Carr-Purcell-Meiboom-Gill (CPMG) protocols, and of nuclear spins via electron-nuclear double resonance. These results provide a pathway toward on-chip integration and real-field exploitation of quantum functionalities based on 2D material platforms.

quant-ph↗

arXiv2025

On-the-fly Large-scale 3D Reconstruction from Multi-Camera Rigs

Recent advances in 3D Gaussian Splatting (3DGS) have enabled efficient free-viewpoint rendering and photorealistic scene reconstruction. While on-the-fly extensions of 3DGS have shown promise for real-time reconstruction from monocular RGB streams, they often fail to achieve complete 3D coverage due to the limited field of view (FOV). Employing a multi-camera rig fundamentally addresses this limitation. In this paper, we present the first on-the-fly 3D reconstruction framework for multi-camera rigs. Our method incrementally fuses dense RGB streams from multiple overlapping cameras into a unified Gaussian representation, achieving drift-free trajectory estimation and efficient online reconstruction. We propose a hierarchical camera initialization scheme that enables coarse inter-camera alignment without calibration, followed by a lightweight multi-camera bundle adjustment that stabilizes trajectories while maintaining real-time performance. Furthermore, we introduce a redundancy-free Gaussian sampling strategy and a frequency-aware optimization scheduler to reduce the number of Gaussian primitives and the required optimization iterations, thereby maintaining both efficiency and reconstruction fidelity. Our method reconstructs hundreds of meters of 3D scenes within just 2 minutes using only raw multi-camera video streams, demonstrating unprecedented speed, robustness, and Fidelity for on-the-fly 3D scene reconstruction.

cs.CV↗

arXiv2025

A Bit Level Weight Reordering Strategy Based on Column Similarity to Explore Weight Sparsity in RRAM-based NN Accelerator

Compute-in-Memory (CIM) and weight sparsity are two effective techniques to reduce data movement during Neural Network (NN) inference. However, they can hardly be employed in the same accelerator simultaneously because CIM requires structural compute patterns which are disrupted in sparse NNs. In this paper, we partially solve this issue by proposing a bit level weight reordering strategy which can realize compact mapping of sparse NN weight matrices onto Resistive Random Access Memory (RRAM) based NN Accelerators (RRAM-Acc). In specific, when weights are mapped to RRAM crossbars in a binary complement manner, we can observe that, which can also be mathematically proven, bit-level sparsity and similarity commonly exist in the crossbars. The bit reordering method treats bit sparsity as a special case of bit similarity, reserve only one column in a pair of columns that have identical bit values, and then map the compressed weight matrices into Operation Units (OU). The performance of our design is evaluated with typical NNs. Simulation results show a 61.24% average performance improvement and 1.51x-2.52x energy savings under different sparsity ratios, with only slight overhead compared to the state-of-the-art design.

cs.AR↗

arXiv2025

Scientists' First Exam: Probing Cognitive Abilities of MLLM via Perception, Understanding, and Reasoning

Scientific discoveries increasingly rely on complex multimodal reasoning based on information-intensive scientific data and domain-specific expertise. Empowered by expert-level scientific benchmarks, scientific Multimodal Large Language Models (MLLMs) hold the potential to significantly enhance this discovery process in realistic workflows. However, current scientific benchmarks mostly focus on evaluating the knowledge understanding capabilities of MLLMs, leading to an inadequate assessment of their perception and reasoning abilities. To address this gap, we present the Scientists' First Exam (SFE) benchmark, designed to evaluate the scientific cognitive capacities of MLLMs through three interconnected levels: scientific signal perception, scientific attribute understanding, scientific comparative reasoning. Specifically, SFE comprises 830 expert-verified VQA pairs across three question types, spanning 66 multimodal tasks across five high-value disciplines. Extensive experiments reveal that current state-of-the-art GPT-o3 and InternVL-3 achieve only 34.08% and 26.52% on SFE, highlighting significant room for MLLMs to improve in scientific realms. We hope the insights obtained in SFE will facilitate further developments in AI-enhanced scientific discoveries.

cs.AI↗

arXiv2025

Shape-controlled growth of two-dimensional kagome-lattice colloidal crystals through nanoparticle capping

Organic capping ligands can selectively bind to crystal facets to modulate growth kinetics and are important in chemical synthesis of inorganic nanocrystals. Using the capping ligands for shape-controlled growth of colloidal crystals is challenging due to the size mismatch of molecules and nanoparticle building blocks. In existing synthetic pathways, colloidal crystal shapes are determined by their thermodynamically favored phases yet controlling their shapes independent of lattice symmetry is vital to study many solid-state properties. Here, we develop a nanoparticle capping strategy to control colloidal crystal shapes and structural heterogeneity. Au bipyramids were used as building blocks and assembled into rhombohedral colloidal crystals driven by DNA hybridization. In (111) planes of the crystals, bipyramids assembled into kagome lattices, featuring structure cavities organized in a hexagonal lattice. The rhombohedral crystals have truncated tetrahedral crystal habits, and the degree of truncation defines the exposed facets and crystal shapes. Our surface capping strategy is to introduce DNA-modified nanospheres as effective capping agents, which selectively register on the surface vacancies of the kagome facets and resemble the role of organic ligands in classic nanocrystal growth. Such selective capping is driven by maximizing DNA hybridization and leads to slower growth of the (111) kagome facets, changing the crystal shape from three-dimensional truncated tetrahedra to two-dimensional layered microplates with structural heterogeneity and shape anisotropy. This study underpins the importance of capping agents in colloidal crystal growth and inspires effective ways to control the growth kinetics and heterostructures of colloidal crystals.

cond-mat.soft↗

arXiv2025

Nuclear Ptychoscopy: A Ptychographic Framework for Nuclear Spectroscopy

Accessing both amplitude and phase of nuclear response functions is central to fully characterizing light-matter interactions in the X-ray-nuclear regime. Recent work has demonstrated phase retrieval in two-dimensional time- and energy-resolved spectra, establishing the feasibility of phase-sensitive nuclear spectroscopy. Here, we introduce Nuclear Ptychoscopy, a ptychographic framework that adapts algorithms from coherent diffractive imaging to nuclear spectroscopy, enabling reconstruction of the complex response function by exploiting redundancy in two-dimensional spectra. We develop three complementary reconstruction schemes tailored to distinct experimental scenarios: reconstruction with a known analyzer response, blind reconstruction, and reconstruction incorporating partial prior information. In parallel, we develop geometric analysis techniques that elucidate algorithmic behavior and contribute new tools to ptychography. The framework is validated through experimental data and simulations, demonstrating its versatility across diverse nuclear spectroscopy scenarios and bridging nuclear spectroscopy with ptychography. Beyond advancing quantitative nuclear spectroscopy, our framework opens new opportunities for metrology, coherent control, and quantum applications in the X-ray-nuclear regime.

physics.ins-det↗

arXiv2025

FaStfact: Faster, Stronger Long-Form Factuality Evaluations in LLMs

Evaluating the factuality of long-form generations from Large Language Models (LLMs) remains challenging due to efficiency bottlenecks and reliability concerns. Prior efforts attempt this by decomposing text into claims, searching for evidence, and verifying claims, but suffer from critical drawbacks: (1) inefficiency due to overcomplicated pipeline components, and (2) ineffectiveness stemming from inaccurate claim sets and insufficient evidence. To address these limitations, we propose \textbf{FaStfact}, an evaluation framework that achieves the highest alignment with human evaluation and time/token efficiency among existing baselines. FaStfact first employs chunk-level claim extraction integrated with confidence-based pre-verification, significantly reducing the time and token cost while ensuring reliability. For searching and verification, it collects document-level evidence from crawled web-pages and selectively retrieves it during verification. Extensive experiments based on an annotated benchmark \textbf{FaStfact-Bench} demonstrate the reliability of FaStfact in both efficiently and effectively evaluating long-form factuality. Code, benchmark data, and annotation interface tool are available at https://github.com/Yingjia-Wan/FaStfact.

cs.CL↗