arXiv subjects

Jun Wang

Publications and source records attributed to Jun Wang.

At least 415 records · Page 23Linked to original sources

arXiv2025

Re-purposing a modular origami manipulator into an adaptive physical computer for machine learning and robotic perception

Physical computing has emerged as a powerful tool for performing intelligent tasks directly in the mechanical domain of functional materials and robots, reducing our reliance on the more traditional COMS computers. However, no systematic study explains how mechanical design can influence physical computing performance. This study sheds insights into this question by repurposing an origami-inspired modular robotic manipulator into an adaptive physical reservoir and systematically evaluating its computing capacity with different physical configurations, input setups, and computing tasks. By challenging this adaptive reservoir computer to complete the classical NARMA benchmark tasks, this study shows that its time series emulation performance directly correlates to the Peak Similarity Index (PSI), which quantifies the frequency spectrum correlation between the target output and reservoir dynamics. The adaptive reservoir also demonstrates perception capabilities, accurately extracting its payload weight and orientation information from the intrinsic dynamics. Importantly, such information extraction capability can be measured by the spatial correlation between nodal dynamics within the reservoir body. Finally, by integrating shape memory alloy (SMA) actuation, this study demonstrates how to exploit such computing power embodied in the physical body for practical, robotic operations. This study provides a strategic framework for harvesting computing power from soft robots and functional materials, demonstrating how design parameters and input selection can be configured based on computing task requirements. Extending this framework to bio-inspired adaptive materials, prosthetics, and self-adaptive soft robotic systems could enable next-generation embodied intelligence, where the physical structure can compute and interact with their digital counterparts.

cs.RO↗

arXiv2025

Retrieval-augmented in-context learning for multimodal large language models in disease classification

Objectives: We aim to dynamically retrieve informative demonstrations, enhancing in-context learning in multimodal large language models (MLLMs) for disease classification. Methods: We propose a Retrieval-Augmented In-Context Learning (RAICL) framework, which integrates retrieval-augmented generation (RAG) and in-context learning (ICL) to adaptively select demonstrations with similar disease patterns, enabling more effective ICL in MLLMs. Specifically, RAICL examines embeddings from diverse encoders, including ResNet, BERT, BioBERT, and ClinicalBERT, to retrieve appropriate demonstrations, and constructs conversational prompts optimized for ICL. We evaluated the framework on two real-world multi-modal datasets (TCGA and IU Chest X-ray), assessing its performance across multiple MLLMs (Qwen, Llava, Gemma), embedding strategies, similarity metrics, and varying numbers of demonstrations. Results: RAICL consistently improved classification performance. Accuracy increased from 0.7854 to 0.8368 on TCGA and from 0.7924 to 0.8658 on IU Chest X-ray. Multi-modal inputs outperformed single-modal ones, with text-only inputs being stronger than images alone. The richness of information embedded in each modality will determine which embedding model can be used to get better results. Few-shot experiments showed that increasing the number of retrieved examples further enhanced performance. Across different similarity metrics, Euclidean distance achieved the highest accuracy while cosine similarity yielded better macro-F1 scores. RAICL demonstrated consistent improvements across various MLLMs, confirming its robustness and versatility. Conclusions: RAICL provides an efficient and scalable approach to enhance in-context learning in MLLMs for multimodal disease classification.

cs.AI↗

arXiv2025

ST-FlowNet: An Efficient Spiking Neural Network for Event-Based Optical Flow Estimation

Spiking Neural Networks (SNNs) have emerged as a promising tool for event-based optical flow estimation tasks due to their ability to leverage spatio-temporal information and low-power capabilities. However, the performance of SNN models is often constrained, limiting their application in real-world scenarios. In this work, we address this gap by proposing a novel neural network architecture, ST-FlowNet, specifically tailored for optical flow estimation from event-based data. The ST-FlowNet architecture integrates ConvGRU modules to facilitate cross-modal feature augmentation and temporal alignment of the predicted optical flow, improving the network's ability to capture complex motion dynamics. Additionally, to overcome the challenges associated with training SNNs, we introduce a novel approach to derive SNN models from pre-trained artificial neural networks (ANNs) through ANN-to-SNN conversion or our proposed BISNN method. Notably, the BISNN method alleviates the complexities involved in biological parameter selection, further enhancing the robustness of SNNs in optical flow estimation tasks. Extensive evaluations on three benchmark event-based datasets demonstrate that the SNN-based ST-FlowNet model outperforms state-of-the-art methods, delivering superior performance in accurate optical flow estimation across a diverse range of dynamic visual scenes. Furthermore, the inherent energy efficiency of SNN models is highlighted, establishing a compelling advantage for their practical deployment. Overall, our work presents a novel framework for optical flow estimation using SNNs and event-based data, contributing to the advancement of neuromorphic vision applications.

cs.CV↗

arXiv2025

Eisenstein cocycles for imaginary quadratic fields

We construct Eisenstein cocycles for arithmetic subgroups of GL_2 of imaginary quadratic fields valued in second K-groups of products of two CM elliptic curves. We use these to construct maps from the first homology groups of Bianchi spaces to corresponding second K-groups of ray class fields and to verify the Eisenstein property of these maps for prime-to-level Hecke operators.

math.NT↗

arXiv2025

Plug-and-Play Physics-informed Learning using Uncertainty Quantified Port-Hamiltonian Models

The ability to predict trajectories of surrounding agents and obstacles is a crucial component in many robotic applications. Data-driven approaches are commonly adopted for state prediction in scenarios where the underlying dynamics are unknown. However, the performance, reliability, and uncertainty of data-driven predictors become compromised when encountering out-of-distribution observations relative to the training data. In this paper, we introduce a Plug-and-Play Physics-Informed Machine Learning (PnP-PIML) framework to address this challenge. Our method employs conformal prediction to identify outlier dynamics and, in that case, switches from a nominal predictor to a physics-consistent model, namely distributed Port-Hamiltonian systems (dPHS). We leverage Gaussian processes to model the energy function of the dPHS, enabling not only the learning of system dynamics but also the quantification of predictive uncertainty through its Bayesian nature. In this way, the proposed framework produces reliable physics-informed predictions even for the out-of-distribution scenarios.

cs.RO↗

arXiv2025

Mission-driven Exploration for Accelerated Deep Reinforcement Learning with Temporal Logic Task Specifications

This paper addresses the problem of designing control policies for agents with unknown stochastic dynamics and control objectives specified using Linear Temporal Logic (LTL). Recent Deep Reinforcement Learning (DRL) algorithms have aimed to compute policies that maximize the satisfaction probability of LTL formulas, but they often suffer from slow learning performance. To address this, we introduce a novel Deep Q-learning algorithm that significantly improves learning speed. The enhanced sample efficiency stems from a mission-driven exploration strategy that prioritizes exploration towards directions likely to contribute to mission success. Identifying these directions relies on an automaton representation of the LTL task as well as a learned neural network that partially models the agent-environment interaction. We provide comparative experiments demonstrating the efficiency of our algorithm on robot navigation tasks in unseen environments.

cs.RO↗

arXiv2025

Lance: Efficient Random Access in Columnar Storage through Adaptive Structural Encodings

The growing interest in artificial intelligence has created workloads that require both sequential and random access. At the same time, NVMe-backed storage solutions have emerged, providing caching capability for large columnar datasets in cloud storage. Current columnar storage libraries fall short of effectively utilizing an NVMe device's capabilities, especially when it comes to random access. Historically, this has been assumed an implicit weakness in columnar storage formats, but this has not been sufficiently explored. In this paper, we examine the effectiveness of popular columnar formats such as Apache Arrow, Apache Parquet, and Lance in both random access and full scan tasks against NVMe storage. We argue that effective encoding of a column's structure, such as the repetition and validity information, is the key to unlocking the disk's performance. We show that Parquet, when configured correctly, can achieve over 60x better random access performance than default settings. We also show that this high random access performance requires making minor trade-offs in scan performance and RAM utilization. We then describe the Lance structural encoding scheme, which alternates between two different structural encodings based on data width, and achieves better random access performance without making trade-offs in scan performance or RAM utilization.

cs.DB↗

arXiv2025

NTIRE 2025 Challenge on Real-World Face Restoration: Methods and Results

This paper provides a review of the NTIRE 2025 challenge on real-world face restoration, highlighting the proposed solutions and the resulting outcomes. The challenge focuses on generating natural, realistic outputs while maintaining identity consistency. Its goal is to advance state-of-the-art solutions for perceptual quality and realism, without imposing constraints on computational resources or training data. The track of the challenge evaluates performance using a weighted image quality assessment (IQA) score and employs the AdaFace model as an identity checker. The competition attracted 141 registrants, with 13 teams submitting valid models, and ultimately, 10 teams achieved a valid score in the final ranking. This collaborative effort advances the performance of real-world face restoration while offering an in-depth overview of the latest trends in the field.

cs.CV↗

arXiv2025

Combined study of the isospin-violating decay $D^{}_{s}\to D_{s} π^0$ and radiative decay $D^_s\to D_sγ$ with intermediate meson loops

We carry out a combined study of the isospin-violating decay $D_{s}^{*} \to D_{s} π^{0}$ and radiative decay $D^*_s\to D_sγ$ in an effective Lagrangian approach by taking into account the corrections from the one-loop transitions. By distinguishing the transition mechanisms of the long-distance interactions through the intermediate meson loops from the short-distance interactions through the $η-π^{0}$ mixing at the tree level the isospin-violating decay $D_{s}^{*} \to D_{s} π^{0}$ can be well constrained. In our approach the higher order corrections to the isospin-violating effects can involve the intermediate $D^{(*)}$ and $K^{(*)}$ scatterings. We find that the contributions from the destructive interference of intermediate meson loops via $D^{(*)0}(c\bar{u}){K}^{(*)+}(u\bar{s})$ and $D^{(*)+}(c\bar{d}){K}^{(*)0}(d\bar{s})$ rescatterings are significant. Within the commonly accepted ultra-violet (UV) cutoff range we obtain the partial decay width $Γ[D_{s}^{*} \to D_{s} π^{0}] = 9.92^{+0.76}_{-0.66}\,\mathrm{eV}$. This approach allows us to describe the $D_s^*$ radiative decay in the same framework via the vector meson dominance (VMD) model. We demonstrate that both the tree-level and one-loop transitions can be self-consistently determined if we adopt the experimental data for the branching ratio fraction of $D_{s}^{*} \to D_{s} π^{0}$ to $D^*_s\to D_sγ$. It then leads to a reliable estimate of the total decay width of $D_s^*$, i.e. $Γ_{\text{total}}(D_s^{*+})=170^{+ 13}_{-12}\,\mathrm{eV}$.

hep-ph↗

arXiv2025

CMAT: A Multi-Agent Collaboration Tuning Framework for Enhancing Small Language Models

Open large language models (LLMs) have significantly advanced the field of natural language processing, showcasing impressive performance across various tasks.Despite the significant advancements in LLMs, their effective operation still relies heavily on human input to accurately guide the dialogue flow, with agent tuning being a crucial optimization technique that involves human adjustments to the model for better response to such guidance.Addressing this dependency, our work introduces the TinyAgent model, trained on a meticulously curated high-quality dataset. We also present the Collaborative Multi-Agent Tuning (CMAT) framework, an innovative system designed to augment language agent capabilities through adaptive weight updates based on environmental feedback. This framework fosters collaborative learning and real-time adaptation among multiple intelligent agents, enhancing their context-awareness and long-term memory. In this research, we propose a new communication agent framework that integrates multi-agent systems with environmental feedback mechanisms, offering a scalable method to explore cooperative behaviors. Notably, our TinyAgent-7B model exhibits performance on par with GPT-3.5, despite having fewer parameters, signifying a substantial improvement in the efficiency and effectiveness of LLMs.

cs.CL↗

arXiv2025

FaceSpeak: Expressive and High-Quality Speech Synthesis from Human Portraits of Different Styles

Humans can perceive speakers' characteristics (e.g., identity, gender, personality and emotion) by their appearance, which are generally aligned to their voice style. Recently, vision-driven Text-to-speech (TTS) scholars grounded their investigations on real-person faces, thereby restricting effective speech synthesis from applying to vast potential usage scenarios with diverse characters and image styles. To solve this issue, we introduce a novel FaceSpeak approach. It extracts salient identity characteristics and emotional representations from a wide variety of image styles. Meanwhile, it mitigates the extraneous information (e.g., background, clothing, and hair color, etc.), resulting in synthesized speech closely aligned with a character's persona. Furthermore, to overcome the scarcity of multi-modal TTS data, we have devised an innovative dataset, namely Expressive Multi-Modal TTS, which is diligently curated and annotated to facilitate research in this domain. The experimental results demonstrate our proposed FaceSpeak can generate portrait-aligned voice with satisfactory naturalness and quality.

cs.SD↗

arXiv2025

A Highly Efficient Cross-matching Scheme using Learned Index Structure

Spatial data fusion is a bottleneck when it meets the scale of 10 billion records. Cross-matching celestial catalogs is just one example of this. To challenge this, we present a framework that enables efficient cross-matching using Learned Index Structures. Our approach involves a data transformation method to map multi-dimensional data into easily learnable distributions, coupled with a novel search algorithm that leverages the advantages of model pairs, significantly enhancing the efficiency of nearest-neighbor search. In this study, we utilized celestial catalog data derived from astronomical surveys to construct the index and evaluated the speed of the cross-matching process. Using the HEALPix segmentation scheme, we built an independent model object for each tile and developed an end-to-end pipeline to construct a framework with semantic guarantees for record retrieval in query and range search. Our results show that the proposed method improves cross-matching speed by more than four times compared to KD-trees for a radius range between 1 milli-arcseconds and 100 arcseconds.

astro-ph.IM↗

arXiv2025

AutoStyle-TTS: Retrieval-Augmented Generation based Automatic Style Matching Text-to-Speech Synthesis

With the advancement of speech synthesis technology, users have higher expectations for the naturalness and expressiveness of synthesized speech. But previous research ignores the importance of prompt selection. This study proposes a text-to-speech (TTS) framework based on Retrieval-Augmented Generation (RAG) technology, which can dynamically adjust the speech style according to the text content to achieve more natural and vivid communication effects. We have constructed a speech style knowledge database containing high-quality speech samples in various contexts and developed a style matching scheme. This scheme uses embeddings, extracted by Llama, PER-LLM-Embedder,and Moka, to match with samples in the knowledge database, selecting the most appropriate speech style for synthesis. Furthermore, our empirical research validates the effectiveness of the proposed method. Our demo can be viewed at: https://thuhcsi.github.io/icme2025-AutoStyle-TTS

cs.SD↗

arXiv2025

Extraction of fissile isotope antineutrino spectra using feedforward neural network

The precise measurement of the antineutrino spectra produced by isotope fission in reactors is of great significance for studying neutrino oscillations, refining nuclear databases, and addressing the reactor antineutrino anomaly. In this paper, we report a method that utilizes a feedforward neural network (FNN) model to decompose the prompt energy spectrum observed in a short-baseline reactor neutrino experiment and extract the antineutrino spectra produced by the fission of major isotopes such as $^{235}$U, $^{238}$U, $^{239}$Pu, and $^{241}$Pu in the nuclear reactor. We present two training strategies for the model and compare them with the traditional $χ^2$ minimization method by applying them to the same set of pseudo-data corresponding to a total exposure of $(2.9\times 5\times 1800)~\rm{GW_{th}\cdot tonnes\cdot days}$. The results show that the FNN model not only converges faster and better during the fitting process but also achieves relative errors of less than 1\% in the $2-8$ MeV range in the extracted spectra, outperforming the $χ^2$ minimization method. The feasibility and superiority of this method were validated in the study.

hep-ph↗

arXiv2025

Electric-Field-Controlled Chemical Reaction via Piezo-Chemistry Creates Programmable Material Stiffness

The spatial and temporal control of material properties at a distance has yielded many unique innovations including photo-patterning, 3D-printing, and architected material design. To date, most of these innovations have relied on light, heat, sound, or electric current as stimuli for controlling the material properties. Here, we demonstrate that an electric field can induce chemical reactions and subsequent polymerization in composites via piezoelectrically-mediated transduction. The response to an electric field rather than through direct contact with an electrode is mediated by a nanoparticle transducer, i.e., piezoelectric ZnO, which mediates reactions between thiol and alkene monomers, resulting in tunable moduli as a function of voltage, time, and the frequency of the applied AC power. The reactivity of the mixture and the modulus of a naïve material containing these elements can be programmed based on the distribution of the electric field strength. This programmability results in multi-stiffness gels. Additionally, the system can be adjusted for the formation of an electro-adhesive. This simple and generalizable design opens new avenues for facile application in adaptive damping and variable-rigidity materials, adhesive, soft robotics, and potentially tissue engineering.

physics.chem-ph↗

arXiv2025

Mixture of Attentions For Speculative Decoding

The growth in the number of parameters of Large Language Models (LLMs) has led to a significant surge in computational requirements, making them challenging and costly to deploy. Speculative decoding (SD) leverages smaller models to efficiently propose future tokens, which are then verified by the LLM in parallel. Small models that utilise activations from the LLM currently achieve the fastest decoding speeds. However, we identify several limitations of SD models including the lack of on-policyness during training and partial observability. To address these shortcomings, we propose a more grounded architecture for small models by introducing a Mixture of Attentions for SD. Our novel architecture can be applied in two scenarios: a conventional single device deployment and a novel client-server deployment where the small model is hosted on a consumer device and the LLM on a server. In a single-device scenario, we demonstrate state-of-the-art speedups improving EAGLE-2 by 9.5% and its acceptance length by 25%. In a client-server setting, our experiments demonstrate: 1) state-of-the-art latencies with minimal calls to the server for different network conditions, and 2) in the event of a complete disconnection, our approach can maintain higher accuracy compared to other SD methods and demonstrates advantages over API calls to LLMs, which would otherwise be unable to continue the generation process.

cs.CL↗

arXiv2025

SimDC: A High-Fidelity Device Simulation Platform for Device-Cloud Collaborative Computing

The advent of edge intelligence and escalating concerns for data privacy protection have sparked a surge of interest in device-cloud collaborative computing. Large-scale device deployments to validate prototype solutions are often prohibitively expensive and practically challenging, resulting in a pronounced demand for simulation tools that can emulate realworld scenarios. However, existing simulators predominantly rely solely on high-performance servers to emulate edge computing devices, overlooking (1) the discrepancies between virtual computing units and actual heterogeneous computing devices and (2) the simulation of device behaviors in real-world environments. In this paper, we propose a high-fidelity device simulation platform, called SimDC, which uses a hybrid heterogeneous resource and integrates high-performance servers and physical mobile phones. Utilizing this platform, developers can simulate numerous devices for functional testing cost-effectively and capture precise operational responses from varied real devices. To simulate real behaviors of heterogeneous devices, we offer a configurable device behavior traffic controller that dispatches results on devices to the cloud using a user-defined operation strategy. Comprehensive experiments on the public dataset show the effectiveness of our simulation platform and its great potential for application. The code is available at https://github.com/opas-lab/olearning-sim.

cs.DC↗

arXiv2025

SPA-Bench: A Comprehensive Benchmark for SmartPhone Agent Evaluation

Smartphone agents are increasingly important for helping users control devices efficiently, with (Multimodal) Large Language Model (MLLM)-based approaches emerging as key contenders. Fairly comparing these agents is essential but challenging, requiring a varied task scope, the integration of agents with different implementations, and a generalisable evaluation pipeline to assess their strengths and weaknesses. In this paper, we present SPA-Bench, a comprehensive SmartPhone Agent Benchmark designed to evaluate (M)LLM-based agents in an interactive environment that simulates real-world conditions. SPA-Bench offers three key contributions: (1) A diverse set of tasks covering system and third-party apps in both English and Chinese, focusing on features commonly used in daily routines; (2) A plug-and-play framework enabling real-time agent interaction with Android devices, integrating over ten agents with the flexibility to add more; (3) A novel evaluation pipeline that automatically assesses agent performance across multiple dimensions, encompassing seven metrics related to task completion and resource consumption. Our extensive experiments across tasks and agents reveal challenges like interpreting mobile user interfaces, action grounding, memory retention, and execution costs. We propose future research directions to ease these difficulties, moving closer to real-world smartphone agent applications. SPA-Bench is available at https://ai-agents-2030.github.io/SPA-Bench/.

cs.AI↗