arXiv subjects

Han Zhang

Publications and source records attributed to Han Zhang.

At least 217 records · Page 12Linked to original sources

arXiv2025

BACON: Improving Clarity of Image Captions via Bag-of-Concept Graphs

Advancements in large Vision-Language Models have brought precise, accurate image captioning, vital for advancing multi-modal image understanding and processing. Yet these captions often carry lengthy, intertwined contexts that are difficult to parse and frequently overlook essential cues, posing a great barrier for models like GroundingDINO and SDXL, which lack the strong text encoding and syntax analysis needed to fully leverage dense captions. To address this, we propose BACON, a prompting method that breaks down VLM-generated captions into disentangled, structured elements such as objects, relationships, styles, and themes. This approach not only minimizes confusion from handling complex contexts but also allows for efficient transfer into a JSON dictionary, enabling models without linguistic processing capabilities to easily access key information. We annotated 100,000 image-caption pairs using BACON with GPT-4V and trained an LLaVA captioner on this dataset, enabling it to produce BACON-style captions without relying on costly GPT-4V. Evaluations of overall quality, precision, and recall-as well as user studies-demonstrate that the resulting caption model consistently outperforms other SOTA VLM models in generating high-quality captions. Besides, we show that BACON-style captions exhibit better clarity when applied to various models, enabling them to accomplish previously unattainable tasks or surpass existing SOTA solutions without training. For example, BACON-style captions help GroundingDINO achieve 1.51x higher recall scores on open-vocabulary object detection tasks compared to leading methods.

cs.CV↗

arXiv2025

Chemistry-aware battery degradation prediction under simulated real-world cyclic protocols

Battery degradation is governed by complex and randomized cyclic conditions, yet existing modeling and prediction frameworks usually rely on rigid, unchanging protocols that fail to capture real-world dynamics. The stochastic electrical signals make such prediction extremely challenging, while, on the other hand, they provide abundant additional information, such as voltage fluctuations, which may probe the degradation mechanisms. Here, we present chemistry-aware battery degradation prediction under dynamic conditions with machine learning, which integrates hidden Markov processes for realistic power simulations, an automated batch-testing system that generates a large electrochemical dataset under randomized conditions, an interfacial chemistry database derived from high-throughput X-ray photoelectron spectroscopy for mechanistic probing, and a machine learning model for prediction. By automatically constructing a polynomial-scale feature space from irregular electrochemical curves, our model accurately predicts both battery life and critical knee points. This feature space also predicts the composition of the solid electrolyte interphase, revealing six distinct failure mechanisms-demonstrating a viable approach to use electrical signals to infer interfacial chemistry. This work establishes a scalable and adaptive framework for integrating chemical engineering and data science to advance noninvasive diagnostics and optimize processes for more durable and sustainable energy storage technologies.

eess.SP↗

arXiv2025

Beam Selection in ISAC using Contextual Bandit with Multi-modal Transformer and Transfer Learning

Sixth generation (6G) wireless technology is anticipated to introduce Integrated Sensing and Communication (ISAC) as a transformative paradigm. ISAC unifies wireless communication and RADAR or other forms of sensing to optimize spectral and hardware resources. This paper presents a pioneering framework that leverages ISAC sensing data to enhance beam selection processes in complex indoor environments. By integrating multi-modal transformer models with a multi-agent contextual bandit algorithm, our approach utilizes ISAC sensing data to improve communication performance and achieves high spectral efficiency (SE). Specifically, the multi-modal transformer can capture inter-modal relationships, enhancing model generalization across diverse scenarios. Experimental evaluations on the DeepSense 6G dataset demonstrate that our model outperforms traditional deep reinforcement learning (DRL) methods, achieving superior beam prediction accuracy and adaptability. In the single-user scenario, we achieve an average SE regret improvement of 49.6% as compared to DRL. Furthermore, we employ transfer reinforcement learning to reduce training time and improve model performance in multi-user environments. In the multi-user scenario, this approach enhances the average SE regret, which is a measure to demonstrate how far the learned policy is from the optimal SE policy, by 19.7% compared to training from scratch, even when the latter is trained 100 times longer.

eess.SP↗

arXiv2025

Protein Large Language Models: A Comprehensive Survey

Protein-specific large language models (Protein LLMs) are revolutionizing protein science by enabling more efficient protein structure prediction, function annotation, and design. While existing surveys focus on specific aspects or applications, this work provides the first comprehensive overview of Protein LLMs, covering their architectures, training datasets, evaluation metrics, and diverse applications. Through a systematic analysis of over 100 articles, we propose a structured taxonomy of state-of-the-art Protein LLMs, analyze how they leverage large-scale protein sequence data for improved accuracy, and explore their potential in advancing protein engineering and biomedical research. Additionally, we discuss key challenges and future directions, positioning Protein LLMs as essential tools for scientific discovery in protein science. Resources are maintained at https://github.com/Yijia-Xiao/Protein-LLM-Survey.

q-bio.BM↗

arXiv2025

MITracker: Multi-View Integration for Visual Object Tracking

Multi-view object tracking (MVOT) offers promising solutions to challenges such as occlusion and target loss, which are common in traditional single-view tracking. However, progress has been limited by the lack of comprehensive multi-view datasets and effective cross-view integration methods. To overcome these limitations, we compiled a Multi-View object Tracking (MVTrack) dataset of 234K high-quality annotated frames featuring 27 distinct objects across various scenes. In conjunction with this dataset, we introduce a novel MVOT method, Multi-View Integration Tracker (MITracker), to efficiently integrate multi-view object features and provide stable tracking outcomes. MITracker can track any object in video frames of arbitrary length from arbitrary viewpoints. The key advancements of our method over traditional single-view approaches come from two aspects: (1) MITracker transforms 2D image features into a 3D feature volume and compresses it into a bird's eye view (BEV) plane, facilitating inter-view information fusion; (2) we propose an attention mechanism that leverages geometric information from fused 3D feature volume to refine the tracking results at each view. MITracker outperforms existing methods on the MVTrack and GMTD datasets, achieving state-of-the-art performance. The code and the new dataset will be available at https://mii-laboratory.github.io/MITracker/.

cs.CV↗

arXiv2025

DiffKAN-Inpainting: KAN-based Diffusion model for brain tumor inpainting

Brain tumors delay the standard preprocessing workflow for further examination. Brain inpainting offers a viable, although difficult, solution for tumor tissue processing, which is necessary to improve the precision of the diagnosis and treatment. Most conventional U-Net-based generative models, however, often face challenges in capturing the complex, nonlinear latent representations inherent in brain imaging. In order to accomplish high-quality healthy brain tissue reconstruction, this work proposes DiffKAN-Inpainting, an innovative method that blends diffusion models with the Kolmogorov-Arnold Networks architecture. During the denoising process, we introduce the RePaint method and tumor information to generate images with a higher fidelity and smoother margin. Both qualitative and quantitative results demonstrate that as compared to the state-of-the-art methods, our proposed DiffKAN-Inpainting inpaints more detailed and realistic reconstructions on the BraTS dataset. The knowledge gained from ablation study provide insights for future research to balance performance with computing cost.

eess.IV↗

arXiv2025

ArrayBot: Reinforcement Learning for Generalizable Distributed Manipulation through Touch

We present ArrayBot, a distributed manipulation system consisting of a $16 \times 16$ array of vertically sliding pillars integrated with tactile sensors, which can simultaneously support, perceive, and manipulate the tabletop objects. Towards generalizable distributed manipulation, we leverage reinforcement learning (RL) algorithms for the automatic discovery of control policies. In the face of the massively redundant actions, we propose to reshape the action space by considering the spatially local action patch and the low-frequency actions in the frequency domain. With this reshaped action space, we train RL agents that can relocate diverse objects through tactile observations only. Surprisingly, we find that the discovered policy can not only generalize to unseen object shapes in the simulator but also transfer to the physical robot without any domain randomization. Leveraging the deployed policy, we present abundant real-world manipulation tasks, illustrating the vast potential of RL on ArrayBot for distributed manipulation.

cs.RO↗

arXiv2025

DOGlove: Dexterous Manipulation with a Low-Cost Open-Source Haptic Force Feedback Glove

Dexterous hand teleoperation plays a pivotal role in enabling robots to achieve human-level manipulation dexterity. However, current teleoperation systems often rely on expensive equipment and lack multi-modal sensory feedback, restricting human operators' ability to perceive object properties and perform complex manipulation tasks. To address these limitations, we present DOGlove, a low-cost, precise, and haptic force feedback glove system for teleoperation and manipulation. DoGlove can be assembled in hours at a cost under 600 USD. It features a customized joint structure for 21-DoF motion capture, a compact cable-driven torque transmission mechanism for 5-DoF multidirectional force feedback, and a linear resonate actuator for 5-DoF fingertip haptic feedback. Leveraging action and haptic force retargeting, DOGlove enables precise and immersive teleoperation of dexterous robotic hands, achieving high success rates in complex, contact-rich tasks. We further evaluate DOGlove in scenarios without visual feedback, demonstrating the critical role of haptic force feedback in task performance. In addition, we utilize the collected demonstrations to train imitation learning policies, highlighting the potential and effectiveness of DOGlove. DOGlove's hardware and software system will be fully open-sourced at https://do-glove.github.io/.

cs.RO↗

arXiv2025

Towards AI-driven Sign Language Generation with Non-manual Markers

Sign languages are essential for the Deaf and Hard-of-Hearing (DHH) community. Sign language generation systems have the potential to support communication by translating from written languages, such as English, into signed videos. However, current systems often fail to meet user needs due to poor translation of grammatical structures, the absence of facial cues and body language, and insufficient visual and motion fidelity. We address these challenges by building on recent advances in LLMs and video generation models to translate English sentences into natural-looking AI ASL signers. The text component of our model extracts information for manual and non-manual components of ASL, which are used to synthesize skeletal pose sequences and corresponding video frames. Our findings from a user study with 30 DHH participants and thorough technical evaluations demonstrate significant progress and identify critical areas necessary to meet user needs.

cs.HC↗

arXiv2025

A Comprehensive Review of Protein Language Models

At the intersection of the rapidly growing biological data landscape and advancements in Natural Language Processing (NLP), protein language models (PLMs) have emerged as a transformative force in modern research. These models have achieved remarkable progress, highlighting the need for timely and comprehensive overviews. However, much of the existing literature focuses narrowly on specific domains, often missing a broader analysis of PLMs. This study provides a systematic review of PLMs from a macro perspective, covering key historical milestones and current mainstream trends. We focus on the models themselves and their evaluation metrics, exploring aspects such as model architectures, positional encoding, scaling laws, and datasets. In the evaluation section, we discuss benchmarks and downstream applications. To further support ongoing research, we introduce relevant mainstream tools. Lastly, we critically examine the key challenges and limitations in this rapidly evolving field.

q-bio.BM↗

arXiv2025

Postselection-Free Cavity-Enhanced Narrow-Band Orbital Angular Momentum Entangled Photon Source

Cavity-enhanced spontaneous parametric down-conversion (SPDC) provides a significant way to produce $\sim$10 MHz narrow-band photon pairs, which matches the bandwidth of photon for quantum memory. However, the output photon pairs from the cavity is not entangled and the postselection is required to create the entanglement outside the cavity, so the direct output of cavity-enhanced narrow-band entangled photon pairs is still an open challenge. Here we propose a solution that realizes the first postselection-free cavity-enhanced narrow-band entangled photon pairs. The entanglement is achieved in degree of freedom (DOF) of orbital angular momentum (OAM) by implementing an OAM-conservation SPDC process in an actively and precisely controlled cavity supporting degenerate high-order OAM modes. The measured linewidth for the two photons is 13.8 MHz and the measured fidelity is 0.969(3) for the directly generated OAM entangled two photons. We deterministically transfer the OAM entanglement to polarization one with almost no loss and obtain polarization entangled two photons with a fidelity of 0.948(2). Moreover, we produce narrow-band OAM-polarization hyperentangled photon pairs with a fidelity of 0.850(2) by establishing polarization entanglement with preservation of OAM entanglement, which is realized by interfering the two photons on a polarizing beam splitter (PBS) and post-selecting the events of one and only one photon in on each of the PBS port. Novel cavity may find applications in cavity-based light-matter interaction. Our results provide an efficient and promising approach to create narrow-band entangled photon sources for memory-based long-distance quantum communication and network.

quant-ph↗

arXiv2025

Towards Multimodal Empathetic Response Generation: A Rich Text-Speech-Vision Avatar-based Benchmark

Empathetic Response Generation (ERG) is one of the key tasks of the affective computing area, which aims to produce emotionally nuanced and compassionate responses to user's queries. However, existing ERG research is predominantly confined to the singleton text modality, limiting its effectiveness since human emotions are inherently conveyed through multiple modalities. To combat this, we introduce an avatar-based Multimodal ERG (MERG) task, entailing rich text, speech, and facial vision information. We first present a large-scale high-quality benchmark dataset, \textbf{AvaMERG}, which extends traditional text ERG by incorporating authentic human speech audio and dynamic talking-face avatar videos, encompassing a diverse range of avatar profiles and broadly covering various topics of real-world scenarios. Further, we deliberately tailor a system, named \textbf{Empatheia}, for MERG. Built upon a Multimodal Large Language Model (MLLM) with multimodal encoder, speech and avatar generators, Empatheia performs end-to-end MERG, with Chain-of-Empathetic reasoning mechanism integrated for enhanced empathy understanding and reasoning. Finally, we devise a list of empathetic-enhanced tuning strategies, strengthening the capabilities of emotional accuracy and content, avatar-profile consistency across modalities. Experimental results on AvaMERG data demonstrate that Empatheia consistently shows superior performance than baseline methods on both textual ERG and MERG. Overall, this work is expected to pioneer the MERG research by introducing a novel benchmark and an end-to-end model, laying a solid foundation for future advancements in multimodal empathetic response generation.

cs.MM↗

arXiv2025

Roles of the scalar $f_0(500)$ and $f_0(980)$ in the process $D^0\to π^0π^0 \bar{K}^0$

Motivated by the near-threshold enhancement and the dip structure around 1~GeV in the $π^0π^0$ invariant mass distribution of the process $D^0\to π^0π^0\bar{K}^0$ observed by the CLEO Collaboration, we have investigated this process by taking into account the contribution from the $S$-wave pseudoscalar meson-pseudoscalar meson interactions within the chiral unitary approach, and also the one from the intermediate resonance $K^{*}(892)$. Our results are in good agreement with the CLEO measurements, which implies that, the near-threshold enhancement near the $π^0π^0$ threshold is mainly due to the contributions from the scalar meson $f_0(500)$ and the intermediate $K^*$, and the cusp structure around 1~GeV in the $π^0π^0$ invariant mass distribution should be associated with the scalar meson $f_0(980)$.

hep-ph↗

arXiv2025

A phenomenological estimate of rescattering effects in $B_s\to K^{0}\bar{K}^{0}$

The measurements in $b\to s$ penguin-dominated decays are widely recognized as a powerful test for searching for New Physics by studying the deviation from theoretical estimations within the Standard Model. We examine the final-state rescattering effects on the decay $B_s\to K^{*0}\bar{K}^{*0}$ and provide estimations of the branching ratio and longitudinal polarization of $B_s\to K^{*0}\bar{K}^{*0}$, which is consistent with experimental observations. Our conclusion is that both short- and long-distance interactions contribute significantly in this decay. The small longitudinal polarization in $B\to VV$ modes may not be a signal for New Physics.

hep-ph↗

arXiv2025

Addressing Bias in Generative AI: Challenges and Research Opportunities in Information Management

Generative AI technologies, particularly Large Language Models (LLMs), have transformed information management systems but introduced substantial biases that can compromise their effectiveness in informing business decision-making. This challenge presents information management scholars with a unique opportunity to advance the field by identifying and addressing these biases across extensive applications of LLMs. Building on the discussion on bias sources and current methods for detecting and mitigating bias, this paper seeks to identify gaps and opportunities for future research. By incorporating ethical considerations, policy implications, and sociotechnical perspectives, we focus on developing a framework that covers major stakeholders of Generative AI systems, proposing key research questions, and inspiring discussion. Our goal is to provide actionable pathways for researchers to address bias in LLM applications, thereby advancing research in information management that ultimately informs business practices. Our forward-looking framework and research agenda advocate interdisciplinary approaches, innovative methods, dynamic perspectives, and rigorous evaluation to ensure fairness and transparency in Generative AI-driven information systems. We expect this study to serve as a call to action for information management scholars to tackle this critical issue, guiding the improvement of fairness and effectiveness in LLM-based systems for business practice.

cs.CY↗

arXiv2025

Generative AI-enabled Blockage Prediction for Robust Dual-Band mmWave Communication

In mmWave wireless networks, signal blockages present a significant challenge due to the susceptibility to environmental moving obstructions. Recently, the availability of visual data has been leveraged to enhance blockage prediction accuracy in mmWave networks. In this work, we propose a Vision Transformer (ViT)-based approach for visual-aided blockage prediction that intelligently switches between mmWave and Sub-6 GHz frequencies to maximize network throughput and maintain reliable connectivity. Given the computational demands of processing visual data, we implement our solution within a hierarchical fog-cloud computing architecture, where fog nodes collaborate with cloud servers to efficiently manage computational tasks. This structure incorporates a generative AI-based compression technique that significantly reduces the volume of visual data transmitted between fog nodes and cloud centers. Our proposed method is tested with the real-world DeepSense 6G dataset, and according to the simulation results, it achieves a blockage prediction accuracy of 92.78% while reducing bandwidth usage by 70.31%.

eess.SP↗

arXiv2025

Rethinking Early-Fusion Strategies for Improved Multimodal Image Segmentation

RGB and thermal image fusion have great potential to exhibit improved semantic segmentation in low-illumination conditions. Existing methods typically employ a two-branch encoder framework for multimodal feature extraction and design complicated feature fusion strategies to achieve feature extraction and fusion for multimodal semantic segmentation. However, these methods require massive parameter updates and computational effort during the feature extraction and fusion. To address this issue, we propose a novel multimodal fusion network (EFNet) based on an early fusion strategy and a simple but effective feature clustering for training efficient RGB-T semantic segmentation. In addition, we also propose a lightweight and efficient multi-scale feature aggregation decoder based on Euclidean distance. We validate the effectiveness of our method on different datasets and outperform previous state-of-the-art methods with lower parameters and computation.

cs.CV↗

arXiv2025

NLO EW corrections to tau pair production via photon fusion in Pb-Pb ultraperipheral collision

We study the next-to-leading order (NLO) electroweak (EW) corrections to the $γγ\to τ^+ τ^-$ process in Pb-Pb ultraperipheral collision (UPC). We find that the EW correction $δσ_{\mathrm{EW}}$ decreases the total cross section $σ_{\mathrm{NLO}} = σ_{\mathrm{LO}} + δσ_{\mathrm{EW}}$ by -3\% at Pb-Pb center-of-mass energy $\sqrt{s_{NN}}=5.02$ TeV. The weak correction plays significant role whose contribution is about -4 times of that of QED. The CMS and ATLAS collaborations use the reaction $γγ\to τ^+ τ^-$ in Pb-Pb and proton-proton UPC to constrain tau's anomalous magnetic moment $a_τ$. By parameterizing the $γττ$ vertex with two form factors $F_{1,2}$, the cross section can be written as $σ_{a_τ} = σ_{\mathrm{LO}} + δσ_{a_τ}$, where $δσ_{a_τ}$ is proportional to $a_τ$. The impact of NLO EW corrections on $a_τ$ bounds in Pb-Pb UPC is limited, as the current experimental bounds are loose. We also find that various differential distributions of the two ratios $\mathrm{d} σ_{\mathrm{NLO}}/ \mathrm{d} σ_{\mathrm{LO}}$ and $\mathrm{d} σ_{a_τ}/ \mathrm{d} σ_{\mathrm{LO}}$ have different lineshapes. This work is significant to precisely study the interaction of $γττ$ via $γγ\to τ^+ τ^-$ process.

hep-ph↗