arXiv subjects

Tong Wu

Publications and source records attributed to Tong Wu.

At least 145 records · Page 8Linked to original sources

arXiv2025

Instructional Segment Embedding: Improving LLM Safety with Instruction Hierarchy

Large Language Models (LLMs) are susceptible to security and safety threats, such as prompt injection, prompt extraction, and harmful requests. One major cause of these vulnerabilities is the lack of an instruction hierarchy. Modern LLM architectures treat all inputs equally, failing to distinguish between and prioritize various types of instructions, such as system messages, user prompts, and data. As a result, lower-priority user prompts may override more critical system instructions, including safety protocols. Existing approaches to achieving instruction hierarchy, such as delimiters and instruction-based training, do not address this issue at the architectural level. We introduce the Instructional Segment Embedding (ISE) technique, inspired by BERT, to modern large language models, which embeds instruction priority information directly into the model. This approach enables models to explicitly differentiate and prioritize various instruction types, significantly improving safety against malicious prompts that attempt to override priority rules. Our experiments on the Structured Query and Instruction Hierarchy benchmarks demonstrate an average robust accuracy increase of up to 15.75% and 18.68%, respectively. Furthermore, we observe an improvement in instruction-following capability of up to 4.1% evaluated on AlpacaEval. Overall, our approach offers a promising direction for enhancing the safety and effectiveness of LLM architectures.

cs.LG↗

arXiv2025

CoMT: Chain-of-Medical-Thought Reduces Hallucination in Medical Report Generation

Automatic medical report generation (MRG), which possesses significant research value as it can aid radiologists in clinical diagnosis and report composition, has garnered increasing attention. Despite recent progress, generating accurate reports remains arduous due to the requirement for precise clinical comprehension and disease diagnosis inference. Furthermore, owing to the limited accessibility of medical data and the imbalanced distribution of diseases, the underrepresentation of rare diseases in training data makes large-scale medical visual language models (LVLMs) prone to hallucinations, such as omissions or fabrications, severely undermining diagnostic performance and further intensifying the challenges for MRG in practice. In this study, to effectively mitigate hallucinations in medical report generation, we propose a chain-of-medical-thought approach (CoMT), which intends to imitate the cognitive process of human doctors by decomposing diagnostic procedures. The radiological features with different importance are structured into fine-grained medical thought chains to enhance the inferential ability during diagnosis, thereby alleviating hallucination problems and enhancing the diagnostic accuracy of MRG. The code and dataset have been released at https://github.com/FRENKIE-CHIANG/CoMT.

cs.CV↗

arXiv2025

ByTheWay: Boost Your Text-to-Video Generation Model to Higher Quality in a Training-free Way

The text-to-video (T2V) generation models, offering convenient visual creation, have recently garnered increasing attention. Despite their substantial potential, the generated videos may present artifacts, including structural implausibility, temporal inconsistency, and a lack of motion, often resulting in near-static video. In this work, we have identified a correlation between the disparity of temporal attention maps across different blocks and the occurrence of temporal inconsistencies. Additionally, we have observed that the energy contained within the temporal attention maps is directly related to the magnitude of motion amplitude in the generated videos. Based on these observations, we present ByTheWay, a training-free method to improve the quality of text-to-video generation without introducing additional parameters, augmenting memory or sampling time. Specifically, ByTheWay is composed of two principal components: 1) Temporal Self-Guidance improves the structural plausibility and temporal consistency of generated videos by reducing the disparity between the temporal attention maps across various decoder blocks. 2) Fourier-based Motion Enhancement enhances the magnitude and richness of motion by amplifying the energy of the map. Extensive experiments demonstrate that ByTheWay significantly improves the quality of text-to-video generation with negligible additional cost.

cs.CV↗

arXiv2025

RIS-Aided Monitoring With Cooperative Jamming: Design and Performance Analysis

We investigate a reconfigurable intelligent surface (RIS) aided wireless surveillance system. In this system, a monitor not only receives signal from suspicious transmitter via a RIS-enhanced legitimate surveillance (LS) link but also simultaneously takes control of multiple jammers to degrade the quality of received suspicious signal. Under this setup, to enhance monitoring performance requires improvements of both the received signal quality at the monitor and the cooperative jamming (CJ). Considering that the surveillance system is aided by one RIS, whose phase shift optimization involves both channel state information (CSI) of the LS and CJ links, we utilize partial CSI to alleviate the CSI acquisition burden in our design. We propose two RIS-aided monitoring schemes with optimal jammer selection (OJS), and derive their closed-form expressions of surveillance success probability (SSP), respectively. Furthermore, we consider RIS-aided monitoring schemes with random jammer selection as corresponding benchmarks. Thereafter, we analyze special cases where the jammers are using power control to avoid being found, making it appears like passive monitoring. Also, the effect of RIS is highlighted by considering asymptotically large number of RIS elements. Numerical results verify that the proposed OJS strategy further enhances the RIS-aided monitoring performance compared with non-jammer-selection RISLR and RISCR schemes, where the superiority comes at the cost of CSI knowledge and becomes marginal in the region of high jamming power. In addition, the RISLO shows surveillance performance advantage overRISCOwhen the suspicious power is low or when the number of RIS elements is large.

eess.SY↗

arXiv2025

LayerPano3D: Layered 3D Panorama for Hyper-Immersive Scene Generation

3D immersive scene generation is a challenging yet critical task in computer vision and graphics. A desired virtual 3D scene should 1) exhibit omnidirectional view consistency, and 2) allow for free exploration in complex scene hierarchies. Existing methods either rely on successive scene expansion via inpainting or employ panorama representation to represent large FOV scene environments. However, the generated scene suffers from semantic drift during expansion and is unable to handle occlusion among scene hierarchies. To tackle these challenges, we introduce Layerpano3D, a novel framework for full-view, explorable panoramic 3D scene generation from a single text prompt. Our key insight is to decompose a reference 2D panorama into multiple layers at different depth levels, where each layer reveals the unseen space from the reference views via diffusion prior. Layerpano3D comprises multiple dedicated designs: 1) We introduce a new panorama dataset Upright360, comprising 9k high-quality and upright panorama images, and finetune the advanced Flux model on Upright360 for high-quality, upright and consistent panorama generation. 2) We pioneer the Layered 3D Panorama as underlying representation to manage complex scene hierarchies and lift it into 3D Gaussians to splat detailed 360-degree omnidirectional scenes with unconstrained viewing paths. Extensive experiments demonstrate that our framework generates state-of-the-art 3D panoramic scene in both full view consistency and immersive exploratory experience. We believe that Layerpano3D holds promise for advancing 3D panoramic scene creation with numerous applications.

cs.CV↗

arXiv2025

Anisotropic galvanomagnetic effects in single-crystal Fe(001) films elucidated by a phenomenological theory

Utilizing the phenomenological theory based on crystal symmetry operation, we have established the complete angular dependencies of the galvanomagnetic effects, encompassing both anisotropic magnetoresistance (AMR) and the planar Hall effect (PHE), for the ferromagnetic films with C4v symmetry. These dependencies were experimentally confirmed via comprehensive angular-mapping of AMR and PHE in single-crystal Fe(001) films at room temperature. We demonstrated that the intrinsic magnetization-induced effects are independent of the field strength by carefully separating the field-induced and magnetization-induced galvanomagnetic effects. Our theoretical and experimental findings highlight the absence of in-plane four-fold angular dependence in PHE, a feature prohibited by the Onsager relation in systems with C4 symmetry. This study affirms that the universal angular dependencies of AMR and PHE in single crystals can be accurately predicted by the conventional phenomenological theory.

cond-mat.mtrl-sci↗

arXiv2025

RelightVid: Temporal-Consistent Diffusion Model for Video Relighting

Diffusion models have demonstrated remarkable success in image generation and editing, with recent advancements enabling albedo-preserving image relighting. However, applying these models to video relighting remains challenging due to the lack of paired video relighting datasets and the high demands for output fidelity and temporal consistency, further complicated by the inherent randomness of diffusion models. To address these challenges, we introduce RelightVid, a flexible framework for video relighting that can accept background video, text prompts, or environment maps as relighting conditions. Trained on in-the-wild videos with carefully designed illumination augmentations and rendered videos under extreme dynamic lighting, RelightVid achieves arbitrary video relighting with high temporal consistency without intrinsic decomposition while preserving the illumination priors of its image backbone.

cs.CV↗

arXiv2025

Geometric Mean Improves Loss For Few-Shot Learning

Few-shot learning (FSL) is a challenging task in machine learning, demanding a model to render discriminative classification by using only a few labeled samples. In the literature of FSL, deep models are trained in a manner of metric learning to provide metric in a feature space which is well generalizable to classify samples of novel classes; in the space, even a few amount of labeled training examples can construct an effective classifier. In this paper, we propose a novel FSL loss based on \emph{geometric mean} to embed discriminative metric into deep features. In contrast to the other losses such as utilizing arithmetic mean in softmax-based formulation, the proposed method leverages geometric mean to aggregate pair-wise relationships among samples for enhancing discriminative metric across class categories. The proposed loss is not only formulated in a simple form but also is thoroughly analyzed in theoretical ways to reveal its favorable characteristics which are favorable for learning feature metric in FSL. In the experiments on few-shot image classification tasks, the method produces competitive performance in comparison to the other losses.

cs.CV↗

arXiv2025

Proposal of the KOTO II experiment

The KOTO II experiment is proposed to measure the branching ratio of the decay $K_L\toπ^0ν\barν$ at J-PARC. With a beamline to extract long-lived neutral kaons at 5 degrees from a production target, the single event sensitivity of the decay is $8.5\times 10^{-13}$, which is much smaller than the Standard Model prediction $3\times 10^{-11}$. This allows searches for new physics beyond the Standard Model and the first discovery of the decay with a significance exceeding $5σ$. As the only experiment proposed in the world dedicated to rare kaon decays, KOTO II will be indispensable in the quest for a complete understanding of flavor dynamics in the quark sector. Moreover, by combining efforts from the kaon community worldwide, we plan to develop the KOTO II detector further and expand the physics reach of the experiment to include measurements of the branching ratio of the $K_L\toπ^0\ell^+\ell^-$ decays, studies of other $K_L$ decays, and searches for dark photons, axions, and axion-like particles. KOTO II will therefore obtain a comprehensive understanding of $K_L$ decays, providing further constraints on new physics scenarios with existing $K^+$ results.

hep-ex↗

arXiv2025

Digital Twin Online Channel Modeling: Challenges,Principles, and Applications

Different from traditional offline channel modeling, digital twin online channel modeling can sense and accurately characterize dynamic wireless channels in real time, and can therefore greatly assist 6G network optimization. This article proposes a novel promising framework and a step-by-step design procedure of digital twin online channel models (DTOCM). By enabling continuous visualization and accurate prediction of dynamic channel variations, DTOCM can synchronize the performance between simulated and real networks. We first explore the evolution and conceptual advancements of DTOCM, highlighting its visions and associated challenges. Then, we explain its operational principles, construction mechanisms, and applications to typical 6G scenarios. Subsequently, the real-time channel information provisioning and visualization capabilities of DTOCM are illustrated through our DTOCM platform based on practical scenarios. Finally, future research directions and open issues are discussed.

eess.SY↗

arXiv2025

A self-learning magnetic Hopfield neural network with intrinsic gradient descent adaption

Physical neural networks using physical materials and devices to mimic synapses and neurons offer an energy-efficient way to implement artificial neural networks. Yet, training physical neural networks are difficult and heavily relies on external computing resources. An emerging concept to solve this issue is called physical self-learning that uses intrinsic physical parameters as trainable weights. Under external inputs (i.e. training data), training is achieved by the natural evolution of physical parameters that intrinsically adapt modern learning rules via autonomous physical process, eliminating the requirements on external computation resources.Here, we demonstrate a real spintronic system that mimics Hopfield neural networks (HNN) and unsupervised learning is intrinsically performed via the evolution of physical process. Using magnetic texture defined conductance matrix as trainable weights, we illustrate that under external voltage inputs, the conductance matrix naturally evolves and adapts Oja's learning algorithm in a gradient descent manner. The self-learning HNN is scalable and can achieve associative memories on patterns with high similarities. The fast spin dynamics and reconfigurability of magnetic textures offer an advantageous platform towards efficient autonomous training directly in materials.

cond-mat.dis-nn↗

arXiv2025

Diffusion Model-Based Data Synthesis Aided Federated Semi-Supervised Learning

Federated semi-supervised learning (FSSL) is primarily challenged by two factors: the scarcity of labeled data across clients and the non-independent and identically distribution (non-IID) nature of data among clients. In this paper, we propose a novel approach, diffusion model-based data synthesis aided FSSL (DDSA-FSSL), which utilizes a diffusion model (DM) to generate synthetic data, bridging the gap between heterogeneous local data distributions and the global data distribution. In DDSA-FSSL, clients address the challenge of the scarcity of labeled data by employing a federated learning-trained classifier to perform pseudo labeling for unlabeled data. The DM is then collaboratively trained using both labeled and precision-optimized pseudo-labeled data, enabling clients to generate synthetic samples for classes that are absent in their labeled datasets. This process allows clients to generate more comprehensive synthetic datasets aligned with the global distribution. Extensive experiments conducted on multiple datasets and varying non-IID distributions demonstrate the effectiveness of DDSA-FSSL, e.g., it improves accuracy from 38.46% to 52.14% on CIFAR-10 datasets with 10% labeled data.

cs.LG↗

arXiv2024

Automated Progressive Red Teaming

Ensuring the safety of large language models (LLMs) is paramount, yet identifying potential vulnerabilities is challenging. While manual red teaming is effective, it is time-consuming, costly and lacks scalability. Automated red teaming (ART) offers a more cost-effective alternative, automatically generating adversarial prompts to expose LLM vulnerabilities. However, in current ART efforts, a robust framework is absent, which explicitly frames red teaming as an effectively learnable task. To address this gap, we propose Automated Progressive Red Teaming (APRT) as an effectively learnable framework. APRT leverages three core modules: an Intention Expanding LLM that generates diverse initial attack samples, an Intention Hiding LLM that crafts deceptive prompts, and an Evil Maker to manage prompt diversity and filter ineffective samples. The three modules collectively and progressively explore and exploit LLM vulnerabilities through multi-round interactions. In addition to the framework, we further propose a novel indicator, Attack Effectiveness Rate (AER) to mitigate the limitations of existing evaluation metrics. By measuring the likelihood of eliciting unsafe but seemingly helpful responses, AER aligns closely with human evaluations. Extensive experiments with both automatic and human evaluations, demonstrate the effectiveness of ARPT across both open- and closed-source LLMs. Specifically, APRT effectively elicits 54% unsafe yet useful responses from Meta's Llama-3-8B-Instruct, 50% from GPT-4o (API access), and 39% from Claude-3.5 (API access), showcasing its robust attack capability and transferability across LLMs (especially from open-source LLMs to closed-source LLMs).

cs.CR↗

arXiv2024

Sensing Surface Patches in Volume Rendering for Inferring Signed Distance Functions

It is vital to recover 3D geometry from multi-view RGB images in many 3D computer vision tasks. The latest methods infer the geometry represented as a signed distance field by minimizing the rendering error on the field through volume rendering. However, it is still challenging to explicitly impose constraints on surfaces for inferring more geometry details due to the limited ability of sensing surfaces in volume rendering. To resolve this problem, we introduce a method to infer signed distance functions (SDFs) with a better sense of surfaces through volume rendering. Using the gradients and signed distances, we establish a small surface patch centered at the estimated intersection along a ray by pulling points randomly sampled nearby. Hence, we are able to explicitly impose surface constraints on the sensed surface patch, such as multi-view photo consistency and supervision from depth or normal priors, through volume rendering. We evaluate our method by numerical and visual comparisons on scene benchmarks. Our superiority over the latest methods justifies our effectiveness.

cs.CV↗

arXiv2024

The Task Shield: Enforcing Task Alignment to Defend Against Indirect Prompt Injection in LLM Agents

Large Language Model (LLM) agents are increasingly being deployed as conversational assistants capable of performing complex real-world tasks through tool integration. This enhanced ability to interact with external systems and process various data sources, while powerful, introduces significant security vulnerabilities. In particular, indirect prompt injection attacks pose a critical threat, where malicious instructions embedded within external data sources can manipulate agents to deviate from user intentions. While existing defenses based on rule constraints, source spotlighting, and authentication protocols show promise, they struggle to maintain robust security while preserving task functionality. We propose a novel and orthogonal perspective that reframes agent security from preventing harmful actions to ensuring task alignment, requiring every agent action to serve user objectives. Based on this insight, we develop Task Shield, a test-time defense mechanism that systematically verifies whether each instruction and tool call contributes to user-specified goals. Through experiments on the AgentDojo benchmark, we demonstrate that Task Shield reduces attack success rates (2.07\%) while maintaining high task utility (69.79\%) on GPT-4o.

cs.CR↗

arXiv2024

Interfacial Perpendicular Magnetic Anisotropy of Ultrathin Fe(001) Film Grown on CoO(001) Surface

Exploring novel systems with perpendicular magnetic anisotropy (PMA) is vital for advancing memory devices. In this study, we report an intriguing PMA system involving an ultrathin Fe layer on an antiferromagnetic (AFM) CoO(001) surface. The measured perpendicular anisotropy field is inversely proportional to the Fe thickness, indicating an interfacial origin of PMA. Temperature-dependent measurements reveal that the antiferromagnetism of CoO has a negligible effect on the PMA. By leveraging the magneto-optical Kerr effect and birefringence effect, we achieved concurrent visualization of ferromagnetic (FM) and AFM domains. A pronounced coupling effect between these domains was observed near the spin reorientation transition, contrasting sharply with areas of stronger PMA that exhibited weak coupling. This research not only establishes a new FM/AFM bilayer PMA system but also significantly advances the understanding of FM/AFM interfacial interactions.

cond-mat.mtrl-sci↗

arXiv2024

FlashSparse: Minimizing Computation Redundancy for Fast Sparse Matrix Multiplications on Tensor Cores

Sparse Matrix-matrix Multiplication (SpMM) and Sampled Dense-dense Matrix Multiplication (SDDMM) are important sparse operators in scientific computing and deep learning. Tensor Core Units (TCUs) enhance modern accelerators with superior computing power, which is promising to boost the performance of matrix operators to a higher level. However, due to the irregularity of unstructured sparse data, it is difficult to deliver practical speedups on TCUs. To this end, we propose FlashSparse, a novel approach to bridge the gap between sparse workloads and the TCU architecture. Specifically, FlashSparse minimizes the sparse granularity for SpMM and SDDMM on TCUs through a novel swap-and-transpose matrix multiplication strategy. Benefiting from the minimum sparse granularity, the computation redundancy is remarkably reduced while the computing power of TCUs is fully utilized. Besides, FlashSparse is equipped with a memory-efficient thread mapping strategy for coalesced data access and a sparse matrix storage format to save memory footprint. Extensive experimental results on H100 and RTX 4090 GPUs show that FlashSparse sets a new state-of-the-art for sparse matrix multiplications (geometric mean 5.5x speedup over DTC-SpMM and 3.22x speedup over RoDe).

cs.DC↗

arXiv2024

Quasinormal mode as a foundational framework for all electromagnetic Fano resonances

Fano profiles are observed across various fields of wave physics. They emerge from interference phenomena and are quantified by the asymmetry parameter q. In optics, q is usually considered as a phenomenological coefficient obtained by fitting experimental or numerical data. In this work, we introduce an ab initio Maxwellian approach using quasinormal modes to analytically describe line shapes in light scattering problems. We show that the response of each individual quasinormal mode inherently exhibits a Fano profile and derive an explicit analytical formula for the Fano parameter. Experimental and numerical validations confirm the formula's accuracy across a broad spectrum of electromagnetic systems. The general expression for q opens new possibilities for fine-tuning and optimizing spectral line shapes in electromagnetism.

physics.optics↗