arXiv subjects

Yong Zhuang

Publications and source records attributed to Yong Zhuang.

12 recordsLinked to original sources

arXiv2026

MIDR: Enrichment-Augmented Indexing for Multimodal Document Retrieval

Retrieval over visually rich documents has a representation problem: important content often lives in tables, charts, figures, and layout relations that plain OCR linearizes, corrupts, or omits. ColPali-family visual retrievers address this with patch-level multi-vector indexes and late-interaction scoring, keeping image-derived retrieval on the query-time serving path. We introduce MIDR (Multimodal Indexing for Document Retrieval), a training-free framework for enrichment-augmented indexing that shifts multimodal reasoning to index time. During ingestion, a multimodal LLM converts rendered pages into verified textual fields that are indexed with BM25F and optionally fused with dense retrieval, enabling text-centric serving over multimodally grounded evidence. On ViDoRe V3, MIDR Hybrid achieves 0.6219 average nDCG across five English domains, a 23.0% relative gain over BM25, remaining competitive with ColQwen2.5. On two French-document domains, enrichment bridges English queries and French page text, lifting BM25 from 0.1532 to 0.5448 nDCG and outperforming ColQwen2.5. Across all seven domains, MIDR leads ColQwen2.5 on four while using approximately 9x smaller index memory and approximately 2x lower query latency. These results establish index-time multimodal reasoning as a compelling accuracy-deployment alternative to serving-time visual late interaction.

cs.IR↗

arXiv2026

MazzikaAI: A knowledge-based performance-to-prompt compiler for real-time Arabic maqam accompaniment with a streaming text-to-music model

Arabic maqam music microtonal, modal, and built on ornamented call and response is among the traditions most underserved by generative music models, whose training frameworks remain predominantly Western and equaltempered. Real time accompaniment sharpens this gap: an AI partner must listen, adapt dynamically, and respect idiomatic microtonal structures. Streaming text to music models provide strong generative capabilities but lack precise control interfaces. We present MazzikaAI, a knowledge based system that uses natural language as the actuator of a realtime control loop. By compiling live MIDI, gesture, and inferred harmony into continuously updated text prompts, MazzikaAI steers an unmodified streaming generator, Google Lyria RealTime, without requiring model finetuning. The system embeds expert knowledge of six core maqamat, characteristic ornaments, and ensemble dynamics, maintaining realtime responsiveness with subsecond keytoaudibleupdate latency. Empirical evaluations demonstrate that dynamic prompt compilation reliably grounds generation in microtonal scales, significantly increasing offgrid quartertone content over baseline generation. Beyond its core implementation, MazzikaAI illustrates how deterministic knowledgebased rules can effectively bridge expert, nonWestern musical traditions and unfinetuned foundation models. This architecture establishes a scalable paradigm for realtime humanAI cocreation, offering a generalizable blueprint for interactive accompaniment, adaptive music education, and culturally inclusive generative audio across diverse global idioms.

cs.HC↗

arXiv2026

LLM agents security duality: a comprehensive survey of self-security and empowered cybersecurity

Large language model (LLM) agents are rapidly being integrated into real-world systems. Their autonomy and tool-use capabilities generate substantial value while simultaneously expanding the security attack surface. This survey provides a comprehensive overview of the opportunities and challenges of LLM agents in security, focusing on two core areas: (1) threats to LLM agents themselves and corresponding mitigation strategies (LLM agents self-security), and (2) the role of LLM agents in empowering the cybersecurity lifecycle across offense and defense (LLM agents empowered cybersecurity). We first examine the internal and external attack surfaces of agents, propose a taxonomy organized by threat sources, and analyze associated mitigations and evaluation frameworks. We then investigate how agent capabilities are applied in cybersecurity practice and present, to our knowledge, the first agent-empowerment framework aligned with the full cyber offense-defense lifecycle. By systematically surveying these two areas, we are the first to highlight a positive feedback synergy between LLM agents self-security and empowered cybersecurity, offering new insights for the advancement of both. We further identify current limitations and outline promising directions for future research. The insights provided aim to catalyze the coordinated development of LLM agents self-security and agent empowered cybersecurity, paving the way for more capable and robust agent applications.

cs.CR↗

arXiv2024

A Stealthy Wrongdoer: Feature-Oriented Reconstruction Attack against Split Learning

Split Learning (SL) is a distributed learning framework renowned for its privacy-preserving features and minimal computational requirements. Previous research consistently highlights the potential privacy breaches in SL systems by server adversaries reconstructing training data. However, these studies often rely on strong assumptions or compromise system utility to enhance attack performance. This paper introduces a new semi-honest Data Reconstruction Attack on SL, named Feature-Oriented Reconstruction Attack (FORA). In contrast to prior works, FORA relies on limited prior knowledge, specifically that the server utilizes auxiliary samples from the public without knowing any client's private information. This allows FORA to conduct the attack stealthily and achieve robust performance. The key vulnerability exploited by FORA is the revelation of the model representation preference in the smashed data output by victim client. FORA constructs a substitute client through feature-level transfer learning, aiming to closely mimic the victim client's representation preference. Leveraging this substitute client, the server trains the attack model to effectively reconstruct private data. Extensive experiments showcase FORA's superior performance compared to state-of-the-art methods. Furthermore, the paper systematically evaluates the proposed method's applicability across diverse settings and advanced defense strategies.

cs.CR↗

arXiv2023

Dense Retrieval Adaptation using Target Domain Description

In information retrieval (IR), domain adaptation is the process of adapting a retrieval model to a new domain whose data distribution is different from the source domain. Existing methods in this area focus on unsupervised domain adaptation where they have access to the target document collection or supervised (often few-shot) domain adaptation where they additionally have access to (limited) labeled data in the target domain. There also exists research on improving zero-shot performance of retrieval models with no adaptation. This paper introduces a new category of domain adaptation in IR that is as-yet unexplored. Here, similar to the zero-shot setting, we assume the retrieval model does not have access to the target document collection. In contrast, it does have access to a brief textual description that explains the target domain. We define a taxonomy of domain attributes in retrieval tasks to understand different properties of a source domain that can be adapted to a target domain. We introduce a novel automatic data construction pipeline that produces a synthetic document collection, query set, and pseudo relevance labels, given a textual domain description. Extensive experiments on five diverse target domains show that adapting dense retrieval models using the constructed synthetic data leads to effective retrieval performance on the target domain.

cs.IR↗

arXiv2022

Novelty Controlled Paraphrase Generation with Retrieval Augmented Conditional Prompt Tuning

Paraphrase generation is a fundamental and long-standing task in natural language processing. In this paper, we concentrate on two contributions to the task: (1) we propose Retrieval Augmented Prompt Tuning (RAPT) as a parameter-efficient method to adapt large pre-trained language models for paraphrase generation; (2) we propose Novelty Conditioned RAPT (NC-RAPT) as a simple model-agnostic method of using specialized prompt tokens for controlled paraphrase generation with varying levels of lexical novelty. By conducting extensive experiments on four datasets, we demonstrate the effectiveness of the proposed approaches for retaining the semantic content of the original text while inducing lexical novelty in the generation.

cs.CL↗

arXiv2021

Federated Natural Language Generation for Personalized Dialogue System

Neural conversational models have long suffered from the problem of inconsistency and lacking coherent personality. To address the issue, persona-based models capturing individual characteristics have been proposed, but they still face the dilemma of model adaption and data privacy. To break this dilemma, we propose a novel Federated Natural Language Generation (FedNLG) framework, which learns personalized representations from various dataset on distributed devices, and thus implements the personalized dialogue system efficiently and safely. FedNLG first pre-trains parameters of standard neural conversational model over a large dialogue corpus, and then fine-tune the model parameters and persona embeddings on specific datasets, in a federated manner. Thus, the model could simultaneously learn the persona embeddings in local clients and learn shared model parameters by federated aggregation, which achieves accuracyprivacy balance. By conducting extensive experiments, we demonstrate the effectiveness of our model by pre-training model over Cornell Movie-Dialogs Corpus and fine-tuning the model over two TV series dataset.

cs.CL↗

arXiv2019

The Effects of Evolutionary Adaptations on Spreading Processes in Complex Networks

A common theme among the proposed models for network epidemics is the assumption that the propagating object, i.e., a virus or a piece of information, is transferred across the nodes without going through any modification or evolution. However, in real-life spreading processes, pathogens often evolve in response to changing environments and medical interventions and information is often modified by individuals before being forwarded. In this paper, we investigate the evolution of spreading processes on complex networks with the aim of i) revealing the role of evolution on the threshold, probability, and final size of epidemics; and ii) exploring the interplay between the structural properties of the network and the dynamics of evolution. In particular, we develop a mathematical theory that accurately predicts the epidemic threshold and the expected epidemic size as functions of the characteristics of the spreading process, the evolutionary dynamics of the pathogen, and the structure of the underlying contact network. In addition to the mathematical theory, we perform extensive simulations on random and real-world contact networks to verify our theory and reveal the significant shortcomings of the classical mathematical models that do not capture evolution. Our results reveal that the classical, single-type bond-percolation models may accurately predict the threshold and final size of epidemics, but their predictions on the probability of emergence are inaccurate on both random and real-world networks. This inaccuracy sheds the light on a fundamental disconnect between the classical bond-percolation models and real-life spreading processes that entail evolution. Finally, we consider the case when co-infection is possible and show that co-infection could lead the order of phase transition to change from second-order to first-order.

physics.soc-ph↗

arXiv2018

Multi-Stage Complex Contagions in Random Multiplex Networks

Complex contagion models have been developed to understand a wide range of social phenomena such as adoption of cultural fads, the diffusion of belief, norms, and innovations in social networks, and the rise of collective action to join a riot. Most existing works focus on contagions where individuals' states are represented by {\em binary} variables, and propagation takes place over a single isolated network. However, characterization of an individual's standing on a given matter as a binary state might be overly simplistic as most of our opinions, feelings, and perceptions vary over more than two states. Also, most real-world contagions take place over multiple networks (e.g., Twitter and Facebook) or involve {\em multiplex} networks where individuals engage in different {\em types} of relationships (e.g., acquaintance, co-worker, family, etc.). To this end, this paper studies {\em multi-stage} complex contagions that take place over multi-layer or multiplex networks. Under a linear threshold based contagion model, we give analytic results for the probability and expected size of \textit{global} cascades, i.e., cases where a randomly chosen node can initiate a propagation that eventually reaches a {\em positive} fraction of the whole population. Analytic results are also confirmed and supported by an extensive numerical study. In particular, we demonstrate how the dynamics of complex contagions is affected by the extra weight exerted by \textit{hyper-active} nodes and by the structural properties of the networks involved. Among other things, we reveal an interesting connection between the assortativity of a network and the impact of \textit{hyper-active} nodes on the cascade size.

physics.soc-ph↗

arXiv2016

Clustering determines the dynamics of complex contagions in multiplex networks

We present the mathematical analysis of generalized complex contagions in clustered multiplex networks for susceptible-infected-recovered (SIR)-like dynamics. The model is intended to understand diffusion of influence, or any other spreading process implying a threshold dynamics, in setups of interconnected networks with significant clustering. The contagion is assumed to be general enough to account for a content-dependent linear threshold model, where each link type has a different weight (for spreading influence) that may depend on the content (e.g., product, rumor, political view) that is being spread. Using the generating functions formalism, we determine the conditions, probability, and expected size of the emergent global cascades. This analysis provides a generalization of previous approaches and is specially useful in problems related to spreading and percolation. The results present non trivial dependencies between the clustering coefficient of the networks and its average degree. In particular, several phase transitions are shown to occur depending on these descriptors. Generally speaking, our findings reveal that increasing clustering decreases the probability of having global cascades and their size, however this tendency changes with the average degree. There exists a certain average degree from which on clustering favours the probability and size of the contagion. By comparing the dynamics of complex contagions over multiplex networks and their monoplex projections, we demonstrate that ignoring link types and aggregating network layers may lead to inaccurate conclusions about contagion dynamics, particularly when the correlation of degrees between layers is high.

physics.soc-ph↗

arXiv2015

Information Propagation in Clustered Multilayer Networks

In today's world, individuals interact with each other in more complicated patterns than ever. Some individuals engage through online social networks (e.g., Facebook, Twitter), while some communicate only through conventional ways (e.g., face-to-face). Therefore, understanding the dynamics of information propagation among humans calls for a multi-layer network model where an online social network is conjoined with a physical network. In this work, we initiate a study of information diffusion in a clustered multi-layer network model, where all constituent layers are random networks with high clustering. We assume that information propagates according to the SIR model and with different information transmissibility across the networks. We give results for the conditions, probability, and size of information epidemics, i.e., cases where information starts from a single individual and reaches a positive fraction of the population. We show that increasing the level of clustering in either one of the layers increases the epidemic threshold and decreases the final epidemic size in the whole system. An interesting finding is that information with low transmissibility spreads more effectively with a small but densely connected social network, whereas highly transmissible information spreads better with the help of a large but loosely connected social network.

physics.soc-ph↗

arXiv2011

Effect of depreciation of the public goods in spatial public goods games

In this work, depreciated effect of the public goods is considered in the public goods games, which is realized by rescaling the multiplication factor r of each group as r' = r(nc/G)^beta (beat>= 0). It is assumed that each individual enjoys the full profit of the public goods if all the players of this group are cooperators, otherwise, the value of the public goods is reduced to r'. It is found that compared with the original version (beta = 0), emergence of cooperation is remarkably promoted for beta > 0, and there exit optimal values of beta inducing the best cooperation. Moreover, the optimal plat of beta broadens as r increases. Furthermore, effect of noise on the evolution of cooperation is studied, it is presented that variation of cooperator density with the noise is dependent of the value of beta and r, and cooperation dominates over most of the range of noise at an intermediate value of beta = 1.0. We study the initial distribution of the multiplication factor at beta = 1.0, and find that all the distributions can be described as Gauss distribution.

physics.soc-ph↗