arXiv subjects

Hongyu Zhu

Publications and source records attributed to Hongyu Zhu.

At least 37 records · Page 2Linked to original sources

arXiv2025

PromptLA: Towards Integrity Verification of Black-box Text-to-Image Diffusion Models

Despite the impressive synthesis quality of text-to-image (T2I) diffusion models, their black-box deployment poses significant regulatory challenges: Malicious actors can fine-tune these models to generate illegal content, circumventing existing safeguards through parameter manipulation. Therefore, it is essential to verify the integrity of T2I diffusion models. To this end, considering the randomness within the outputs of generative models and the high costs in interacting with them, we discern model tampering via the KL divergence between the distributions of the features of generated images. We propose a novel prompt selection algorithm based on learning automaton (PromptLA) for efficient and accurate verification. Evaluations on four advanced T2I models (e.g., SDXL, FLUX.1) demonstrate that our method achieves a mean AUC of over 0.96 in integrity detection, exceeding baselines by more than 0.2, showcasing strong effectiveness and generalization. Additionally, our approach achieves lower cost and is robust against image-level post-processing. To the best of our knowledge, this paper is the first work addressing the integrity verification of T2I diffusion models, which establishes quantifiable standards for AI copyright litigation in practice.

cs.CV↗

arXiv2025

The Borel complexity of the class of models of first-order theories

We investigate the descriptive complexity of the set of models of first-order theories. Using classical results of Knight and Solovay, we give a sharp condition for complete theories to have a $\pmbΠ_ω^0$-complete set of models. In particular, any sequential theory (a class of foundational theories isolated by Pudlák) has a $\pmbΠ_ω^0$-complete set of models. We also give sharp conditions for theories to have a $\pmbΠ^0_n$-complete set of models.

math.LO↗

arXiv2025

RGAR: Recurrence Generation-augmented Retrieval for Factual-aware Medical Question Answering

Medical question answering requires extensive access to specialized conceptual knowledge. The current paradigm, Retrieval-Augmented Generation (RAG), acquires expertise medical knowledge through large-scale corpus retrieval and uses this knowledge to guide a general-purpose large language model (LLM) for generating answers. However, existing retrieval approaches often overlook the importance of factual knowledge, which limits the relevance of retrieved conceptual knowledge and restricts its applicability in real-world scenarios, such as clinical decision-making based on Electronic Health Records (EHRs). This paper introduces RGAR, a recurrence generation-augmented retrieval framework that retrieves both relevant factual and conceptual knowledge from dual sources (i.e., EHRs and the corpus), allowing them to interact and refine each another. Through extensive evaluation across three factual-aware medical question answering benchmarks, RGAR establishes a new state-of-the-art performance among medical RAG systems. Notably, the Llama-3.1-8B-Instruct model with RGAR surpasses the considerably larger, RAG-enhanced GPT-3.5. Our findings demonstrate the benefit of extracting factual knowledge for retrieval, which consistently yields improved generation quality.

cs.CL↗

arXiv2025

EM-DARTS: Hierarchical Differentiable Architecture Search for Eye Movement Recognition

Eye movement biometrics has received increasing attention thanks to its highly secure identification. Although deep learning (DL) models have shown success in eye movement recognition, their architectures largely rely on human prior knowledge. Differentiable Neural Architecture Search (DARTS) automates the manual process of architecture design with high search efficiency. However, DARTS typically stacks multiple cells to form a convolutional network, which limits the diversity of architecture. Furthermore, DARTS generally searches for architectures using shallower networks than those used in the evaluation, creating a significant disparity in architecture depth between the search and evaluation phases. To address this issue, we propose EM-DARTS, a hierarchical differentiable architecture search algorithm to automatically design the DL architecture for eye movement recognition. First, we define a supernet and propose a global and local alternate Neural Architecture Search method to search the optimal architecture alternately with a differentiable neural architecture search. The local search strategy aims to find an optimal architecture for different cells while the global search strategy is responsible for optimizing the architecture of the target network. To minimize redundancy, transfer entropy is proposed to compute the information amount of each layer, thereby further simplifying the network search process. Experimental results on three public datasets demonstrate that the proposed EM-DARTS is capable of producing an optimal architecture that leads to state-of-the-art recognition performance, {Specifically, the recognition models developed using EM-DARTS achieved the lowest EERs of 0.0453 on the GazeBase dataset, 0.0377 on the JuDo1000 dataset, and 0.1385 on the EMglasses dataset.

cs.CV↗

arXiv2024

Variational Quantum Framework for Nonlinear PDE Constrained Optimization Using Carleman Linearization

We present a novel variational quantum framework for nonlinear partial differential equation (PDE) constrained optimization problems. The proposed work extends the recently introduced bi-level variational quantum PDE constrained optimization (BVQPCO) framework for linear PDE to a nonlinear setting by leveraging Carleman linearization (CL). CL framework allows one to transform a system of polynomial ordinary differential equations (ODE), i,e. ODE with polynomial vector field, into an system of infinite but linear system of ODE. For instance, such polynomial ODEs naturally arise when the PDE are semi-discretized in the spatial dimensions. By truncating the CL system to a finite order, one obtains a finite system of linear ODE to which the linear BVQPCO framework can be applied. In particular, the finite system of linear ODE is discretized in time and embedded as a system of linear equations. The variational quantum linear solver (VQLS) is used to solve the linear system for given optimization parameters, and evaluate the design cost/objective function, and a classical black box optimizer is used to select next set of parameter values based on this evaluated cost. We present detailed computational error and complexity analysis and prove that under suitable assumptions, our proposed framework can provide potential advantage over classical techniques. We implement our framework using the PennyLane library and apply it to solve inverse Burgers' problem. We also explore an alternative tensor product decomposition which exploits the sparsity/structure of linear system arising from PDE discretization to facilitate the computation of VQLS cost functions.

quant-ph↗

arXiv2024

StarLKNet: Star Mixup with Large Kernel Networks for Palm Vein Identification

As a representative of a new generation of biometrics, vein identification technology offers a high level of security and convenience.Convolutional neural networks (CNNs), a prominent class of deep learning architectures, have been extensively utilized for vein identification. Since their performance and robustness are limited by small \emph{Effective Receptive Fields} (\emph{e.g.}, 3$\times$3 kernels) and insufficient training samples, however, they are unable to extract global feature representations from vein images effectively. To address these issues, we propose \textbf{StarLKNet}, a large kernel convolution-based palm-vein identification network, with the Mixup approach.Our StarMix learns effectively the distribution of vein features to expand samples. To enable CNNs to capture comprehensive feature representations from palm-vein images, we explored the effect of convolutional kernel size on the performance of palm-vein identification networks and designed LaKNet, a network leveraging large kernel convolution and gating mechanism. In light of the current state of knowledge, this represents an inaugural instance of the deployment of a CNN with large kernels in the domain of vein identification. Extensive experiments were conducted to validate the performance of StarLKNet on two public palm-vein datasets. The results demonstrated that \textbf{StarMix} provided superior augmentation, and \textbf{LakNet} exhibited more stable performance gains compared to mainstream approaches, resulting in the highest identification accuracy and lowest identification error.

cs.CV↗

arXiv2024

Efficient and Effective Model Extraction

Model extraction aims to create a functionally similar copy from a machine learning as a service (MLaaS) API with minimal overhead, typically for illicit profit or as a precursor to further attacks, posing a significant threat to the MLaaS ecosystem. However, recent studies have shown that model extraction is highly inefficient, particularly when the target task distribution is unavailable. In such cases, even substantially increasing the attack budget fails to produce a sufficiently similar replica, reducing the adversary's motivation to pursue extraction attacks. In this paper, we revisit the elementary design choices throughout the extraction lifecycle. We propose an embarrassingly simple yet dramatically effective algorithm, Efficient and Effective Model Extraction (E3), focusing on both query preparation and training routine. E3 achieves superior generalization compared to state-of-the-art methods while minimizing computational costs. For instance, with only 0.005 times the query budget and less than 0.2 times the runtime, E3 outperforms classical generative model based data-free model extraction by an absolute accuracy improvement of over 50% on CIFAR-10. Our findings underscore the persistent threat posed by model extraction and suggest that it could serve as a valuable benchmarking algorithm for future security evaluations.

cs.CR↗

arXiv2024

SUMix: Mixup with Semantic and Uncertain Information

Mixup data augmentation approaches have been applied for various tasks of deep learning to improve the generalization ability of deep neural networks. Some existing approaches CutMix, SaliencyMix, etc. randomly replace a patch in one image with patches from another to generate the mixed image. Similarly, the corresponding labels are linearly combined by a fixed ratio $λ$ by l. The objects in two images may be overlapped during the mixing process, so some semantic information is corrupted in the mixed samples. In this case, the mixed image does not match the mixed label information. Besides, such a label may mislead the deep learning model training, which results in poor performance. To solve this problem, we proposed a novel approach named SUMix to learn the mixing ratio as well as the uncertainty for the mixed samples during the training process. First, we design a learnable similarity function to compute an accurate mix ratio. Second, an approach is investigated as a regularized term to model the uncertainty of the mixed samples. We conduct experiments on five image benchmarks, and extensive experimental results imply that our method is capable of improving the performance of classifiers with different cutting-based mixup approaches. The source code is available at https://github.com/JinXins/SUMix.

cs.CV↗

arXiv2024

Relax DARTS: Relaxing the Constraints of Differentiable Architecture Search for Eye Movement Recognition

Eye movement biometrics is a secure and innovative identification method. Deep learning methods have shown good performance, but their network architecture relies on manual design and combined priori knowledge. To address these issues, we introduce automated network search (NAS) algorithms to the field of eye movement recognition and present Relax DARTS, which is an improvement of the Differentiable Architecture Search (DARTS) to realize more efficient network search and training. The key idea is to circumvent the issue of weight sharing by independently training the architecture parameters $α$ to achieve a more precise target architecture. Moreover, the introduction of module input weights $β$ allows cells the flexibility to select inputs, to alleviate the overfitting phenomenon and improve the model performance. Results on four public databases demonstrate that the Relax DARTS achieves state-of-the-art recognition performance. Notably, Relax DARTS exhibits adaptability to other multi-feature temporal classification tasks.

cs.CV↗

arXiv2024

Data-driven network analysis using local delay embeddings

Data-driven methods for the identification of the governing equations of dynamical systems or the computation of reduced surrogate models play an increasingly important role in many application areas such as physics, chemistry, biology, and engineering. Given only measurement or observation data, data-driven modeling techniques allow us to gain important insights into the characteristic properties of a system, without requiring detailed mechanistic models. However, most methods assume that we have access to the full state of the system, which might be too restrictive. We show that it is possible to learn certain global dynamical features from local observations using delay embedding techniques, provided that the system satisfies a localizability condition -- a property that is closely related to the observability and controllability of linear time-invariant systems.

math.DS↗

arXiv2024

EmMixformer: Mix transformer for eye movement recognition

Eye movement (EM) is a new highly secure biometric behavioral modality that has received increasing attention in recent years. Although deep neural networks, such as convolutional neural network (CNN), have recently achieved promising performance, current solutions fail to capture local and global temporal dependencies within eye movement data. To overcome this problem, we propose in this paper a mixed transformer termed EmMixformer to extract time and frequency domain information for eye movement recognition. To this end, we propose a mixed block consisting of three modules, transformer, attention Long short-term memory (attention LSTM), and Fourier transformer. We are the first to attempt leveraging transformer to learn long temporal dependencies within eye movement. Second, we incorporate the attention mechanism into LSTM to propose attention LSTM with the aim to learn short temporal dependencies. Third, we perform self attention in the frequency domain to learn global features. As the three modules provide complementary feature representations in terms of local and global dependencies, the proposed EmMixformer is capable of improving recognition accuracy. The experimental results on our eye movement dataset and two public eye movement datasets show that the proposed EmMixformer outperforms the state of the art by achieving the lowest verification error.

cs.CV↗

arXiv2024

Reliable Model Watermarking: Defending Against Theft without Compromising on Evasion

With the rise of Machine Learning as a Service (MLaaS) platforms,safeguarding the intellectual property of deep learning models is becoming paramount. Among various protective measures, trigger set watermarking has emerged as a flexible and effective strategy for preventing unauthorized model distribution. However, this paper identifies an inherent flaw in the current paradigm of trigger set watermarking: evasion adversaries can readily exploit the shortcuts created by models memorizing watermark samples that deviate from the main task distribution, significantly impairing their generalization in adversarial settings. To counteract this, we leverage diffusion models to synthesize unrestricted adversarial examples as trigger sets. By learning the model to accurately recognize them, unique watermark behaviors are promoted through knowledge injection rather than error memorization, thus avoiding exploitable shortcuts. Furthermore, we uncover that the resistance of current trigger set watermarking against removal attacks primarily relies on significantly damaging the decision boundaries during embedding, intertwining unremovability with adverse impacts. By optimizing the knowledge transfer properties of protected models, our approach conveys watermark behaviors to extraction surrogates without aggressively decision boundary perturbation. Experimental results on CIFAR-10/100 and Imagenette datasets demonstrate the effectiveness of our method, showing not only improved robustness against evasion adversaries but also superior resistance to watermark removal attacks compared to state-of-the-art solutions.

cs.CR↗

arXiv2023

Improve Deep Forest with Learnable Layerwise Augmentation Policy Schedule

As a modern ensemble technique, Deep Forest (DF) employs a cascading structure to construct deep models, providing stronger representational power compared to traditional decision forests. However, its greedy multi-layer learning procedure is prone to overfitting, limiting model effectiveness and generalizability. This paper presents an optimized Deep Forest, featuring learnable, layerwise data augmentation policy schedules. Specifically, We introduce the Cut Mix for Tabular data (CMT) augmentation technique to mitigate overfitting and develop a population-based search algorithm to tailor augmentation intensity for each layer. Additionally, we propose to incorporate outputs from intermediate layers into a checkpoint ensemble for more stable performance. Experimental results show that our method sets new state-of-the-art (SOTA) benchmarks in various tabular classification tasks, outperforming shallow tree ensembles, deep forests, deep neural network, and AutoML competitors. The learned policies also transfer effectively to Deep Forest variants, underscoring its potential for enhancing non-differentiable deep learning modules in tabular signal processing.

cs.LG↗

arXiv2022

Modeling the impact of tissue oxygen profiles and oxygen depletion parameter uncertainties on biological response and therapeutic benefit of FLASH

FLASH radiation has been reported to efficiently suppress tumor growth while sparing normal tissue, however, the mechanism of the differential tissue sparing effect is still not known. Oxygen has long been known to profoundly impact radiobiological responses, and radiolytic oxygen depletion has been considered to be a possible cause or contributor to the FLASH phenomenon. This work investigates the impact of tissue pO2 profiles, oxygen depletion per unit dose (g), and the oxygen concentration yielding half-maximal radiosensitization (k) in tumor and normal tissue. We developed a model that considers the dependent relationship between oxygen depletion and change of radiosensitivity during FLASH irradiation. Cell survival was calculated based on the LQ-L model and the radiosensitivity related parameters were adjusted while delivering fractional doses of FLASH irradiation The model reproduced published experimental data and was used to analyze the impact of parameter uncertainties on the radiobiological responses. This study expands the oxygen depletion analysis of FLASH to normal human tissue and tumor based on clinically determined aggregate and individual patient pO2 profiles. The results show that the pO2 profile is the most essential factor that affects biological response and analyses based on the median pO2 rather than the full pO2 profile can be unreliable and misleading. Additionally, the presence of a small fraction of cells on the threshold of radiobiologic hypoxia substantially alters biological response to FLASH irradiation. We found that an increment in the k value is generally more protective of tumor than normal tissue due to a higher frequency of lower pO2 values in tumors. Variation in the g value affects the dose at which oxygen depletion impacts response, but does not alter the dose dependent response trends, if the g value is identical in both tumor and normal tissue.

physics.med-ph↗

arXiv2022

A Dynamic Mode Decomposition Approach for Decentralized Spectral Clustering of Graphs

We propose a novel robust decentralized graph clustering algorithm that is provably equivalent to the popular spectral clustering approach. Our proposed method uses the existing wave equation clustering algorithm that is based on propagating waves through the graph. However, instead of using a fast Fourier transform (FFT) computation at every node, our proposed approach exploits the Koopman operator framework. Specifically, we show that propagating waves in the graph followed by a local dynamic mode decomposition (DMD) computation at every node is capable of retrieving the eigenvalues and the local eigenvector components of the graph Laplacian, thereby providing local cluster assignments for all nodes. We demonstrate that the DMD computation is more robust than the existing FFT based approach and requires 20 times fewer steps of the wave equation to accurately recover the clustering information and reduces the relative error by orders of magnitude. We demonstrate the decentralized approach on a range of graph clustering problems.

cs.LG↗

arXiv2022

An Input-Aware Mimic Defense Theory and its Practice

The current security problems in cyberspace are characterized by strong and complex threats. Defenders face numerous problems such as lack of prior knowledge, various threats, and unknown vulnerabilities, which urgently need new fundamental theories to support. To address these issues, this article proposes a generic theoretical model for cyberspace defense and a new mimic defense framework, that is, Spatiotemporally heterogeneous, Input aware, and Dynamically updated Mimic Defense (SIDMD). We make the following contributions: (1) We first redefine vulnerabilities from the input space perspective to normalize the diverse cyberspace security problem. (2) We propose a novel unknown vulnerability discovery method and a dynamic scheduling strategy considering temporal and spatial dimensions without prior knowledge. Theoretical analysis and experimental results show that SIDMD has the best security performance in complex attack scenarios, and the probability of successful attacks is greatly reduced compared to the state-of-the-art.

cs.CR↗

arXiv2022

DuQM: A Chinese Dataset of Linguistically Perturbed Natural Questions for Evaluating the Robustness of Question Matching Models

In this paper, we focus on studying robustness evaluation of Chinese question matching. Most of the previous work on analyzing robustness issue focus on just one or a few types of artificial adversarial examples. Instead, we argue that it is necessary to formulate a comprehensive evaluation about the linguistic capabilities of models on natural texts. For this purpose, we create a Chinese dataset namely DuQM which contains natural questions with linguistic perturbations to evaluate the robustness of question matching models. DuQM contains 3 categories and 13 subcategories with 32 linguistic perturbations. The extensive experiments demonstrate that DuQM has a better ability to distinguish different models. Importantly, the detailed breakdown of evaluation by linguistic phenomenon in DuQM helps us easily diagnose the strength and weakness of different models. Additionally, our experiment results show that the effect of artificial adversarial examples does not work on the natural texts.

cs.CL↗

arXiv2021

An Analysis of COVID-19 Knowledge Graph Construction and Applications

The construction and application of knowledge graphs have seen a rapid increase across many disciplines in recent years. Additionally, the problem of uncovering relationships between developments in the COVID-19 pandemic and social media behavior is of great interest to researchers hoping to curb the spread of the disease. In this paper we present a knowledge graph constructed from COVID-19 related tweets in the Los Angeles area, supplemented with federal and state policy announcements and disease spread statistics. By incorporating dates, topics, and events as entities, we construct a knowledge graph that describes the connections between these useful information. We use natural language processing and change point analysis to extract tweet-topic, tweet-date, and event-date relations. Further analysis on the constructed knowledge graph provides insight into how tweets reflect public sentiments towards COVID-19 related topics and how changes in these sentiments correlate with real-world events.

cs.SI↗