arXiv subjects

Zixuan Hu

Publications and source records attributed to Zixuan Hu.

At least 37 records · Page 2Linked to original sources

arXiv2025

Learning to Learn from APIs: Black-Box Data-Free Meta-Learning

Data-free meta-learning (DFML) aims to enable efficient learning of new tasks by meta-learning from a collection of pre-trained models without access to the training data. Existing DFML work can only meta-learn from (i) white-box and (ii) small-scale pre-trained models (iii) with the same architecture, neglecting the more practical setting where the users only have inference access to the APIs with arbitrary model architectures and model scale inside. To solve this issue, we propose a Bi-level Data-free Meta Knowledge Distillation (BiDf-MKD) framework to transfer more general meta knowledge from a collection of black-box APIs to one single meta model. Specifically, by just querying APIs, we inverse each API to recover its training data via a zero-order gradient estimator and then perform meta-learning via a novel bi-level meta knowledge distillation structure, in which we design a boundary query set recovery technique to recover a more informative query set near the decision boundary. In addition, to encourage better generalization within the setting of limited API budgets, we propose task memory replay to diversify the underlying task distribution by covering more interpolated tasks. Extensive experiments in various real-world scenarios show the superior performance of our BiDf-MKD framework.

cs.LG↗

arXiv2025

Quantum encryption design overcomes Shannon's theorem to achieve perfect secrecy with reusable keys

Shannon's perfect-secrecy theorem states that a perfect encryption system that yields zero information to the adversary must be a one-time pad (OTP) with the keys randomly generated and never reused. In this work we design the first encryption method (classical or quantum) that overcomes Shannon's theorem to achieve perfect secrecy with reusable keys. Because the mechanisms used are fundamentally quantum, Shannon's theorem remains true in the classical regime. Consequently, the quantum encryption design demonstrates decisive quantum advantage by achieving a goal impossible for classical systems. Finally, the design has major practical advantages by not requiring authentication and having silent tampering detection.

quant-ph↗

arXiv2024

FREE: Faster and Better Data-Free Meta-Learning

Data-Free Meta-Learning (DFML) aims to extract knowledge from a collection of pre-trained models without requiring the original data, presenting practical benefits in contexts constrained by data privacy concerns. Current DFML methods primarily focus on the data recovery from these pre-trained models. However, they suffer from slow recovery speed and overlook gaps inherent in heterogeneous pre-trained models. In response to these challenges, we introduce the Faster and Better Data-Free Meta-Learning (FREE) framework, which contains: (i) a meta-generator for rapidly recovering training tasks from pre-trained models; and (ii) a meta-learner for generalizing to new unseen tasks. Specifically, within the module Faster Inversion via Meta-Generator, each pre-trained model is perceived as a distinct task. The meta-generator can rapidly adapt to a specific task in just five steps, significantly accelerating the data recovery. Furthermore, we propose Better Generalization via Meta-Learner and introduce an implicit gradient alignment algorithm to optimize the meta-learner. This is achieved as aligned gradient directions alleviate potential conflicts among tasks from heterogeneous pre-trained models. Empirical experiments on multiple benchmarks affirm the superiority of our approach, marking a notable speed-up (20$\times$) and performance enhancement (1.42%$\sim$4.78%) in comparison to the state-of-the-art.

cs.LG↗

arXiv2024

Task Groupings Regularization: Data-Free Meta-Learning with Heterogeneous Pre-trained Models

Data-Free Meta-Learning (DFML) aims to derive knowledge from a collection of pre-trained models without accessing their original data, enabling the rapid adaptation to new unseen tasks. Current methods often overlook the heterogeneity among pre-trained models, which leads to performance degradation due to task conflicts. In this paper, we empirically and theoretically identify and analyze the model heterogeneity in DFML. We find that model heterogeneity introduces a heterogeneity-homogeneity trade-off, where homogeneous models reduce task conflicts but also increase the overfitting risk. Balancing this trade-off is crucial for learning shared representations across tasks. Based on our findings, we propose Task Groupings Regularization that benefits from model heterogeneity by grouping and aligning conflicting tasks. Specifically, we embed pre-trained models into a task space to compute dissimilarity, and group heterogeneous models together based on this measure. Then, we introduce implicit gradient regularization within each group to mitigate potential conflicts. By encouraging a gradient direction suitable for all tasks, the meta-model captures shared representations that generalize across tasks. Comprehensive experiments showcase the superiority of our approach in multiple benchmarks, effectively tackling the model heterogeneity in challenging multi-domain and multi-architecture scenarios.

cs.LG↗

arXiv2024

Unlocking Tuning-Free Few-Shot Adaptability in Visual Foundation Models by Recycling Pre-Tuned LoRAs

Large Language Models (LLMs) such as ChatGPT demonstrate strong few-shot adaptability without requiring fine-tuning, positioning them ideal for data-limited and real-time applications. However, this adaptability has not yet been replicated in current Visual Foundation Models (VFMs), which require explicit fine-tuning with sufficient tuning data. Besides, the pretraining-finetuning paradigm has led to the surge of numerous task-specific modular components, such as Low-Rank Adaptation (LoRA). For the first time, we explore the potential of reusing diverse pre-tuned LoRAs without accessing their original training data, to achieve tuning-free few-shot adaptation in VFMs. Our framework, LoRA Recycle, distills a meta-LoRA from diverse pre-tuned LoRAs with a meta-learning objective, using surrogate data generated inversely from pre-tuned LoRAs themselves. The VFM, once equipped with the meta-LoRA, is empowered to solve new few-shot tasks in a single forward pass, akin to the in-context learning of LLMs. Additionally, we incorporate a double-efficient mechanism tailored to our framework, significantly accelerating the meta-training process while maintaining or even improving performance. Extensive experiments across various few-shot classification benchmarks across both in- and cross-domain scenarios demonstrate the superiority of our framework.

cs.CV↗

arXiv2024

Coding for Intelligence from the Perspective of Category

Coding, which targets compressing and reconstructing data, and intelligence, often regarded at an abstract computational level as being centered around model learning and prediction, interweave recently to give birth to a series of significant progress. The recent trends demonstrate the potential homogeneity of these two fields, especially when deep-learning models aid these two categories for better probability modeling. For better understanding and describing from a unified perspective, inspired by the basic generally recognized principles in cognitive psychology, we formulate a novel problem of Coding for Intelligence from the category theory view. Based on the three axioms: existence of ideal coding, existence of practical coding, and compactness promoting generalization, we derive a general framework to understand existing methodologies, namely that, coding captures the intrinsic relationships of objects as much as possible, while ignoring information irrelevant to downstream tasks. This framework helps identify the challenges and essential elements in solving the specific derived Minimal Description Length (MDL) optimization problem from a broader range, providing opportunities to build a more intelligent system for handling multiple tasks/applications with coding ideas/tools. Centering on those elements, we systematically review recent processes of towards optimizing the MDL problem in more comprehensive ways from data, model, and task perspectives, and reveal their impacts on the potential CfI technical routes. After that, we also present new technique paths to fulfill CfI and provide potential solutions with preliminary experimental evidence. Last, further directions and remaining issues are discussed as well. The discussion shows our theory can reveal many phenomena and insights about large foundation models, which mutually corroborate with recent practices in feature learning.

cs.CV↗

arXiv2024

Quantum Algorithms and Applications for Open Quantum Systems

Accurate models for open quantum systems -- quantum states that have non-trivial interactions with their environment -- may aid in the advancement of a diverse array of fields, including quantum computation, informatics, and the prediction of static and dynamic molecular properties. In recent years, quantum algorithms have been leveraged for the computation of open quantum systems as the predicted quantum advantage of quantum devices over classical ones may allow previously inaccessible applications. Accomplishing this goal will require input and expertise from different research perspectives, as well as the training of a diverse quantum workforce, making a compilation of current quantum methods for treating open quantum systems both useful and timely. In this Review, we first provide a succinct summary of the fundamental theory of open quantum systems and then delve into a discussion on recent quantum algorithms. We conclude with a discussion of pertinent applications, demonstrating the applicability of this field to realistic chemical, biological, and material systems.

quant-ph↗

arXiv2024

Designing variational ansatz for quantum-enabled simulation of non-unitary dynamical evolution -- an excursion into Dicke supperradiance

Adaptive Variational Quantum Dynamics (AVQD) algorithms offer a promising approach to providing quantum-enabled solutions for systems treated within the purview of open quantum dynamical evolution. In this study, we employ the unrestricted vectorization variant of AVQD to simulate and benchmark various non-unitarily evolving systems. We exemplify how construction of an expressible ansatz unitary and the associated operator pool can be implemented to analyze examples such as the Fenna Matthews Olson complex (FMO) and even the permutational invariant Dicke model of quantum optics. We furthermore show an efficient decomposition scheme for the ansatz used, which can extend its applications to a wide range of other open quantum system scenarios in near future. In all cases the results obtained are in excellent agreement with exact numerical computations which bolsters the effectiveness of this technique. Our successful demonstrations pave the way for utilizing this adaptive variational technique to study complex systems in chemistry and physics, like light harvesting devices, thermal, and opto mechanical switches, to name a few.

quant-ph↗

arXiv2024

The qubit information logic theory for understanding multi-qubit entanglement and designing exotic entangled states

We develop a "qubit information logic" (QIL) theory that uses the "qubit information equation" (QIE) and logic to describe the correlation behaviors of multi-qubit entanglement. Introducing the "global information status" and "local information availability", the QIL gives an alternative and natural interpretation of the "spooky action" and the quantum no-communication theorem. Compared to the conventional entropy-based entanglement theories, the QIL directly describes the correlation of each possible pair of qubits and how the correlation changes when other qubits are measured. This makes the QIL more advantageous in describing the correlation properties of multi-qubit entanglement, which is illustrated by studying the dormant entanglement phenomenon. The QIL theory's usefulness is further demonstrated by designing an exotic quantum state where two qubits can be entangled but not correlated in any arbitrary basis. Overall the QIL provides an alternative and intuitive understanding of multi-qubit entanglement that is, compared to the conventional theories, directly focused on the correlation behaviors between qubits and thus more suitable for designing exotic quantum states that may be used in quantum algorithms.

quant-ph↗

arXiv2023

Exploring Model Transferability through the Lens of Potential Energy

Transfer learning has become crucial in computer vision tasks due to the vast availability of pre-trained deep learning models. However, selecting the optimal pre-trained model from a diverse pool for a specific downstream task remains a challenge. Existing methods for measuring the transferability of pre-trained models rely on statistical correlations between encoded static features and task labels, but they overlook the impact of underlying representation dynamics during fine-tuning, leading to unreliable results, especially for self-supervised models. In this paper, we present an insightful physics-inspired approach named PED to address these challenges. We reframe the challenge of model selection through the lens of potential energy and directly model the interaction forces that influence fine-tuning dynamics. By capturing the motion of dynamic representations to decline the potential energy within a force-driven physical model, we can acquire an enhanced and more stable observation for estimating transferability. The experimental results on 10 downstream tasks and 12 self-supervised models demonstrate that our approach can seamlessly integrate into existing ranking techniques and enhance their performances, revealing its effectiveness for the model selection task and its potential for understanding the mechanism in transfer learning. Code will be available at https://github.com/lixiaotong97/PED.

cs.CV↗

arXiv2023

Simulating Open Quantum System Dynamics on NISQ Computers with Generalized Quantum Master Equations

We present a quantum algorithm based on the Generalized Quantum Master Equation (GQME) approach to simulate open quantum system dynamics on noisy intermediate-scale quantum (NISQ) computers. This approach overcomes the limitations of the Lindblad equation, which assumes weak system-bath coupling and Markovity, by providing a rigorous derivation of the equations of motion for any subset of elements of the reduced density matrix. The memory kernel resulting from the effect of the remaining degrees of freedom is used as input to calculate the corresponding non-unitary propagator. We demonstrate how the Sz.-Nagy dilation theorem can be employed to transform the non-unitary propagator into a unitary one in a higher-dimensional Hilbert space, which can then be implemented on quantum circuits of NISQ computers. We validate our quantum algorithm as applied to the spin-boson benchmark model by analyzing the impact of the quantum circuit depth on the accuracy of the results when the subset is limited to the diagonal elements of the reduced density matrix. Our findings demonstrate that our approach yields reliable results on NISQ IBM computers.

quant-ph↗

arXiv2023

Dormant entanglement that can be activated or destroyed by the basis choice of measurements on an external system

We propose a new form of entanglement called the dormant entanglement that can be activated or destroyed by the basis choice of measurements on an external system. The dormant entanglement without activation cannot be used as a quantum communication resource and has reduced correlation as compared to the Bell states. A particular form of the dormant entanglement is so weak that, without activation, no correlation in any basis can be observed between the entangled qubits. The dormant entanglement showcases a unique quantum behavior that the physical description of a local system remains incomplete until the information on all external systems entangled with the local system becomes available. For a potential application, we propose an n-party collective quantum communication channel that allows any 2 out of the n parties to activate an entanglement pair with the complete consensus of all other parties.

quant-ph↗

arXiv2023

Mapping Molecular Hamiltonians into Hamiltonians of Modular cQED Processors

We introduce a general method based on the operators of the Dyson-Masleev transformation to map the Hamiltonian of an arbitrary model system into the Hamiltonian of a circuit Quantum Electrodynamics (cQED) processor. Furthermore, we introduce a modular approach to program a cQED processor with components corresponding to the mapping Hamiltonian. The method is illustrated as applied to quantum dynamics simulations of the Fenna-Matthews-Olson (FMO) complex and the spin-boson model of charge transfer. Beyond applications to molecular Hamiltonians, the mapping provides a general approach to implement any unitary operator in terms of a sequence of unitary transformations corresponding to powers of creation and annihilation operators of a single bosonic mode in a cQED processor.

quant-ph↗

arXiv2023

Modeling Uncertain Feature Representation for Domain Generalization

Though deep neural networks have achieved impressive success on various vision tasks, obvious performance degradation still exists when models are tested in out-of-distribution scenarios. In addressing this limitation, we ponder that the feature statistics (mean and standard deviation), which carry the domain characteristics of the training data, can be properly manipulated to improve the generalization ability of deep learning models. Existing methods commonly consider feature statistics as deterministic values measured from the learned features and do not explicitly model the uncertain statistics discrepancy caused by potential domain shifts during testing. In this paper, we improve the network generalization ability by modeling domain shifts with uncertainty (DSU), i.e., characterizing the feature statistics as uncertain distributions during training. Specifically, we hypothesize that the feature statistic, after considering the potential uncertainties, follows a multivariate Gaussian distribution. During inference, we propose an instance-wise adaptation strategy that can adaptively deal with the unforeseeable shift and further enhance the generalization ability of the trained model with negligible additional cost. We also conduct theoretical analysis on the aspects of generalization error bound and the implicit regularization effect, showing the efficacy of our method. Extensive experiments demonstrate that our method consistently improves the network generalization ability on multiple vision tasks, including image classification, semantic segmentation, instance retrieval, and pose estimation. Our methods are simple yet effective and can be readily integrated into networks without additional trainable parameters or loss constraints. Code will be released in https://github.com/lixiaotong97/DSU.

cs.CV↗

arXiv2022

The wave-particle duality of the qudit quantum space and the quantum wave gates

We propose three core ideas: 1. the wave-particle duality of the qudit quantum space; 2. the classification of all elementary quantum gates by ordered pairs of qudit functionals; 3. a new type of quantum gates called the "quantum wave gates". We first study the quantum functionals whose relation to the quantum states is analogous to that between the momentum and position wavefunctions in fundamental quantum physics: a Fourier transform and an entropic uncertainty principle can be defined between the dual representations. The quantum functionals are not just mathematical constructs but have clear physical meanings and quantum circuit realizations. Connecting the partition interpretation of the qudit functionals to the effects of quantum gates we classify all elementary quantum gates by ordered pairs of qudit functionals. By generalizing the qudit functionals to quantum functionals, the new type of "quantum wave gates" are discovered as quantum versions of the conventional quantum gates.

quant-ph↗

arXiv2022

Quantum Simulation of the Radical Pair Dynamics of the Avian Compass

The simulation of open quantum dynamics on quantum circuits has attracted wide interests recently with a variety of quantum algorithms developed and demonstrated. Among these, one particular design of a unitary-dilation-based quantum algorithm is capable of simulating general and complex physical systems. In this paper, we apply this quantum algorithm to simulating the dynamics of the radical pair mechanism in the avian compass. This application is demonstrated on the IBM QASM quantum simulator. This work is the first application of any quantum algorithm to simulating the radical pair mechanism in the avian compass, which not only demonstrates the generality of the quantum algorithm, but also opens new opportunities for studying the avian compass with quantum computing devices.

quant-ph↗

arXiv2022

mc-BEiT: Multi-choice Discretization for Image BERT Pre-training

Image BERT pre-training with masked image modeling (MIM) becomes a popular practice to cope with self-supervised representation learning. A seminal work, BEiT, casts MIM as a classification task with a visual vocabulary, tokenizing the continuous visual signals into discrete vision tokens using a pre-learned dVAE. Despite a feasible solution, the improper discretization hinders further improvements of image pre-training. Since image discretization has no ground-truth answers, we believe that the masked patch should not be assigned with a unique token id even if a better tokenizer can be obtained. In this work, we introduce an improved BERT-style image pre-training method, namely mc-BEiT, which performs MIM proxy tasks towards eased and refined multi-choice training objectives. Specifically, the multi-choice supervision for the masked image patches is formed by the soft probability vectors of the discrete token ids, which are predicted by the off-the-shelf image tokenizer and further refined by high-level inter-patch perceptions resorting to the observation that similar patches should share their choices. Extensive experiments on classification, segmentation, and detection tasks demonstrate the superiority of our method, e.g., the pre-trained ViT-B achieves 84.1% top-1 fine-tuning accuracy on ImageNet-1K classification, 49.2% AP^b and 44.0% AP^m of object detection and instance segmentation on COCO, 50.8% mIOU on ADE20K semantic segmentation, outperforming the competitive counterparts. The code will be available at https://github.com/lixiaotong97/mc-BEiT.

cs.CV↗

arXiv2022

A general quantum algorithm for open quantum dynamics demonstrated with the Fenna-Matthews-Olson complex

Using quantum algorithms to simulate complex physical processes and correlations in quantum matter has been a major direction of quantum computing research, towards the promise of a quantum advantage over classical approaches. In this work we develop a generalized quantum algorithm to simulate any dynamical process represented by either the operator sum representation or the Lindblad master equation. We then demonstrate the quantum algorithm by simulating the dynamics of the Fenna-Matthews-Olson (FMO) complex on the IBM QASM quantum simulator. This work represents a first demonstration of a quantum algorithm for open quantum dynamics with a moderately sophisticated dynamical process involving a realistic biological structure. We discuss the complexity of the quantum algorithm relative to the classical method for the same purpose, presenting a decisive query complexity advantage of the quantum approach based on the unique property of quantum measurement.

quant-ph↗