arXiv subjects

Tao Li

Publications and source records attributed to Tao Li.

At least 289 records · Page 16Linked to original sources

arXiv2024

FRCNet Frequency and Region Consistency for Semi-supervised Medical Image Segmentation

Limited labeled data hinder the application of deep learning in medical domain. In clinical practice, there are sufficient unlabeled data that are not effectively used, and semi-supervised learning (SSL) is a promising way for leveraging these unlabeled data. However, existing SSL methods ignore frequency domain and region-level information and it is important for lesion regions located at low frequencies and with significant scale changes. In this paper, we introduce two consistency regularization strategies for semi-supervised medical image segmentation, including frequency domain consistency (FDC) to assist the feature learning in frequency domain and multi-granularity region similarity consistency (MRSC) to perform multi-scale region-level local context information feature learning. With the help of the proposed FDC and MRSC, we can leverage the powerful feature representation capability of them in an effective and efficient way. We perform comprehensive experiments on two datasets, and the results show that our method achieves large performance gains and exceeds other state-of-the-art methods.

cs.CV↗

arXiv2024

Inflationary Krylov complexity

In this work, we have systematically investigated the Krylov complexity of curvature perturbation for the modified dispersion relation in inflation, using the algorithm in closed system and open system. Our analysis could be applied to the most inflationary models. Following the Lanczos algorithm, we find the very early universe is an infinite, many-body, and maximal chaotic system. Our numerics shows that the Lanczos coefficient and Lyapunov index of the standard dispersion relation are mainly determined by the scale factor. As for the modified case, it is nearly determined by the momentum. In a method of the closed system, we discover that the Krylov complexity will show irregular oscillation before the horizon exits. The modified case will present faster growth after the horizon exists. Since the whole universe is an open system, the approach of an open system is more realistic and reliable. Then, we construct the exact wave function which is very robust only requiring the Lanczos coefficient proportional to $n$ (main quantum number). Based on it, we find the Krylov complexity and Krylov entropy could nicely recover in the case of a closed system under the weak dissipative approximation, in which our analysis shows that the evolution of Krylov complexity will not be the same with the original situation. We also find the inflationary period is a strong dissipative system. Meanwhile, our numerics clearly shows the Krylov complexity will grow during the whole inflationary period. But for the small scales, there will be a peak after the horizon exits. Our analysis reveals that the dramatic change in background (inflation) will significantly impact the evolution of Krylov complexity. Since the curvature perturbation will transit from the quantum level to the classical level.

hep-th↗

arXiv2024

Cosmological complexity of the modified dispersion relation

Complexity will be more and more essential in high-energy physics. It is naturally extended into the very early universe. Considering the universe as a quantum chaotic system, the curvature perturbation of the scalar field is identified with the two-mode squeezed state. By solving the Schr$\ddot{o}$dinger equation, one can obtain the numerical solutions of the angle parameter and squeezing parameter. The solution of the squeezing parameter mainly determines the evolution of complexity. Our numeric indicates that the complexity of the modified dispersion relation will have a non-linear pattern after the horizon exits. Meanwhile, its corresponding Lyapunov index is also larger compared with the standard case. During the inflationary period, the complexity will irregularly oscillate and its scrambling time is also shorter compared with the standard case. Since the modified dispersion relation can be dubbed as the consequences of various frameworks of quantum gravity, it could be applicable to these frameworks. Finally, one can expect the framework of quantum gravity will lead to the fruitful evolution of complexity, which guides us in distinguishing various inflationary models.

gr-qc↗

arXiv2024

Searching for Two-Neutrino and Neutrinoless Double Beta Decay of $^{134}$Xe with the PandaX-4T Experiment

$^{134}$Xe is a candidate isotope for neutrinoless double beta decay~($0νββ$) search. In addition, the two-neutrino case ($2νββ$) allowed by the Standard Model of particle physics has not yet been observed. Utilizing the 10.4% of $^{134}$Xe in the natural xenon in the PandaX-4T detector and its first 94.9-day exposure, we have established the most stringent constraints on $2νββ$ and $0νββ$ of $^{134}$Xe half-lives, with limits of $2.8\times10^{22}$ yr and $3.0\times10^{23}$ yr at 90% confidence level, respectively. The $2νββ$ ($0νββ$) limit surpasses the previously reported best result by a factor of 32 (2.7), highlighting the potential of large monolithic natural xenon detectors.

nucl-ex↗

arXiv2024

Automatic Macro Mining from Interaction Traces at Scale

Macros are building block tasks of our everyday smartphone activity (e.g., "login", or "booking a flight"). Effectively extracting macros is important for understanding mobile interaction and enabling task automation. These macros are however difficult to extract at scale as they can be comprised of multiple steps yet hidden within programmatic components of mobile apps. In this paper, we introduce a novel approach based on Large Language Models (LLMs) to automatically extract semantically meaningful macros from both random and user-curated mobile interaction traces. The macros produced by our approach are automatically tagged with natural language descriptions and are fully executable. We conduct multiple studies to validate the quality of extracted macros, including user evaluation, comparative analysis against human-curated tasks, and automatic execution of these macros. These experiments and analyses show the effectiveness of our approach and the usefulness of extracted macros in various downstream applications.

cs.HC↗

arXiv2024

Be Bayesian by Attachments to Catch More Uncertainty

Bayesian Neural Networks (BNNs) have become one of the promising approaches for uncertainty estimation due to the solid theorical foundations. However, the performance of BNNs is affected by the ability of catching uncertainty. Instead of only seeking the distribution of neural network weights by in-distribution (ID) data, in this paper, we propose a new Bayesian Neural Network with an Attached structure (ABNN) to catch more uncertainty from out-of-distribution (OOD) data. We first construct a mathematical description for the uncertainty of OOD data according to the prior distribution, and then develop an attached Bayesian structure to integrate the uncertainty of OOD data into the backbone network. ABNN is composed of an expectation module and several distribution modules. The expectation module is a backbone deep network which focuses on the original task, and the distribution modules are mini Bayesian structures which serve as attachments of the backbone. In particular, the distribution modules aim at extracting the uncertainty from both ID and OOD data. We further provide theoretical analysis for the convergence of ABNN, and experimentally validate its superiority by comparing with some state-of-the-art uncertainty estimation methods Code will be made available.

cs.LG↗

arXiv2024

Minimize Quantization Output Error with Bias Compensation

Quantization is a promising method that reduces memory usage and computational intensity of Deep Neural Networks (DNNs), but it often leads to significant output error that hinder model deployment. In this paper, we propose Bias Compensation (BC) to minimize the output error, thus realizing ultra-low-precision quantization without model fine-tuning. Instead of optimizing the non-convex quantization process as in most previous methods, the proposed BC bypasses the step to directly minimize the quantizing output error by identifying a bias vector for compensation. We have established that the minimization of output error through BC is a convex problem and provides an efficient strategy to procure optimal solutions associated with minimal output error,without the need for training or fine-tuning. We conduct extensive experiments on Vision Transformer models and Large Language Models, and the results show that our method notably reduces quantization output error, thereby permitting ultra-low-precision post-training quantization and enhancing the task performance of models. Especially, BC improves the accuracy of ViT-B with 4-bit PTQ4ViT by 36.89% on the ImageNet-1k task, and decreases the perplexity of OPT-350M with 3-bit GPTQ by 5.97 on WikiText2.The code is in https://github.com/GongCheng1919/bias-compensation.

cs.CV↗

arXiv2024

Revisiting Random Weight Perturbation for Efficiently Improving Generalization

Improving the generalization ability of modern deep neural networks (DNNs) is a fundamental challenge in machine learning. Two branches of methods have been proposed to seek flat minima and improve generalization: one led by sharpness-aware minimization (SAM) minimizes the worst-case neighborhood loss through adversarial weight perturbation (AWP), and the other minimizes the expected Bayes objective with random weight perturbation (RWP). While RWP offers advantages in computation and is closely linked to AWP on a mathematical basis, its empirical performance has consistently lagged behind that of AWP. In this paper, we revisit the use of RWP for improving generalization and propose improvements from two perspectives: i) the trade-off between generalization and convergence and ii) the random perturbation generation. Through extensive experimental evaluations, we demonstrate that our enhanced RWP methods achieve greater efficiency in enhancing generalization, particularly in large-scale problems, while also offering comparable or even superior performance to SAM. The code is released at https://github.com/nblt/mARWP.

cs.LG↗

arXiv2024

On the Variational Interpretation of Mirror Play in Monotone Games

Mirror play (MP) is a well-accepted primal-dual multi-agent learning algorithm where all agents simultaneously implement mirror descent in a distributed fashion. The advantage of MP over vanilla gradient play lies in its usage of mirror maps that better exploit the geometry of decision domains. Despite extensive literature dedicated to the asymptotic convergence of MP to equilibrium, the understanding of the finite-time behavior of MP before reaching equilibrium is still rudimentary. To facilitate the study of MP's non-equilibrium performance, this work establishes an equivalence between MP's finite-time primal-dual path (mirror path) in monotone games and the closed-loop Nash equilibrium path of a finite-horizon differential game, referred to as mirror differential game (MDG). Our construction of MDG rests on the Brezis-Ekeland variational principle, and the stage cost functional for MDG is Fenchel coupling between MP's iterates and associated gradient updates. The variational interpretation of mirror path in static games as the equilibrium path in MDG holds in deterministic and stochastic cases. Such a variational interpretation translates the non-equilibrium studies of learning dynamics into a more tractable equilibrium analysis of dynamic games, as demonstrated in a case study on the Cournot game, where MP dynamics corresponds to a linear quadratic game.

cs.GT↗

arXiv2024

Friendly Sharpness-Aware Minimization

Sharpness-Aware Minimization (SAM) has been instrumental in improving deep neural network training by minimizing both training loss and loss sharpness. Despite the practical success, the mechanisms behind SAM's generalization enhancements remain elusive, limiting its progress in deep learning optimization. In this work, we investigate SAM's core components for generalization improvement and introduce "Friendly-SAM" (F-SAM) to further enhance SAM's generalization. Our investigation reveals the key role of batch-specific stochastic gradient noise within the adversarial perturbation, i.e., the current minibatch gradient, which significantly influences SAM's generalization performance. By decomposing the adversarial perturbation in SAM into full gradient and stochastic gradient noise components, we discover that relying solely on the full gradient component degrades generalization while excluding it leads to improved performance. The possible reason lies in the full gradient component's increase in sharpness loss for the entire dataset, creating inconsistencies with the subsequent sharpness minimization step solely on the current minibatch data. Inspired by these insights, F-SAM aims to mitigate the negative effects of the full gradient component. It removes the full gradient estimated by an exponentially moving average (EMA) of historical stochastic gradients, and then leverages stochastic gradient noise for improved generalization. Moreover, we provide theoretical validation for the EMA approximation and prove the convergence of F-SAM on non-convex problems. Extensive experiments demonstrate the superior generalization performance and robustness of F-SAM over vanilla SAM. Code is available at https://github.com/nblt/F-SAM.

cs.LG↗

arXiv2024

Memory-Efficient and Secure DNN Inference on TrustZone-enabled Consumer IoT Devices

Edge intelligence enables resource-demanding Deep Neural Network (DNN) inference without transferring original data, addressing concerns about data privacy in consumer Internet of Things (IoT) devices. For privacy-sensitive applications, deploying models in hardware-isolated trusted execution environments (TEEs) becomes essential. However, the limited secure memory in TEEs poses challenges for deploying DNN inference, and alternative techniques like model partitioning and offloading introduce performance degradation and security issues. In this paper, we present a novel approach for advanced model deployment in TrustZone that ensures comprehensive privacy preservation during model inference. We design a memory-efficient management method to support memory-demanding inference in TEEs. By adjusting the memory priority, we effectively mitigate memory leakage risks and memory overlap conflicts, resulting in 32 lines of code alterations in the trusted operating system. Additionally, we leverage two tiny libraries: S-Tinylib (2,538 LoCs), a tiny deep learning library, and Tinylibm (827 LoCs), a tiny math library, to support efficient inference in TEEs. We implemented a prototype on Raspberry Pi 3B+ and evaluated it using three well-known lightweight DNN models. The experimental results demonstrate that our design significantly improves inference speed by 3.13 times and reduces power consumption by over 66.5% compared to non-memory optimization method in TEEs.

cs.CR↗

arXiv2024

An exact formula for the optical conductivity of the two dimensional Hubbard model and its application to the cuprate superconductors

Understanding the origin of electron incoherence is believed to be the first step toward the resolution of the mysteries of the high-T$_{c}$ cuprate superconductors. Such electron incoherence manifests itself most evidently in the non-Drude form of the optical absorption spectrum of the system. The spectral weight transfer related to such dissipative response, which is absent in conventional Fermi liquid metal, has direct consequence on the dc transport property of the system. However, a theoretical study of the optical conductivity of a strongly correlated model is a formidable task. Here we present an exact formula for the optical conductivity of the 2D Hubbard model from the low energy effective theory perspective. We show that the optical conductivity in Matsubara frequency of the 2D Hubbard model can be represented as the ensemble average of the optical conductivity of non-interacting systems in the background of fluctuating local moment. We find that such an ensemble average can be done exactly with a sign-problem-free Monte Carlo simulation if we assume the widely adopted Millis-Monien-Pines spin susceptibility for the fluctuating local moment. For thermal fluctuation of the local moment, our formula can be used to calculate directly the optical conductivity in real frequency which can be compared with the result of optical measurements in the cuprate superconductors.

cond-mat.str-el↗

arXiv2024

Detecting Neutrinos from Supernova Bursts in PandaX-4T

Neutrinos from core-collapse supernovae are essential for the understanding of neutrino physics and stellar evolution. The dual-phase xenon dark matter detectors can provide a way to track explosions of galactic supernovae by detecting neutrinos through coherent elastic neutrino-nucleus scatterings. In this study, a variation of progenitor masses as well as explosion models are assumed to predict the neutrino fluxes and spectra, which result in the number of expected neutrino events ranging from 6.6 to 13.7 at a distance of 10 kpc over a 10-second duration with negligible backgrounds at PandaX-4T. Two specialized triggering alarms for monitoring supernova burst neutrinos are built. The efficiency of detecting supernova explosions at various distances in the Milky Way is estimated. These alarms will be implemented in the real-time supernova monitoring system at PandaX-4T in the near future, providing the astronomical communities with supernova early warnings.

hep-ex↗

arXiv2024

Bound-extended mode transition in type-II synthetic photonic Weyl heterostructures

Photonic structures with Weyl points (WPs), including type-I and type-II, promise nontrivial surface modes and intriguing light manipulations for their three-dimensional topological bands. While previous studies mainly focus on exploring WPs in a uniform Weyl structure, here we establish Weyl heterostructures (i.e., a nonuniform Weyl lattice) with different rotational orientations in the synthetic dimension by nanostructured photonic waveguides. In this work, we unveil a transition between bound and extended modes on the interface of type-II Weyl heterostructures by tuning their rotational phases, despite the reversed topological order across the interface. This mode transition is also manifested from the total transmission to total reflection at the interface. All of these unconventional effects are attributed to the tilted dispersion of type-II Weyl band structure that can lead to mismatched bands and gaps across the interface. As a comparison, the type-I Weyl heterostructures lack the phase transition due to the untilted band structure. This work establishes a flexible scheme of artificial Weyl heterostructures that opens a new avenue towards high-dimensional topological effects and significantly enhances our capabilities in on-chip light manipulations.

physics.optics↗

arXiv2024

Observation of topology transition in Floquet non-Hermitian skin effects in silicon photonics

Non-Hermitian physics has greatly enriched our understanding of nonequilibrium phenomena and uncovered novel effects such as the non-Hermitian skin effect (NHSE) that has profoundly revolutionized the field. NHSE is typically predicted in systems with nonreciprocal couplings which, however, are difficult to realize in experiments. Without nonreciprocal couplings, the NHSE can also emerge in systems with coexisting gauge fields and loss or gain (e.g., in Floquet non-Hermitian systems). However, such Floquet NHSE remains largely unexplored in experiments. Here, we realize the Floquet NHSEs in periodically modulated optical waveguides integrated on a silicon photonics platform. By engineering the artificial gauge fields induced by the periodical modulation, we observe various Floquet NHSEs and unveil their rich topological transitions. Remarkably, we discover the transitions between the normal unipolar NHSEs and an unconventional bipolar NHSE which is accompanied by the directional reversal of the NHSEs. The underlying physics is revealed by the band winding in complex quasienergy space which undergoes a topology change from isolated loops with the same winding to linked loops with opposite windings. Our work unfolds a new route toward Floquet NHSEs originating from the interplay between gauge fields and dissipation effects and offers fundamentally new ways for steering light and other waves.

physics.optics↗

arXiv2024

Hilbert Curve Projection Distance for Distribution Comparison

Distribution comparison plays a central role in many machine learning tasks like data classification and generative modeling. In this study, we propose a novel metric, called Hilbert curve projection (HCP) distance, to measure the distance between two probability distributions with low complexity. In particular, we first project two high-dimensional probability distributions using Hilbert curve to obtain a coupling between them, and then calculate the transport distance between these two distributions in the original space, according to the coupling. We show that HCP distance is a proper metric and is well-defined for probability measures with bounded supports. Furthermore, we demonstrate that the modified empirical HCP distance with the $L_p$ cost in the $d$-dimensional space converges to its population counterpart at a rate of no more than $O(n^{-1/2\max\{d,p\}})$. To suppress the curse-of-dimensionality, we also develop two variants of the HCP distance using (learnable) subspace projections. Experiments on both synthetic and real-world data show that our HCP distance works as an effective surrogate of the Wasserstein distance with low complexity and overcomes the drawbacks of the sliced Wasserstein distance.

cs.LG↗

arXiv2024

QQMR: A Structure-Preserving Quaternion Quasi-Minimal Residual Method for Non-Hermitian Quaternion Linear Systems

The quaternion biconjugate gradient (QBiCG) method, as a novel variant of quaternion Lanczos-type methods for solving the non-Hermitian quaternion linear systems, does not yield a minimization property. This means that the method possesses a rather irregular convergence behavior, which leads to numerical instability. In this paper, we propose a new structure-preserving quaternion quasi-minimal residual method, based on the quaternion biconjugate orthonormalization procedure with coupled two-term recurrences, which overcomes the drawback of QBiCG. The computational cost and storage required by the proposed method are much less than the traditional QMR iterations for the real representation of quaternion linear systems. Some convergence properties of which are also established. Finally, we report the numerical results to show the robustness and effectiveness of the proposed method compared with QBiCG.

math.NA↗

arXiv2024

MindOpt Adapter for CPLEX Benchmarking Performance Analysis

This report provides a comprehensive analysis of the performance of MindOpt Adapter for CPLEX 12.9 in benchmark testing. CPLEX, recognized as a robust Mixed Integer Programming (MIP) solver, has faced some scrutiny regarding its performance on MIPLIB 2017 when configured to default settings. MindOpt Adapter aims to enhance CPLEX's performance by automatically applying improved configurations for solving optimization problems. Our testing demonstrates that MindOpt Adapter for CPLEX yields successfully solved 232 of the 240 problems in the MIPLIB 2017 benchmark set. This performance surpasses all the other solvers in terms of the number of problems solved and the geometric mean of running times. The report provides a comparison of the benchmark results against the outcomes achieved by CPLEX under its default configuration.

cs.MS↗