Searcharxiv⌕ Search

arXiv subjects

Ziqiao Wang

Publications and source records attributed to Ziqiao Wang.

At least 55 records · Page 3Linked to original sources

Two Facets of SDE Under an Information-Theoretic Lens: Generalization of SGD via Training Trajectories and via Terminal States

Stochastic differential equations (SDEs) have been shown recently to characterize well the dynamics of training machine learning models with SGD. When the generalization error of the SDE approximation closely aligns with that of SGD in expectation, it provides two opportunities for understanding better the generalization behaviour of SGD through its SDE approximation. Firstly, viewing SGD as full-batch gradient descent with Gaussian gradient noise allows us to obtain trajectory-based generalization bound using the information-theoretic bound from Xu and Raginsky [2017]. Secondly, assuming mild conditions, we estimate the steady-state weight distribution of SDE and use information-theoretic bounds from Xu and Raginsky [2017] and Negrea et al. [2019] to establish terminal-state-based generalization bounds. Our proposed bounds have some advantages, notably the trajectory-based bound outperforms results in Wang and Mao [2022], and the terminal-state-based bound exhibits a fast decay rate comparable to stability-based bounds.

cs.LG↗

Cross-Modal and Uni-Modal Soft-Label Alignment for Image-Text Retrieval

Current image-text retrieval methods have demonstrated impressive performance in recent years. However, they still face two problems: the inter-modal matching missing problem and the intra-modal semantic loss problem. These problems can significantly affect the accuracy of image-text retrieval. To address these challenges, we propose a novel method called Cross-modal and Uni-modal Soft-label Alignment (CUSA). Our method leverages the power of uni-modal pre-trained models to provide soft-label supervision signals for the image-text retrieval model. Additionally, we introduce two alignment techniques, Cross-modal Soft-label Alignment (CSA) and Uni-modal Soft-label Alignment (USA), to overcome false negatives and enhance similarity recognition between uni-modal samples. Our method is designed to be plug-and-play, meaning it can be easily applied to existing image-text retrieval models without changing their original architectures. Extensive experiments on various image-text retrieval models and datasets, we demonstrate that our method can consistently improve the performance of image-text retrieval and achieve new state-of-the-art results. Furthermore, our method can also boost the uni-modal retrieval performance of image-text retrieval models, enabling it to achieve universal retrieval. The code and supplementary files can be found at https://github.com/lerogo/aaai24_itr_cusa.

cs.CV↗

Quantum phase transitions in two-dimensional superconductors: a review on recent experimental progress

Superconductor-insulator/metal transition (SIT/SMT) as a paradigm of quantum phase transition has been a research highlight over the last three decades. Benefit from recent developments in the fabrication and measurements of 2D superconducting films and nanodevices, unprecedented quantum phenomena have been revealed in the quantum phase transitions of 2D superconductors. In this review, we introduce the recent progress on quantum phase transitions in 2D superconductors, focusing on the quantum Griffiths singularity (QGS) and anomalous metal state. Characterized by a divergent critical exponent when approaching zero temperature, QGS of SMT is discovered in ultrathin crystalline Ga films and subsequently detected in various 2D superconductors. The universality of QGS indicates the profound influence of quenched disorder on quantum phase transitions. Besides, in a 2D superconducting system, whether a metallic ground state can exist is a long-sought mystery. Recently, the charge-2e quantum oscillations are observed in nanopatterned superconducting films, indicating the bosonic nature of the anomalous metal state and ending the debate on whether bosons can exist as a metal. The evidences of the anomalous metal states have also been reported in crystalline epitaxial thin films and exfoliated nanoflakes, as well as granular composite films. High quality filters are used in these works to exclude the influence of external high frequency noises in ultralow temperature measurements. The observations of QGS and metallic ground states in 2D superconductors not only reveal the prominent role of quantum fluctuations and dissipations but also provide new perspective to explore quantum phase transitions in superconducting systems.

cond-mat.supr-con↗

Sample-Conditioned Hypothesis Stability Sharpens Information-Theoretic Generalization Bounds

We present new information-theoretic generalization guarantees through the a novel construction of the "neighboring-hypothesis" matrix and a new family of stability notions termed sample-conditioned hypothesis (SCH) stability. Our approach yields sharper bounds that improve upon previous information-theoretic bounds in various learning scenarios. Notably, these bounds address the limitations of existing information-theoretic bounds in the context of stochastic convex optimization (SCO) problems, as explored in the recent work by Haghifam et al. (2023).

stat.ML↗

Tighter Information-Theoretic Generalization Bounds from Supersamples

In this work, we present a variety of novel information-theoretic generalization bounds for learning algorithms, from the supersample setting of Steinke & Zakynthinou (2020)-the setting of the "conditional mutual information" framework. Our development exploits projecting the loss pair (obtained from a training instance and a testing instance) down to a single number and correlating loss values with a Rademacher sequence (and its shifted variants). The presented bounds include square-root bounds, fast-rate bounds, including those based on variance and sharpness, and bounds for interpolating algorithms etc. We show theoretically or empirically that these bounds are tighter than all information-theoretic bounds known to date on the same supersample setting.

stat.ML↗

Information-Theoretic Analysis of Unsupervised Domain Adaptation

This paper uses information-theoretic tools to analyze the generalization error in unsupervised domain adaptation (UDA). We present novel upper bounds for two notions of generalization errors. The first notion measures the gap between the population risk in the target domain and that in the source domain, and the second measures the gap between the population risk in the target domain and the empirical risk in the source domain. While our bounds for the first kind of error are in line with the traditional analysis and give similar insights, our bounds on the second kind of error are algorithm-dependent, which also provide insights into algorithm designs. Specifically, we present two simple techniques for improving generalization in UDA and validate them experimentally.

cs.LG↗

Over-training with Mixup May Hurt Generalization

Mixup, which creates synthetic training instances by linearly interpolating random sample pairs, is a simple and yet effective regularization technique to boost the performance of deep models trained with SGD. In this work, we report a previously unobserved phenomenon in Mixup training: on a number of standard datasets, the performance of Mixup-trained models starts to decay after training for a large number of epochs, giving rise to a U-shaped generalization curve. This behavior is further aggravated when the size of original dataset is reduced. To help understand such a behavior of Mixup, we show theoretically that Mixup training may introduce undesired data-dependent label noises to the synthesized data. Via analyzing a least-square regression problem with a random feature model, we explain why noisy labels may cause the U-shaped curve to occur: Mixup improves generalization through fitting the clean patterns at the early training stage, but as training progresses, Mixup becomes over-fitting to the noise in the synthetic data. Extensive experiments are performed on a variety of benchmark datasets, validating this explanation.

cs.LG↗

Cluster Attack: Query-based Adversarial Attacks on Graphs with Graph-Dependent Priors

While deep neural networks have achieved great success in graph analysis, recent work has shown that they are vulnerable to adversarial attacks. Compared with adversarial attacks on image classification, performing adversarial attacks on graphs is more challenging because of the discrete and non-differential nature of the adjacent matrix for a graph. In this work, we propose Cluster Attack -- a Graph Injection Attack (GIA) on node classification, which injects fake nodes into the original graph to degenerate the performance of graph neural networks (GNNs) on certain victim nodes while affecting the other nodes as little as possible. We demonstrate that a GIA problem can be equivalently formulated as a graph clustering problem; thus, the discrete optimization problem of the adjacency matrix can be solved in the context of graph clustering. In particular, we propose to measure the similarity between victim nodes by a metric of Adversarial Vulnerability, which is related to how the victim nodes will be affected by the injected fake node, and to cluster the victim nodes accordingly. Our attack is performed in a practical and unnoticeable query-based black-box manner with only a few nodes on the graphs that can be accessed. Theoretical analysis and extensive experiments demonstrate the effectiveness of our method by fooling the node classifiers with only a small number of queries.

cs.LG↗

On the Generalization of Models Trained with SGD: Information-Theoretic Bounds and Implications

This paper follows up on a recent work of Neu et al. (2021) and presents some new information-theoretic upper bounds for the generalization error of machine learning models, such as neural networks, trained with SGD. We apply these bounds to analyzing the generalization behaviour of linear and two-layer ReLU networks. Experimental study of these bounds provide some insights on the SGD training of neural networks. They also point to a new and simple regularization scheme which we show performs comparably to the current state of the art.

cs.LG↗

Capping layer influence and isotropic in-plane upper critical field of the superconductivity at the FeSe/SrTiO3 interface

Understanding the superconductivity at the interface of FeSe/SrTiO3 is a problem of great contemporary interest due to the significant increase in critical temperature (Tc) compared to that of bulk FeSe, as well as the possibility of an unconventional pairing mechanism and topological superconductivity. We report a study of the influence of a capping layer on superconductivity in thin films of FeSe grown on SrTiO3 using molecular beam epitaxy. We used in vacuo four-probe electrical resistance measurements and ex situ magneto-transport measurements to examine the effect of three capping layers that provide distinctly different charge transfer into FeSe: compound FeTe, non-metallic Te, and metallic Zr. Our results show that FeTe provides an optimal cap that barely influences the inherent Tc found in pristine FeSe/SrTiO3, while the transfer of holes from a non-metallic Te cap completely suppresses superconductivity and leads to insulating behavior. Finally, we used ex situ magnetoresistance measurements in FeTe-capped FeSe films to extract the angular dependence of the in-plane upper critical magnetic field. Our observations reveal an almost isotropic in-plane upper critical field, providing insight into the symmetry and pairing mechanism of high temperature superconductivity in FeSe.

cond-mat.supr-con↗

Anomalous linear magnetoresistance in high quality crystalline lead thin films

Intriguing novel phenomena in lead films inspire new understanding of quantum physics, such as quantum size effect and quantum phase transitions etc. The improvement of the sample quality makes it even more promising to explore the intrinsic properties in two-dimensional system. In this paper, we show that the crystalline interfacial striped incommensurate layer can increase the quality of the lead films and significantly enhance the magnitude of magnetoresistance. By performing systematic transport measurement, a predominant anomalous linear magnetoresistance is revealed, and the widely used Parish-Littlewood model and Abrikosov's explanation fail to describe the observation. Instead, we propose a new model of linear magnetoresistance based on linear band structure, which shows a good agreement with the experimental results. Our studies reveal a novel origin of linear magnetoresistance which may also be helpful to understand the linear magnetoresistance in other materials with linear dispersion of electronic structure.

cond-mat.mes-hall↗

Zero-energy bound states in the high-temperature superconductors at the two-dimensional limit

Majorana zero modes (MZMs) that obey the non-Abelian statistics have been intensively investigated for potential applications in topological quantum computing. The prevailing signals in tunneling experiments "fingerprinting" the existence of MZMs are the zero-energy bound states (ZEBSs). However, nearly all of the previously reported ZEBSs showing signatures of the MZMs are observed in difficult-to-fabricate heterostructures at very low temperatures and additionally require applied magnetic field. Here, by using in-situ scanning tunneling spectroscopy, we detect the ZEBSs upon the interstitial Fe adatoms deposited on two different high-temperature superconducting one-unit-cell-thick iron chalcogenides on SrTiO3(001). The spectroscopic results resemble the phenomenological characteristics of the MZMs inside the vortex cores of topological superconductors. Our experimental findings may extend the MZM explorations in connate topological superconductors towards an applicable temperature regime and down to the two-dimensional limit. While a concrete understanding of the observations is lacking, possible explanations involving novel 2D superconducting states with spin-orbit coupling, spontaneous nucleation of anomalous vortices at the magnetic sites, and noncoplanar magnetic ordering may further stimulate theoretical understandings of the scarcely captured ZEBSs in strongly correlated systems with multiband Cooper pairing.

cond-mat.supr-con↗

Anomalous quantum Griffiths singularity in ultrathin crystalline lead films

We study the magnetic field induced superconductor-metal transition (SMT) in ultrathin crystalline Pb films. By performing ultralow temperature transport measurement, the divergent critical exponent as an indication of quantum Griffiths singularity (QGS) is observed when approaching zero temperature quantum critical point. Distinctively, the phase boundary of SMT exhibits an anomalous behavior in low temperature regime that the onset critical field decreases with decreasing temperatures, which distinguishes our observation from earlier reports of QGS in various two-dimensional superconductors. We demonstrate that this observed anomalous phase boundary has its origin from the superconducting fluctuations in ultrathin Pb films. Our findings reveal a novel aspect of the QGS of SMT in two-dimentional superconducting systems with anomalous phase boundary.

cond-mat.supr-con↗

Detection of bosonic mode as a signature of magnetic excitation in one unit cell FeSe on SrTiO3

We report an in situ scanning tunneling spectroscopy study of one-unit-cell (1-UC) FeSe film on SrTiO3(001) (STO) substrate. In quasiparticle density of states, bosonic excitation mode characterized by the "dip-hump" structure is detected outside the larger superconducting gap with energy comparable with phonon and spin resonance modes in heavily electron-doped iron selenides. Statistically, the excitation mode, which is intimately correlated with superconductivity, shows an anticorrelation with pairing strength and yields an energy scale upper-bounded by twice the superconducting gap coinciding with the characteristics of magnetic resonance in cuprates and iron-based superconductors. The local response of tunneling spectra to magnetically different Se defects all exhibits the induced in-gap quasiparticle bound states, indicating an unconventional sign-reversing pairing. These results support the magnetic nature of the excitation mode and possibly reveal a signature of electron-magnetic-excitation coupling in high-temperature superconductivity of 1-UC FeSe/STO.

cond-mat.supr-con↗

Novel voltage signal at proximity effect induced superconducting transition temperature in gold nanowires

We observed a novel voltage peak in the proximity effect induced superconducting gold (Au) nanowire while cooling the sample through the superconducting transition temperature. The voltage peak turned into dip in the warming process. The voltage peak (or dip) was found to be closely related to the emergence (vanishing) of the proximity induced superconductivity in the Au nanowire. The amplitude of the voltage signal depends on the temperature scanning rate and it cannot be detected when the temperature is changed slower than 0.03 K/min. This transient feature suggests the non-equilibrium property of the effect. The voltage peak could be understood by Ginzburg-Landau model as a combined result of the emergence of Cooper pairs with relatively lower free energy in W contact and the non-equilibrium diffusion of Cooper pairs and quasiparticles.

cond-mat.supr-con↗

Interface induced Zeeman-protected superconductivity in ultrathin crystalline lead films

Two dimensional (2D) superconducting systems are of great importance to exploring exotic quantum physics. Recent development of fabrication techniques stimulates the studies of high quality single crystalline 2D superconductors, where intrinsic properties give rise to unprecedented physical phenomena. Here we report the observation of Zeeman-type spin-orbit interaction protected superconductivity (Zeeman-protected superconductivity) in 4 monolayer (ML) to 6 ML crystalline Pb films grown on striped incommensurate (SIC) Pb layers on Si(111) substrates by molecular beam epitaxy (MBE). Anomalous large in-plane critical field far beyond the Pauli limit is detected, which can be attributed to the Zeeman-protected superconductivity due to the in-plane inversion symmetry breaking at the interface. Our work demonstrates that in superconducting heterostructures the interface can induce Zeeman-type spin-orbit interaction (SOI) and modulate the superconductivity.

cond-mat.supr-con↗

Spin Fluctuation Induced Linear Magnetoresistance in Ultrathin Superconducting FeSe Films

The discovery of high-temperature superconductivity in FeSe/STO has trigged great research interest to reveal a range of exotic physical phenomena in this novel material. Here we present a temperature dependent magnetotransport measurement for ultrathin FeSe/STO films with different thickness and protection layers. Remarkably, a surprising linear magnetoresistance (LMR) is observed around the superconducting transition temperatures but absent otherwise. The experimental LMR can be reproduced by magnetotransport calculations based on a model of magnetic field dependent disorder induced by spin fluctuation. Thus, the observed LMR in coexistence with superconductivity provides the first magnetotransport signature for spin fluctuation around the superconducting transition region in ultrathin FeSe/STO films.

cond-mat.mes-hall↗

High-temperature superconductivity in one-unit-cell FeSe films

Since the dramatic interface enhancement of superconducting transition temperature (Tc) was reported in one unit-cell FeSe film grown on SrTiO3 substrate (1-UC FeSe/STO) by molecular beam epitaxy (MBE), related research on this system has become a new frontier in condensed matter physics. In this paper, we present a brief review on this rapidly developing field, mainly focusing on the superconducting properties of 1-UC FeSe/STO. Experimental evidences for the high-temperature superconductivity in 1-UC FeSe/STO, including the direct evidences revealed by transport and diamagnetic measurements, and other evidences from scanning tunneling microscope (STM) and angle-resolved photoemission spectroscopy (ARPES), are overviewed. Potential mechanisms of the enhanced superconductivity are discussed. There are accumulating arguments suggesting that the strengthened Cooper pairing in 1-UC FeSe/STO originates from the interface effects, specifically charge transfer and coupling to phonon modes in TiO2 plane. The study of superconductivity in 1-UC FeSe/STO not only sheds a new light on the mechanism of high-temperature superconductors with layered structures, but also provides the insight to explore new superconductors by interface engineering.

cond-mat.supr-con↗