arXiv subjects

Hao Zheng

Publications and source records attributed to Hao Zheng.

At least 163 records · Page 9Linked to original sources

arXiv2023

Categorical computation

In quantum computing, the computation is achieved by linear operators in or between Hilbert spaces. In this work, we explore a new computation scheme, in which the linear operators in quantum computing are replaced by (higher) functors between two (higher) categories. If from Turing computing to quantum computing is the first quantization of computation, then this new scheme can be viewed as the second quantization of computation. The fundamental problem in realizing this idea is how to realize a (higher) functor physically. We provide a theoretical idea of realizing (higher) functors physically based on the physics of topological orders.

quant-ph↗

arXiv2023

TAPS: Topology-Aware Intra-Operator Parallelism Strategy Searching Algorithm for Deep Neural Networks

TAPS is a Topology-Aware intra-operator Parallelism strategy Searching algorithm that generates intra-operator parallelism strategies by considering both intra-node and inter-node bandwidth. Most of the existing auto-parallelism works use the communication volume as the communication cost directly when generating strategies, which we prove to be sub-optimal in multi-nodes cases. We design a topology-aware cost model for multi-node intra-operator parallelism strategy searching. Numerical experiments demonstrate that TAPS can generate strategies with up to 85% fewer communication costs, which outperform the latest baselines.

cs.DC↗

arXiv2022

Real-World Compositional Generalization with Disentangled Sequence-to-Sequence Learning

Compositional generalization is a basic mechanism in human language learning, which current neural networks struggle with. A recently proposed Disentangled sequence-to-sequence model (Dangle) shows promising generalization capability by learning specialized encodings for each decoding step. We introduce two key modifications to this model which encourage more disentangled representations and improve its compute and memory efficiency, allowing us to tackle compositional generalization in a more realistic setting. Specifically, instead of adaptively re-encoding source keys and values at each time step, we disentangle their representations and only re-encode keys periodically, at some interval. Our new architecture leads to better generalization performance across existing tasks and datasets, and a new machine translation benchmark which we create by detecting naturally occurring compositional patterns in relation to a training set. We show this methodology better emulates real-world requirements than artificial challenges.

cs.CL↗

arXiv2022

Keep Your Friends Close & Enemies Farther: Debiasing Contrastive Learning with Spatial Priors in 3D Radiology Images

Understanding of spatial attributes is central to effective 3D radiology image analysis where crop-based learning is the de facto standard. Given an image patch, its core spatial properties (e.g., position & orientation) provide helpful priors on expected object sizes, appearances, and structures through inherent anatomical consistencies. Spatial correspondences, in particular, can effectively gauge semantic similarities between inter-image regions, while their approximate extraction requires no annotations or overbearing computational costs. However, recent 3D contrastive learning approaches either neglect correspondences or fail to maximally capitalize on them. To this end, we propose an extensible 3D contrastive framework (Spade, for Spatial Debiasing) that leverages extracted correspondences to select more effective positive & negative samples for representation learning. Our method learns both globally invariant and locally equivariant representations with downstream segmentation in mind. We also propose separate selection strategies for global & local scopes that tailor to their respective representational requirements. Compared to recent state-of-the-art approaches, Spade shows notable improvements on three downstream segmentation tasks (CT Abdominal Organ, CT Heart, MR Heart).

cs.CV↗

arXiv2022

A Point in the Right Direction: Vector Prediction for Spatially-aware Self-supervised Volumetric Representation Learning

High annotation costs and limited labels for dense 3D medical imaging tasks have recently motivated an assortment of 3D self-supervised pretraining methods that improve transfer learning performance. However, these methods commonly lack spatial awareness despite its centrality in enabling effective 3D image analysis. More specifically, position, scale, and orientation are not only informative but also automatically available when generating image crops for training. Yet, to date, no work has proposed a pretext task that distills all key spatial features. To fulfill this need, we develop a new self-supervised method, VectorPOSE, which promotes better spatial understanding with two novel pretext tasks: Vector Prediction (VP) and Boundary-Focused Reconstruction (BFR). VP focuses on global spatial concepts (i.e., properties of 3D patches) while BFR addresses weaknesses of recent reconstruction methods to learn more effective local representations. We evaluate VectorPOSE on three 3D medical image segmentation tasks, showing that it often outperforms state-of-the-art methods, especially in limited annotation settings.

cs.CV↗

arXiv2022

Dance of SNN and ANN: Solving binding problem by combining spike timing and reconstructive attention

The binding problem is one of the fundamental challenges that prevent the artificial neural network (ANNs) from a compositional understanding of the world like human perception, because disentangled and distributed representations of generative factors can interfere and lead to ambiguity when complex data with multiple objects are presented. In this paper, we propose a brain-inspired hybrid neural network (HNN) that introduces temporal binding theory originated from neuroscience into ANNs by integrating spike timing dynamics (via spiking neural networks, SNNs) with reconstructive attention (by ANNs). Spike timing provides an additional dimension for grouping, while reconstructive feedback coordinates the spikes into temporal coherent states. Through iterative interaction of ANN and SNN, the model continuously binds multiple objects at alternative synchronous firing times in the SNN coding space. The effectiveness of the model is evaluated on synthetic datasets of binary images. By visualization and analysis, we demonstrate that the binding is explainable, soft, flexible, and hierarchical. Notably, the model is trained on single object datasets without explicit supervision on grouping, but successfully binds multiple objects on test datasets, showing its compositional generalization capability. Further results show its binding ability in dynamic situations.

cs.AI↗

arXiv2022

Quasiparticle characteristics of the weakly ferromagnetic Hund's metal MnSi

Hund's metals are multi-orbital systems with $3d$ or $4d$ electrons exhibiting both itinerant character and local moments, and they feature Kondo-like screenings of local orbital and spin moments, with suppressed coherence temperature driven by Hund's coupling $J_H$. They often exhibit magnetic order at low temperature, but how the interaction between the Kondo-like screening and long-range magnetic order is manifested in the quasiparticle spectrum remains an open question. Here we present spectroscopic signature of such interaction in a Hund's metal candidate MnSi exhibiting weak ferromagnetism. Our photoemission measurements reveal renormalized quasiparticle bands near the Fermi level with strong momentum dependence: the ferromagnetism manifests through possibly exchange-split bands (Q1) below $T_C$ , while the spin/orbital screenings lead to gradual development of quasiparticles (Q2) upon cooling. Our results demonstrate how the characteristic spin/orbital coherence in a Hund's metal could coexist and compete with the magnetic order to form a weak itinerant ferromagnet, via quasiparticle bands that are well separated in momentum space and exhibit distinct temperature dependence. Our results imply that the competition between the spin/orbital screening and the magnetic order in a Hund's metal bears intriguing similarity to the Kondo lattice systems.

cond-mat.str-el↗

arXiv2022

Revising the Hubble constant, spatial curvature and dark energy dynamics with the latest observations of quasars

In this paper, we use a newly compiled sample of ultra-compact structure in radio quasars and strong gravitational lensing systems with quasars acting as background sources to constrain six spatially flat and non-flat cosmological models ($Λ$CDM, PEDE and DGP). These two sets of quasar data (the time-delay measurements of six strong lensing systems and 120 intermediate-luminosity quasars calibrated as standard rulers) could break the degeneracy between cosmological parameters ($H_0$, $Ω_m$ and $Ω_k$) and therefore provide more stringent cosmological constraints for the six cosmological models we study. A joint analysis of the quasar sample provides model-independent estimations of the Hubble constant $H_0$, which is strongly consistent with that derived from the local distance ladder by SH0ES collaboration in the $Λ$CDM and PEDE model. However, in the framework of a DGP cosmology (especially for the flat universe), the measured Hubble constant is in good agreement with that derived from the the recent Planck 2018 results. In addition, our results show that zero spatial curvature is supported by the current lensed and unlensed quasar observations and there is no significant deviation from a flat universe. For most of cosmological model we study (the flat $Λ$CDM, non-flat $Λ$CDM, flat PEDE, and non-flat PEDE models), the derived matter density parameter is completely consistent with $Ω_m\sim 0.30$ in all the data sets, as expected by the latest cosmological observations. Finally, according to the the statistical criteria DIC, although the joint constraints provide substantial observational support to the flat PEDE model, they do not rule out dark energy being a cosmological constant and non-flat spatial hypersurfaces.

astro-ph.CO↗

arXiv2022

Mining SoC Message Flows with Attention Model

High-quality system-level message flow specifications are necessary for comprehensive validation of system-on-chip (SoC) designs. However, manual development and maintenance of such specifications are daunting tasks. We propose a disruptive method that utilizes deep sequence modeling with the attention mechanism to infer accurate flow specifications from SoC communication traces. The proposed method can overcome the inherent complexity of SoC traces induced by the concurrent executions of SoC designs that existing mining tools often find extremely challenging. We conduct experiments on five highly concurrent traces and find that the proposed approach outperforms several existing state-of-the-art trace mining tools.

cs.AI↗

arXiv2022

Observation of magnetism induced topological edge state in antiferromagnetic topological insulator MnBi4Te7

Breaking time reversal symmetry in a topological insulator may lead to quantum anomalous Hall effect and axion insulator phase. MnBi4Te7 is a recently discovered antiferromagnetic topological insulator with TN ~12.5 K, which is constituted of alternatively stacked magnetic layer (MnBi2Te4) and non-magnetic layer (Bi2Te3). By means of scanning tunneling spectroscopy, we clearly observe the electronic state present at a step edge of a magnetic MnBi2Te4 layer but absent at non-magnetic Bi2Te3 layers at 4.5 K. Furthermore, we find that as the temperature rises above TN, the edge state vanishes, while the point defect induced state persists upon temperature increasing. These results confirm the observation of magnetism induced edge states. Our analysis based on an axion insulator theory reveals that the nontrivial topological nature of the observed edge state.

cond-mat.mtrl-sci↗

arXiv2022

Quantum phase transition in magnetic nanographenes on a lead superconductor

Quantum spins, referred to the spin operator preserved by full SU(2) symmetry in the absence of the magnetic anistropy, have been proposed to host exotic interactions with superconductivity4. However, spin orbit coupling and crystal field splitting normally cause a significant magnetic anisotropy for d/f-shell spins on surfaces6,9, breaking SU(2) symmetry and fabricating the spins with Ising properties10. Recently, magnetic nanographenes have been proven to host intrinsic quantum magnetism due to their negligible spin orbital coupling and crystal field splitting. Here, we fabricate three atomically precise nanographenes with the same magnetic ground state of spin S=1/2 on Pb(111) through engineering sublattice imbalance in graphene honeycomb lattice. Scanning tunneling spectroscopy reveals the coexistence of magnetic bound states and Kondo screening in such hybridized system. Through engineering the magnetic exchange strength between the unpaired spin in nanographenes and cooper pairs, quantum phase transition from the singlet to the doublet state has been observed, in consistent with quantum models of spins on superconductors. Our work demonstrates delocalized graphene magnetism host highly tunable magnetic bound states with cooper pairs, which can be further developed to study the Majorana bound states and other rich quantum physics of low-dimensional quantum spins on superconductors.

cond-mat.mes-hall↗

arXiv2022

Counterexample Generation for Infinite-State Chemical Reaction Networks

Counterexample generation is an indispensable part of model checking process. In stochastic model checking, counterexample generation is a challenging problem as it is not enough to find a single trace that violates the given property. Instead, a potentially large set of traces with enough probability to violate the property needs to be found. This paper considers counterexample generation for chemical reaction network (CRN) models with potentially infinite state space. A method based on bounded model checking using SMT solving is developed for counterexample generation for CRNs. It intends to find a small set of property violating paths of a given model such that they collectively have a total probability that is above a given threshold. A unique challenge is due to the highly connected state space of CRNs where a counterexample is only a tiny subset of all property violating paths. To address such challenges, this paper presents a number of optimizations including a divide-and-conquer technique to scale up the counterexample generation method for large CRN models. This paper reports results from experiments on a number of infinite-state CRN models.

cs.LO↗

arXiv2022

Usable Region Estimate for Assessing Practical Usability of Medical Image Segmentation Models

We aim to quantitatively measure the practical usability of medical image segmentation models: to what extent, how often, and on which samples a model's predictions can be used/trusted. We first propose a measure, Correctness-Confidence Rank Correlation (CCRC), to capture how predictions' confidence estimates correlate with their correctness scores in rank. A model with a high value of CCRC means its prediction confidences reliably suggest which samples' predictions are more likely to be correct. Since CCRC does not capture the actual prediction correctness, it alone is insufficient to indicate whether a prediction model is both accurate and reliable to use in practice. Therefore, we further propose another method, Usable Region Estimate (URE), which simultaneously quantifies predictions' correctness and reliability of confidence assessments in one estimate. URE provides concrete information on to what extent a model's predictions are usable. In addition, the sizes of usable regions (UR) can be utilized to compare models: A model with a larger UR can be taken as a more usable and hence better model. Experiments on six datasets validate that the proposed evaluation methods perform well, providing a concrete and concise measure for the practical usability of medical image segmentation models. Code is made available at https://github.com/yizhezhang2000/ure.

eess.IV↗

arXiv2022

Disentangled Sequence to Sequence Learning for Compositional Generalization

There is mounting evidence that existing neural network models, in particular the very popular sequence-to-sequence architecture, struggle to systematically generalize to unseen compositions of seen components. We demonstrate that one of the reasons hindering compositional generalization relates to representations being entangled. We propose an extension to sequence-to-sequence models which encourages disentanglement by adaptively re-encoding (at each time step) the source input. Specifically, we condition the source representations on the newly decoded target context which makes it easier for the encoder to exploit specialized information for each prediction rather than capturing it all in a single forward pass. Experimental results on semantic parsing and machine translation empirically show that our proposal delivers more disentangled representations and better generalization.

cs.CL↗

arXiv2022

One dimensional gapped quantum phases and enriched fusion categories

In this work, we use Ising chain and Kitaev chain to check the validity of an earlier proposal in arXiv:2011.02859 that enriched fusion (higher) categories provide a unified categorical description of all gapped/gapless quantum liquid phases, including symmetry-breaking phases, topological orders, SPT/SET orders and certain gapless quantum phases. In particular, we show explicitly that, in each gapped phase realized by these two models, the spacetime observables form a fusion category enriched in a braided fusion category. We also study the categorical descriptions of the boundaries of these models. In the end, we provide a classification of and the categorical descriptions of all 1-dimensional (the spatial dimension) gapped quantum phases with a finite onsite symmetry.

cond-mat.str-el↗

arXiv2022

Deep Bidirectional Transformers for SoC Flow Specification Mining

High-quality system-level message flow specifications can lead to comprehensive validation of system-on-chip (SoC) designs. We propose a disruptive method that utilizes an attention mechanism to produce accurate flow specifications from SoC IP communication traces. The proposed method can overcome the inherent complexity of SoC traces induced by the concurrency and parallelism of multicore designs that existing flow specification mining tools often find extremely challenging. Experiments on highly interleaved traces show promising flow reconstruction compared to several tools dedicated to the flow specification mining problem.

cs.LG↗

arXiv2022

LTSP: Long-Term Slice Propagation for Accurate Airway Segmentation

Purpose: Bronchoscopic intervention is a widely-used clinical technique for pulmonary diseases, which requires an accurate and topological complete airway map for its localization and guidance. The airway map could be extracted from chest computed tomography (CT) scans automatically by airway segmentation methods. Due to the complex tree-like structure of the airway, preserving its topology completeness while maintaining the segmentation accuracy is a challenging task. Methods: In this paper, a long-term slice propagation (LTSP) method is proposed for accurate airway segmentation from pathological CT scans. We also design a two-stage end-to-end segmentation framework utilizing the LTSP method in the decoding process. Stage 1 is used to generate a coarse feature map by an encoder-decoder architecture. Stage 2 is to adopt the proposed LTSP method for exploiting the continuity information and enhancing the weak airway features in the coarse feature map. The final segmentation result is predicted from the refined feature map. Results: Extensive experiments were conducted to evaluate the performance of the proposed method on 70 clinical CT scans. The results demonstrate the considerable improvements of the proposed method compared to some state-of-the-art methods as most breakages are eliminated and more tiny bronchi are detected. The ablation studies further confirm the effectiveness of the constituents of the proposed method. Conclusion: Slice continuity information is beneficial to accurate airway segmentation. Furthermore, by propagating the long-term slice feature, the airway topology connectivity is preserved with overall segmentation accuracy maintained.

eess.IV↗

arXiv2022

Topological Defects Induced High-Spin Quartet State in Truxene-Based Molecular Graphenoids

Topological defects in graphene materials introduce exotic properties which are absent in their defect-free counterparts with both fundamental importance and technological implications. Although individual topological defects have been widely studied, collective magnetic behaviors originating from well-organized multiple topological defects remain a great challenge. Here, we studied the collective magnetic properties originating from three pentagon topological defects in truxene-based molecular graphenoids by using scanning tunneling microscopy and non-contact atomic force microscopy. Unpaired $π$ electrons are introduced into the aromatic topology of truxene molecular graphenoids one by one by dissociating hydrogen atoms at the pentagon defects via atom manipulation. Scanning tunneling spectroscopy measurements together with density functional theory calculations suggest that the unpaired electrons are ferromagnetically coupled, forming a collective high-spin quartet state of S=3/2. Our work demonstrates that the collective spin ordering can be realized through engineering regular patterned topological defects in molecular graphenoids, providing a new platform for designer one-dimensional ferromagnetic spin chains and two-dimensional ferromagnetic networks.

cond-mat.mes-hall↗