arXiv subjects

Weifeng Zhang

Publications and source records attributed to Weifeng Zhang.

35 records · Page 2Linked to original sources

arXiv2023

A Classical Architecture For Digital Quantum Computers

Scaling bottlenecks the making of digital quantum computers, posing challenges from both the quantum and the classical components. We present a classical architecture to cope with a comprehensive list of the latter challenges {\em all at once}, and implement it fully in an end-to-end system by integrating a multi-core RISC-V CPU with our in-house control electronics. Our architecture enables scalable, high-precision control of large quantum processors and accommodates evolving requirements of quantum hardware. A central feature is a microarchitecture executing quantum operations in parallel on arbitrary predefined qubit groups. Another key feature is a reconfigurable quantum instruction set that supports easy qubit re-grouping and instructions extensions. As a demonstration, we implement the widely-studied surface code quantum computing workflow, which is instructive for being demanding on both the controllers and the integrated classical computation. Our design, for the first time, reduces instruction issuing and transmission costs to constants, which do not scale with the number of qubits, without adding any overheads in decoding or dispatching. Rather than relying on specialized hardware for syndrome decoding, our system uses a dedicated multi-core CPU for both qubit control and classical computation, including syndrome decoding. This simplifies the system design and facilitates load-balancing between the quantum and classical components. We implement recent proposals as decoding firmware on a RISC-V system-on-chip (SoC) that parallelizes general inner decoders. By using our in-house Union-Find and PyMatching 2 implementations, we can achieve unprecedented decoding capabilities of up to distances 47 and 67 with the currently available SoCs, under realistic and optimistic assumptions of physical error rate $p=0.001 and p=0.0001, respectively, all in just 1 \textmu s.

quant-ph↗

arXiv2022

Learning to Affiliate: Mutual Centralized Learning for Few-shot Classification

Few-shot learning (FSL) aims to learn a classifier that can be easily adapted to accommodate new tasks not seen during training, given only a few examples. To handle the limited-data problem in few-shot regimes, recent methods tend to collectively use a set of local features to densely represent an image instead of using a mixed global feature. They generally explore a unidirectional query-to-support paradigm in FSL, e.g., find the nearest/optimal support feature for each query feature and aggregate these local matches for a joint classification. In this paper, we propose a new method Mutual Centralized Learning (MCL) to fully affiliate the two disjoint sets of dense features in a bidirectional paradigm. We associate each local feature with a particle that can bidirectionally random walk in a discrete feature space by the affiliations. To estimate the class probability, we propose the features' accessibility that measures the expected number of visits to the support features of that class in a Markov process. We relate our method to learning a centrality on an affiliation network and demonstrate its capability to be plugged in existing methods by highlighting centralized local features. Experiments show that our method achieves the state-of-the-art on both miniImageNet and tieredImageNet.

cs.CV↗

arXiv2021

N3H-Core: Neuron-designed Neural Network Accelerator via FPGA-based Heterogeneous Computing Cores

Accelerating the neural network inference by FPGA has emerged as a popular option, since the reconfigurability and high performance computing capability of FPGA intrinsically satisfies the computation demand of the fast-evolving neural algorithms. However, the popular neural accelerators on FPGA (e.g., Xilinx DPU) mainly utilize the DSP resources for constructing their processing units, while the rich LUT resources are not well exploited. Via the software-hardware co-design approach, in this work, we develop an FPGA-based heterogeneous computing system for neural network acceleration. From the hardware perspective, the proposed accelerator consists of DSP- and LUT-based GEneral Matrix-Multiplication (GEMM) computing cores, which forms the entire computing system in a heterogeneous fashion. The DSP- and LUT-based GEMM cores are computed w.r.t a unified Instruction Set Architecture (ISA) and unified buffers. Along the data flow of the neural network inference path, the computation of the convolution/fully-connected layer is split into two portions, handled by the DSP- and LUT-based GEMM cores asynchronously. From the software perspective, we mathematically and systematically model the latency and resource utilization of the proposed heterogeneous accelerator, regarding varying system design configurations. Through leveraging the reinforcement learning technique, we construct a framework to achieve end-to-end selection and optimization of the design specification of target heterogeneous accelerator, including workload split strategy, mixed-precision quantization scheme, and resource allocation of DSP- and LUT-core. In virtue of the proposed design framework and heterogeneous computing system, our design outperforms the state-of-the-art Mix&Match design with latency reduced by 1.12-1.32x with higher inference accuracy. The N3H-core is open-sourced at: https://github.com/elliothe/N3H_Core.

cs.AR↗

arXiv2021

Magnetic moment preservation and emergent Kondo resonance of Co-phthalocyanine on semimetallic Sb(111)

Magnetic molecules on surfaces have been widely investigated to reveal delicate interfacial couplings and for potential technological applications. In these endeavors, one prevailing challenge is how to preserve or recover the molecular spins, especially on highly metallic substrates that can readily quench the magnetic moments of the ad-molecules. Here we use scanning tunneling microscopy/spectroscopy to exploit the semimetallic nature of antimony and observe, surprisingly yet pleasantly, that the spin of Co-phthalocyanine is well preserved on Sb(111), as unambiguously evidenced by the emergent strong Kondo resonance across the molecule. Our first-principles calculations further confirm that the optimal density of states near the Fermi level of the semimetal is a decisive factor, weakening the overall interfacial coupling, while still ensuring sufficiently effective electron-spin scattering in the many-body system. Beyond isolated ad-molecules, we discover that each of the magnetic moments in a molecular dimer or a densely packed island is distinctly preserved as well, rendering such molecular magnets immense potentials for ultra-high density memory devices.

cond-mat.mtrl-sci↗

arXiv2020

Cross-modal Knowledge Reasoning for Knowledge-based Visual Question Answering

Knowledge-based Visual Question Answering (KVQA) requires external knowledge beyond the visible content to answer questions about an image. This ability is challenging but indispensable to achieve general VQA. One limitation of existing KVQA solutions is that they jointly embed all kinds of information without fine-grained selection, which introduces unexpected noises for reasoning the correct answer. How to capture the question-oriented and information-complementary evidence remains a key challenge to solve the problem. Inspired by the human cognition theory, in this paper, we depict an image by multiple knowledge graphs from the visual, semantic and factual views. Thereinto, the visual graph and semantic graph are regarded as image-conditioned instantiation of the factual graph. On top of these new representations, we re-formulate Knowledge-based Visual Question Answering as a recurrent reasoning process for obtaining complementary evidence from multimodal information. To this end, we decompose the model into a series of memory-based reasoning steps, each performed by a G raph-based R ead, U pdate, and C ontrol ( GRUC ) module that conducts parallel reasoning over both visual and semantic information. By stacking the modules multiple times, our model performs transitive reasoning and obtains question-oriented concept representations under the constrain of different modalities. Finally, we perform graph neural networks to infer the global-optimal answer by jointly considering all the concepts. We achieve a new state-of-the-art performance on three popular benchmark datasets, including FVQA, Visual7W-KB and OK-VQA, and demonstrate the effectiveness and interpretability of our model with extensive experiments.

cs.AI↗

arXiv2020

Regularized Training and Tight Certification for Randomized Smoothed Classifier with Provable Robustness

Recently smoothing deep neural network based classifiers via isotropic Gaussian perturbation is shown to be an effective and scalable way to provide state-of-the-art probabilistic robustness guarantee against $\ell_2$ norm bounded adversarial perturbations. However, how to train a good base classifier that is accurate and robust when smoothed has not been fully investigated. In this work, we derive a new regularized risk, in which the regularizer can adaptively encourage the accuracy and robustness of the smoothed counterpart when training the base classifier. It is computationally efficient and can be implemented in parallel with other empirical defense methods. We discuss how to implement it under both standard (non-adversarial) and adversarial training scheme. At the same time, we also design a new certification algorithm, which can leverage the regularization effect to provide tighter robustness lower bound that holds with high probability. Our extensive experimentation demonstrates the effectiveness of the proposed training and certification approaches on CIFAR-10 and ImageNet datasets.

stat.ML↗

arXiv2020

Software-defined Design Space Exploration for an Efficient DNN Accelerator Architecture

Deep neural networks (DNNs) have been shown to outperform conventional machine learning algorithms across a wide range of applications, e.g., image recognition, object detection, robotics, and natural language processing. However, the high computational complexity of DNNs often necessitates extremely fast and efficient hardware. The problem gets worse as the size of neural networks grows exponentially. As a result, customized hardware accelerators have been developed to accelerate DNN processing without sacrificing model accuracy. However, previous accelerator design studies have not fully considered the characteristics of the target applications, which may lead to sub-optimal architecture designs. On the other hand, new DNN models have been developed for better accuracy, but their compatibility with the underlying hardware accelerator is often overlooked. In this article, we propose an application-driven framework for architectural design space exploration of DNN accelerators. This framework is based on a hardware analytical model of individual DNN operations. It models the accelerator design task as a multi-dimensional optimization problem. We demonstrate that it can be efficaciously used in application-driven accelerator architecture design. Given a target DNN, the framework can generate efficient accelerator design solutions with optimized performance and area. Furthermore, we explore the opportunity to use the framework for accelerator configuration optimization under simultaneous diverse DNN applications. The framework is also capable of improving neural network models to best fit the underlying hardware resources.

cs.DC↗

arXiv2019

Finite-dimensional bistable topological insulators: From small to large

Photonic topological insulators supporting unidirectional topologically protected edge states represent attractive platform for realization of disorder- and backscattering-immune transport of edge excitations in both linear and nonlinear regimes. In many realizations of topological insulators structured periodic materials are used, since they may admit specific Dirac degeneracy in the spectrum, around which unidirectional edge states appear under the action of physical effects breaking time-reversal symmetry. While properties of the edge states at unclosed interfaces of two bulk media with different topology are known, the existence of the edge states in practical finite-dimensional topological insulators fully immersed in nontopological environment remains largely unexplored. In this work using as an example realistic polariton topological insulators built from small-size honeycomb arrays of microcavity pillars, we illustrate how topological properties of the system build up upon gradual increase of its dimensionality. To account for dissipative nature of polariton condensate forming in the array of microcavity pillars, we consider the impact of losses and resonant pump leading to rich bistability effects in this system. We describe the mechanism in accordance with which trivial-phase pump "selects" and excites specific nonlinear topological edge states circulating along the periphery of the structure in the azimuthal direction dictated by the direction of the external applied magnetic field. We also show the possibility of utilization of vortex pump with different topological charges for selective excitation of different edge currents.

physics.optics↗

arXiv2019

Semantic Modeling of Textual Relationships in Cross-Modal Retrieval

Feature modeling of different modalities is a basic problem in current research of cross-modal information retrieval. Existing models typically project texts and images into one embedding space, in which semantically similar information will have a shorter distance. Semantic modeling of textural relationships is notoriously difficult. In this paper, we propose an approach to model texts using a featured graph by integrating multi-view textual relationships including semantic relations, statistical co-occurrence, and prior relations in the knowledge base. A dual-path neural network is adopted to learn multi-modal representations of information and cross-modal similarity measure jointly. We use a Graph Convolutional Network (GCN) for generating relation-aware text representations, and use a Convolutional Neural Network (CNN) with non-linearities for image representations. The cross-modal similarity measure is learned by distance metric learning. Experimental results show that, by leveraging the rich relational semantics in texts, our model can outperform the state-of-the-art models by 3.4% and 6.3% on accuracy on two benchmark datasets.

cs.MM↗

arXiv2019

Sionnx: Automatic Unit Test Generator for ONNX Conformance

Open Neural Network Exchange (ONNX) is an open format to represent AI models and is supported by many machine learning frameworks. While ONNX defines unified and portable computation operators across various frameworks, the conformance tests for those operators are insufficient, which makes it difficult to verify if an operator's behavior in an ONNX backend implementation complies with the ONNX standard. In this paper, we present the first automatic unit test generator named Sionnx for verifying the compliance of ONNX implementation. First, we propose a compact yet complete set of rules to describe the operator's attributes and the properties of its operands. Second, we design an Operator Specification Language (OSL) to provide a high-level description for the operator's syntax. Finally, through this easy-to-use specification language, we are able to build a full testing specification which leverages LLVM TableGen to automatically generate unit tests for ONNX operators with much large coverage. Sionnx is lightweight and flexible to support cross-framework verification. The Sionnx framework is open-sourced in the github repository (https://github.com/alibaba/Sionnx).

cs.SE↗

arXiv2019

Bloch oscillations of topological edge modes

Under the action of a weak constant force a wavepacket in periodic potential undergoes periodic oscillations in space, returning to the initial position after one oscillation cycle. This wave phenomenon, known as Bloch oscillations (BOs), pertains to many physical systems. Can BOs also occur in topological insulators with topologically protected edge states? This question is highly nontrivial, because in topological insulators with broken time-reversal symmetry, the edge states propagate unidirectionally without backscattering, hence BOs that typically involve stages, where wavepacket moves along and against the direction of the force, seem to be impossible in such systems when force acts parallel to the edge of the insulator. Here we reveal that BOs still occur with topological edge states, but in a nonconventional way: they are accompanied not only by oscillations along the edge in the direction of force, but also by oscillations in the direction transverse to that of the force. Full BO cycle involves switching between edge states at the opposite edges through delocalized bulk modes. Bloch oscillations of the topological edge states require to scan the first Brillouin zone twice to complete one cycle, thus they have a period that is two times larger than the period of usual BOs. All these unusual properties are in contrast to BOs in non-topological systems.

cond-mat.mes-hall↗

arXiv2019

Hardware-Guided Symbiotic Training for Compact, Accurate, yet Execution-Efficient LSTM

Many long short-term memory (LSTM) applications need fast yet compact models. Neural network compression approaches, such as the grow-and-prune paradigm, have proved to be promising for cutting down network complexity by skipping insignificant weights. However, current compression strategies are mostly hardware-agnostic and network complexity reduction does not always translate into execution efficiency. In this work, we propose a hardware-guided symbiotic training methodology for compact, accurate, yet execution-efficient inference models. It is based on our observation that hardware may introduce substantial non-monotonic behavior, which we call the latency hysteresis effect, when evaluating network size vs. inference latency. This observation raises question about the mainstream smaller-dimension-is-better compression strategy, which often leads to a sub-optimal model architecture. By leveraging the hardware-impacted hysteresis effect and sparsity, we are able to achieve the symbiosis of model compactness and accuracy with execution efficiency, thus reducing LSTM latency while increasing its accuracy. We have evaluated our algorithms on language modeling and speech recognition applications. Relative to the traditional stacked LSTM architecture obtained for the Penn Treebank dataset, we reduce the number of parameters by 18.0x (30.5x) and measured run-time latency by up to 2.4x (5.2x) on Nvidia GPUs (Intel Xeon CPUs) without any accuracy degradation. For the DeepSpeech2 architecture obtained for the AN4 dataset, we reduce the number of parameters by 7.0x (19.4x), word error rate from 12.9% to 9.9% (10.4%), and measured run-time latency by up to 1.7x (2.4x) on Nvidia GPUs (Intel Xeon CPUs). Thus, our method yields compact, accurate, yet execution-efficient inference models.

cs.NE↗

arXiv2018

Modeling Text with Graph Convolutional Network for Cross-Modal Information Retrieval

Cross-modal information retrieval aims to find heterogeneous data of various modalities from a given query of one modality. The main challenge is to map different modalities into a common semantic space, in which distance between concepts in different modalities can be well modeled. For cross-modal information retrieval between images and texts, existing work mostly uses off-the-shelf Convolutional Neural Network (CNN) for image feature extraction. For texts, word-level features such as bag-of-words or word2vec are employed to build deep learning models to represent texts. Besides word-level semantics, the semantic relations between words are also informative but less explored. In this paper, we model texts by graphs using similarity measure based on word2vec. A dual-path neural network model is proposed for couple feature learning in cross-modal information retrieval. One path utilizes Graph Convolutional Network (GCN) for text modeling based on graph representations. The other path uses a neural network with layers of nonlinearities for image modeling based on off-the-shelf features. The model is trained by a pairwise similarity loss function to maximize the similarity of relevant text-image pairs and minimize the similarity of irrelevant pairs. Experimental results show that the proposed model outperforms the state-of-the-art methods significantly, with 17% improvement on accuracy for the best case.

cs.IR↗

arXiv2017

Experimental realization of graphene-like borophene

We report a successful preparation of a purely honeycomb, graphene-like borophene, by using an Al(111) surface as the substrate and molecular beam epitaxy (MBE) growth in ultrahigh vacuum. Our scanning tunneling microscopy (STM) reveals perfect monolayer borophene with planar, non-buckled honeycomb lattice similar as graphene. Theoretical calculations show that the honeycomb borophene on Al(111) is energetically stable. Remarkably, nearly one electron charge is transferred to each boron atom from the Al(111) substrate and stabilizes the honeycomb borophene structure, in contrast to the little charge transfer in borophene/Ag(111) case. This work demonstrates that the borophene lattice can be manipulated by controlling the charge transfer between the substrate and the borophene film.

cond-mat.mes-hall↗

arXiv2017

A Silicon Photonic Integrated Frequency-Tunable Microwave Photonic Bandpass Filter

We report a fully integrated microwave photonic passband filter on a silicon photonic chip. The on-chip integrated microwave photonic filter (IMPF) consists of a high-speed phase modulator (PM), a thermally tunable high-Q micro-disk resonator (MDR), and a high-speed photodetector (PD). The bandpass filtering function of the IMPF is implemented based on phase modulation and phase-modulation to intensity-modulation conversion, to translate the spectral response of an optical filter, which is the MDR in the IMPF, to the spectral response of a microwave filter. By injecting a CW light from a laser diode (LD) to the PM, a passband IMPF with a broad tuning range is demonstrated. Thanks to the ultra-narrow notch of the MDR, a passband IMPF with a 3-dB bandwidth of 2.3 GHz is realized. By thermally tuning the MDR, the center frequency of the IMPF is tuned from 7 to 25 GHz, achieving a tuning range of 18 GHz at a power consumption of 1.58 mW. This successful implementation of an IMPF marks a significant step forward in full integration of microwave photonic systems on a single chip and opens up avenues toward real applications of microwave photonic filters.

physics.app-ph↗

arXiv2016

Facet-dependent giant spin orbit torque in single crystalline antiferromagnetic Ir-Mn / ferromagnetic permalloy bilayers

There has been considerable interest in spin-orbit torques for the purpose of manipulating the magnetization of ferromagnetic (FM) films or nano-elements for spintronic technologies. Spin-orbit torques are derived from spin currents created from charge currents in materials with significant spin-orbit coupling that diffuse into an adjacent FM material. There have been intensive efforts to search for candidate materials that exhibit large spin Hall angles, i.e. efficient charge to spin current conversion. Here we report, using spin torque ferromagnetic resonance, the observation of a giant spin Hall angle of up to ~0.35 in (100) oriented single crystalline antiferromagnetic (AF) IrMn3 thin films, coupled to ferromagnetic permalloy layers, and a spin Hall angle that is about three times smaller in (111) oriented films. For the (100) oriented samples we show that the magnitude of the spin Hall angle can be significantly changed by manipulating the populations of the various AF domains through field annealing. Using ab-initio calculations we show that the triangular AF structure of IrMn3 gives rise to a substantial intrinsic spin Hall conductivity that is three times larger for the (100) than for the (111) orientations, consistent with our experimental findings.

cond-mat.mtrl-sci↗

arXiv2015

Role of transparency of platinum-ferromagnet interface in determining intrinsic magnitude of spin Hall effect

The spin Hall effect (SHE) converts charge current to pure spin currents in orthogonal directions in materials that have significant spin-orbit coupling.The efficiency of the conversion is described by the spin Hall Angle (SHA). The SHA can most readily be inferred by using the generated spin currents to excite or rotate the magnetization of ferromagnetic films or nano-elements via spin-transfer torques.Some of the largest spin torque derived spin Hall angles (ST-SHA) have been reported in platinum. Here we show, using spin torque ferromagnetic resonance (ST-FMR) measurements, that the transparency of the Pt-ferromagnet interface to the spin current plays a central role in determining the magnitude of the ST-SHA. We measure a much larger ST-SHA in Pt/cobalt (~0.11) compared to Pt/permalloy (~0.05) bilayers when the interfaces are assumed to be completely transparent. Taking into account the transparency of these interfaces, as derived from spin-mixing conductances, we find that the intrinsic SHA in platinum has a much higher value of 0.19 +- 0.04 as compared to the ST-SHA. The importance of the interface transparency is further exemplified by the insertion of atomically thin magnetic layers at the Pt/permalloy interface that we show strongly modulates the magnitude of the ST-SHA.

cond-mat.mes-hall↗