arXiv subjects

Xiao Zhang

Publications and source records attributed to Xiao Zhang.

At least 379 records · Page 21Linked to original sources

arXiv2022

Training Large-Vocabulary Neural Language Models by Private Federated Learning for Resource-Constrained Devices

Federated Learning (FL) is a technique to train models using data distributed across devices. Differential Privacy (DP) provides a formal privacy guarantee for sensitive data. Our goal is to train a large neural network language model (NNLM) on compute-constrained devices while preserving privacy using FL and DP. However, the DP-noise introduced to the model increases as the model size grows, which often prevents convergence. We propose Partial Embedding Updates (PEU), a novel technique to decrease noise by decreasing payload size. Furthermore, we adopt Low Rank Adaptation (LoRA) and Noise Contrastive Estimation (NCE) to reduce the memory demands of large models on compute-constrained devices. This combination of techniques makes it possible to train large-vocabulary language models while preserving accuracy and privacy.

cs.LG↗

arXiv2022

Interpretable Deep Learning: Interpretation, Interpretability, Trustworthiness, and Beyond

Deep neural networks have been well-known for their superb handling of various machine learning and artificial intelligence tasks. However, due to their over-parameterized black-box nature, it is often difficult to understand the prediction results of deep models. In recent years, many interpretation tools have been proposed to explain or reveal how deep models make decisions. In this paper, we review this line of research and try to make a comprehensive survey. Specifically, we first introduce and clarify two basic concepts -- interpretations and interpretability -- that people usually get confused about. To address the research efforts in interpretations, we elaborate the designs of a number of interpretation algorithms, from different perspectives, by proposing a new taxonomy. Then, to understand the interpretation results, we also survey the performance metrics for evaluating interpretation algorithms. Further, we summarize the current works in evaluating models' interpretability using "trustworthy" interpretation algorithms. Finally, we review and discuss the connections between deep models' interpretations and other factors, such as adversarial robustness and learning from interpretations, and we introduce several open-source libraries for interpretation algorithms and evaluation approaches.

cs.LG↗

arXiv2022

Real Non-Hermitian Energy Spectra Without Any Symmetry

Non-Hermitian models with real eigenenergies are highly desirable for their stability. Yet, most of the currently known ones are constrained by symmetries such as PT-symmetry, which is incompatible with realizing some of the most exotic non-Hermitian phenomena. In this work, we investigate how the non-Hermitian skin effect provides an alternative route towards enforcing real spectra and system stability. We showcase, for different classes of energy dispersions, various ansatz models that possess large parameter space regions with real spectra, despite not having any obvious symmetry. These minimal local models can be quickly implemented in non-reciprocal experimental setups such as electrical circuits with operational amplifiers.

cond-mat.mes-hall↗

arXiv2022

A Model-Agnostic Causal Learning Framework for Recommendation using Search Data

Machine-learning based recommender systems(RSs) has become an effective means to help people automatically discover their interests. Existing models often represent the rich information for recommendation, such as items, users, and contexts, as embedding vectors and leverage them to predict users' feedback. In the view of causal analysis, the associations between these embedding vectors and users' feedback are a mixture of the causal part that describes why an item is preferred by a user, and the non-causal part that merely reflects the statistical dependencies between users and items, for example, the exposure mechanism, public opinions, display position, etc. However, existing RSs mostly ignored the striking differences between the causal parts and non-causal parts when using these embedding vectors. In this paper, we propose a model-agnostic framework named IV4Rec that can effectively decompose the embedding vectors into these two parts, hence enhancing recommendation results. Specifically, we jointly consider users' behaviors in search scenarios and recommendation scenarios. Adopting the concepts in causal analysis, we embed users' search behaviors as instrumental variables (IVs), to help decompose original embedding vectors in recommendation, i.e., treatments. IV4Rec then combines the two parts through deep neural networks and uses the combined results for recommendation. IV4Rec is model-agnostic and can be applied to a number of existing RSs such as DIN and NRHUB. Experimental results on both public and proprietary industrial datasets demonstrate that IV4Rec consistently enhances RSs and outperforms a framework that jointly considers search and recommendation.

cs.IR↗

arXiv2022

Feature Forgetting in Continual Representation Learning

In continual and lifelong learning, good representation learning can help increase performance and reduce sample complexity when learning new tasks. There is evidence that representations do not suffer from "catastrophic forgetting" even in plain continual learning, but little further fact is known about its characteristics. In this paper, we aim to gain more understanding about representation learning in continual learning, especially on the feature forgetting problem. We devise a protocol for evaluating representation in continual learning, and then use it to present an overview of the basic trends of continual representation learning, showing its consistent deficiency and potential issues. To study the feature forgetting problem, we create a synthetic dataset to identify and visualize the prevalence of feature forgetting in neural networks. Finally, we propose a simple technique using gating adapters to mitigate feature forgetting. We conclude by discussing that improving representation learning benefits both old and new tasks in continual learning.

cs.LG↗

arXiv2022

GeV Gamma-ray Emission and Molecular Clouds towards Supernova Remnant G35.6$-$0.4 and the TeV Source HESS J1858+020

It is difficult to distinguish hadronic process from the leptonic one in $γ$-ray observation, which is however crucial in revealing the origin of cosmic rays. As an endeavor in the regard, we focus in this work on the complex $γ$-ray emitting region, which partially overlaps with the unidentified TeV source HESS~J1858+020 and includes supernova remnant (SNR) G35.6$-$0.4 and HII region G35.6$-$0.5. We reanalyze CO-line, HI, and Fermi-LAT GeV $γ$-ray emission data of this region. The analysis of the molecular and HI data suggests that SNR G35.6$-$0.4 and HII region G35.6$-$0.5 are located at different distances. The analysis the GeV $γ$-rays shows that GeV emission arises from two point sources: one (SrcA) coincident with the SNR, and the other (SrcB) coincident with both HESS J1858+020 and HII region G35.6$-$0.5. The GeV emission of SrcA can be explained by the hadronic process in the SNR-MC association scenario. The GeV-band spectrum of SrcB and the TeV-band spectrum of HESS J1858+020 can be smoothly connected by a power-law function, with an index of $\sim$2.2. The connected spectrum is well explained with a hadronic emission, with the cutoff energy of protons above 1 PeV. It thus indicates that there is a potential PeVatron in the HII region and should be further verified with ultra-high energy observations with, e.g., LHAASO.

astro-ph.HE↗

arXiv2022

Cross-Lingual Phrase Retrieval

Cross-lingual retrieval aims to retrieve relevant text across languages. Current methods typically achieve cross-lingual retrieval by learning language-agnostic text representations in word or sentence level. However, how to learn phrase representations for cross-lingual phrase retrieval is still an open problem. In this paper, we propose XPR, a cross-lingual phrase retriever that extracts phrase representations from unlabeled example sentences. Moreover, we create a large-scale cross-lingual phrase retrieval dataset, which contains 65K bilingual phrase pairs and 4.2M example sentences in 8 English-centric language pairs. Experimental results show that XPR outperforms state-of-the-art baselines which utilize word-level or sentence-level representations. XPR also shows impressive zero-shot transferability that enables the model to perform retrieval in an unseen language pair during training. Our dataset, code, and trained models are publicly available at www.github.com/cwszz/XPR/.

cs.CL↗

arXiv2022

Simultaneously Enhanced Tenacity, Rupture Work, and Thermal Conductivity of Carbon Nanotubes Fibers by Increasing the Effective Tube Contribution

Although individual carbon nanotubes (CNTs) are superior as constituents to polymer chains, the mechanical and thermal properties of CNT fibers (CNTFs) remain inferior to commercial synthetic fibers due to the lack of synthesis methods to embed CNTs effectively in superstructures. The application of conventional techniques for mechanical enhancement resulted in a mild improvement of target properties while achieving parity at best on others. In this work, a Double-Drawing technique is developed to deform continuously grown CNTFs and rearrange the constituent CNTs in both mesoscale and nanoscale morphology. Consequently, the mechanical and thermal properties of the resulting CNTFs can be jointly improved, and simultaneously reach their highest performances with specific strength (tenacity) $\rm\sim3.30\,N\,tex^{-1}$, work of rupture $\rm\sim70\,J\,g^{-1}$, and thermal conductivity $\rm\sim354\,W\,m^{-1}\,K^{-1}$, despite starting from commercial low-crystallinity materials ($I{\rm_G}:I{\rm_D}\sim5$). The processed CNTFs are more versatile than comparable carbon fiber, Zylon, Dyneema, and Kevlar. Furthermore, based on evidence of load transfer efficiency on individual CNTs measured with In-Situ Stretching Raman, we find the main contributors to property enhancements are (1) the increased proportion of load-bearing CNT bundles and (2) the extension of effective length of tubes attached on these bundles.

cond-mat.mtrl-sci↗

arXiv2022

A Possible Gamma-Ray Enhancement Event in Tycho's Supernova Remnant

We report a possible $γ$-ray enhancement event detected from Tycho's supernova remnant (SNR), the outcome of a type Ia supernova explosion that occurred in year 1572. The event lasted for 1.5 years and showed a factor of 3.6 flux increase mainly in the energy range of 4--100 GeV, while notably accompanied with two 478\,GeV photons. Several young SNRs (including Tycho's SNR) were previously found to show peculiar X-ray structures with flux variations in one- or several-year timescales, such an event at $γ$-ray energies is for the first time seen. The year-long timescale of the event suggests a synchrotron radiation process, but the hard $γ$-ray emission requires extreme conditions of either ultra-high energies for the electrons upto $\sim$10 PeV (well above the cosmic-ray "knee" energy) or high inhomogeneity of the magnetic field in the SNR. This event in Tycho's SNR is likely analogous to the $γ$-ray flares observed in the Crab nebula, the comparably short timescales of them both requiring a synchrotron process, and similar magnetohydrodynamic processes such as magnetic reconnection would be at work as well in the SNR to accelerate particles to ultra-relativistic energies. The event, if confirmed, helps reveal the more complicated side of the physical processes that can occur in young SNRs.

astro-ph.HE↗

arXiv2022

Understanding Intrinsic Robustness Using Label Uncertainty

A fundamental question in adversarial machine learning is whether a robust classifier exists for a given task. A line of research has made some progress towards this goal by studying the concentration of measure, but we argue standard concentration fails to fully characterize the intrinsic robustness of a classification problem since it ignores data labels which are essential to any classification task. Building on a novel definition of label uncertainty, we empirically demonstrate that error regions induced by state-of-the-art models tend to have much higher label uncertainty than randomly-selected subsets. This observation motivates us to adapt a concentration estimation algorithm to account for label uncertainty, resulting in more accurate intrinsic robustness measures for benchmark image classification problems.

cs.LG↗

arXiv2022

High-harmonic generation approaching the quantum critical point of strongly correlated systems

By employing the exact diagonalization method, we investigate the high-harmonic generation (HHG) of the correlated systems under the strong laser irradiation. For the extended Hubbard model on a periodic chain, HHG close to the quantum critical point (QCP) is more significant compared to two neighboring gapped phases (i.e., charge-density-wave and spin-density wave states), especially in low-frequencies. We confirm that the systems in the vicinity of the QCP are supersensitive to the external field and more optical-transition channels via excited states are responsible for HHG. This feature holds the potential of obtaining high-efficiency harmonics by making use of materials approaching to QCP. Based on two-dimensional Haldane model, we further propose that the even- or odd-order components of generated harmonics can be promisingly regarded as spectral signals to distinguish the topologically ordered phases from locally ordered ones. Our findings in this work pave the way to achieve ultrafast light source from HHG in strongly correlated materials and to study quantum phase transition by nonlinear optics in strong laser fields.

cond-mat.str-el↗

arXiv2022

A Remark on Evolution Equation of Stochastic Logical Dynamic Systems

Modelling is an essential procedure in analyzing and controlling a given logical dynamic system (LDS). It has been proved that deterministic LDS can be modeled as a linear-like system using algebraic state space representation. However, due to the inherently non-linear, it is difficult to obtain the algebraic expression of a stochastic LDS. This paper provides a unified framework for transition analysis of LDSs with deterministic and stochastic dynamics. First, modelling of LDS with deterministic dynamics is reviewed. Then modeling of LDS with stochastic dynamics is considered, and non-equivalence between subsystems and global system is proposed. Next, the reason for the non-equivalence is provided. Finally, consistency condition is presented for independent model and conditional independent model.

math.OC↗

arXiv2022

The Large High Altitude Air Shower Observatory (LHAASO) Science Book (2021 Edition)

Since the science white paper of the Large High Altitude Air Shower Observatory (LHAASO) published on arXiv in 2019 [e-Print: 1905.02773 (astro-ph.HE)], LHAASO has completed the transition from a project to an operational gamma-ray astronomical observatory LHAASO is a new generation multi-component facility located in Daocheng, Sichuan province of China, at an altitude of 4410 meters. It aims at measuring with unprecedented sensitivity the spectrum, composition, and anisotropy of cosmic rays in the energy range between 10$^{12}$ and 10$^{18}$~eV, and acting simultaneously as a wide aperture (one stereoradiant) continuously operating gamma-ray telescope in the energy range between 10$^{11}$ and $10^{15}$~eV with the designed sensitivity of 1.3\% of the Crab Unit (CU) above 100 TeV. LHAASO's capability of measuring simultaneously different shower components (electrons, muons, and Cherenkov/fluorescence light), will allow it to investigate the origin, acceleration, and propagation of CR through measurement of the energy spectrum, elemental composition, and anisotropy with unprecedented resolution. The remarkable sensitivity of LHAASO will play a key role in CR physics and gamma-ray astronomy for a general and comprehensive exploration of the high energy universe and will allow important studies of fundamental physics (such as indirect dark matter search, Lorentz invariance violation, quantum gravity) and solar and heliospheric physics. The LHAASO Collaboration organized an editorial working group and finished all editorial work of this science book, to summarize the instrumental features and outline the prospects of scientific researches with the LHAASO experiment.

astro-ph.HE↗

arXiv2022

Beam pointing stabilization of an acousto-optic modulator with thermal control

Diffraction beams generated by an acousto-optic modulator (AOM) are widely used in various optical experiments, some of which require high angular stability with the temporal modulation of optical power. Usually, it is difficult to realize both angular stability and high-power modulation in a passive setup without a servo system of radio-frequency compensation. Here, we present a method to suppress the angular drift and pointing noise only with the thermal management of the AOM crystal. We analyze the dependence of the angular drift on the refractive index variation, and find that the angular drift is very sensitivity to the temperature gradient which could induce the refractive index gradient inside the AOM crystal. It reminds us such angular drift could be significantly suppressed by carefully overlapping the zero temperature gradient area with the position of the acousto-optic interaction zone. We implement a water-cooling setup, and find that the angular drift of an AOM is reduced over 100 times during the thermal transient, and the angular noise is also suppressed to 1/3 of the non-cooled case. It should be emphasized that this thermal control method is a general to suppress the beam drift in both the diffraction and the perpendicular-to-diffraction directions. The refractive index thermal coefficient of tellurium dioxide crystal at 1064 nm determined by this angular drift-temperature model is 16$\times$10$^{-6}$ K$^{-1}$ consistent with previous studies. This thermal control technique provides potential applications for optical trapping and remote sensoring that demand for intensity ramps.

physics.ins-det↗

arXiv2022

ASM2TV: An Adaptive Semi-Supervised Multi-Task Multi-View Learning Framework for Human Activity Recognition

Many real-world scenarios, such as human activity recognition (HAR) in IoT, can be formalized as a multi-task multi-view learning problem. Each specific task consists of multiple shared feature views collected from multiple sources, either homogeneous or heterogeneous. Common among recent approaches is to employ a typical hard/soft sharing strategy at the initial phase separately for each view across tasks to uncover common knowledge, underlying the assumption that all views are conditionally independent. On the one hand, multiple views across tasks possibly relate to each other under practical situations. On the other hand, supervised methods might be insufficient when labeled data is scarce. To tackle these challenges, we introduce a novel framework ASM2TV for semi-supervised multi-task multi-view learning. We present a new perspective named gating control policy, a learnable task-view-interacted sharing policy that adaptively selects the most desirable candidate shared block for any view across any task, which uncovers more fine-grained task-view-interacted relatedness and improves inference efficiency. Significantly, our proposed gathering consistency adaption procedure takes full advantage of large amounts of unlabeled fragmented time-series, making it a general framework that accommodates a wide range of applications. Experiments on two diverse real-world HAR benchmark datasets collected from various subjects and sources demonstrate our framework's superiority over other state-of-the-arts. The detailed codes are available at https://github.com/zachstarkk/ASM2TV.

cs.LG↗

arXiv2022

Nodal lines in momentum space: topological invariants and recent realizations in photonic and other systems

Topological insulators constitute one of the most intriguing phenomena in modern condensed matter theory. The unique and exotic properties of topological states of matter allow for unidirectional gapless electron transport and extremely accurate measurements of the Hall conductivity. Recently, new topological effects occurring at Dirac/Weyl points have been better understood and demonstrated using artificial materials such as photonic and phononic crystals, metamaterials and electrical circuits. In comparison, the topological properties of nodal lines, which are one-dimensional degeneracies in momentum space, remain less explored. Here, we explain the theoretical concept of topological nodal lines and review recent and ongoing progress using artificial materials. The review includes recent demonstrations of non-Abelian topological charges of nodal lines in momentum space and examples of nodal lines realized in photonic and other systems. Finally, we will address the challenges involved in both experimental demonstration and theoretical understanding of topological nodal lines.

cond-mat.mtrl-sci↗

arXiv2022

Learning Graph Structures with Transformer for Multivariate Time Series Anomaly Detection in IoT

Many real-world IoT systems, which include a variety of internet-connected sensory devices, produce substantial amounts of multivariate time series data. Meanwhile, vital IoT infrastructures like smart power grids and water distribution networks are frequently targeted by cyber-attacks, making anomaly detection an important study topic. Modeling such relatedness is, nevertheless, unavoidable for any efficient and effective anomaly detection system, given the intricate topological and nonlinear connections that are originally unknown among sensors. Furthermore, detecting anomalies in multivariate time series is difficult due to their temporal dependency and stochasticity. This paper presented GTA, a new framework for multivariate time series anomaly detection that involves automatically learning a graph structure, graph convolution, and modeling temporal dependency using a Transformer-based architecture. The connection learning policy, which is based on the Gumbel-softmax sampling approach to learn bi-directed links among sensors directly, is at the heart of learning graph structure. To describe the anomaly information flow between network nodes, we introduced a new graph convolution called Influence Propagation convolution. In addition, to tackle the quadratic complexity barrier, we suggested a multi-branch attention mechanism to replace the original multi-head self-attention method. Extensive experiments on four publicly available anomaly detection benchmarks further demonstrate the superiority of our approach over alternative state-of-the-arts. Codes are available at https://github.com/ZEKAICHEN/GTA.

cs.LG↗

arXiv2022

Combining Reinforcement Learning and Inverse Reinforcement Learning for Asset Allocation Recommendations

We suggest a simple practical method to combine the human and artificial intelligence to both learn best investment practices of fund managers, and provide recommendations to improve them. Our approach is based on a combination of Inverse Reinforcement Learning (IRL) and RL. First, the IRL component learns the intent of fund managers as suggested by their trading history, and recovers their implied reward function. At the second step, this reward function is used by a direct RL algorithm to optimize asset allocation decisions. We show that our method is able to improve over the performance of individual fund managers.

cs.LG↗