arXiv subjects

Shen Zhang

Publications and source records attributed to Shen Zhang.

At least 55 records · Page 3Linked to original sources

arXiv2023

Joint Token Pruning and Squeezing Towards More Aggressive Compression of Vision Transformers

Although vision transformers (ViTs) have shown promising results in various computer vision tasks recently, their high computational cost limits their practical applications. Previous approaches that prune redundant tokens have demonstrated a good trade-off between performance and computation costs. Nevertheless, errors caused by pruning strategies can lead to significant information loss. Our quantitative experiments reveal that the impact of pruned tokens on performance should be noticeable. To address this issue, we propose a novel joint Token Pruning & Squeezing module (TPS) for compressing vision transformers with higher efficiency. Firstly, TPS adopts pruning to get the reserved and pruned subsets. Secondly, TPS squeezes the information of pruned tokens into partial reserved tokens via the unidirectional nearest-neighbor matching and similarity-based fusing steps. Compared to state-of-the-art methods, our approach outperforms them under all token pruning intensities. Especially while shrinking DeiT-tiny&small computational budgets to 35%, it improves the accuracy by 1%-6% compared with baselines on ImageNet classification. The proposed method can accelerate the throughput of DeiT-small beyond DeiT-tiny, while its accuracy surpasses DeiT-tiny by 4.78%. Experiments on various transformers demonstrate the effectiveness of our method, while analysis experiments prove our higher robustness to the errors of the token pruning policy. Code is available at https://github.com/megvii-research/TPS-CVPR2023.

cs.CV↗

arXiv2023

Inverse problems for mean field games

The theory of mean field games studies the limiting behaviors of large systems where the agents interact with each other in a certain symmetric way. The running and terminal costs are critical for the agents to decide the strategies. However, in practice they are often partially known or totally unknown for the agents, while the total cost is known at the end of the game. To address this challenging issue, we propose and study several inverse problems for mean field games. When the Lagrangian is a kinetic energy, we first establish unique identifiability results, showing that one can recover either the running cost or the terminal cost from knowledge of the total cost. If the running cost is limited to the time-independent class, we can further prove that one can simultaneously recover both the running and the terminal costs. Finally, we extend the results to the setup with general Lagrangians.

math.OC↗

arXiv2023

Magnetoresistance signature of two-dimensional electronic states in Co$_3$Sn$_2$S$_2$

Two-dimensional (2D) Dirac bands and flat bands are characteristics of a kagome lattice. However, experimental studies on their electrical transport are few, because three-dimensional (3D) bulk bands of kagome materials, consisting of stacked 2D kagome layers, dominate the transport. We report a magnetoresistance (MR) study of a kagome material, Co$_3$Sn$_2$S$_2$. Based on analysis of the temperature, magnetic field, and field angle dependence of the resistivity, we obtain a complete anatomy of MR. Besides a magnon MR, a chirality-dependent MR, and a chiral-anomaly-induced MR, the most intriguing feature is an orbital MR that scales only with the out-of-plane field, which strongly indicates its 2D nature. We attribute it to the Dirac band of the kagome lattice.

cond-mat.mes-hall↗

arXiv2023

Machine Learning for the Control and Monitoring of Electric Machine Drives: Advances and Trends

This review paper systematically summarizes the existing literature on utilizing machine learning (ML) techniques for the control and monitoring of electric machine drives. It is anticipated that with the rapid progress in learning algorithms and specialized embedded hardware platforms, machine learning-based data-driven approaches will become standard tools for the automated high-performance control and monitoring of electric drives. Additionally, this paper also provides some outlook toward promoting its widespread application in the industry with a focus on deploying ML algorithms onto embedded system-on-chip (SoC) field-programmable gate array (FPGA) devices.

eess.SY↗

arXiv2022

On an inverse boundary problem for mean field games

In this paper, we propose and study an inverse boundary problem for the mean field games (MFGs) governed by the first-order master equation in a bounded domain. We establish the unique identifiability result by showing that the running cost function within any given proper (state) space-time subdomain is uniquely determined by the MFG boundary data of this subdomain. There are several salient features of our study. The MFG system couples two nonlinear parabolic PDEs with one forward in time and the other one backward in time, and moreover there is a probability measure constraint. These make the inverse problem study new to the literature and highly challenging. We develop an effective and efficient scheme in tackling the inverse problem including the high-order variation in the probability space around a nontrivial distribution, and the construction of a novel class of CGO solutions to serve as the ``probing modes". Our study opens up a new field of research on inverse problems for MFGs.

math.AP↗

arXiv2022

A simple consistent Bayes factor for testing the Kendall rank correlation coefficient

In this paper, we propose a simple and easy-to-implement Bayesian hypothesis test for the presence of an association, described by Kendall's τcoefficient, between two variables measured on at least an ordinal scale. Owing to the absence of the likelihood functions for the data, we employ the asymptotic sampling distributions of the test statistic as the working likelihoods and then specify a truncated normal prior distribution on the noncentrality parameter of the alternative hypothesis, which results in the Bayes factor available in closed form in terms of the cumulative distribution function of the standard normal distribution. Investigating the asymptotic behavior of the Bayes factor we find the conditions of the priors so that it is consistent to whichever the hypothesis is true. Simulation studies and a real-data application are used to illustrate the effectiveness of the proposed Bayes factor. It deserves mentioning that the proposed method can be easily covered in undergraduate and graduate courses in nonparametric statistics with an emphasis on students' Bayesian thinking for data analysis.

stat.ME↗

arXiv2022

Magnetic-field modulation of topological electronic states and emergent magneto-transport in a magnetic Weyl semimetal

The modulation of topological electronic states by an external magnetic field is highly desired for condensed matter physics. Schemes to achieve this have been proposed theoretically, but few can be realized experimentally. Here, combining transverse transport, theoretical calculations, and scanning tunneling microscopy/spectroscopy (STM/S) investigations, we provide an observation that the topological electronic states, accompanied by an emergent magneto-transport phenomenon, were modulated by applying magnetic fields through induced non-collinear magnetism in the magnetic Weyl semimetal EuB6. A giant unconventional anomalous Hall effect (UAHE) is found during the magnetisation re-orientation from easy axis to hard ones in magnetic fields, with a UAHE peak around the low field of 5 kOe. Under the reasonable spin-canting effect, the folding of the topological anti-crossing bands occurs, generating a strong Berry curvature that accounts for the observed UAHE. Field-dependent STM/S reveals a highly synchronous evolution of electronic density of states, with a dI/dV peak around the same field of 5 kOe, which provides evidence to the folded bands and excited UAHE by external magnetic fields. This finding elucidates the connection between the real-space non-collinear magnetism and the k-space topological electronic states and establishes a novel manner to engineer the magneto-transport behaviours of correlated electrons for future topological spintronics.

cond-mat.mtrl-sci↗

arXiv2022

Scaling of Berry-curvature monopole dominated large linear positive magnetoresistance

The linear positive magnetoresistance (LPMR) is a widely observed phenomenon in topological materials, which is promising for potential applications on topological spintronics. However, its mechanism remains ambiguous yet and the effect is thus uncontrollable. Here, we report a quantitative scaling model that correlates the LPMR with the Berry curvature, based on a ferromagnetic Weyl semimetal CoS2 that bears the largest LPMR of over 500% at 2 Kelvin and 9 Tesla, among known magnetic topological semimetals. In this system, masses of Weyl nodes existing near the Fermi level, revealed by theoretical calculations, serve as Berry-curvature monopoles and low-effective-mass carriers. Based on the Weyl picture, we propose a relation \[\text{MR}=\frac{e}{\hbar }B{{Ω}_{\text{F}}}\], with B being the applied magnetic field and \[{{Ω}_{\text{F}}}\] the average Berry curvature near the Fermi surface, and further introduce temperature factor to both MR/B slope (MR per unit field) and anomalous Hall conductivity, which establishes the connection between the model and experimental measurements. A clear picture of the linearly slowing down of carriers, i.e., the LPMR effect, is demonstrated under the cooperation of the k-space Berry curvature and real-space magnetic field. Our study not only provides an experimental evidence of Berry curvature induced LPMR for the first time, but also promotes the common understanding and functional designing of the large Berry-curvature MR in topological Dirac/Weyl systems for magnetic sensing or information storage.

cond-mat.mtrl-sci↗

arXiv2022

Three-step Formation of Diamonds in Shock-compressed Hydrocarbons: Decomposition, Species Separation, and Nucleation

The accumulation and circulation of carbon-hydrogen dictate the chemical evolution of ice giant planets. Species separation and diamond precipitation have been reported in carbon-hydrogen systems, verified by static and shock-compression experiments. Nevertheless, the dynamic formation processes for the above-mentioned phenomena are still insufficiently understood. Here, combing deep learning model, we demonstrate that diamonds form through a three-step process involving decomposition, species separation and nucleation procedures. Under shock condition of 125 GPa and 4590 K, hydrocarbons are decomposed to give hydrogen and low-molecular-weight alkanes (CH4 and C2H6), which escape from the carbon chains resulting in C/H species separation. The remaining carbon atoms without C-H bonds accumulate and nucleate to form diamond crystals. The process of diamond growth is found to associated with a critical nucleus size where dynamic energy barrier plays a key role. These dynamic processes for diamonds formation are insightful in establishing the model for ice giant planet evolution.

astro-ph.EP↗

arXiv2022

Antisymmetric Seebeck effect in a tilted Weyl semimetal

Tilting the Weyl cone breaks the Lorentz invariance and enriches the Weyl physics. Here, we report the observation of a magnetic-field-antisymmetric Seebeck effect in a tilted Weyl semimetal, Co$_3$Sn$_2$S$_2$. Moreover, it is found that the Seebeck effect and the Nernst effect are antisymmetric in both the in-plane magnetic field and the magnetization. We attribute these exotic effects to the one-dimensional chiral anomaly and phase space correction due to the Berry curvature. The observation is further reproduced by a theoretical calculation, taking into account the orbital magnetization.

cond-mat.mes-hall↗

arXiv2022

Towards Large-Scale and Spatio-temporally Resolved Diagnosis of Electronic Density of States by Deep Learning

Modern laboratory techniques like ultrafast laser excitation and shock compression can bring matter into highly nonequilibrium states with complex structural transformation, metallization and dissociation dynamics. To understand and model the dramatic change of both electronic structures and ion dynamics during such dynamic processes, the traditional method faces difficulties. Here, we demonstrate the ability of deep neural network (DNN) to capture the atomic local-environment dependence of electronic density of states (DOS) for both multicomponent system under exoplanet thermodynamic condition and nonequilibrium system during super-heated melting process. Large scale and time-resolved diagnosis of DOS can be efficiently achieved within the accuracy of ab initio method. Moreover, the atomic contribution to DOS given by DNN model accurately reveals the information of local neighborhood for selected atom, thus can serve as robust order parameters to identify different phases and intermediate local structures, strongly highlights the efficacy of this DNN model in studying dynamic processes.

physics.comp-ph↗

arXiv2022

Deep Learning in Human Activity Recognition with Wearable Sensors: A Review on Advances

Mobile and wearable devices have enabled numerous applications, including activity tracking, wellness monitoring, and human--computer interaction, that measure and improve our daily lives. Many of these applications are made possible by leveraging the rich collection of low-power sensors found in many mobile and wearable devices to perform human activity recognition (HAR). Recently, deep learning has greatly pushed the boundaries of HAR on mobile and wearable devices. This paper systematically categorizes and summarizes existing work that introduces deep learning methods for wearables-based HAR and provides a comprehensive analysis of the current advancements, developing trends, and major challenges. We also present cutting-edge frontiers and future directions for deep learning-based HAR.

cs.HC↗

arXiv2021

A comparative study on machine learning models combining with outlier detection and balanced sampling methods for credit scoring

Peer-to-peer (P2P) lending platforms have grown rapidly over the past decade as the network infrastructure has improved and the demand for personal lending has grown. Such platforms allow users to create peer-to-peer lending relationships without the help of traditional financial institutions. Assessing the borrowers' credit is crucial to reduce the default rate and benign development of P2P platforms. Building a personal credit scoring machine learning model can effectively predict whether users will repay loans on the P2P platform. And the handling of data outliers and sample imbalance problems can affect the final effect of machine learning models. There have been some studies on balanced sampling methods, but the effect of outlier detection methods and their combination with balanced sampling methods on the effectiveness of machine learning models has not been fully studied. In this paper, the influence of using different outlier detection methods and balanced sampling methods on commonly used machine learning models is investigated. Experiments on 44,487 Lending Club samples show that proper outlier detection can improve the effectiveness of the machine learning model, and the balanced sampling method only has a good effect on a few machine learning models, such as MLP.

cs.LG↗

arXiv2021

Simultaneous recoveries for semilinear parabolic systems

In this paper, we study inverse boundary problems associated with semilinear parabolic systems in several scenarios where both the nonlinearities and the initial data can be unknown. We establish several simultaneous recovery results showing that the passive or active boundary Dirichlet-to-Neumann operators can uniquely recover both of the unknowns, even stably in a certain case. It turns out that the nonlinearities play a critical role in deriving these recovery results. If the nonlinear term belongs to a general $C^1$ class but fulfilling a certain growth condition, the recovery results are established by the control approach via Carleman estimates. If the nonlinear term belongs to an analytic class, the recovery results are established through successive linearization in combination with special CGO (Complex Geometrical Optics) solutions for the parabolic system.

math.AP↗

arXiv2021

Continuous-time Gaussian Process Trajectory Generation for Multi-robot Formation via Probabilistic Inference

In this paper, we extend a famous motion planning approach GPMP2 to multi-robot cases, yielding a novel centralized trajectory generation method for the multi-robot formation. A sparse Gaussian Process model is employed to represent the continuous-time trajectories of all robots as a limited number of states, which improves computational efficiency due to the sparsity. We add constraints to guarantee collision avoidance between individuals as well as formation maintenance, then all constraints and kinematics are formulated on a factor graph. By introducing a global planner, our proposed method can generate trajectories efficiently for a team of robots which have to get through a width-varying area by adaptive formation change. Finally, we provide the implementation of an incremental replanning algorithm to demonstrate the online operation potential of our proposed framework. The experiments in simulation and real world illustrate the feasibility, efficiency and scalability of our approach.

cs.RO↗

arXiv2021

Atomistic Mechanism of Phase Transition in Shock Compressed Gold Revealed by Deep Potential

A detailed understanding of the material response to rapid compression is challenging and demanding. For instance, the element gold under dynamic compression exhibits complex phase transformations where there exist some large discrepancies between experimental and theoretical studies. Here, we combined large-scale molecular dynamics simulations with a deep potential to elucidate the dynamic compression processes of gold from an atomic level. The potential is constructed by accurately reproducing the free energy surfaces of density-functional-theory calculations for gold, from ambient conditions to 15 500 K and 500 GPa. Within this framework, we extend the simulations up to 200 000 atoms size, and found a much lower pressure threshold for phase transitioning from face-centered cubic (FCC) to body-centered (BCC), as compared to previous calculations. Furthermore, the transition pressure is strongly dependent on the shock direction, namely 159 GPa for (100) orientation and 219 GPa for (110) orientation, respectively. Most importantly, the accurate atomistic perspective presents that the shocked BCC structure contains unique features of medium-range and short-range orders, which is named disorders here. We propose a model and demonstrate that the existence of disorders significantly reduces the Gibbs free energies of shocked structures, therefore leading to the lowering of the phase transition pressure. The present study provides a new path to understand the structure dynamics under extreme conditions.

cond-mat.mtrl-sci↗

arXiv2021

Few-Shot Bearing Fault Diagnosis Based on Model-Agnostic Meta-Learning

The rapid development of artificial intelligence and deep learning has provided many opportunities to further enhance the safety, stability, and accuracy of industrial Cyber-Physical Systems (CPS). As indispensable components to many mission-critical CPS assets and equipment, mechanical bearings need to be monitored to identify any trace of abnormal conditions. Most of the data-driven approaches applied to bearing fault diagnosis up-to-date are trained using a large amount of fault data collected a priori. In many practical applications, however, it can be unsafe and time-consuming to collect sufficient data samples for each fault category, making it challenging to train a robust classifier. In this paper, we propose a few-shot learning framework for bearing fault diagnosis based on model-agnostic meta-learning (MAML), which targets for training an effective fault classifier using limited data. In addition, it can leverage the training data and learn to identify new fault scenarios more efficiently. Case studies on the generalization to new artificial faults show that the proposed framework achieves an overall accuracy up to 25% higher than a Siamese network-based benchmark study. Finally, the robustness and the generalization capability of the proposed framework are further validated by applying it to identify real bearing damages using data from artificial damages, which compares favorably against 6 state-of-the-art few-shot learning algorithms using consistent test environments.

cs.LG↗

arXiv2021

Rotor Thermal Monitoring Scheme for Direct-Torque-Controlled Interior Permanent Magnet Synchronous Machines via High-Frequency Rotating Flux or Torque Injection

Interior permanent magnet synchronous machine drives are widely employed in electric traction systems and various industrial processes. However, prolonged exposure to high temperatures while operating can demagnetize the permanent magnets to the point of irreversible demagnetization. In addition, direct measurements with infrared sensors or contact-type sensors with wireless communication can be expensive and intrusive to the motor drive systems. This paper thus proposes a nonintrusive thermal monitoring scheme for the permanent magnets inside the direct-torque-controlled interior permanent magnet synchronous machines. By applying an external high-frequency rotating flux or torque signal to the hysteresis torque controller in the motor drive, the high-frequency currents can be injected into the stator windings. The permanent magnet temperature can thus be monitored based on the induced high-frequency resistance. The nonintrusive nature of the method is indicated by the elimination of the extra sensors and no hardware change to the existing system. Finally, the effectiveness of the proposed method is validated with experimental results.

eess.SY↗