arXiv subjects

Shi Pu

Publications and source records attributed to Shi Pu.

At least 109 records · Page 6Linked to original sources

arXiv2020

Distributed Stochastic Gradient Tracking Methods

In this paper, we study the problem of distributed multi-agent optimization over a network, where each agent possesses a local cost function that is smooth and strongly convex. The global objective is to find a common solution that minimizes the average of all cost functions. Assuming agents only have access to unbiased estimates of the gradients of their local cost functions, we consider a distributed stochastic gradient tracking method (DSGT) and a gossip-like stochastic gradient tracking method (GSGT). We show that, in expectation, the iterates generated by each agent are attracted to a neighborhood of the optimal solution, where they accumulate exponentially fast (under a constant stepsize choice). Under DSGT, the limiting (expected) error bounds on the distance of the iterates from the optimal solution decrease with the network size $n$, which is a comparable performance to a centralized stochastic gradient algorithm. Moreover, we show that when the network is well-connected, GSGT incurs lower communication cost than DSGT while maintaining a similar computational cost. Numerical example further demonstrates the effectiveness of the proposed methods.

math.OC↗

arXiv2020

Asymptotic Network Independence in Distributed Stochastic Optimization for Machine Learning

We provide a discussion of several recent results which, in certain scenarios, are able to overcome a barrier in distributed stochastic optimization for machine learning. Our focus is the so-called asymptotic network independence property, which is achieved whenever a distributed method executed over a network of n nodes asymptotically converges to the optimal solution at a comparable rate to a centralized method with the same computational power as the entire network. We explain this property through an example involving the training of ML models and sketch a short mathematical analysis for comparing the performance of distributed stochastic gradient descent (DSGD) with centralized stochastic gradient decent (SGD).

math.OC↗

arXiv2020

Push-Pull Gradient Methods for Distributed Optimization in Networks

In this paper, we focus on solving a distributed convex optimization problem in a network, where each agent has its own convex cost function and the goal is to minimize the sum of the agents' cost functions while obeying the network connectivity structure. In order to minimize the sum of the cost functions, we consider new distributed gradient-based methods where each node maintains two estimates, namely, an estimate of the optimal decision variable and an estimate of the gradient for the average of the agents' objective functions. From the viewpoint of an agent, the information about the gradients is pushed to the neighbors, while the information about the decision variable is pulled from the neighbors hence giving the name "push-pull gradient methods". The methods utilize two different graphs for the information exchange among agents, and as such, unify the algorithms with different types of distributed architecture, including decentralized (peer-to-peer), centralized (master-slave), and semi-centralized (leader-follower) architecture. We show that the proposed algorithms and their many variants converge linearly for strongly convex and smooth objective functions over a network (possibly with unidirectional data links) in both synchronous and asynchronous random-gossip settings. In particular, under the random-gossip setting, "push-pull" is the first class of algorithms for distributed optimization over directed graphs. Moreover, we numerically evaluate our proposed algorithms in both scenarios, and show that they outperform other existing linearly convergent schemes, especially for ill-conditioned problems and networks that are not well balanced.

math.OC↗

arXiv2019

Towards a full solution of relativistic Boltzmann equation for quark-gluon matter on GPUs

We have developed a numerical framework for a full solution of the relativistic Boltzmann equations for the quark-gluon matter using the multiple Graphics Processing Units (GPUs) on distributed clusters. Including all the $2 \to 2$ scattering processes of 3-flavor quarks and gluons, we compute the time evolution of distribution functions in both coordinate and momentum spaces for the cases of pure gluons, quarks and the mixture of quarks and gluons. By introducing a symmetrical sampling method on GPUs which ensures the particle number conservation, our framework is able to perform the space-time evolution of quark-gluon system towards thermal equilibrium with high performance. We also observe that the gluons naturally accumulate in the soft region at the early time, which may indicate the gluon condensation.

hep-ph↗

arXiv2019

A Push-Pull Gradient Method for Distributed Optimization in Networks

In this paper, we focus on solving a distributed convex optimization problem in a network, where each agent has its own convex cost function and the goal is to minimize the sum of the agents' cost functions while obeying the network connectivity structure. In order to minimize the sum of the cost functions, we consider a new distributed gradient-based method where each node maintains two estimates, namely, an estimate of the optimal decision variable and an estimate of the gradient for the average of the agents' objective functions. From the viewpoint of an agent, the information about the decision variable is pushed to the neighbors, while the information about the gradients is pulled from the neighbors (hence giving the name "push-pull gradient method"). The method unifies the algorithms with different types of distributed architecture, including decentralized (peer-to-peer), centralized (master-slave), and semi-centralized (leader-follower) architecture. We show that the algorithm converges linearly for strongly convex and smooth objective functions over a directed static network. In our numerical test, the algorithm performs well even for time-varying directed networks.

math.OC↗

arXiv2019

A Distributed Stochastic Gradient Tracking Method

In this paper, we study the problem of distributed multi-agent optimization over a network, where each agent possesses a local cost function that is smooth and strongly convex. The global objective is to find a common solution that minimizes the average of all cost functions. Assuming agents only have access to unbiased estimates of the gradients of their local cost functions, we consider a distributed stochastic gradient tracking method. We show that, in expectation, the iterates generated by each agent are attracted to a neighborhood of the optimal solution, where they accumulate exponentially fast (under a constant step size choice). More importantly, the limiting (expected) error bounds on the distance of the iterates from the optimal solution decrease with the network size, which is a comparable performance to a centralized stochastic gradient algorithm. Numerical examples further demonstrate the effectiveness of the method.

math.OC↗

arXiv2019

Eddy magnetization from the chiral Barnett effect

We discuss the spin, the angular momentum, and the magnetic moment of rotating chiral fermions using a kinetic theory. We find that, in addition to the chiral vortical contribution along the rotation axis, finite circular spin polarization is induced by the spin-momentum correlation of chiral fermions, which is canceled by a change in the orbital angular momentum. We point out that the eddy magnetic moment is nonvanishing due to the $g$-factors, exhibiting the chiral Barnett effect.

hep-ph↗

arXiv2019

Anomalous magnetohydrodynamics with longitudinal boost invariance and chiral magnetic effect

We study relativistic magnetohydrodynamics with longitudinal boost invariance in the presence of chiral magnetic effects and finite electric conductivity. With initial magnetic fields parallel or anti-parallel to electric fields, we derive the analytic solutions of electromagnetic fields and the chiral number and energy density in an expansion of several parameters determined by initial conditions. The numerical solutions show that such analytic solutions work well in weak fields or large chiral fluctuations. We also discuss the properties of electromagnetic fields in the laboratory frame.

hep-ph↗

arXiv2018

Deep Attentive Tracking via Reciprocative Learning

Visual attention, derived from cognitive neuroscience, facilitates human perception on the most pertinent subset of the sensory data. Recently, significant efforts have been made to exploit attention schemes to advance computer vision systems. For visual tracking, it is often challenging to track target objects undergoing large appearance changes. Attention maps facilitate visual tracking by selectively paying attention to temporal robust features. Existing tracking-by-detection approaches mainly use additional attention modules to generate feature weights as the classifiers are not equipped with such mechanisms. In this paper, we propose a reciprocative learning algorithm to exploit visual attention for training deep classifiers. The proposed algorithm consists of feed-forward and backward operations to generate attention maps, which serve as regularization terms coupled with the original classification loss function for training. The deep classifier learns to attend to the regions of target objects robust to appearance changes. Extensive experiments on large-scale benchmark datasets show that the proposed attentive tracking method performs favorably against the state-of-the-art approaches.

cs.CV↗

arXiv2018

Swarming for Faster Convergence in Stochastic Optimization

We study a distributed framework for stochastic optimization which is inspired by models of collective motion found in nature (e.g., swarming) with mild communication requirements. Specifically, we analyze a scheme in which each one of $N > 1$ independent threads, implements in a distributed and unsynchronized fashion, a stochastic gradient-descent algorithm which is perturbed by a swarming potential. Assuming the overhead caused by synchronization is not negligible, we show the swarming-based approach exhibits better performance than a centralized algorithm (based upon the average of $N$ observations) in terms of (real-time) convergence speed. We also derive an error bound that is monotone decreasing in network size and connectivity. We characterize the scheme's finite-time performances for both convex and non-convex objective functions.

math.OC↗

arXiv2018

Non-Equilibrium Quantum Transport of Chiral Fluids from Kinetic Theory

We introduce the quantum-field-theory (QFT) derivation of chiral kinetic theory (CKT) from the Wigner-function approach, which manifests side jumps and non-scalar distribution functions associated with Lorentz covariance and incorporates both background fields and collisions. The formalism is utilized to investigate second-order responses of chiral fluids near local equilibrium. Such non-equilibrium anomalous transport is dissipative and affected by interactions. Contributions from both quantum corrections in anomalous hydrodynamic equations (EOM) of motion and those from the CKT and Wigner functions (WF) are considered in a relaxation-time approximation (RTA). Anomalous charged Hall currents engendered by background electric fields and temperature/chemical-potential gradients are obtained. Furthermore, chiral magnetic/vortical effects (CME/CVE) receive viscous corrections as non-equilibrium modifications stemming from the interplay between side jumps, magnetic-moment coupling, and chiral anomaly.

nucl-th↗

arXiv2018

Axial Ward identity and the Schwinger mechanism -- Applications to the real-time chiral magnetic effect and condensates

We elucidate chirality production under parity breaking constant electromagnetic fields, with which we clarify qualitative differences in and out of equilibrium. For a strong magnetic field the pair production from the Schwinger mechanism increments the chirality. The pair production rate is exponentially suppressed with mass according to the Schwinger formula, while the mass dependence of chirality production in the axial Ward identity appears in the pesudo-scalar term. We demonstrate that in equilibrium field theory calculus the axial anomaly is canceled by the pseudo-scalar condensate for any mass. In a real-time formulation with in- and out-states, we show that the axial Ward identity leads to the chirality production rate consistent with the Schwinger formula. We illuminate that such an in- and out-states formulation makes clear the chiral magnetic effect in and out of equilibrium, and we discuss further applications to real-time condensates.

hep-th↗

arXiv2018

Abelian and non-Abelian Berry curvatures in lattice QCD

We studied the Berry curvature of the massive Dirac fermion in 3+1 dimensions. For the non-interacting Dirac fermion, the Berry curvature is non-Abelian because of the degeneracy of positive and negative helicity modes. We calculated the non-Abelian Berry curvature analytically and numerically. For the interacting Dirac fermion in QCD, the degeneracy is lost because gluons carry helicity and color charge. We calculated the Abelian Berry curvature in lattice QCD.

hep-lat↗

arXiv2018

Nonlinear Responses of Chiral Fluids from Kinetic Theory

The second-order nonlinear responses of inviscid chiral fluids near local equilibrium are investigated by applying the chiral kinetic theory (CKT) incorporating side-jump effects. It is shown that the local equilibrium distribution function can be non-trivially introduced in a co-moving frame with respect to the fluid velocity when the quantum corrections in collisions are involved. For the study of anomalous transport, contributions from both quantum corrections in anomalous hydrodynamic equations of motion and those from the CKT and Wigner functions are considered under the relaxation-time (RT) approximation, which result in anomalous charge Hall currents propagating along the cross product of the background electric field and the temperature (or chemical-potential) gradient and of the temperature and chemical-potential gradients. On the other hand, the nonlinear quantum correction on the charge density vanishes in the classical RT approximation, which in fact satisfies the matching condition given by the anomalous equation obtained from the CKT.

hep-th↗

arXiv2017

Effect of intense magnetic fields on reduced-MHD evolution in $\sqrt{s_{\rm NN}}$ = 200 GeV Au+Au collisions

We investigate the effect of large magnetic fields on the $2+1$ dimensional reduced-magnetohydrodynamical expansion of hot and dense nuclear matter produced in $\sqrt{s_{\rm NN}}$ = 200 GeV Au+Au collisions. For the sake of simplicity, we consider the case where the magnetic field points in the direction perpendicular to the reaction plane. We also consider this field to be external, with energy density parametrized as a two-dimensional Gaussian. The width of the Gaussian along the directions orthogonal to the beam axis varies with the centrality of the collision. The dependence of the magnetic field on proper time ($τ$) for the case of zero electrical conductivity of the QGP is parametrized following [Deng 2012], and for finite electrical conductivity following [Tuchin 2013]. We solve the equations of motion of ideal hydrodynamics for such an external magnetic field. For collisions with non-zero impact parameter we observe considerable changes in the evolution of the momentum eccentricities of the fireball when comparing the case when the magnetic field decays in a conducting QGP medium and when no magnetic field is present. The elliptic-flow coefficient $v_2$ of $π^{-}$ is shown to increase in the presence of an external magnetic field and the increment in $v_2$ is found to depend on the evolution and the initial magnitude of the magnetic field.

nucl-th↗

arXiv2017

Thermal vorticity production in relativistic dissipative fluids

We have computed the circulation integrations of thermal vorticity with and without charged currents in dissipative fluids. We find that the relativistic Kelvin circulation theorem will be modified by the dissipative effects, therefore, the circulation integrations of thermal vorticity may not be conserved during the fluid evolution.

hep-ph↗

arXiv2017

A Flocking-based Approach for Distributed Stochastic Optimization

In recent years, the paradigm of cloud computing has emerged as an architecture for computing that makes use of distributed (networked) computing resources. In this paper, we consider a distributed computing algorithmic scheme for stochastic optimization which relies on modest communication requirements amongst processors and most importantly, does not require synchronization. Specifically, we analyze a scheme with $N>1$ independent threads implementing each a stochastic gradient algorithm. The threads are coupled via a perturbation of the gradient (with attractive and repulsive forces) in a similar manner to mathematical models of flocking, swarming and other group formations found in nature with mild communication requirements. When the objective function is convex, we show that a flocking-like approach for distributed stochastic optimization provides a noise reduction effect similar to that of a centralized stochastic gradient algorithm based upon the average of $N$ gradient samples at each step. The distributed nature of flocking makes it an appealing computational alternative. We show that when the overhead related to the time needed to gather $N$ samples and synchronization is not negligible, the flocking implementation outperforms a centralized stochastic gradient algorithm based upon the average of $N$ gradient samples at each step. When the objective function is not convex, the flocking-based approach seems better suited to escape locally optimal solutions due to the repulsive force which enforces a certain level of diversity in the set of candidate solutions. Here again, we show that the noise reduction effect is similar to that associated to the centralized stochastic gradient algorithm based upon the average of $N$ gradient samples at each step.

math.OC↗

arXiv2017

Boost invariant formulation of the chiral kinetic theory

We formulate the chiral kinetic equation with the longitudinal boost invariance. We particularly focus on the physical interpretation of the particle number conservation. There appear two terms associated with the expansion, which did not exist in the non-chiral kinetic equation. One is a contribution to the transverse current arising from the side-jump effect, and the other is a change in the density whose flow makes the longitudinal current. We point out a characteristic pattern in the transverse current driven by the expansion, which we call the chiral circular displacement.

hep-ph↗