arXiv subjects

Igor Sokolov

Publications and source records attributed to Igor Sokolov.

31 records · Page 2Linked to original sources

arXiv2022

3PC: Three Point Compressors for Communication-Efficient Distributed Training and a Better Theory for Lazy Aggregation

We propose and study a new class of gradient communication mechanisms for communication-efficient training -- three point compressors (3PC) -- as well as efficient distributed nonconvex optimization algorithms that can take advantage of them. Unlike most established approaches, which rely on a static compressor choice (e.g., Top-$K$), our class allows the compressors to {\em evolve} throughout the training process, with the aim of improving the theoretical communication complexity and practical efficiency of the underlying methods. We show that our general approach can recover the recently proposed state-of-the-art error feedback mechanism EF21 (Richtárik et al., 2021) and its theoretical properties as a special case, but also leads to a number of new efficient methods. Notably, our approach allows us to improve upon the state of the art in the algorithmic and theoretical foundations of the {\em lazy aggregation} literature (Chen et al., 2018). As a by-product that may be of independent interest, we provide a new and fundamental link between the lazy aggregation and error feedback literature. A special feature of our work is that we do not require the compressors to be unbiased.

cs.LG↗

arXiv2021

EF21: A New, Simpler, Theoretically Better, and Practically Faster Error Feedback

Error feedback (EF), also known as error compensation, is an immensely popular convergence stabilization mechanism in the context of distributed training of supervised machine learning models enhanced by the use of contractive communication compression mechanisms, such as Top-$k$. First proposed by Seide et al (2014) as a heuristic, EF resisted any theoretical understanding until recently [Stich et al., 2018, Alistarh et al., 2018]. However, all existing analyses either i) apply to the single node setting only, ii) rely on very strong and often unreasonable assumptions, such global boundedness of the gradients, or iterate-dependent assumptions that cannot be checked a-priori and may not hold in practice, or iii) circumvent these issues via the introduction of additional unbiased compressors, which increase the communication cost. In this work we fix all these deficiencies by proposing and analyzing a new EF mechanism, which we call EF21, which consistently and substantially outperforms EF in practice. Our theoretical analysis relies on standard assumptions only, works in the distributed heterogeneous data setting, and leads to better and more meaningful rates. In particular, we prove that EF21 enjoys a fast $O(1/T)$ convergence rate for smooth nonconvex problems, beating the previous bound of $O(1/T^{2/3})$, which was shown a bounded gradients assumption. We further improve this to a fast linear rate for PL functions, which is the first linear convergence result for an EF-type method not relying on unbiased compressors. Since EF has a large number of applications where it reigns supreme, we believe that our 2021 variant, EF21, can a large impact on the practice of communication efficient distributed learning.

cs.LG↗

arXiv2020

Max-compound Cox processes. III

Extreme values are considered in samples with random size that has a mixed Poisson distribution being generated by a doubly stochastic Poisson process. We prove some inequalities providing bounds on the rate of convergence in limit theorems for the distributions of max-compound Cox processes.

math.PR↗

arXiv2018

Synthetic Radio Imaging for Quiescent and CME-flare Scenarios

Radio observations grant access to a wide range of physical processes through different emission mechanisms. These processes range from thermal and quiescent to eruptive phenomena, such as shock waves and particle beams. We present a new synthetic radio imaging tool that calculates and visualizes the Bremsstrahlung radio emission. This tool works concurrently with state-of-the-art Magnetohydrodynamic (MHD) simulations of the solar corona using the code BATS-R-US. Our model produces results that are in good agreement with both high and low frequency observations of the solar disk. In this study, a ray-tracing algorithm is used and the radio intensity is computed along the actual curved ray trajectories. We illustrate the importance of refraction in locating the radio emitting source by comparison of the radio imaging illustrations when the line-of-sight instead of the refracted paths are considered. We are planning to incorporate non-thermal radio emission mechanisms in a future version of the radio imaging tool.

astro-ph.SR↗

arXiv2018

Exoplanet Modulation of Stellar Coronal Radio Emission

The search for exoplanets in the radio bands has been focused on detecting radio emissions produced by the interaction between magnetized planets and the stellar wind (auroral emission). Here we introduce a new tool, which is part of our MHD stellar corona model, to predict the ambient coronal radio emission and its modulations induced by a close planet. For simplicity, the present work assumes that the exoplanet is stationary in the frame rotating with the stellar rotation. We explore the radio flux modulations using a limited parameter space of idealized cases by changing the magnitude of the planetary field, its polarity, the planetary orbital separation, and the strength of the stellar field. We find that the modulations induced by the planet could be significant and observable in the case of hot Jupiter planets --- above 100% modulation with respect to the ambient flux in the $10-100~MHz$ range in some cases, and 2-10% in the frequency bands above $250~MHz$ for some cases. Thus, our work indicates that radio signature of exoplanets might not be limited to low-frequency radio range. We find that the intensity modulations are sensitive to the planetary magnetic field polarity for short-orbit planets, and to the stellar magnetic field strength for all cases. The new radio tool, when applied to real systems, could provide predictions for the frequency range at which the modulations can be observed by current facilities.

astro-ph.SR↗

arXiv2018

Quantum algorithms for electronic structure calculations: particle/hole Hamiltonian and optimized wavefunction expansions

In this work we investigate methods to improve the efficiency and scalability of quantum algorithms for quantum chemistry applications. We propose a transformation of the electronic structure Hamiltonian in the second quantization framework into the particle-hole (p/h) picture, which offers a better starting point for the expansion of the trial wavefunction. The state of the molecular system at study is parametrized in a way to efficiently explore the sector of the molecular Fock space that contains the desired solution. To this end, we explore several trial wavefunctions to identify the most efficient parameterization of the molecular ground state. Taking advantage of known post-Hartree Fock quantum chemistry approaches and heuristic Hilbert space search quantum algorithms, we propose a new family of quantum circuits based on exchange-type gates that enable accurate calculations while keeping the gate count (i.e., the circuit depth) low. The particle-hole implementation of the Unitary Coupled Cluster (UCC) method within the Variational Quantum Eigensolver approach gives rise to an efficient quantum algorithm, named q-UCC , with important advantages compared to the straightforward 'translation' of the classical Coupled Cluster counterpart. In particular, we show how a single Trotter step can accurately and efficiently reproduce the ground state energies of simple molecular systems.

quant-ph↗

arXiv2014

Taming Lévy flights in confined crowded geometries

We study a two-dimensional diffusive motion of a tracer particle in restricted, crowded anisotropic geometries. The underlying medium is the same as in our previous work [J. Chem. Phys. 140, 044706 (2014)] in which standard, gaussian diffusion was studied. Here, a tracer is allowed to perform Cauchy random walk with uncorrelated steps. Our analysis shows that presence of obstacles significantly influences motion, which in an obstacle-free space would be of a superdiffusive type. At the same time, the selfdiffusive process reveals different anomalous properties, both at the level of a single trajectory realization and after the ensemble averaging. In particular, due to obstacles, the sample mean squared displacement asymptotically grows sublinearly in time, suggesting non-Markov character of motion. Closer inspection of survival probabilities indicates however that underlying diffusion is memoryless over long time scales despite strong inhomogeneity of motion induced by orientational ordering.

physics.chem-ph↗

arXiv2014

The energy conservation law in hydrodynamics vs the pseudo-law of alternative energy. Comment on 'Alternative energy vs pseudoscience' and the papers cited and not cited therein

A series of papers in the ISJAEE Journal on 'dam-free hydroelectric power station' concluded by paper: Zotyev, D. B., Alternative energy vs pseudo-science, ISJAEE. 2013. 8(130). P.131-136, is reviewed and commented. A comparison with the generally accepted energy conservation law in hydrodynamics reveals a disappointingly low scientific level of the reviewed papers (both pro- and contra- the dam-free concept), not excluding the published peer-reviewer reports. In the present version we emphasize that the ISJAEE journal published several papers, which neglect and reject the basic physical concepts, such as the Bernoulli integral, the energy conservation law in hydrodynamics, the wave function of photon and some others, with all these rejected concepts being far beyond the scientific scope of the journal. Some readers will be more concerned about the energy equation in hydrodynamics, in its theoretical form or in the Bernoulli integral form as being more traditional in technical hydromechanics. Some readers will be shocked with the statement that a photon does not possess the wave function. Overall, in this way or in that way, all readers will hardly stay unsurprised.

physics.gen-ph↗

arXiv2010

Algorithm for Tracing Radio Rays in Solar Corona and Chromosphere

In this paper a new efficient algorithm for computation of radio wave ray trajectories is described. The algorithm is based on an original second-order difference scheme with a specific "length-conservation" property, which allows to resolve the ray shape even in the regions where its curvature is high. Besides the scheme, the algorithm includes a number of mechanisms ensuring its correctness and stability. The algorithm is intended for obtaining multi-pixel solar images in wide range of radio frequencies, and it is designed to be used in studies of the solar environment with modern high-resolution radio interferometers and radio telescopes such as the Murchison Widefield Array

astro-ph.IM↗

arXiv2008

Three-Dimensional MHD Simulation of the 2003 October 28 Coronal Mass Ejection: Comparison with LASCO Coronagraph Observations

We numerically model the coronal mass ejection (CME) event of October 28, 2003 that erupted from active region 10486 and propagated to Earth in less than 20 hours causing severe geomagnetic storms. The magnetohydrodynamic (MHD) model is formulated by first arriving at a steady state corona and solar wind employing synoptic magnetograms. We initiate two CMEs from the same active region, one approximately a day earlier that preconditions the solar wind for the much faster CME on the 28th. This second CME travels through the corona at a rate of over 2500 km s$^{-1}$ driving a strong forward shock. We clearly identify this shock in an image produced by the Large Angle Spectrometric Coronagraph (LASCO) C3, and reproduce the shock and its appearance in synthetic white light images from the simulation. We find excellent agreement with both the general morphology and the quantitative brightness of the model CME with LASCO observations. These results demonstrate that the CME shape is largely determined by its interaction with the ambient solar wind and may not be sensitive to the initiation process. We then show how the CME would appear as observed by wide-angle coronagraphs onboard the Solar Terrestrial Relations Observatory (STEREO) spacecraft. We find complex time evolution of the white-light images as a result of the way in which the density structures pass through the Thomson sphere. The simulation is performed with the Space Weather Modeling Framework (SWMF).

astro-ph↗

arXiv2006

Modeling of Self-Healing Polymer Composites Reinforced with Nanoporous Glass Fibers

We report on our progress towards continuum rate equation modeling, as well as numerical simulations, of self-healing of fatigue in composites reinforced with glue carrying nanoporous fibers. We conclude that with the proper choice of the material parameters, effects of fatigue can be partially overcome and degradation of mechanical properties can be delayed.

cond-mat.mtrl-sci↗

arXiv2004

Nanosurgery: Observation of Peptidoglycan Strands in Lactobacillus helveticus Cell Walls

The internal cell wall structure of the bacterium Lactobacillus helveticus has been observed in situ in aqueous solution using an atomic force microscope (AFM). The AFM tip was used not only for imaging but presumably to remove mechanically large patches of the outer cell wall after appropriate chemical treatment, which typically leaves the bacteria alive. The surface exposed after such a surgery revealed ca. 26 nm thick twisted strands within the cell wall. The structure and location of the observed strands are consistent with the glycan backbone of peptidoglycan fibers that give strength to the cell wall. The found structural organization of these fibers has not been observed previously.

q-bio.SC↗

arXiv2002

Levy Flights in External Force Fields: From Models to Equations

We consider different generalizations of the Fokker-Planck-equation devised to describe Levy processes in potential force fields. We show that such generalizations can proceed along different lines. On one hand, Levy statistics can emerge from the fractal temporal nature of the underlying process, i.e. a high variability in the rate of microscopic events. On the other hand, they may be a direct consequence of the scale-free spatial structure on which the process evolves. Although both forms considered lead to Boltzmann equilibrium, the relaxation patterns are quite different. As an example, generalized diffusion in a double-well potential is considered.

cond-mat.stat-mech↗