SearcharxivSearch

arXiv subjects

Igor Polyakov

Publications and source records attributed to Igor Polyakov.

2 recordsLinked to original sources

The Tara Polaris scientific vision: Advancing our understanding of the central Arctic Ocean to better address life in the Earth System

The Arctic Ocean is currently experiencing, at the forefront of global concerns, the pressures of climate change and global pollution. To boost our ability to understand the state of this ecosystem, its evolution in this context and its resilience, the Tara Ocean Foundation has built the Tara Polar Station (TPS), intended to become a permanent observatory of the central Arctic Ocean. The objective of this initiative is threefold: to deepen our knowledge of the foundations of life in an ice-covered polar ocean, to better understand the dynamics of the coupled ocean-ice-atmosphere system and the role of living organisms, and to identify long-term trends in the main characteristics of the Central Arctic Ocean ecosystem resulting from global change. In this article, we describe the vision that guided the development of the Tara Polaris scientific programme, and more specifically the first of ten transpolar drifts that will be undertaken over the next 20 years aboard the TPS (Tara Polaris I, II, III, etc.). The research activities of the Tara Polaris I expedition will be grouped under four specific but interrelated themes: biosphere-atmosphere interactions, epi- and mesopelagic life in an ice-covered ocean, life in sea ice, and pollution. In addition, a theme that cuts across all environmental compartments and disciplines, and is implemented on all Tara Polaris expeditions, is the establishment of an observatory that will monitor the main sentinels of this ecosystem. This umbrella article introduces these different themes, which are then described in more detail in four other articles in this Special Feature, in addition to an article describing the technical characteristics of the TPS.

physics.ao-ph

TAGC: Optimizing Gradient Communication in Distributed Transformer Training

The increasing complexity of large language models (LLMs) necessitates efficient training strategies to mitigate the high computational costs associated with distributed training. A significant bottleneck in this process is gradient synchronization across multiple GPUs, particularly in the zero-redundancy parallelism mode. In this paper, we introduce Transformer-Aware Gradient Compression (TAGC), an optimized gradient compression algorithm designed specifically for transformer-based models. TAGC extends the lossless homomorphic compression method by adapting it for sharded models and incorporating transformer-specific optimizations, such as layer-selective compression and dynamic sparsification. Our experimental results demonstrate that TAGC accelerates training by up to 15% compared to the standard Fully Sharded Data Parallel (FSDP) approach, with minimal impact on model quality. We integrate TAGC into the PyTorch FSDP framework, the implementation is publicly available at https://github.com/ipolyakov/TAGC.

cs.LG