arXiv subjects

Zhiwei Tao

Publications and source records attributed to Zhiwei Tao.

5 recordsLinked to original sources

arXiv2024

MoNTA: Accelerating Mixture-of-Experts Training with Network-Traffc-Aware Parallel Optimization

The Mixture of Experts (MoE) is an advanced model architecture in the industry that combines multiple specialized expert models from various domains into a single supermodel. This approach enables the model to scale without significantly increasing the computational costs of training and inference, while maximizing model performance. However, current distributed training frameworks do not consider the ultimate optimization of communication, especially for large base models. This paper proposes a network-traffic-aware parallel optimization method that selects the optimal parallel strategy based on the communication volume, and the training cluster's inter-node and intra-node network topologies. Compared to the DeepSpeed, MoNTA achieves an 8x increase in AllToAll communication performance under 8-card tensor parallelism. Compared to the baseline, training a 2x70B model using 16 A800 cards, with an 8K sequence, results in a 13% overall latency performance improvement. Project Page: https://github.com/EnflameTechnology/DeepSpeed.

cs.LG↗

arXiv2023

Atmospheric turbulence does not change the degree of polarization of vector beams

We propose a novel theoretical framework to demonstrate vector beams whose degree of polarization does not change on atmospheric propagation. Inspired by the Fresnel equations, we derive the reflective and refractive field of vector beams propagating through a phase screen by employing the continuity of electromagnetic field. We generalize the conventional split-step beam propagation method by considering the vectorial properties in the vacuum diffraction and the refractive properties of a single phase screen. Based on this vectorial propagation model, we extensively calculate the change of degree of polarization (DOP) of vector beams under different beam parameters and turbulence parameters both in free-space and satellite-mediated links. Our result is that whatever in the free-space or satellite-mediated regime, the change of DOP mainly fluctuates around the order of $10^{-13}$ to $10^{-6}$, which is almost negligible.

physics.optics↗

arXiv2021

Physical meaning of the deviation scale under arbitrary turbulence strengths of optical orbital angular momentum

The recently so-called deviation scale [C. M. Mabena et al., Phys. Rev. A 99, 013828 (2019)] bridges the connection between the result of the infinitesimal propagation equation (IPE) prediction and that of the single phase screen (SPS) approximation. Thanks to the multiple phase screen (MPS) approach, in this paper we elaborate the physical meaning of the deviation scale: the spatial accumulation of slight intensity modulation of incident orbital angular momentum (OAM) carrying beam splits the original vortex into multiple individual vortices with a topological charge (TC) of +1 and re-generates the vortex-antivortex pairs with a TC of +1 and with a TC of -1, leading to a significant deviation between these two different results only when the disruption of this compound effect on the phase distribution of the incident OAM-carrying beam becomes more significant. Other than that, we also show that the appearance of the deviation scale cannot be predicted only by the Rytov variance, which can be predicted through the vortex-splitting ratio of the received optical field alone or with the help of the normalized propagation distance.

physics.optics↗

arXiv2021

Phase-sensitive nonclassical properties in quantum metrology with a displaced squeezed vacuum state

We predict that the phase-dependent error distribution of locally unentangled quantum states directly affects quantum parameter estimation accuracy. Therefore, we employ the displaced squeezed vacuum (DSV) state as a probe state and investigate an interesting question of the phase-sensitive nonclassical properties in DSV's metrology. We found that the accuracy limit of parameter estimation is a function of the phase-sensitive parameter $ϕ-θ/2$ with a period $π$. We show that when $ϕ-θ/2$ $\in \left[ kπ/2,3kπ/4\right) \left( k\in \mathbb{Z}\right)$, we can obtain the accuracy of parameter estimation approaching the ultimate quantum limit through using the DSV state with the larger displacement and squeezing strength, whereas $ϕ-θ/2$ $\in \left(3kπ/4,kπ\right] \left( k\in \mathbb{Z}\right) $, the optimal estimation accuracy can be acquired only when the DSV state degenerates to a squeezed-vacuum state.

quant-ph↗

arXiv2021

Mitigating the effect of atmospheric turbulence on orbital angular momentum-based quantum key distribution using real-time adaptive optics with phase unwrapping

Quantum key distribution (QKD) employed orbital angular momentum (OAM) for high-dimensional encoding enhances the system security and information capacity between two communication parties. However, such advantages significantly degrade because of the fragility of OAM states in atmospheric turbulence. Unlike previous researches, we first investigate the performance degradation of OAM-based QKD by infinitely long phase screen (ILPS), which offers a feasible way to study how adaptive optics (AO) dynamically corrects the turbulence-induced aberrations in real time. Secondly, considering the failure of AO while encountering phase cuts, we evaluate the quality enhancement of OAM-based QKD under moderate turbulence strengths by AO after implementing the wrapped cuts elimination. Finally, we simulate that, with more realistic considerations, real-time AO can still mitigate the impact of atmospheric turbulence on OAM-based QKD even in the large wind velocity regime.

quant-ph↗