Evgeny Shevelev | Searcharxiv

arXiv2026

ML-assisted Subband Learned Digital Backpropagation for Nonlinearity Compensation in Wideband Optical Systems

Digital backpropagation (DBP) is one of the most effective techniques for compensating nonlinear distortions in coherent optical fiber communication systems. However, its practical application to wideband transmission remains limited by high computational complexity caused by large channel memory and the requirement for fine spatial discretization. In this work, we propose a subband-based learned digital backpropagation (SbL-DBP) framework for wideband optical transmission systems. The received signal is decomposed into multiple subbands, enabling independent frequency-domain compensation of the chromatic dispersion with reduced effective channel memory and lower computational complexity. Nonlinear intra- and inter-subband interactions are addressed in the time domain using a trainable multi-input multi-output filtering structure. The parameters of the proposed framework are jointly optimized using end-to-end gradient-based learning. In addition, sparsification techniques are employed to remove insignificant coefficients and further reduce computational complexity. Numerical simulations of an 11$\times$40~Gbaud WDM RRC-16QAM 20$\times$100 km transmission system demonstrate that the proposed method provides a superior performance--complexity trade-off compared to conventional DBP and enhanced DBP. In the low- and medium-complexity regimes, SbL-DBP provides higher signal-to-noise ratio gains while requiring fewer propagation steps.

physics.optics↗

arXiv2024

CNN-Assisted Particle Swarm Optimization of a Perturbation-Based Model for Nonlinearity Compensation in Optical Transmission Systems

Nonlinear signal distortions are one of the primary factors limiting the capacity and reach of optical transmission systems. Currently, several approaches exist for compensating nonlinear distortions, but for practical implementation, algorithms must be simultaneously accurate, fast, and robust against various interferences. One established approach involves applying perturbation theory methods to the nonlinear Schr\"{o}dinger equation, which enables the determination of the relation between transmitted and received symbols. In most studies, gradient methods are used to find perturbation coefficients by minimizing the mean squared error between symbols. However, the main parameter characterizing the quality of information transmission is the bit error rate. We propose a modification of the conventional perturbation-based approach for fiber nonlinearity compensation in the form of a two-stage scheme for calculating perturbation coefficients. In the first stage, the coefficients are computed using a convolutional neural network by minimizing the mean squared error. In the second stage, the obtained solution is used as an initial approximation for minimizing the bit error rate using the particle swarm optimization method. In numerical experiments, using the nonlinearity compensation algorithm based on the proposed scheme, we achieved a 0.8~dB gain in the signal-to-noise ratio for a 16QAM 20$\times$100 km link with a channel rate of 267~Gbit/s and demonstrated improved accuracy compared to the single-stage scheme. We estimated computational complexity of the algorithm and demonstrated the relation between its complexity and accuracy. Additionally, we developed a method for learning perturbation coefficients without relying on ideal symbols from the transmitter, instead using the received symbols after hard decision detection.

physics.optics↗