arXiv subjects

Feixiang Ren

Publications and source records attributed to Feixiang Ren.

3 recordsLinked to original sources

arXiv2026

Towards Critical Branching Mechanism in Recurrent Neural Networks

Criticality has been proposed as a key organizing principle in biological neural systems, yet its origin and relevance in artificial neural networks remain unclear. We analyze hidden-state dynamics in trained long short-term memory (LSTM) networks and show that small networks near their optimal training epochs (steps) exhibit scale-free avalanche statistics and branching parameters close to unity, indicative of near-critical dynamics, while larger models remain subcritical. To explain the coexistence of subcritical branching with robust $1/f^{\beta}$ noise, we introduce a mixture branching process framework that links heterogeneous branching dynamics to long-range temporal correlations. These results identify critical-like behavior in LSTMs as an emergent, capacity-dependent dynamical regime.

nlin.AO↗

arXiv2024

Unveiling the Relationship Between Amplitude-only Transmission Matrix and Third-Order Correlation of Light Fields in Wavefront Shaping

In the regime of wavefront shaping (WFS) techniques, both the transmission matrix (TM) methods and a recently proposed third-order correlation of light fields (TCLF) method are effective in overcoming light scattering. This letter details the relationship between the amplitude-only TM method and the amplitude-only TCLF. The random fluctuations of different pixels on a digital micromirror device (DMD) in the amplitude-only TCLF method can be regarded as orthogonal bases, sharing the same role as the Hadamard bases in the amplitude-only TM method. This insight explains why the computational complexity of the TCLF method is significantly lower than that of the TM methods and also indicates that the amplitude-only TM is essentially a special case of the TCLF method.

physics.optics↗

arXiv2023

Distilling Token-Pruned Pose Transformer for 2D Human Pose Estimation

Human pose estimation has seen widespread use of transformer models in recent years. Pose transformers benefit from the self-attention map, which captures the correlation between human joint tokens and the image. However, training such models is computationally expensive. The recent token-Pruned Pose Transformer (PPT) solves this problem by pruning the background tokens of the image, which are usually less informative. However, although it improves efficiency, PPT inevitably leads to worse performance than TokenPose due to the pruning of tokens. To overcome this problem, we present a novel method called Distilling Pruned-Token Transformer for human pose estimation (DPPT). Our method leverages the output of a pre-trained TokenPose to supervise the learning process of PPT. We also establish connections between the internal structure of pose transformers and PPT, such as attention maps and joint features. Our experimental results on the MPII datasets show that our DPPT can significantly improve PCK compared to previous PPT models while still reducing computational complexity.

cs.CV↗