arXiv subjects

Liyang Jiang

Publications and source records attributed to Liyang Jiang.

3 recordsLinked to original sources

arXiv2023

Real time spectrogram inversion on mobile phone

We present two methods of real time magnitude spectrogram inversion: streaming Griffin Lim(GL) and streaming MelGAN. We demonstrate the impact of looking ahead on perceptual quality of MelGAN. As little as one hop size (12.5ms) of lookahead is able to significantly improve perceptual quality in comparison to its causal version. We compare streaming GL with the streaming MelGAN and show different trade-offs in terms of perceptual quality, on-device latency, algorithmic delay, memory footprint and noise sensitivity. For fair quality assessment of the GL approach, we use input log magnitude spectrogram without mel transformation. We evaluate presented real time spectrogram inversion approaches on clean, noisy and atypical speech. We specified conditions when streaming GL has comparable quality with MelGAN: noisy audio and no mel transformation. Streaming GL is 2.4x faster than real time on the ARM CPU of a Pixel4 and it uses 4.5x times less memory than MelGAN.

eess.AS↗

arXiv2023

Streaming Parrotron for on-device speech-to-speech conversion

We present a fully on-device streaming Speech2Speech conversion model that normalizes a given input speech directly to synthesized output speech. Deploying such a model on mobile devices pose significant challenges in terms of memory footprint and computation requirements. We present a streaming-based approach to produce an acceptable delay, with minimal loss in speech conversion quality, when compared to a reference state of the art non-streaming approach. Our method consists of first streaming the encoder in real time while the speaker is speaking. Then, as soon as the speaker stops speaking, we run the spectrogram decoder in streaming mode along the side of a streaming vocoder to generate output speech. To achieve an acceptable delay-quality trade-off, we propose a novel hybrid approach for look-ahead in the encoder which combines a look-ahead feature stacker with a look-ahead self-attention. We show that our streaming approach is almost 2x faster than real time on the Pixel4 CPU.

eess.AS↗

arXiv2019

The Influence of Mineral Texture on Fracture Geometry in Layered Geo-Architected Rock

Rock is a complicated material because of the inherent heterogeneity in mineral phases and composition, even when extracted from the same rock mass. The spatial variability in compositional and structural features prevent reproducible measurements of deformation, fracture formation and other physical and chemical properties. Here, we use geo-architected 3D printed synthetic gypsum rock to show that mineral texture orientation governs the isotropy or anisotropy in fracture surface roughness and volumetric flow rate through tensile fractures. Failure load is governed by the orientation of depositional layering relative to loading direction but this effect can be hidden when an orientated mineral texture is present within the layers. We find that samples with certain mineral orientations are twice as strong as the weakest samples and that high values of sample strength correlate with rougher fracture surfaces. Mineral texture oriented parallel to the failure plane gives rise to anisotropic surface roughness and in turn to anisotropic flow rates. The uniqueness of induced fracture roughness and the degree of anisotropy is a potential method for assessment of the orientation and relative bonding strengths of minerals in a rock. With this information, we will be able to predict isotropic or anisotropic flow rates through fractures which is vital to induced fracturing, geothermal energy production and CO2 sequestration.

physics.geo-ph↗