Zhengxiang Yan | Searcharxiv

arXiv2021

Talking Head Generation with Audio and Speech Related Facial Action Units

The task of talking head generation is to synthesize a lip synchronized talking head video by inputting an arbitrary face image and audio clips. Most existing methods ignore the local driving information of the mouth muscles. In this paper, we propose a novel recurrent generative network that uses both audio and speech-related facial action units (AUs) as the driving information. AU information related to the mouth can guide the movement of the mouth more accurately. Since speech is highly correlated with speech-related AUs, we propose an Audio-to-AU module in our system to predict the speech-related AU information from speech. In addition, we use AU classifier to ensure that the generated images contain correct AU information. Frame discriminator is also constructed for adversarial training to improve the realism of the generated face. We verify the effectiveness of our model on the GRID dataset and TCD-TIMIT dataset. We also conduct an ablation study to verify the contribution of each component in our model. Quantitative and qualitative experiments demonstrate that our method outperforms existing methods in both image quality and lip-sync accuracy.

cs.CV↗

arXiv2011

Singularity of the extremal solution for supercritical biharmonic equations with power-type nonlinearity

Let $λ^{*}>0$ denote the largest possible value of $λ$ such that $$ \{{array}{lllllll} Δ^{2}u=λ(1+u)^{p} & {in}\ \ \B, %0 \frac{n+4}{n-4}$ and $n$ is the exterior unit normal vector. We show that for $λ=λ^{*}$ this problem possesses a unique weak solution $u^{*}$, called the extremal solution. We prove that $u^{*}$ is singular when $n\geq 13$ for $p$ large enough, in which case $u^{*}(x)\leq r^{-\frac{4}{p-1}}-1$ on the unit ball.

math.AP↗