arXiv subjects

Yutong Chen

Publications and source records attributed to Yutong Chen.

At least 55 records · Page 3Linked to original sources

arXiv2024

Within the Dynamic Context: Inertia-aware 3D Human Modeling with Pose Sequence

Neural rendering techniques have significantly advanced 3D human body modeling. However, previous approaches often overlook dynamics induced by factors such as motion inertia, leading to challenges in scenarios like abrupt stops after rotation, where the pose remains static while the appearance changes. This limitation arises from reliance on a single pose as conditional input, resulting in ambiguity in mapping one pose to multiple appearances. In this study, we elucidate that variations in human appearance depend not only on the current frame's pose condition but also on past pose states. Therefore, we introduce Dyco, a novel method utilizing the delta pose sequence representation for non-rigid deformations and canonical space to effectively model temporal appearance variations. To prevent a decrease in the model's generalization ability to novel poses, we further propose low-dimensional global context to reduce unnecessary inter-body part dependencies and a quantization operation to mitigate overfitting of the delta pose sequence by the model. To validate the effectiveness of our approach, we collected a novel dataset named I3D-Human, with a focus on capturing temporal changes in clothing appearance under approximate poses. Through extensive experiments on both I3D-Human and existing datasets, our approach demonstrates superior qualitative and quantitative performance. In addition, our inertia-aware 3D human method can unprecedentedly simulate appearance changes caused by inertia at different velocities.

cs.CV↗

arXiv2024

Self-Supervised Spatial-Temporal Normality Learning for Time Series Anomaly Detection

Time Series Anomaly Detection (TSAD) finds widespread applications across various domains such as financial markets, industrial production, and healthcare. Its primary objective is to learn the normal patterns of time series data, thereby identifying deviations in test samples. Most existing TSAD methods focus on modeling data from the temporal dimension, while ignoring the semantic information in the spatial dimension. To address this issue, we introduce a novel approach, called Spatial-Temporal Normality learning (STEN). STEN is composed of a sequence Order prediction-based Temporal Normality learning (OTN) module that captures the temporal correlations within sequences, and a Distance prediction-based Spatial Normality learning (DSN) module that learns the relative spatial relations between sequences in a feature space. By synthesizing these two modules, STEN learns expressive spatial-temporal representations for the normal patterns hidden in the time series data. Extensive experiments on five popular TSAD benchmarks show that STEN substantially outperforms state-of-the-art competing methods. Our code is available at https://github.com/mala-lab/STEN.

cs.LG↗

arXiv2024

Dynamic feature selection in medical predictive monitoring by reinforcement learning

In this paper, we investigate dynamic feature selection within multivariate time-series scenario, a common occurrence in clinical prediction monitoring where each feature corresponds to a bio-test result. Many existing feature selection methods fall short in effectively leveraging time-series information, primarily because they are designed for static data. Our approach addresses this limitation by enabling the selection of time-varying feature subsets for each patient. Specifically, we employ reinforcement learning to optimize a policy under maximum cost restrictions. The prediction model is subsequently updated using synthetic data generated by trained policy. Our method can seamlessly integrate with non-differentiable prediction models. We conducted experiments on a sizable clinical dataset encompassing regression and classification tasks. The results demonstrate that our approach outperforms strong feature selection baselines, particularly when subjected to stringent cost limitations. Code will be released once paper is accepted.

cs.LG↗

arXiv2024

PriorNet: A Novel Lightweight Network with Multidimensional Interactive Attention for Efficient Image Dehazing

Hazy images degrade visual quality, and dehazing is a crucial prerequisite for subsequent processing tasks. Most current dehazing methods rely on neural networks and face challenges such as high computational parameter pressure and weak generalization capabilities. This paper introduces PriorNet--a novel, lightweight, and highly applicable dehazing network designed to significantly improve the clarity and visual quality of hazy images while avoiding excessive detail extraction issues. The core of PriorNet is the original Multi-Dimensional Interactive Attention (MIA) mechanism, which effectively captures a wide range of haze characteristics, substantially reducing the computational load and generalization difficulties associated with complex systems. By utilizing a uniform convolutional kernel size and incorporating skip connections, we have streamlined the feature extraction process. Simplifying the number of layers and architecture not only enhances dehazing efficiency but also facilitates easier deployment on edge devices. Extensive testing across multiple datasets has demonstrated PriorNet's exceptional performance in dehazing and clarity restoration, maintaining image detail and color fidelity in single-image dehazing tasks. Notably, with a model size of just 18Kb, PriorNet showcases superior dehazing generalization capabilities compared to other methods. Our research makes a significant contribution to advancing image dehazing technology, providing new perspectives and tools for the field and related domains, particularly emphasizing the importance of improving universality and deployability.

cs.CV↗

arXiv2024

Expressive Speech-driven Facial Animation with controllable emotions

It is in high demand to generate facial animation with high realism, but it remains a challenging task. Existing approaches of speech-driven facial animation can produce satisfactory mouth movement and lip synchronization, but show weakness in dramatic emotional expressions and flexibility in emotion control. This paper presents a novel deep learning-based approach for expressive facial animation generation from speech that can exhibit wide-spectrum facial expressions with controllable emotion type and intensity. We propose an emotion controller module to learn the relationship between the emotion variations (e.g., types and intensity) and the corresponding facial expression parameters. It enables emotion-controllable facial animation, where the target expression can be continuously adjusted as desired. The qualitative and quantitative evaluations show that the animation generated by our method is rich in facial emotional expressiveness while retaining accurate lip movement, outperforming other state-of-the-art methods.

cs.CV↗

arXiv2023

Real-time VaR Calculations for Crypto Derivatives in kdb+/q

Cryptocurrency market is known for exhibiting significantly higher volatility than traditional asset classes. Efficient and adequate risk calculation is vital for managing risk exposures in such market environments where extreme price fluctuations occur in short timeframes. The objective of this thesis is to build a real-time computation workflow that provides VaR estimates for non-linear portfolios of cryptocurrency derivatives. Many researchers have examined the predictive capabilities of time-series models within the context of cryptocurrencies. In this work, we applied three commonly used models - EMWA, GARCH and HAR - to capture and forecast volatility dynamics, in conjunction with delta-gamma-theta approach and Cornish-Fisher expansion to crypto derivatives, examining their performance from the perspectives of calculation efficiency and accuracy. We present a calculation workflow which harnesses the information embedded in high-frequency market data and the computation simplicity inherent in analytical estimation procedures. This workflow yields reasonably robust VaR estimates with calculation latencies on the order of milliseconds.

q-fin.ST↗

arXiv2023

Improving Continuous Sign Language Recognition with Cross-Lingual Signs

This work dedicates to continuous sign language recognition (CSLR), which is a weakly supervised task dealing with the recognition of continuous signs from videos, without any prior knowledge about the temporal boundaries between consecutive signs. Data scarcity heavily impedes the progress of CSLR. Existing approaches typically train CSLR models on a monolingual corpus, which is orders of magnitude smaller than that of speech recognition. In this work, we explore the feasibility of utilizing multilingual sign language corpora to facilitate monolingual CSLR. Our work is built upon the observation of cross-lingual signs, which originate from different sign languages but have similar visual signals (e.g., hand shape and motion). The underlying idea of our approach is to identify the cross-lingual signs in one sign language and properly leverage them as auxiliary training data to improve the recognition capability of another. To achieve the goal, we first build two sign language dictionaries containing isolated signs that appear in two datasets. Then we identify the sign-to-sign mappings between two sign languages via a well-optimized isolated sign language recognition model. At last, we train a CSLR model on the combination of the target data with original labels and the auxiliary data with mapped labels. Experimentally, our approach achieves state-of-the-art performance on two widely-used CSLR datasets: Phoenix-2014 and Phoenix-2014T.

cs.CV↗

arXiv2023

A radio pulsar phase from SGR J1935+2154 provides clues to the magnetar FRB mechanism

The megajansky radio burst, FRB 20200428, and other bright radio bursts detected from the Galactic source SGR J1935+2154 suggest that magnetars can make fast radio bursts (FRBs), but the emission site and mechanism of FRB-like bursts are still unidentified. Here we report the emergence of a radio pulsar phase of the magnetar five months after FRB 20200428. 795 pulses were detected in 16.5 hours over 13 days by the Five-hundred-meter Aperture Spherical Radio telescope, with luminosities about eight decades fainter than FRB 20200428. The pulses were emitted in a narrow phase window anti-aligned with the X-ray pulsation profile observed by the X-ray telescopes. The bursts, conversely, appear in random phases. This dichotomy suggests that radio pulses originate from a fixed region within the magnetosphere, but bursts occur in random locations and are possibly associated with explosive events in a dynamically evolving magnetosphere. This picture reconciles the lack of periodicity in cosmological repeating FRBs within the magnetar engine model.

astro-ph.HE↗

arXiv2023

Efficient Cross-Lingual Transfer for Chinese Stable Diffusion with Images as Pivots

Diffusion models have made impressive progress in text-to-image synthesis. However, training such large-scale models (e.g. Stable Diffusion), from scratch requires high computational costs and massive high-quality text-image pairs, which becomes unaffordable in other languages. To handle this challenge, we propose IAP, a simple but effective method to transfer English Stable Diffusion into Chinese. IAP optimizes only a separate Chinese text encoder with all other parameters fixed to align Chinese semantics space to the English one in CLIP. To achieve this, we innovatively treat images as pivots and minimize the distance of attentive features produced from cross-attention between images and each language respectively. In this way, IAP establishes connections of Chinese, English and visual semantics in CLIP's embedding space efficiently, advancing the quality of the generated image with direct Chinese prompts. Experimental results show that our method outperforms several strong Chinese diffusion models with only 5%~10% training data.

cs.CV↗

arXiv2023

A Simple Multi-Modality Transfer Learning Baseline for Sign Language Translation

This paper proposes a simple transfer learning baseline for sign language translation. Existing sign language datasets (e.g. PHOENIX-2014T, CSL-Daily) contain only about 10K-20K pairs of sign videos, gloss annotations and texts, which are an order of magnitude smaller than typical parallel data for training spoken language translation models. Data is thus a bottleneck for training effective sign language translation models. To mitigate this problem, we propose to progressively pretrain the model from general-domain datasets that include a large amount of external supervision to within-domain datasets. Concretely, we pretrain the sign-to-gloss visual network on the general domain of human actions and the within-domain of a sign-to-gloss dataset, and pretrain the gloss-to-text translation network on the general domain of a multilingual corpus and the within-domain of a gloss-to-text corpus. The joint model is fine-tuned with an additional module named the visual-language mapper that connects the two networks. This simple baseline surpasses the previous state-of-the-art results on two sign language translation benchmarks, demonstrating the effectiveness of transfer learning. With its simplicity and strong performance, this approach can serve as a solid baseline for future research. Code and models are available at: https://github.com/FangyunWei/SLRT.

cs.CV↗

arXiv2023

Two-Stream Network for Sign Language Recognition and Translation

Sign languages are visual languages using manual articulations and non-manual elements to convey information. For sign language recognition and translation, the majority of existing approaches directly encode RGB videos into hidden representations. RGB videos, however, are raw signals with substantial visual redundancy, leading the encoder to overlook the key information for sign language understanding. To mitigate this problem and better incorporate domain knowledge, such as handshape and body movement, we introduce a dual visual encoder containing two separate streams to model both the raw videos and the keypoint sequences generated by an off-the-shelf keypoint estimator. To make the two streams interact with each other, we explore a variety of techniques, including bidirectional lateral connection, sign pyramid network with auxiliary supervision, and frame-level self-distillation. The resulting model is called TwoStream-SLR, which is competent for sign language recognition (SLR). TwoStream-SLR is extended to a sign language translation (SLT) model, TwoStream-SLT, by simply attaching an extra translation network. Experimentally, our TwoStream-SLR and TwoStream-SLT achieve state-of-the-art performance on SLR and SLT tasks across a series of datasets including Phoenix-2014, Phoenix-2014T, and CSL-Daily. Code and models are available at: https://github.com/FangyunWei/SLRT.

cs.CV↗

arXiv2022

Efficient photocatalytic nitrogen fixation from air under sunlight via iron-doped WO$_3$

Photocatalytic nitrogen fixation from air directly under sunlight can contribute significantly to carbon neutralization. It is an ideal pathway to replace the industrial Haber Bosch process in future. A Fe-doped layered WO$_3$ photocatalyst containing oxygen vacancies was developed which can fix nitrogen from air directly under sunlight at atmospheric pressure. The iron doping enhances the transport efficiency of photogenerated electrons. The photocatalytic efficiency is around 4 times higher than that of pure WO$_3$. The optimum nitrogen fixation conditions were examined by orthogonal experiments and its nitrogen fixation performance could reach up to 477 $μ\text{g} \cdot \text{g}_{\text{cat}}^{-1} \cdot \text{h}^{-1}$ under sunlight. In addition, the process of nitrogen fixation was detected by situ infrared, which confirmed the reliability of nitrogen fixation. Also, modelling on the interactions between light and the photocatalyst was carried out to study the distribution of surface charge and validate the light absorption of the photocatalyst. This work provides a simple and cheap strategy for photocatalytic nitrogen fixation from air under mild conditions.

cond-mat.mtrl-sci↗

arXiv2022

Efficient sunlight promoted nitrogen fixation from air under room temperature and ambient pressure via Ti/Mo composites

Photocatalytic nitrogen fixation is an important pathway for carbon neutralization and sustainable development. Inspired by nitrogenase, the participation of molybdenum can effectively activate nitrogen. A novel Ti/Mo composites photocatalyst is designed by sintering the molybdenum acetylacetonate precursor with TiO$_{2}$. The special carbon-coated hexagonal photocatalyst is obtained which photocatalytic nitrogen fixation performance is enhanced 16 times compared to pure TiO$_{2}$ at room temperature and ambient pressure. The abundant surface defects in this composite were confirmed to be the key factor for nitrogen fixation. The $^{15}$N$_{2}$ isotope labeling experiment was used to demonstrate the feasibility of nitrogen to ammonia conversion. Also, modelling on the interactions between light and the synthesized photocatalyst particle was examined for the light absorption. The optimum nitrogen fixation conditions have been examined, and the nitrogen fixation performance can reach up to 432 $μ$g$\cdot$g$_{\text{cat}}^{-1}\cdot$h$^{-1}$. Numerical simulations via the field-only surface integral method were also carried out to study the interactions between light and the photocatalytic particles to further confirm that it can be a useful material for photocatalyst. This newly developed Ti/Mo composites provide a simple and effective strategy for photocatalytic nitrogen fixation from air directly under ambient conditions.

cond-mat.mtrl-sci↗

arXiv2021

AI-based Reconstruction for Fast MRI -- A Systematic Review and Meta-analysis

Compressed sensing (CS) has been playing a key role in accelerating the magnetic resonance imaging (MRI) acquisition process. With the resurgence of artificial intelligence, deep neural networks and CS algorithms are being integrated to redefine the state of the art of fast MRI. The past several years have witnessed substantial growth in the complexity, diversity, and performance of deep learning-based CS techniques that are dedicated to fast MRI. In this meta-analysis, we systematically review the deep learning-based CS techniques for fast MRI, describe key model designs, highlight breakthroughs, and discuss promising directions. We have also introduced a comprehensive analysis framework and a classification system to assess the pivotal role of deep learning in CS-based acceleration for MRI.

eess.IV↗

arXiv2021

Generative Adversarial Networks (GAN) Powered Fast Magnetic Resonance Imaging -- Mini Review, Comparison and Perspectives

Magnetic Resonance Imaging (MRI) is a vital component of medical imaging. When compared to other image modalities, it has advantages such as the absence of radiation, superior soft tissue contrast, and complementary multiple sequence information. However, one drawback of MRI is its comparatively slow scanning and reconstruction compared to other image modalities, limiting its usage in some clinical applications when imaging time is critical. Traditional compressive sensing based MRI (CS-MRI) reconstruction can speed up MRI acquisition, but suffers from a long iterative process and noise-induced artefacts. Recently, Deep Neural Networks (DNNs) have been used in sparse MRI reconstruction models to recreate relatively high-quality images from heavily undersampled k-space data, allowing for much faster MRI scanning. However, there are still some hurdles to tackle. For example, directly training DNNs based on L1/L2 distance to the target fully sampled images could result in blurry reconstruction because L1/L2 loss can only enforce overall image or patch similarity and does not take into account local information such as anatomical sharpness. It is also hard to preserve fine image details while maintaining a natural appearance. More recently, Generative Adversarial Networks (GAN) based methods are proposed to solve fast MRI with enhanced image perceptual quality. The encoder obtains a latent space for the undersampling image, and the image is reconstructed by the decoder using the GAN loss. In this chapter, we review the GAN powered fast MRI methods with a comparative study on various anatomical datasets to demonstrate the generalisability and robustness of this kind of fast MRI while providing future perspectives.

eess.IV↗

arXiv2021

Pure annihilation decays of $B_s^0 \to a_0^+ a_0^-$ and $B_d^0 \to K_0^{+} K_0^{-}$ in the PQCD approach

We study the {\it CP}-averaged branching fractions and the {\it CP}-violating asymmetries in the pure annihilation decays of $B_s^0 \to a_0^+ a_0^-$ and $B_d^0 \to K_0^{*+} K_0^{*-}$, where $a_0\;[K_0^*]$ denotes the scalar $a_0(980)$ and $a_0(1450)$ [$K_0^*(800) ( {\rm or}\; κ)$ and $K_0^*(1430)$], with the perturbative QCD factorization approach under the assumption of two-quark structure for the $a_0$ and $K_0^*$ states. The numerical results show that the branching ratios of the $B_d^0 \to K_0^{*+} K_0^{*-}$ decays are in the order of $10^{-6}$, while the decay rates of the $B_s^0 \to a_0^+ a_0^-$ modes are in the order of $10^{-5}$. In light of the measured modes with the same quark components in the pseudoscalar sector, namely, $B_d^0 \to K^+ K^-$ and $B_s^0 \to π^+ π^-$, the predictions for the considered decay modes in this work are expected to be measured at the Large Hadron Collider beauty and/or Belle-II experiments in the (near) future. Meanwhile, it is of great interest to find that the twist-3 distribution amplitudes $ϕ^{S}$ and $ϕ^{T}$ with inclusion of the Gegenbauer polynomials for the scalar $a_0(1450)$ and $K_0^*(1430)$ states in scenario 2 contribute slightly to the branching ratios while significantly to the {\it CP} violations in the $B_d^0 \to {K_0^*}(1430)^+ {K_0^*}(1430)^-$ and $B_s^0 \to a_0(1450)^+ a_0(1450)^-$ decays, which indicates that, compared to the asymptotic $ϕ^S$ and $ϕ^T$, these Gegenbauer polynomials could change the strong phases evidently in these pure annihilation decay channels. These predictions await for the future confirmation experimentally, which could further provide useful information to help explore the inner structure of the scalars and shed light on the annihilation decay mechanism.

hep-ph↗

arXiv2020

Learning to Manipulate Individual Objects in an Image

We describe a method to train a generative model with latent factors that are (approximately) independent and localized. This means that perturbing the latent variables affects only local regions of the synthesized image, corresponding to objects. Unlike other unsupervised generative models, ours enables object-centric manipulation, without requiring object-level annotations, or any form of annotation for that matter. The key to our method is the combination of spatial disentanglement, enforced by a Contextual Information Separation loss, and perceptual cycle-consistency, enforced by a loss that penalizes changes in the image partition in response to perturbations of the latent factors. We test our method's ability to allow independent control of spatial and semantic factors of variability on existing datasets and also introduce two new ones that highlight the limitations of current methods.

cs.CV↗

arXiv2015

Non-unique games over compact groups and orientation estimation in cryo-EM

Let $\mathcal{G}$ be a compact group and let $f_{ij} \in L^2(\mathcal{G})$. We define the Non-Unique Games (NUG) problem as finding $g_1,\dots,g_n \in \mathcal{G}$ to minimize $\sum_{i,j=1}^n f_{ij} \left( g_i g_j^{-1}\right)$. We devise a relaxation of the NUG problem to a semidefinite program (SDP) by taking the Fourier transform of $f_{ij}$ over $\mathcal{G}$, which can then be solved efficiently. The NUG framework can be seen as a generalization of the little Grothendieck problem over the orthogonal group and the Unique Games problem and includes many practically relevant problems, such as the maximum likelihood estimator} to registering bandlimited functions over the unit sphere in $d$-dimensions and orientation estimation in cryo-Electron Microscopy.

cs.CV↗