arXiv subjects

Yao Yao

Publications and source records attributed to Yao Yao.

At least 109 records · Page 6Linked to original sources

arXiv2024

Bipolar blobs as evidence of hidden AGN activities in the low-mass galaxies

We report the evidence of a hidden black hole (BH) in a low-mass galaxy, MaNGA 9885-9102, and provide a new method to identify active BH in low mass galaxies. This galaxy is originally selected from the MaNGA survey with distinctive bipolar H$α$ blobs at the minor axis. The bipolar feature can be associated with AGN activity, while the two blobs are classified as the H II regions on the BPT diagram, making the origins confusing. The Swift UV continuum shows that the two blobs do not have UV counterparts, suggesting that the source of ionization is out of the blobs. Consistent with this, the detailed photoionization models prefer to AGN rather than star-forming origin with a significance of 5.8$σ$. The estimated BH mass is $M_{\rm BH}\sim$7.2$\times 10^5 M_\odot$ from the $M_{\rm BH}-σ_*$ relationship. This work introduces a novel method for detecting the light echo of BHs, potentially extending to intermediate mass, in low metallicity environments where the traditional BPT diagram fails.

astro-ph.GA↗

arXiv2024

Multifunctional Bistable Ultrathin Composite Booms with Flexible Electronics

Small satellites such as CubeSats pose demanding requirements on the weight, size, and multifunctionality of their structures due to extreme constraints on the payload mass and volume. To address this challenge, we introduce a concept of multifunctional deployable space structures for CubeSats based on ultrathin, elastically foldable, and self-deployable bistable composite structures integrated with flexible electronics. The multifunctional bistable booms can be stored in a coiled configuration and self-deploy into a long structure upon initiation by releasing the stored strain energy. The boom demonstrates the capabilities of delivering power and transmitting data from the CubeSat to the flexible devices on the boom tip. The boom also shows the ability to monitor the dynamics and vibration during and after the deployment. A payload boom has been installed in a 3U CubeSat as flight hardware for in-space testing and demonstration. This effort combines morphable ultrathin composite structures with flexible electronics.

physics.app-ph↗

arXiv2024

Deep Joint Denoising and Detection for Enhanced Intracellular Particle Analysis

Reliable analysis of intracellular dynamic processes in time-lapse fluorescence microscopy images requires complete and accurate tracking of all small particles in all time frames of the image sequences. A fundamental first step towards this goal is particle detection. Given the small size of the particles, their detection is greatly affected by image noise. Recent studies have shown that applying image denoising as a preprocessing step indeed improves particle detection and their subsequent tracking. Deep learning based particle detection methods have shown superior results compared to traditional detection methods. However, they do not explicitly aim to remove noise from the images to facilitate detection. Thus we hypothesize that their performance could be further improved. In this paper, we propose a new deep neural network, called DENODET (denoising-detection network), which performs image denoising and particle detection simultaneously. We show that integrative denoising and detection yields more accurate detection results. Our method achieves superior results compared to state-of-the-art particle detection methods on the particle tracking challenge dataset and our own real fluorescence microscopy image data.

eess.IV↗

arXiv2024

Relightable 3D Gaussians: Realistic Point Cloud Relighting with BRDF Decomposition and Ray Tracing

In this paper, we present a novel differentiable point-based rendering framework to achieve photo-realistic relighting. To make the reconstructed scene relightable, we enhance vanilla 3D Gaussians by associating extra properties, including normal vectors, BRDF parameters, and incident lighting from various directions. From a collection of multi-view images, the 3D scene is optimized through 3D Gaussian Splatting while BRDF and lighting are decomposed by physically based differentiable rendering. To produce plausible shadow effects in photo-realistic relighting, we introduce an innovative point-based ray tracing with the bounding volume hierarchies for efficient visibility pre-computation. Extensive experiments demonstrate our improved BRDF estimation, novel view synthesis and relighting results compared to state-of-the-art approaches. The proposed framework showcases the potential to revolutionize the mesh-based graphics pipeline with a point-based pipeline enabling editing, tracing, and relighting.

cs.CV↗

arXiv2024

Classical-quantum correspondence in the noise-based dissipative systems

We investigate the correspondence between classical noise and quantum environments. Although it has been known that the classical noise can be mapped to the quantum environments only for pure dephasing and infinite-temperature dissipation processes, we describe that this limitation can be circumvented by introducing auxiliary systems and conservation. Taking a two-level system as an example, we construct the so-called central spin model with its couplings fluctuating as the classical noise, and then acquire its statistical-average dynamics which captures the dissipations beyond the infinite temperature. By adjusting the number of the auxiliary systems and their initial states, the noise-based model reproduces both Markovian and non-Markovian evolutions. It is also found that different quantities of the two-level system are governed by different model parameters, indicating that the constructed model is an efficient simulator for specific observables, rather than an equivalent form of a realistic open system. In addition, the model is also applicable to investigate topical mechanisms of the open systems, e.g. negative temperatures and asymmetric equidistant quenches.

quant-ph↗

arXiv2024

Head360: Learning a Parametric 3D Full-Head for Free-View Synthesis in 360°

Creating a 360° parametric model of a human head is a very challenging task. While recent advancements have demonstrated the efficacy of leveraging synthetic data for building such parametric head models, their performance remains inadequate in crucial areas such as expression-driven animation, hairstyle editing, and text-based modifications. In this paper, we build a dataset of artist-designed high-fidelity human heads and propose to create a novel parametric 360° renderable parametric head model from it. Our scheme decouples the facial motion/shape and facial appearance, which are represented by a classic parametric 3D mesh model and an attached neural texture, respectively. We further propose a training method for decompositing hairstyle and facial appearance, allowing free-swapping of the hairstyle. A novel inversion fitting method is presented based on single image input with high generalization and fidelity. To the best of our knowledge, our model is the first parametric 3D full-head that achieves 360° free-view synthesis, image-based fitting, appearance editing, and animation within a single model. Experiments show that facial motions and appearances are well disentangled in the parametric space, leading to SOTA performance in rendering and animating quality. The code and SynHead100 dataset are released at https://nju-3dv.github.io/projects/Head360.

cs.CV↗

arXiv2024

EmoTalk3D: High-Fidelity Free-View Synthesis of Emotional 3D Talking Head

We present a novel approach for synthesizing 3D talking heads with controllable emotion, featuring enhanced lip synchronization and rendering quality. Despite significant progress in the field, prior methods still suffer from multi-view consistency and a lack of emotional expressiveness. To address these issues, we collect EmoTalk3D dataset with calibrated multi-view videos, emotional annotations, and per-frame 3D geometry. By training on the EmoTalk3D dataset, we propose a \textit{`Speech-to-Geometry-to-Appearance'} mapping framework that first predicts faithful 3D geometry sequence from the audio features, then the appearance of a 3D talking head represented by 4D Gaussians is synthesized from the predicted geometry. The appearance is further disentangled into canonical and dynamic Gaussians, learned from multi-view videos, and fused to render free-view talking head animation. Moreover, our model enables controllable emotion in the generated talking heads and can be rendered in wide-range views. Our method exhibits improved rendering quality and stability in lip motion generation while capturing dynamic facial details such as wrinkles and subtle expressions. Experiments demonstrate the effectiveness of our approach in generating high-fidelity and emotion-controllable 3D talking heads. The code and EmoTalk3D dataset are released at https://nju-3dv.github.io/projects/EmoTalk3D.

cs.CV↗

arXiv2024

Deterministic and Stochastic Accelerated Gradient Method for Convex Semi-Infinite Optimization

This paper explores numerical methods for solving a convex differentiable semi-infinite program. We introduce a primal-dual gradient method which performs three updates iteratively: a momentum gradient ascend step to update the constraint parameters, a momentum gradient ascend step to update the dual variables, and a gradient descend step to update the primal variables. Our approach also extends to scenarios where gradients and function values are accessible solely through stochastic oracles. This method extends the recent primal-dual methods, for example, Hamedani and Aybat (2021); Boob et al. (2022), for optimization with a finite number of constraints. We show the iteration complexity of the proposed method for finding an $ε$-optimal solution under different convexity and concavity assumptions on the functions.

math.OC↗

arXiv2024

Secure Key from Quantum Discord

The study of quantum information processing seeks to characterize the resources that enable quantum information processing to perform tasks that are unfeasible or inefficient for classical information processing. Quantum cryptography is one such task, and researchers have identified entanglement as a sufficient resource for secure key generation. However, quantum discord, another type of quantum correlation beyond entanglement, has been found to be necessary for guaranteeing secure communication due to its direct relation to information leakage. Despite this, it is a long-standing problem how to make use of discord to analyze security in a specific quantum cryptography protocol. Here, based on our proposed quantum discord witness recently, we successfully address this issue by considering a BB84-like quantum key distribution protocol and its equivalent entanglement-based version. Our method is robust against imperfections in qubit sources and qubit measurements as well as basis misalignment due to quantum channels, which results in a better key rate than standard BB84 protocol. Those advantages are experimentally demonstrated via photonic phase encoding systems, which shows the practicality of our results.

quant-ph↗

arXiv2024

Stereo Risk: A Continuous Modeling Approach to Stereo Matching

We introduce Stereo Risk, a new deep-learning approach to solve the classical stereo-matching problem in computer vision. As it is well-known that stereo matching boils down to a per-pixel disparity estimation problem, the popular state-of-the-art stereo-matching approaches widely rely on regressing the scene disparity values, yet via discretization of scene disparity values. Such discretization often fails to capture the nuanced, continuous nature of scene depth. Stereo Risk departs from the conventional discretization approach by formulating the scene disparity as an optimal solution to a continuous risk minimization problem, hence the name "stereo risk". We demonstrate that $L^1$ minimization of the proposed continuous risk function enhances stereo-matching performance for deep networks, particularly for disparities with multi-modal probability distributions. Furthermore, to enable the end-to-end network training of the non-differentiable $L^1$ risk optimization, we exploited the implicit function theorem, ensuring a fully differentiable network. A comprehensive analysis demonstrates our method's theoretical soundness and superior performance over the state-of-the-art methods across various benchmark datasets, including KITTI 2012, KITTI 2015, ETH3D, SceneFlow, and Middlebury 2014.

cs.CV↗

arXiv2024

Indications of superconductivities in blend of variant apatite and covellite

Through heavily doping sulfur into an apatite framework, we synthesize a new blend mainly comprising variant apatite and covellite (copper sulfide). Magnetic measurement exhibits that significant diamagnetism appears at around 260 K and drops dramatically below 30 K implying coexistence of two superconducting phases. The upper critical magnetic field is larger than 1000 Oe at 250 K. Electric measurement manifests that the current-voltage curves deviate from the normal linear lineshape suggesting the presence of zero-resistance effect, and the critical current is around 50 $μ$A at 140 K. These exotic magnetic and electric features strongly indicate these two components, variant apatite and covellite, individually trigger two superconducting phases at near-room and low temperatures.

cond-mat.supr-con↗

arXiv2024

Small scale creation for 2D free boundary Euler equations with surface tension

In this paper, we study the 2D free boundary incompressible Euler equations with surface tension, where the fluid domain is periodic in $x_1$, and has finite depth. We construct initial data with a flat free boundary and arbitrarily small velocity, such that the gradient of vorticity grows at least double-exponentially for all times during the lifespan of the associated solution. This work generalizes the celebrated result by Kiselev--{Š}ver{á}k to the free boundary setting. The free boundary introduces some major challenges in the proof due to the deformation of the fluid domain and the fact that the velocity field cannot be reconstructed from the vorticity using the Biot-Savart law. We overcome these issues by deriving uniform-in-time control on the free boundary and obtaining pointwise estimates on an approximate Biot-Savart law.

math.AP↗

arXiv2024

JEN-1 DreamStyler: Customized Musical Concept Learning via Pivotal Parameters Tuning

Large models for text-to-music generation have achieved significant progress, facilitating the creation of high-quality and varied musical compositions from provided text prompts. However, input text prompts may not precisely capture user requirements, particularly when the objective is to generate music that embodies a specific concept derived from a designated reference collection. In this paper, we propose a novel method for customized text-to-music generation, which can capture the concept from a two-minute reference music and generate a new piece of music conforming to the concept. We achieve this by fine-tuning a pretrained text-to-music model using the reference music. However, directly fine-tuning all parameters leads to overfitting issues. To address this problem, we propose a Pivotal Parameters Tuning method that enables the model to assimilate the new concept while preserving its original generative capabilities. Additionally, we identify a potential concept conflict when introducing multiple concepts into the pretrained model. We present a concept enhancement strategy to distinguish multiple concepts, enabling the fine-tuned model to generate music incorporating either individual or multiple concepts simultaneously. Since we are the first to work on the customized music generation task, we also introduce a new dataset and evaluation protocol for the new task. Our proposed Jen1-DreamStyler outperforms several baselines in both qualitative and quantitative evaluations. Demos will be available at https://www.jenmusic.ai/research#DreamStyler.

cs.SD↗

arXiv2024

Hallo: Hierarchical Audio-Driven Visual Synthesis for Portrait Image Animation

The field of portrait image animation, driven by speech audio input, has experienced significant advancements in the generation of realistic and dynamic portraits. This research delves into the complexities of synchronizing facial movements and creating visually appealing, temporally consistent animations within the framework of diffusion-based methodologies. Moving away from traditional paradigms that rely on parametric models for intermediate facial representations, our innovative approach embraces the end-to-end diffusion paradigm and introduces a hierarchical audio-driven visual synthesis module to enhance the precision of alignment between audio inputs and visual outputs, encompassing lip, expression, and pose motion. Our proposed network architecture seamlessly integrates diffusion-based generative models, a UNet-based denoiser, temporal alignment techniques, and a reference network. The proposed hierarchical audio-driven visual synthesis offers adaptive control over expression and pose diversity, enabling more effective personalization tailored to different identities. Through a comprehensive evaluation that incorporates both qualitative and quantitative analyses, our approach demonstrates obvious enhancements in image and video quality, lip synchronization precision, and motion diversity. Further visualization and access to the source code can be found at: https://fudan-generative-vision.github.io/hallo.

cs.CV↗

arXiv2024

Reconfigurable, Multifunctional Origami Electronic Membranes for Mechanical and Environmental Sensing

This work introduces a concept of origami electronic membranes that leverages the design and fabrication of flexible electronics and the mechanical behavior of engineering origami to achieve unique multifunctional, shape-reconfigurable, and adaptive membranes for mechanical and environmental sensing in benign and harsh conditions. This paper presents the materials, design, and fabrication methods for realizing six origami electronic membranes capable of reconfiguring planar or three-dimensional shapes based on the modified flasher, Kresling, Miura-ori, circular, letter, and Tachi-Miura origami patterns. These origami-based, thin-film flexible electronics can obtain both expansion and folding of their shapes, as well as transformation between different geometries. The origami electronic membranes can achieve mechanical and environmental sensing functions such as measuring motions, mechanical strains, temperatures, UV light, and humidity. The results reported here demonstrate the promise of combining engineering origami with flexible electronics to advance the state-of-the-art in multifunctional foldable and deployable electronics and systems.

physics.app-ph↗

arXiv2024

A Vision to Enhance Trust Requirements for Peer Support Systems by Revisiting Trust Theories

This vision paper focuses on the mental health crisis impacting healthcare workers (HCWs), which exacerbated by the COVID-19 pandemic, leads to increased stress and psychological issues like burnout. Peer Support Programs (PSP) are a recognized intervention for mitigating these issues. These programs are increasingly being delivered virtually through Peer Support Systems (PSS) for increased convenience and accessibility. However, HCWs perception of these systems results in fear of information sharing, perceived lack of safety, and low participation rate, which challenges these systems ability to achieve their goals. In line with the rich body of research on the requirements and properties of trustworthy systems, we posit that increasing HCWs trust in PSS could address these challenges. However, extant research focuses on objectively defined trustworthiness rather than perceptual trust because trustworthy requirements are viewed as more controllable and easier to operationalize. This study proposes a novel approach to elicit perceptual trust requirements by proposing a trust framework anchored in recognized trust theories from different disciplines that unpacks trust into its recognized types and their antecedents. This approach allows the identification of trust requirements beyond those already proposed for trustworthy systems, providing a strong foundation for improving the effectiveness of PSS for HCWs. Keywords: Trust Requirements, Requirements elicitation, Peer support systems, Healthcare workers

cs.CY↗

arXiv2024

Champ: Controllable and Consistent Human Image Animation with 3D Parametric Guidance

In this study, we introduce a methodology for human image animation by leveraging a 3D human parametric model within a latent diffusion framework to enhance shape alignment and motion guidance in curernt human generative techniques. The methodology utilizes the SMPL(Skinned Multi-Person Linear) model as the 3D human parametric model to establish a unified representation of body shape and pose. This facilitates the accurate capture of intricate human geometry and motion characteristics from source videos. Specifically, we incorporate rendered depth images, normal maps, and semantic maps obtained from SMPL sequences, alongside skeleton-based motion guidance, to enrich the conditions to the latent diffusion model with comprehensive 3D shape and detailed pose attributes. A multi-layer motion fusion module, integrating self-attention mechanisms, is employed to fuse the shape and motion latent representations in the spatial domain. By representing the 3D human parametric model as the motion guidance, we can perform parametric shape alignment of the human body between the reference image and the source video motion. Experimental evaluations conducted on benchmark datasets demonstrate the methodology's superior ability to generate high-quality human animations that accurately capture both pose and shape variations. Furthermore, our approach also exhibits superior generalization capabilities on the proposed in-the-wild dataset. Project page: https://fudan-generative-vision.github.io/champ.

cs.CV↗

arXiv2024

Direct3D: Scalable Image-to-3D Generation via 3D Latent Diffusion Transformer

Generating high-quality 3D assets from text and images has long been challenging, primarily due to the absence of scalable 3D representations capable of capturing intricate geometry distributions. In this work, we introduce Direct3D, a native 3D generative model scalable to in-the-wild input images, without requiring a multiview diffusion model or SDS optimization. Our approach comprises two primary components: a Direct 3D Variational Auto-Encoder (D3D-VAE) and a Direct 3D Diffusion Transformer (D3D-DiT). D3D-VAE efficiently encodes high-resolution 3D shapes into a compact and continuous latent triplane space. Notably, our method directly supervises the decoded geometry using a semi-continuous surface sampling strategy, diverging from previous methods relying on rendered images as supervision signals. D3D-DiT models the distribution of encoded 3D latents and is specifically designed to fuse positional information from the three feature maps of the triplane latent, enabling a native 3D generative model scalable to large-scale 3D datasets. Additionally, we introduce an innovative image-to-3D generation pipeline incorporating semantic and pixel-level image conditions, allowing the model to produce 3D shapes consistent with the provided conditional image input. Extensive experiments demonstrate the superiority of our large-scale pre-trained Direct3D over previous image-to-3D approaches, achieving significantly better generation quality and generalization ability, thus establishing a new state-of-the-art for 3D content creation. Project page: https://nju-3dv.github.io/projects/Direct3D/.

cs.CV↗