Searcharxiv⌕ Search

arXiv subjects

Jiawei Huang

Publications and source records attributed to Jiawei Huang.

At least 55 records · Page 3Linked to original sources
arXiv2024

Online Neural Path Guiding with Normalized Anisotropic Spherical Gaussians

The variance reduction speed of physically-based rendering is heavily affected by the adopted importance sampling technique. In this paper we propose a novel online framework to learn the spatial-varying density model with a single small neural network using stochastic ray samples. To achieve this task, we propose a novel closed-form density model called the normalized anisotropic spherical gaussian mixture, that can express complex irradiance fields with a small number of parameters. Our framework learns the distribution in a progressive manner and does not need any warm-up phases. Due to the compact and expressive representation of our density model, our framework can be implemented entirely on the GPU, allowing it produce high quality images with limited computational resources.

cs.CV↗
arXiv2024

Cas-DiffCom: Cascaded diffusion model for infant longitudinal super-resolution 3D medical image completion

Early infancy is a rapid and dynamic neurodevelopmental period for behavior and neurocognition. Longitudinal magnetic resonance imaging (MRI) is an effective tool to investigate such a crucial stage by capturing the developmental trajectories of the brain structures. However, longitudinal MRI acquisition always meets a serious data-missing problem due to participant dropout and failed scans, making longitudinal infant brain atlas construction and developmental trajectory delineation quite challenging. Thanks to the development of an AI-based generative model, neuroimage completion has become a powerful technique to retain as much available data as possible. However, current image completion methods usually suffer from inconsistency within each individual subject in the time dimension, compromising the overall quality. To solve this problem, our paper proposed a two-stage cascaded diffusion model, Cas-DiffCom, for dense and longitudinal 3D infant brain MRI completion and super-resolution. We applied our proposed method to the Baby Connectome Project (BCP) dataset. The experiment results validate that Cas-DiffCom achieves both individual consistency and high fidelity in longitudinal infant brain image completion. We further applied the generated infant brain images to two downstream tasks, brain tissue segmentation and developmental trajectory delineation, to declare its task-oriented potential in the neuroscience field.

eess.IV↗
arXiv2023

On Robust Wasserstein Barycenter: The Model and Algorithm

The Wasserstein barycenter problem is to compute the average of $m$ given probability measures, which has been widely studied in many different areas; however, real-world data sets are often noisy and huge, which impedes its applications in practice. Hence, in this paper, we focus on improving the computational efficiency of two types of robust Wasserstein barycenter problem (RWB): fixed-support RWB (fixed-RWB) and free-support RWB (free-RWB); actually, the former is a subroutine of the latter. Firstly, we improve efficiency through model reducing; we reduce RWB as an augmented Wasserstein barycenter problem, which works for both fixed-RWB and free-RWB. Especially, fixed-RWB can be computed within $\widetilde{O}(\frac{mn^2}{ε_+})$ time by using an off-the-shelf solver, where $ε_+$ is the pre-specified additive error and $n$ is the size of locations of input measures. Then, for free-RWB, we leverage a quality guaranteed data compression technique, coreset, to accelerate computation by reducing the data set size $m$. It shows that running algorithms on the coreset is enough instead of on the original data set. Next, by combining the model reducing and coreset techniques above, we propose an algorithm for free-RWB by updating the weights and locations alternatively. Finally, our experiments demonstrate the efficiency of our techniques.

cs.LG↗
arXiv2023

Competing charge transfer and screening effects in two-dimensional ferroelectric capacitors

Two-dimensional (2D) ferroelectrics offer the potential for ultrathin flexible nanoelectronics, typically utilizing a metal-ferroelectric-metal sandwich structure as the functional unit. Electrodes can either contribute free carriers to screen the depolarization field, enhancing nanoscale ferroelectricity, or they can induce charge doping, disrupting the long-range crystalline order. Here, we explore the dual roles of electrodes in 2D ferroelectric capacitors, supported by extensive first-principles calculations covering a range of electrode work functions. Our results reveal volcano-type relationships between ferroelectric-electrode binding affinity and work function, which are further unified by a quadratic scaling between the binding energy and the transferred interfacial charge. At the monolayer limit, the charge transfer dictates the ferroelectric stability and switching properties. This is a general characteristic confirmed in various 2D ferroelectrics including $α$-In$_2$Se$_3$, CuInP$_2$S$_6$, and SnTe. As the ferroelectric layer's thickness increases, the stability of the capacitor evolves from a charge transfer-dominated to a screening-dominated state. The delicate interplay between these two effects will have important implications for the applications of 2D ferroelectric capacitors.

cond-mat.mtrl-sci↗
arXiv2023

Is Simple Uniform Sampling Effective for Center-Based Clustering with Outliers: When and Why?

Real-world datasets often contain outliers, and the presence of outliers can make the clustering problems to be much more challenging. In this paper, we propose a simple uniform sampling framework for solving three representative center-based clustering with outliers problems: $k$-center/median/means clustering with outliers. Our analysis is fundamentally different from the previous (uniform and non-uniform) sampling based ideas. To explain the effectiveness of uniform sampling in theory, we introduce a measure of "significance" and prove that the performance of our framework depends on the significance degree of the given instance. In particular, the sample size can be independent of the input data size $n$ and the dimensionality $d$, if we assume the given instance is "significant", which is in fact a fairly reasonable assumption in practice. Due to its simplicity, the uniform sampling approach also enjoys several significant advantages over the non-uniform sampling approaches in practice. To the best of our knowledge, this is the first work that systematically studies the effectiveness of uniform sampling from both theoretical and experimental aspects.

cs.LG↗
arXiv2023

Make-An-Audio 2: Temporal-Enhanced Text-to-Audio Generation

Large diffusion models have been successful in text-to-audio (T2A) synthesis tasks, but they often suffer from common issues such as semantic misalignment and poor temporal consistency due to limited natural language understanding and data scarcity. Additionally, 2D spatial structures widely used in T2A works lead to unsatisfactory audio quality when generating variable-length audio samples since they do not adequately prioritize temporal information. To address these challenges, we propose Make-an-Audio 2, a latent diffusion-based T2A method that builds on the success of Make-an-Audio. Our approach includes several techniques to improve semantic alignment and temporal consistency: Firstly, we use pre-trained large language models (LLMs) to parse the text into structured pairs for better temporal information capture. We also introduce another structured-text encoder to aid in learning semantic alignment during the diffusion denoising process. To improve the performance of variable length generation and enhance the temporal information extraction, we design a feed-forward Transformer-based diffusion denoiser. Finally, we use LLMs to augment and transform a large amount of audio-label data into audio-text datasets to alleviate the problem of scarcity of temporal data. Extensive experiments show that our method outperforms baseline models in both objective and subjective metrics, and achieves significant gains in temporal information understanding, semantic consistency, and sound quality.

cs.SD↗
arXiv2023

Coresets for Wasserstein Distributionally Robust Optimization Problems

Wasserstein distributionally robust optimization (\textsf{WDRO}) is a popular model to enhance the robustness of machine learning with ambiguous data. However, the complexity of \textsf{WDRO} can be prohibitive in practice since solving its ``minimax'' formulation requires a great amount of computation. Recently, several fast \textsf{WDRO} training algorithms for some specific machine learning tasks (e.g., logistic regression) have been developed. However, the research on designing efficient algorithms for general large-scale \textsf{WDRO}s is still quite limited, to the best of our knowledge. \textit{Coreset} is an important tool for compressing large dataset, and thus it has been widely applied to reduce the computational complexities for many optimization problems. In this paper, we introduce a unified framework to construct the $ε$-coreset for the general \textsf{WDRO} problems. Though it is challenging to obtain a conventional coreset for \textsf{WDRO} due to the uncertainty issue of ambiguous data, we show that we can compute a ``dual coreset'' by using the strong duality property of \textsf{WDRO}. Also, the error introduced by the dual coreset can be theoretically guaranteed for the original \textsf{WDRO} objective. To construct the dual coreset, we propose a novel grid sampling approach that is particularly suitable for the dual formulation of \textsf{WDRO}. Finally, we implement our coreset approach and illustrate its effectiveness for several \textsf{WDRO} problems in the experiments.

cs.LG↗
arXiv2023

GeneFace++: Generalized and Stable Real-Time Audio-Driven 3D Talking Face Generation

Generating talking person portraits with arbitrary speech audio is a crucial problem in the field of digital human and metaverse. A modern talking face generation method is expected to achieve the goals of generalized audio-lip synchronization, good video quality, and high system efficiency. Recently, neural radiance field (NeRF) has become a popular rendering technique in this field since it could achieve high-fidelity and 3D-consistent talking face generation with a few-minute-long training video. However, there still exist several challenges for NeRF-based methods: 1) as for the lip synchronization, it is hard to generate a long facial motion sequence of high temporal consistency and audio-lip accuracy; 2) as for the video quality, due to the limited data used to train the renderer, it is vulnerable to out-of-domain input condition and produce bad rendering results occasionally; 3) as for the system efficiency, the slow training and inference speed of the vanilla NeRF severely obstruct its usage in real-world applications. In this paper, we propose GeneFace++ to handle these challenges by 1) utilizing the pitch contour as an auxiliary feature and introducing a temporal loss in the facial motion prediction process; 2) proposing a landmark locally linear embedding method to regulate the outliers in the predicted motion sequence to avoid robustness issues; 3) designing a computationally efficient NeRF-based motion-to-video renderer to achieves fast training and real-time inference. With these settings, GeneFace++ becomes the first NeRF-based method that achieves stable and real-time talking face generation with generalized audio-lip synchronization. Extensive experiments show that our method outperforms state-of-the-art baselines in terms of subjective and objective evaluation. Video samples are available at https://genefaceplusplus.github.io .

cs.CV↗
arXiv2023

AudioGPT: Understanding and Generating Speech, Music, Sound, and Talking Head

Large language models (LLMs) have exhibited remarkable capabilities across a variety of domains and tasks, challenging our understanding of learning and cognition. Despite the recent success, current LLMs are not capable of processing complex audio information or conducting spoken conversations (like Siri or Alexa). In this work, we propose a multi-modal AI system named AudioGPT, which complements LLMs (i.e., ChatGPT) with 1) foundation models to process complex audio information and solve numerous understanding and generation tasks; and 2) the input/output interface (ASR, TTS) to support spoken dialogue. With an increasing demand to evaluate multi-modal LLMs of human intention understanding and cooperation with foundation models, we outline the principles and processes and test AudioGPT in terms of consistency, capability, and robustness. Experimental results demonstrate the capabilities of AudioGPT in solving AI tasks with speech, music, sound, and talking head understanding and generation in multi-round dialogues, which empower humans to create rich and diverse audio content with unprecedented ease. Our system is publicly available at \url{https://github.com/AIGC-Audio/AudioGPT}.

cs.CL↗
arXiv2023

Company Competition Graph

Financial market participants frequently rely on numerous business relationships to make investment decisions. Investors can learn about potential risks and opportunities associated with other connected entities through these corporate connections. Nonetheless, human annotation of a large corpus to extract such relationships is highly time-consuming, not to mention that it requires a considerable amount of industry expertise and professional training. Meanwhile, we have yet to observe means to generate reliable knowledge graphs of corporate relationships due to the lack of impartial and granular data sources. This study proposes a system to process financial reports and construct the public competitor graph to fill the void. Our method can retrieve more than 83\% competition relationship of the S\&P 500 index companies. Based on the output from our system, we construct a knowledge graph with more than 700 nodes and 1200 edges. A demo interactive graph interface is available.

q-fin.GN↗
arXiv2023

Structure-aware registration network for liver DCE-CT images

Image registration of liver dynamic contrast-enhanced computed tomography (DCE-CT) is crucial for diagnosis and image-guided surgical planning of liver cancer. However, intensity variations due to the flow of contrast agents combined with complex spatial motion induced by respiration brings great challenge to existing intensity-based registration methods. To address these problems, we propose a novel structure-aware registration method by incorporating structural information of related organs with segmentation-guided deep registration network. Existing segmentation-guided registration methods only focus on volumetric registration inside the paired organ segmentations, ignoring the inherent attributes of their anatomical structures. In addition, such paired organ segmentations are not always available in DCE-CT images due to the flow of contrast agents. Different from existing segmentation-guided registration methods, our proposed method extracts structural information in hierarchical geometric perspectives of line and surface. Then, according to the extracted structural information, structure-aware constraints are constructed and imposed on the forward and backward deformation field simultaneously. In this way, all available organ segmentations, including unpaired ones, can be fully utilized to avoid the side effect of contrast agent and preserve the topology of organs during registration. Extensive experiments on an in-house liver DCE-CT dataset and a public LiTS dataset show that our proposed method can achieve higher registration accuracy and preserve anatomical structure more effectively than state-of-the-art methods.

eess.IV↗
arXiv2023

Semiconducting nonperovskite ferroelectric oxynitride designed ab initio

Recent discovery of HfO2-based and nitride-based ferroelectrics that are compatible to the semiconductor manufacturing process have revitalized the field of ferroelectric-based nanoelectronics. Guided by a simple design principle of charge compensation and density functional theory calculations, we discover HfO2-like mixed-anion materials, TaON and NbON, can crystallize in the polar Pca21 phase with a strong thermodynamic driving force to adopt anion ordering spontaneously. Both oxynitrides possess large remnant polarization, low switching barriers, and unconventional negative piezoelectric effect, making them promising piezoelectrics and ferroelectrics. Distinct from HfO2 that has a wide band gap, both TaON and NbON can absorb visible light and have high charge carrier mobilities, suitable for ferroelectric photovoltaic and photocatalytic applications. This new class of multifunctional nonperovskite oxynitride containing economical and environmentally benign elements offer a platform to design and optimize high-performing ferroelectric semiconductors for integrated systems.

cond-mat.mtrl-sci↗
arXiv2023

Tiered Reinforcement Learning: Pessimism in the Face of Uncertainty and Constant Regret

We propose a new learning framework that captures the tiered structure of many real-world user-interaction applications, where the users can be divided into two groups based on their different tolerance on exploration risks and should be treated separately. In this setting, we simultaneously maintain two policies $π^{\text{O}}$ and $π^{\text{E}}$: $π^{\text{O}}$ ("O" for "online") interacts with more risk-tolerant users from the first tier and minimizes regret by balancing exploration and exploitation as usual, while $π^{\text{E}}$ ("E" for "exploit") exclusively focuses on exploitation for risk-averse users from the second tier utilizing the data collected so far. An important question is whether such a separation yields advantages over the standard online setting (i.e., $π^{\text{E}}=π^{\text{O}}$) for the risk-averse users. We individually consider the gap-independent vs.~gap-dependent settings. For the former, we prove that the separation is indeed not beneficial from a minimax perspective. For the latter, we show that if choosing Pessimistic Value Iteration as the exploitation algorithm to produce $π^{\text{E}}$, we can achieve a constant regret for risk-averse users independent of the number of episodes $K$, which is in sharp contrast to the $Ω(\log K)$ regret for any online RL algorithms in the same setting, while the regret of $π^{\text{O}}$ (almost) maintains its online regret optimality and does not need to compromise for the success of $π^{\text{E}}$.

cs.LG↗
arXiv2023

Make-An-Audio: Text-To-Audio Generation with Prompt-Enhanced Diffusion Models

Large-scale multimodal generative modeling has created milestones in text-to-image and text-to-video generation. Its application to audio still lags behind for two main reasons: the lack of large-scale datasets with high-quality text-audio pairs, and the complexity of modeling long continuous audio data. In this work, we propose Make-An-Audio with a prompt-enhanced diffusion model that addresses these gaps by 1) introducing pseudo prompt enhancement with a distill-then-reprogram approach, it alleviates data scarcity with orders of magnitude concept compositions by using language-free audios; 2) leveraging spectrogram autoencoder to predict the self-supervised audio representation instead of waveforms. Together with robust contrastive language-audio pretraining (CLAP) representations, Make-An-Audio achieves state-of-the-art results in both objective and subjective benchmark evaluation. Moreover, we present its controllability and generalization for X-to-Audio with "No Modality Left Behind", for the first time unlocking the ability to generate high-definition, high-fidelity audios given a user-defined modality input. Audio samples are available at https://Text-to-Audio.github.io

cs.SD↗
arXiv2022

One Dimensional Ferroelectric Nanothreads with Axial and Radial Polarization

Long-range ferroelectric crystalline order usually fades away as the spatial dimension decreases, hence there are few two-dimensional (2D) ferroelectrics and far fewer one-dimensional (1D) ferroelectrics. Due to the depolarization field, low-dimensional ferroelectrics rarely possess the polarization along the direction of reduced dimensionality. Here, using first-principles density functional theory, we explore the structural evolution of nanoribbons of varying widths constructed by cutting the 2D sheet of ferroelectric $α$-III$_2$IV$_3$ (III=Al, Ga, In; VI=S, Se, Te). We discover a one-dimensional ferroelectric nanothread (1DFENT) of ultrasmall diameter with both axial and radial polarization, potentially enabling ultra-dense data storage with a 1D domain of just three unit cells being the functional unit. The polarization in 1DFENT of Ga$_2$Se$_3$ exhibits an unusual piezoelectric response: a stretching stress along the axial direction will increase both axial and radial polarization, referred to as auxetic piezoelectric effect. Utilizing the intrinsically flat electronic bands, we demonstrate the coexistence of ferroelectricity and ferromagnetism in 1DFENT and a counterintuitive charge-doping-induced metal-to-insulator transition. The 1DFENT with both axial and radial polarization offers a counterexample to the Mermin--Wagner theorem in 1D and suggests a new platform for the design of ultrahigh-density memory and the exploration of exotic states of matter.

cond-mat.mtrl-sci↗
arXiv2022

A Novel Sequential Coreset Method for Gradient Descent Algorithms

A wide range of optimization problems arising in machine learning can be solved by gradient descent algorithms, and a central question in this area is how to efficiently compress a large-scale dataset so as to reduce the computational complexity. {\em Coreset} is a popular data compression technique that has been extensively studied before. However, most of existing coreset methods are problem-dependent and cannot be used as a general tool for a broader range of applications. A key obstacle is that they often rely on the pseudo-dimension and total sensitivity bound that can be very high or hard to obtain. In this paper, based on the ''locality'' property of gradient descent algorithms, we propose a new framework, termed ''sequential coreset'', which effectively avoids these obstacles. Moreover, our method is particularly suitable for sparse optimization whence the coreset size can be further reduced to be only poly-logarithmically dependent on the dimension. In practice, the experimental results suggest that our method can save a large amount of running time compared with the baseline algorithms.

cs.LG↗
arXiv2022

Towards Deployment-Efficient Reinforcement Learning: Lower Bound and Optimality

Deployment efficiency is an important criterion for many real-world applications of reinforcement learning (RL). Despite the community's increasing interest, there lacks a formal theoretical formulation for the problem. In this paper, we propose such a formulation for deployment-efficient RL (DE-RL) from an "optimization with constraints" perspective: we are interested in exploring an MDP and obtaining a near-optimal policy within minimal \emph{deployment complexity}, whereas in each deployment the policy can sample a large batch of data. Using finite-horizon linear MDPs as a concrete structural model, we reveal the fundamental limit in achieving deployment efficiency by establishing information-theoretic lower bounds, and provide algorithms that achieve the optimal deployment efficiency. Moreover, our formulation for DE-RL is flexible and can serve as a building block for other practically relevant settings; we give "Safe DE-RL" and "Sample-Efficient DE-RL" as two examples, which may be worth future investigation.

cs.LG↗
arXiv2022

Origin of ferroelectricity in magnesium doped zinc oxide

Recent experiments demonstrated robust ferroelectricity in Mg-doped ZnO (ZMO) of the wurtzite structure, hinting at a promising strategy to substantially expand the list of ferroelectrics by doping conventional piezoelectrics. We investigate the origin of ferroelectricity in ZMO with first-principles density functional theory (DFT). The general argument that the Mg alloying could soften the ionic potential energy surface of ZMO for polarization reversal is overly simplified. Our DFT calculations reveal that even at a high Mg concentration, the energy difference ($ΔU$) between the polar and nonpolar phases remains prohibitively large for ZMO systems when the strain is fixed to the polar phase. Interestingly, the magnitude of $ΔU$ becomes substantially smaller when the strain relaxation is allowed, approaching the value of typical perovskite ferroelectrics such as PbTiO$_3$ with increasing Mg doping concentrations. The enabled switchability of ZMO systems is attributed to a hexagonal phase of MgO that is much lower in energy than its wurtzite counterpart. Detailed orbital and bonding analysis supports that the intra-atomic $3d_{z^2}$-$4p_z$ orbital self-mixing of Zn plays an important role in stabilizing the polar wurtzite phase, the lack of which is responsible for the low-energy nonpolar hexagonal phase of MgO.

cond-mat.mtrl-sci↗