arXiv subjects

Xiaoyang Huang

Publications and source records attributed to Xiaoyang Huang.

36 records · Page 2Linked to original sources

arXiv2023

Frequency-Modulated Point Cloud Rendering with Easy Editing

We develop an effective point cloud rendering pipeline for novel view synthesis, which enables high fidelity local detail reconstruction, real-time rendering and user-friendly editing. In the heart of our pipeline is an adaptive frequency modulation module called Adaptive Frequency Net (AFNet), which utilizes a hypernetwork to learn the local texture frequency encoding that is consecutively injected into adaptive frequency activation layers to modulate the implicit radiance signal. This mechanism improves the frequency expressive ability of the network with richer frequency basis support, only at a small computational budget. To further boost performance, a preprocessing module is also proposed for point cloud geometry optimization via point opacity estimation. In contrast to implicit rendering, our pipeline supports high-fidelity interactive editing based on point cloud manipulation. Extensive experimental results on NeRF-Synthetic, ScanNet, DTU and Tanks and Temples datasets demonstrate the superior performances achieved by our method in terms of PSNR, SSIM and LPIPS, in comparison to the state-of-the-art.

cs.CV↗

arXiv2023

AudioEar: Single-View Ear Reconstruction for Personalized Spatial Audio

Spatial audio, which focuses on immersive 3D sound rendering, is widely applied in the acoustic industry. One of the key problems of current spatial audio rendering methods is the lack of personalization based on different anatomies of individuals, which is essential to produce accurate sound source positions. In this work, we address this problem from an interdisciplinary perspective. The rendering of spatial audio is strongly correlated with the 3D shape of human bodies, particularly ears. To this end, we propose to achieve personalized spatial audio by reconstructing 3D human ears with single-view images. First, to benchmark the ear reconstruction task, we introduce AudioEar3D, a high-quality 3D ear dataset consisting of 112 point cloud ear scans with RGB images. To self-supervisedly train a reconstruction model, we further collect a 2D ear dataset composed of 2,000 images, each one with manual annotation of occlusion and 55 landmarks, named AudioEar2D. To our knowledge, both datasets have the largest scale and best quality of their kinds for public use. Further, we propose AudioEarM, a reconstruction method guided by a depth estimation network that is trained on synthetic data, with two loss functions tailored for ear data. Lastly, to fill the gap between the vision and acoustics community, we develop a pipeline to integrate the reconstructed ear mesh with an off-the-shelf 3D human body and simulate a personalized Head-Related Transfer Function (HRTF), which is the core of spatial audio rendering. Code and data are publicly available at https://github.com/seanywang0408/AudioEar.

cs.CV↗

arXiv2022

Boosting Point Clouds Rendering via Radiance Mapping

Recent years we have witnessed rapid development in NeRF-based image rendering due to its high quality. However, point clouds rendering is somehow less explored. Compared to NeRF-based rendering which suffers from dense spatial sampling, point clouds rendering is naturally less computation intensive, which enables its deployment in mobile computing device. In this work, we focus on boosting the image quality of point clouds rendering with a compact model design. We first analyze the adaption of the volume rendering formulation on point clouds. Based on the analysis, we simplify the NeRF representation to a spatial mapping function which only requires single evaluation per pixel. Further, motivated by ray marching, we rectify the the noisy raw point clouds to the estimated intersection between rays and surfaces as queried coordinates, which could avoid \textit{spatial frequency collapse} and neighbor point disturbance. Composed of rasterization, spatial mapping and the refinement stages, our method achieves the state-of-the-art performance on point clouds rendering, outperforming prior works by notable margins, with a smaller model size. We obtain a PSNR of 31.74 on NeRF-Synthetic, 25.88 on ScanNet and 30.81 on DTU. Code and data are publicly available at https://github.com/seanywang0408/RadianceMapping.

cs.CV↗

arXiv2022

Hydrodynamics with helical symmetry

We present the hydrodynamics of fluids in three spatial dimensions with helical symmetry, wherein only a linear combination of a rotation and translation is conserved in one of the three directions. The hydrodynamic degrees of freedom consist of scalar densities (e.g. energy or charge) along with two velocity fields transverse to the helical axis when the corresponding momenta are conserved. Nondissipative hydrodynamic coefficients reminiscent of chiral vortical coefficients arise. We write down microscopic Hamiltonian dynamical systems exhibiting helical symmetry, and we demonstrate using kinetic theory that these systems will generically exhibit the new helical phenomena that we predicted within hydrodynamics. We also confirm our findings using modern effective field theory techniques for hydrodynamics. We postulate regimes where pinned cholesteric liquid crystals may possess transport coefficients of a helical fluid, which appear to have been overlooked in previous literature.

cond-mat.stat-mech↗

arXiv2022

Fingerprints of quantum criticality in locally resolved transport

Understanding electrical transport in strange metals, including the seeming universality of Planckian $T$-linear resistivity, remains a longstanding challenge in condensed matter physics. We propose that local imaging techniques, such as nitrogen vacancy center magnetometry, can locally identify signatures of quantum critical response which are invisible in measurements of a bulk electrical resistivity. As an illustrative example, we use a minimal holographic model for a strange metal in two spatial dimensions to predict how electrical current will flow in regimes dominated by quantum critical dynamics on the Planckian length scale. We describe the crossover between quantum critical transport and hydrodynamic transport (including Ohmic regimes), both in charge neutral and finite density systems. We compare our holographic predictions to experiments on charge neutral graphene, finding quantitative agreement with available data; we suggest further experiments which may determine the relevance of our framework to transport on Planckian scales in this material. More broadly, we propose that locally imaged transport be used to test the universality (or lack thereof) of microscopic dynamics in the diverse set of quantum materials exhibiting $T$-linear resistivity.

cond-mat.str-el↗

arXiv2022

Representation-Agnostic Shape Fields

3D shape analysis has been widely explored in the era of deep learning. Numerous models have been developed for various 3D data representation formats, e.g., MeshCNN for meshes, PointNet for point clouds and VoxNet for voxels. In this study, we present Representation-Agnostic Shape Fields (RASF), a generalizable and computation-efficient shape embedding module for 3D deep learning. RASF is implemented with a learnable 3D grid with multiple channels to store local geometry. Based on RASF, shape embeddings for various 3D shape representations (point clouds, meshes and voxels) are retrieved by coordinate indexing. While there are multiple ways to optimize the learnable parameters of RASF, we provide two effective schemes among all in this paper for RASF pre-training: shape reconstruction and normal estimation. Once trained, RASF becomes a plug-and-play performance booster with negligible cost. Extensive experiments on diverse 3D representation formats, networks and applications, validate the universal effectiveness of the proposed RASF. Code and pre-trained models are publicly available https://github.com/seanywang0408/RASF

cs.CV↗

arXiv2022

Hydrodynamic effective field theories with discrete rotational symmetry

We develop a hydrodynamic effective field theory on the Schwinger-Keldysh contour for fluids with charge, energy, and momentum conservation, but only discrete rotational symmetry. The consequences of anisotropy on thermodynamics and first-order dissipative hydrodynamics are detailed in some simple examples in two spatial dimensions, but our construction extends to any spatial dimension and any rotation group (discrete or continuous). We find many possible terms in the equations of motion which are compatible with the existence of an entropy current, but not with the ability to couple the fluid to background gauge fields and vielbein.

hep-th↗

arXiv2021

Electron-phonon hydrodynamics

We develop the theory of hydrodynamics of an isotropic Fermi liquid of electrons coupled to isotropic acoustic phonons, assuming that umklapp processes may be neglected. At low temperatures, the fluid is approximately Galilean invariant; at high temperatures, the fluid is nearly relativistic; at intermediate temperatures, there are seven additional temperature regimes with unconventional thermodynamic properties and hydrodynamic transport coefficients in a three-dimensional system. We predict qualitative signatures of electron-phonon fluids in incoherent transport coefficients, shear and Hall viscosity, and plasmon dispersion relations. Our theory may be relevant for numerous quantum materials where strong electron-phonon scattering has been proposed to underlie a hydrodynamic regime, including $\mathrm{WTe}_2$, $\mathrm{WP}_2$, and $\mathrm{PtSn}_4$.

cond-mat.str-el↗

arXiv2021

Deep reinforcement learning for quantum Hamiltonian engineering

Engineering desired Hamiltonian in quantum many-body systems is essential for applications such as quantum simulation, computation and sensing. Conventional quantum Hamiltonian engineering sequences are designed using human intuition based on perturbation theory, which may not describe the optimal solution and is unable to accommodate complex experimental imperfections. Here we numerically search for Hamiltonian engineering sequences using deep reinforcement learning (DRL) techniques and experimentally demonstrate that they outperform celebrated sequences on a solid-state nuclear magnetic resonance quantum simulator. As an example, we aim at decoupling strongly-interacting spin-1/2 systems. We train DRL agents in the presence of different experimental imperfections and verify robustness of the output sequences both in simulations and experiments. Surprisingly, many of the learned sequences exhibit a common pattern that had not been discovered before, to our knowledge, but has an meaningful analytical description. We can thus restrict the searching space based on this control pattern, allowing to search for longer sequences, ultimately leading to sequences that are robust against dominant imperfections in our experiments. Our results not only demonstrate a general method for quantum Hamiltonian engineering, but also highlight the importance of combining black-box artificial intelligence with understanding of physical system in order to realize experimentally feasible applications.

quant-ph↗

arXiv2021

Reinventing 2D Convolutions for 3D Images

There have been considerable debates over 2D and 3D representation learning on 3D medical images. 2D approaches could benefit from large-scale 2D pretraining, whereas they are generally weak in capturing large 3D contexts. 3D approaches are natively strong in 3D contexts, however few publicly available 3D medical dataset is large and diverse enough for universal 3D pretraining. Even for hybrid (2D + 3D) approaches, the intrinsic disadvantages within the 2D / 3D parts still exist. In this study, we bridge the gap between 2D and 3D convolutions by reinventing the 2D convolutions. We propose ACS (axial-coronal-sagittal) convolutions to perform natively 3D representation learning, while utilizing the pretrained weights on 2D datasets. In ACS convolutions, 2D convolution kernels are split by channel into three parts, and convoluted separately on the three views (axial, coronal and sagittal) of 3D representations. Theoretically, ANY 2D CNN (ResNet, DenseNet, or DeepLab) is able to be converted into a 3D ACS CNN, with pretrained weight of a same parameter size. Extensive experiments on several medical benchmarks (including classification, segmentation and detection tasks) validate the consistent superiority of the pretrained ACS CNNs, over the 2D / 3D CNN counterparts with / without pretraining. Even without pretraining, the ACS convolution can be used as a plug-and-play replacement of standard 3D convolution, with smaller model size and less computation.

eess.IV↗

arXiv2020

Learning Black-Box Attackers with Transferable Priors and Query Feedback

This paper addresses the challenging black-box adversarial attack problem, where only classification confidence of a victim model is available. Inspired by consistency of visual saliency between different vision models, a surrogate model is expected to improve the attack performance via transferability. By combining transferability-based and query-based black-box attack, we propose a surprisingly simple baseline approach (named SimBA++) using the surrogate model, which significantly outperforms several state-of-the-art methods. Moreover, to efficiently utilize the query feedback, we update the surrogate model in a novel learning scheme, named High-Order Gradient Approximation (HOGA). By constructing a high-order gradient computation graph, we update the surrogate model to approximate the victim model in both forward and backward pass. The SimBA++ and HOGA result in Learnable Black-Box Attack (LeBA), which surpasses previous state of the art by considerable margins: the proposed LeBA significantly reduces queries, while keeping higher attack success rates close to 100% in extensive ImageNet experiments, including attacking vision benchmarks and defensive models. Code is open source at https://github.com/TrustworthyDL/LeBA.

cs.CR↗

arXiv2020

AlignShift: Bridging the Gap of Imaging Thickness in 3D Anisotropic Volumes

This paper addresses a fundamental challenge in 3D medical image processing: how to deal with imaging thickness. For anisotropic medical volumes, there is a significant performance gap between thin-slice (mostly 1mm) and thick-slice (mostly 5mm) volumes. Prior arts tend to use 3D approaches for the thin-slice and 2D approaches for the thick-slice, respectively. We aim at a unified approach for both thin- and thick-slice medical volumes. Inspired by recent advances in video analysis, we propose AlignShift, a novel parameter-free operator to convert theoretically any 2D pretrained network into thickness-aware 3D network. Remarkably, the converted networks behave like 3D for the thin-slice, nevertheless degenerate to 2D for the thick-slice adaptively. The unified thickness-aware representation learning is achieved by shifting and fusing aligned "virtual slices" as per the input imaging thickness. Extensive experiments on public large-scale DeepLesion benchmark, consisting of 32K lesions for universal lesion detection, validate the effectiveness of our method, which outperforms previous state of the art by considerable margins without whistles and bells. More importantly, to our knowledge, this is the first method that bridges the performance gap between thin- and thick-slice volumes by a unified framework. To improve research reproducibility, our code in PyTorch is open source at https://github.com/M3DV/AlignShift.

cs.CV↗

arXiv2020

Observation of Floquet prethermalization in dipolar spin chains

Periodically driven Floquet quantum systems provide a promising platform to investigate novel physics out of equilibrium. Unfortunately, the drive generically heats up the system to a featureless infinite temperature state. For large driving frequency, the heat absorption rate is predicted to be exponentially small, giving rise to a long-lived prethermal regime which exhibits all the intriguing properties of Floquet systems. Here we experimentally observe Floquet prethermalization using nuclear magnetic resonance techniques. We first show the relaxation of a far-from-equilibrium initial state to a long-lived prethermal state, well described by the time-independent ''prethermal'' Hamiltonian. By measuring the autocorrelation of this prethermal Hamiltonian we can further experimentally confirm the predicted exponentially slow heating rate. More strikingly, we find that in the timescale when the effective Hamiltonian picture breaks down, the Floquet system still possesses other quasi-conservation laws. Our results demonstrate that it is possible to realize robust Floquet engineering, thus enabling the experimental observation of non-trivial Floquet phases of matter.

quant-ph↗

arXiv2020

Prethermal quasiconserved observables in Floquet quantum systems

Prethermalization, by introducing emergent quasiconserved observables, plays a crucial role in protecting Floquet many-body phases over exponentially long time, while the ultimate fate of such quasiconserved operators can signal thermalization to infinite temperature. To elucidate the properties of prethermal quasiconservation in many-body Floquet systems, here we systematically analyze infinite temperature correlations between observables. We numerically show that the late-time behavior of the autocorrelations unambiguously distinguishes quasiconserved observables from non-conserved ones, allowing to single out a set of linearly-independent quasiconserved observables. By investigating two Floquet spin models, we identify two different mechanism underlying the quasi-conservation law. First, we numerically verify energy quasiconservation when the driving frequency is large, so that the system dynamics is approximately described by a static prethermal Hamiltonian. More interestingly, under moderate driving frequency, another quasiconserved observable can still persist if the Floquet driving contains a large global rotation. We show theoretically how to calculate this conserved observable and provide numerical verification. Having systematically identified all quasiconserved observables, we can finally investigate their behavior in the infinite-time limit and thermodynamic limit, using autocorrelations obtained from both numerical simulation and experiments in solid state nuclear magnetic resonance systems.

quant-ph↗

arXiv2020

Relational Learning between Multiple Pulmonary Nodules via Deep Set Attention Transformers

Diagnosis and treatment of multiple pulmonary nodules are clinically important but challenging. Prior studies on nodule characterization use solitary-nodule approaches on multiple nodular patients, which ignores the relations between nodules. In this study, we propose a multiple instance learning (MIL) approach and empirically prove the benefit to learn the relations between multiple nodules. By treating the multiple nodules from a same patient as a whole, critical relational information between solitary-nodule voxels is extracted. To our knowledge, it is the first study to learn the relations between multiple pulmonary nodules. Inspired by recent advances in natural language processing (NLP) domain, we introduce a self-attention transformer equipped with 3D CNN, named {NoduleSAT}, to replace typical pooling-based aggregation in multiple instance learning. Extensive experiments on lung nodule false positive reduction on LUNA16 database, and malignancy classification on LIDC-IDRI database, validate the effectiveness of the proposed method.

eess.IV↗

arXiv2019

Evaluating and Boosting Uncertainty Quantification in Classification

Emergence of artificial intelligence techniques in biomedical applications urges the researchers to pay more attention on the uncertainty quantification (UQ) in machine-assisted medical decision making. For classification tasks, prior studies on UQ are difficult to compare with each other, due to the lack of a unified quantitative evaluation metric. Considering that well-performing UQ models ought to know when the classification models act incorrectly, we design a new evaluation metric, area under Confidence-Classification Characteristic curves (AUCCC), to quantitatively evaluate the performance of the UQ models. AUCCC is threshold-free, robust to perturbation, and insensitive to the classification performance. We evaluate several UQ methods (e.g., max softmax output) with AUCCC to validate its effectiveness. Furthermore, a simple scheme, named Uncertainty Distillation (UDist), is developed to boost the UQ performance, where a confidence model is distilling the confidence estimated by deep ensembles. The proposed method is easy to implement; it consistently outperforms strong baselines on natural and medical image datasets in our experiments.

cs.LG↗

arXiv2018

Bell inequalities violation within non-Bunch-Davies states

We study the quantum nature of non-Bunch-Davies states in de Sitter space by evaluating CHSH inequality on a localized two-atom system. We show that quantum nonlocality can be generated through the Markovian evolution of two-atom, witnessed by a violation of CHSH inequality on its final equilibrium state. We find that the upper bound of inequality violation is determined by different choices of de Sitter-invariant vacua sectors. In particular, with growing Gibbons-Hawking temperature, the CHSH bound degrades monotonously for Bunch-Davies vacuum sector. Due to the intrinsic correlation of non-Bunch-Davies vacua, we find that the related violation of inequality can however drastically increase after certain turning point, and may persist for arbitrarily large environment decoherence. This implies that the CHSH inequality is useful to classify the initial quantum state of the Universe. Finally, we clarify that the witnessed intrinsic correlation of non-Bunch-Davies vacua can be utilized for quantum information applications, e.g., surpassing the Heisenberg uncertainty bound of quantum measurement in de Sitter space.

hep-th↗

arXiv2018

Quantum estimation in an expanding spacetime

We investigate the quantum estimation on the Hubble parameter of an expanding de Sitter space by quantum metrological techniques. By exploring the dynamics of a freely falling Unruh-DeWitt detector, which interacts with a scalar field coupling to curvature, we calculate the Fisher information (FI) and quantum Fisher information (QFI) for the detector, which bound the highest precision of the estimation on Hubble parameter. In standard Bunch-Davies vacuum, we show that the maxima of FI/QFI are located for particular initial state of probe. Beside its dependence on the evolving time of detector and the energy spacing of atom $ω$, we show that the maxima of FI/QFI can be significantly enhanced once a proper coupling of scalar field to curvature is chosen. For instance, we show numerically that the estimation in the scenario with minimally/nearly minimally coupling scalar field can always outperform that with conformally coupling scalar field, corresponding to a higher FI/QFI in estimation. Moreover, we find that for general $α-$vacua of de Sitter space, a further improvement of estimation can be achieved, attributed to the squeezed nature of $α-$vacua that heavily constrains the measurement uncertainty. Some implications of our results are also discussed.

hep-th↗