arXiv subjects

Dong Liu

Publications and source records attributed to Dong Liu.

At least 181 records · Page 10Linked to original sources

arXiv2024

In-Loop Filtering via Trained Look-Up Tables

In-loop filtering (ILF) is a key technology for removing the artifacts in image/video coding standards. Recently, neural network-based in-loop filtering methods achieve remarkable coding gains beyond the capability of advanced video coding standards, which becomes a powerful coding tool candidate for future video coding standards. However, the utilization of deep neural networks brings heavy time and computational complexity, and high demands of high-performance hardware, which is challenging to apply to the general uses of coding scene. To address this limitation, inspired by explorations in image restoration, we propose an efficient and practical in-loop filtering scheme by adopting the Look-up Table (LUT). We train the DNN of in-loop filtering within a fixed filtering reference range, and cache the output values of the DNN into a LUT via traversing all possible inputs. At testing time in the coding process, the filtered pixel is generated by locating input pixels (to-be-filtered pixel with reference pixels) and interpolating cached filtered pixel values. To further enable the large filtering reference range with the limited storage cost of LUT, we introduce the enhanced indexing mechanism in the filtering process, and clipping/finetuning mechanism in the training. The proposed method is implemented into the Versatile Video Coding (VVC) reference software, VTM-11.0. Experimental results show that the ultrafast, very fast, and fast mode of the proposed method achieves on average 0.13%/0.34%/0.51%, and 0.10%/0.27%/0.39% BD-rate reduction, under the all intra (AI) and random access (RA) configurations. Especially, our method has friendly time and computational complexity, only 101%/102%-104%/108% time increase with 0.13-0.93 kMACs/pixel, and only 164-1148 KB storage cost for a single model. Our solution may shed light on the journey of practical neural network-based coding tool evolution.

eess.IV↗

arXiv2024

Diff-INR: Generative Regularization for Electrical Impedance Tomography

Electrical Impedance Tomography (EIT) is a non-invasive imaging technique that reconstructs conductivity distributions within a body from boundary measurements. However, EIT reconstruction is hindered by its ill-posed nonlinear inverse problem, which complicates accurate results. To tackle this, we propose Diff-INR, a novel method that combines generative regularization with Implicit Neural Representations (INR) through a diffusion model. Diff-INR introduces geometric priors to guide the reconstruction, effectively addressing the shortcomings of traditional regularization methods. By integrating a pre-trained diffusion regularizer with INR, our approach achieves state-of-the-art reconstruction accuracy in both simulation and experimental data. The method demonstrates robust performance across various mesh densities and hyperparameter settings, highlighting its flexibility and efficiency. This advancement represents a significant improvement in managing the ill-posed nature of EIT. Furthermore, the method's principles are applicable to other imaging modalities facing similar challenges with ill-posed inverse problems.

eess.IV↗

arXiv2024

Probing the black holes in a dark matter halo of M87 using gravitational wave echoes

Variations at the event horizon structure of a black hole will emit the signals of the gravitational wave echoes associated with the ringdown of the binary black hole merger. In this work, combining mass model of M87 and Einasto profile for dark matter halo, we construct one formal solution of black holes in dark matter halo, and this solution includes the regular black hole for geometric parameter $a < 2M$ and the wormhole for $a \geq 2M$. Then, we test this solution under axial gravitational perturbation and calculate their quasinormal modes (QNM). Our results show that when geometric parameter $a>2M$, a series of gravitational wave echoes appear after the QNM, and one distinctive feature of gravitational wave echoes is the double barrier. Besides, we also study the impacts of shape parameters $α$ of Einasto profile both on the QNM and gravitational wave echoes, and extract their frequencies using WKB method and Prony method. In principle, with the increasing of shape parameter $α$, the frequencies of the QNM and gravitational wave echoes between the different shape parameters must be different. But now there is zero difference of the frequencies between shape parameter $α=0.18$ and $α=0.20$. Zero difference is not allowed, so we give an upper limit for shape parameter $α$, which is approximately $α<0.18$. These signals of gravitational wave echoes in a dark matter halo may be detected in the near future, and these characteristics of gravitational wave echoes may serve as local measurements of dark matter.

gr-qc↗

arXiv2024

KunquDB: An Attempt for Speaker Verification in the Chinese Opera Scenario

This work aims to promote Chinese opera research in both musical and speech domains, with a primary focus on overcoming the data limitations. We introduce KunquDB, a relatively large-scale, well-annotated audio-visual dataset comprising 339 speakers and 128 hours of content. Originating from the Kunqu Opera Art Canon (Kunqu yishu dadian), KunquDB is meticulously structured by dialogue lines, providing explicit annotations including character names, speaker names, gender information, vocal manner classifications, and accompanied by preliminary text transcriptions. KunquDB provides a versatile foundation for role-centric acoustic studies and advancements in speech-related research, including Automatic Speaker Verification (ASV). Beyond enriching opera research, this dataset bridges the gap between artistic expression and technological innovation. Pioneering the exploration of ASV in Chinese opera, we construct four test trials considering two distinct vocal manners in opera voices: stage speech (ST) and singing (S). Implementing domain adaptation methods effectively mitigates domain mismatches induced by these vocal manner variations while there is still room for further improvement as a benchmark.

eess.AS↗

arXiv2024

NVC-1B: A Large Neural Video Coding Model

The emerging large models have achieved notable progress in the fields of natural language processing and computer vision. However, large models for neural video coding are still unexplored. In this paper, we try to explore how to build a large neural video coding model. Based on a small baseline model, we gradually scale up the model sizes of its different coding parts, including the motion encoder-decoder, motion entropy model, contextual encoder-decoder, contextual entropy model, and temporal context mining module, and analyze the influence of model sizes on video compression performance. Then, we explore to use different architectures, including CNN, mixed CNN-Transformer, and Transformer architectures, to implement the neural video coding model and analyze the influence of model architectures on video compression performance. Based on our exploration results, we design the first neural video coding model with more than 1 billion parameters -- NVC-1B. Experimental results show that our proposed large model achieves a significant video compression performance improvement over the small baseline model, and represents the state-of-the-art compression efficiency. We anticipate large models may bring up the video coding technologies to the next level.

cs.CV↗

arXiv2024

Probing black holes in a dark matter spike of M87 using quasinormal modes

Dark matter density can be significantly enhanced by the supermassive black hole at the galactic center, leading to a structure called dark matter spike. Dark matter spike may change the spacetime properties of black holes that constitute deviations from GR black holes. Based on these interesting background, we construct a set of solutions of black holes in a dark matter spike under the Newtonian approximation and full relativity. Combining the mass model of M87, we study the quasinormal modes of black holes in the scalar field and axial gravitational perturbation, then compared them with Schwarzschild black hole. Besides, the impacts of dark matter on the quasinormal mode of black holes have been studied in depth. In particular, in the axial gravitational perturbation, our results show that the impacts of dark matter spike on the quasinormal mode of black holes can reach up to $10^{-4}$. These new features from quasinormal mode of black holes under the Newtonian approximation and full relativity may provide some help for the establishment of the final dark matter model, and provide a new thought for the indirect detection of dark matter.

gr-qc↗

arXiv2024

Uniformly Accelerated Motion Model for Inter Prediction

Inter prediction is a key technology to reduce the temporal redundancy in video coding. In natural videos, there are usually multiple moving objects with variable velocity, resulting in complex motion fields that are difficult to represent compactly. In Versatile Video Coding (VVC), existing inter prediction methods usually assume uniform speed motion between consecutive frames and use the linear models for motion estimation (ME) and motion compensation (MC), which may not well handle the complex motion fields in the real world. To address these issues, we introduce a uniformly accelerated motion model (UAMM) to exploit motion-related elements (velocity, acceleration) of moving objects between the video frames, and further combine them to assist the inter prediction methods to handle the variable motion in the temporal domain. Specifically, first, the theory of UAMM is mentioned. Second, based on that, we propose the UAMM-based parameter derivation and extrapolation schemes in the coding process. Third, we integrate the UAMM into existing inter prediction modes (Merge, MMVD, CIIP) to achieve higher prediction accuracy. The proposed method is implemented into the VVC reference software, VTM version 12.0. Experimental results show that the proposed method achieves up to 0.38% and on average 0.13% BD-rate reduction compared to the VTM anchor, under the Low-delay P configuration, with a slight increase of time complexity on the encoding/decoding side.

eess.IV↗

arXiv2024

Subgroup Identification with Latent Factor Structure

Subgroup analysis has garnered increasing attention for its ability to identify meaningful subgroups within heterogeneous populations, thereby enhancing predictive power. However, in many fields such as social science and biology, covariates are often highly correlated due to common factors. This correlation poses significant challenges for subgroup identification, an issue that is often overlooked in existing literature. In this paper, we aim to address this gap in the ``diverging dimension" regime by proposing a center-augmented subgroup identification method within the Factor Augmented (sparse) Linear Model framework. This method bridges dimension reduction and sparse regression. Our proposed approach is adaptable to the high cross-sectional dependence among covariates and offers computational advantages with a complexity of $O(nK)$, compared to the $O(n^2)$ complexity of the conventional pairwise fusion penalty method in the literature, where $n$ is the sample size and $K$ is the number of subgroups. We also investigate the asymptotic properties of the oracle estimators under conditions on the minimal distance between group centroids. To implement the proposed approach, we introduce a Difference of Convex functions-based Alternating Direction Method of Multipliers (DC-ADMM) algorithm and demonstrate its convergence to a local minimizer in a finite number of steps. We illustrate the superiority of the proposed method through extensive numerical experiments and a real macroeconomic data example. An \texttt{R} package, \texttt{SILFS}, implementing the method is also available on CRAN.

stat.ME↗

arXiv2024

Decomposition Betters Tracking Everything Everywhere

Recent studies on motion estimation have advocated an optimized motion representation that is globally consistent across the entire video, preferably for every pixel. This is challenging as a uniform representation may not account for the complex and diverse motion and appearance of natural videos. We address this problem and propose a new test-time optimization method, named DecoMotion, for estimating per-pixel and long-range motion. DecoMotion explicitly decomposes video content into static scenes and dynamic objects, either of which uses a quasi-3D canonical volume to represent. DecoMotion separately coordinates the transformations between local and canonical spaces, facilitating an affine transformation for the static scene that corresponds to camera motion. For the dynamic volume, DecoMotion leverages discriminative and temporally consistent features to rectify the non-rigid transformation. The two volumes are finally fused to fully represent motion and appearance. This divide-and-conquer strategy leads to more robust tracking through occlusions and deformations and meanwhile obtains decomposed appearances. We conduct evaluations on the TAP-Vid benchmark. The results demonstrate our method boosts the point-tracking accuracy by a large margin and performs on par with some state-of-the-art dedicated point-tracking solutions.

cs.CV↗

arXiv2024

Drantal-NeRF: Diffusion-Based Restoration for Anti-aliasing Neural Radiance Field

Aliasing artifacts in renderings produced by Neural Radiance Field (NeRF) is a long-standing but complex issue in the field of 3D implicit representation, which arises from a multitude of intricate causes and was mitigated by designing more advanced but complex scene parameterization methods before. In this paper, we present a Diffusion-based restoration method for anti-aliasing Neural Radiance Field (Drantal-NeRF). We consider the anti-aliasing issue from a low-level restoration perspective by viewing aliasing artifacts as a kind of degradation model added to clean ground truths. By leveraging the powerful prior knowledge encapsulated in diffusion model, we could restore the high-realism anti-aliasing renderings conditioned on aliased low-quality counterparts. We further employ a feature-wrapping operation to ensure multi-view restoration consistency and finetune the VAE decoder to better adapt to the scene-specific data distribution. Our proposed method is easy to implement and agnostic to various NeRF backbones. We conduct extensive experiments on challenging large-scale urban scenes as well as unbounded 360-degree scenes and achieve substantial qualitative and quantitative improvements.

cs.CV↗

arXiv2024

CompassDB: Pioneering High-Performance Key-Value Store with Perfect Hash

Modern mainstream persistent key-value storage engines utilize Log-Structured Merge tree (LSM-tree) based designs, optimizing read/write performance by leveraging sequential disk I/O. However, the advent of SSDs, with their significant improvements in bandwidth and IOPS, shifts the bottleneck from I/O to CPU. The high compaction cost and large read/write amplification associated with LSM trees have become critical bottlenecks. In this paper, we introduce CompassDB, which utilizes a Two-tier Perfect Hash Table (TPH) design to significantly decrease read/write amplification and compaction costs. CompassDB utilizes a perfect hash algorithm for its in-memory index, resulting in an average index cost of about 6 bytes per key-value pair. This compact index reduces the lookup time complexity from $O(log N)$ to $O(1)$ and decreases the overall cost. Consequently, it allows for the storage of more key-value pairs for reads or provides additional memory for the memtable for writes. This results in substantial improvements in both throughput and latency. Our evaluation using the YCSB benchmark tool shows that CompassDB increases throughput by 2.5x to 4x compared to RocksDB, and by 5x to 17x compared to PebblesDB across six typical workloads. Additionally, CompassDB significantly reduces average and 99th percentile read/write latency, achieving a 50% to 85% reduction in comparison to RocksDB.

cs.DB↗

arXiv2024

Residual resampling-based physics-informed neural network for neutron diffusion equations

The neutron diffusion equation plays a pivotal role in the analysis of nuclear reactors. Nevertheless, employing the Physics-Informed Neural Network (PINN) method for its solution entails certain limitations. Traditional PINN approaches often utilize fully connected network (FCN) architecture, which is susceptible to overfitting, training instability, and gradient vanishing issues as the network depth increases. These challenges result in accuracy bottlenecks in the solution. In response to these issues, the Residual-based Resample Physics-Informed Neural Network(R2-PINN) is proposed, which proposes an improved PINN architecture that replaces the FCN with a Convolutional Neural Network with a shortcut(S-CNN), incorporating skip connections to facilitate gradient propagation between network layers. Additionally, the incorporation of the Residual Adaptive Resampling (RAR) mechanism dynamically increases sampling points, enhancing the spatial representation capabilities and overall predictive accuracy of the model. The experimental results illustrate that our approach significantly improves the model's convergence capability, achieving high-precision predictions of physical fields. In comparison to traditional FCN-based PINN methods, R2-PINN effectively overcomes the limitations inherent in current methods, providing more accurate and robust solutions for neutron diffusion equations.

cs.LG↗

arXiv2024

1-cocycles of the Witt algebra with coefficients in tensor product of modules

In this paper, we classify 1-cocycles of the Witt algebra with coefficients in the tensor product of two arbitrary tensor density modules. In a special case, we recover a theorem originally established by Ng and Taft in \cite{NT}. Furthermore, by these 1-cocycles, we determine Lie bialgebra structures over certain infinite-dimensional Lie algebras containing the Witt algebra.

math.RA↗

arXiv2024

LaCon: Late-Constraint Diffusion for Steerable Guided Image Synthesis

Diffusion models have demonstrated impressive abilities in generating photo-realistic and creative images. To offer more controllability for the generation process, existing studies, termed as early-constraint methods in this paper, leverage extra conditions and incorporate them into pre-trained diffusion models. Particularly, some of them adopt condition-specific modules to handle conditions separately, where they struggle to generalize across other conditions. Although follow-up studies present unified solutions to solve the generalization problem, they also require extra resources to implement, e.g., additional inputs or parameter optimization, where more flexible and efficient solutions are expected to perform steerable guided image synthesis. In this paper, we present an alternative paradigm, namely Late-Constraint Diffusion (LaCon), to simultaneously integrate various conditions into pre-trained diffusion models. Specifically, LaCon establishes an alignment between the external condition and the internal features of diffusion models, and utilizes the alignment to incorporate the target condition, guiding the sampling process to produce tailored results. Experimental results on COCO dataset illustrate the effectiveness and superior generalization capability of LaCon under various conditions and settings. Ablation studies investigate the functionalities of different components in LaCon, and illustrate its great potential to serve as an efficient solution to offer flexible controllability for diffusion models.

cs.CV↗

arXiv2024

Semi-vortex solitons and their excited states in spin-orbit-coupled binary bosonic condensates

It is known that two-dimensional two-component fundamental solitons of the semi-vortex (SV) type, with vorticities $(s_{+},s_{-})=(0,1)$ in their components, are stable ground states (GSs) in the spin-orbit-coupled (SOC) binary Bose-Einstein condensate with the contact self-attraction acting in both components, in spite of the possibility of the critical collapse in the system. However, excited states(ESs) of the SV solitons, with the vorticity set $(s_{+},s_{-})=( S_{+},S_{+}+1)$ and $S_{+}=1,2,3,...$, are unstable in the same system. We construct ESs of SV solitons in the SOC system with opposite signs of the self-interaction in the two components. The main finding is stability of the ES-SV solitons, with the extra vorticity (at least) up to $S_{+}=6$. The threshold value of the norm for the onset of the critical collapse, $N_{\mathrm{thr}}$, in these excited states is higher than the commonly known critical value, $N_{c}\approx 5.85$,associated with the single-component Townes solitons, $N_{\mathrm{thr}}$ increasing with the growth of $S_{+}$. A velocity interval for stable motion of the GS-SV solitons is found too. The results suggest a solution for the challenging problem of the creation of stable vortex solitons with high topological charges.

cond-mat.quant-gas↗

arXiv2024

Hybrid thin-film lithium niobate micro-ring acousto-optic modulator for microwave-to-optical conversion

Highly efficient acousto-optic modulation plays a vital role in the microwave-to-optical conversion. Herein, we demonstrate a hybrid thin-film lithium niobate (TFLN) racetrack micro-ring acousto-optic modulator (AOM) implemented with low-loss chalcogenide (ChG) waveguide. By engineering the electrode configuration of the interdigital transducer, the double-arm micro-ring acousto-optic modulation is experimentally confirmed in nonsuspended ChG loaded TFLN waveguide platform. Varying the position of blue-detuned bias point, the half-wave-voltage-length product VpaiL of the hybrid TFLN micro-ring AOM is as small as 9 mVcm. Accordingly, the acousto-optic coupling strength is estimated to be 0.48 Hz s1/2 at acoustic frequency of 0.84 GHz. By analyzing the generation of phonon number from the piezoelectric transducer, the microwave-to-optical conversion efficiency is calculated to be 0.05%, approximately one order of magnitude larger than that of the state-of-the-art suspended counterpart. Efficient microwave-to-optical conversion thus provides new opportunities for low-power-consumption quantum information transduction using the TFLN-ChG hybrid piezo-optomechanical devices.

physics.optics↗

arXiv2024

DMOFC: Discrimination Metric-Optimized Feature Compression

Feature compression, as an important branch of video coding for machines (VCM), has attracted significant attention and exploration. However, the existing methods mainly focus on intra-feature similarity, such as the Mean Squared Error (MSE) between the reconstructed and original features, while neglecting the importance of inter-feature relationships. In this paper, we analyze the inter-feature relationships, focusing on feature discriminability in machine vision and underscoring its significance in feature compression. To maintain the feature discriminability of reconstructed features, we introduce a discrimination metric for feature compression. The discrimination metric is designed to ensure that the distance between features of the same category is smaller than the distance between features of different categories. Furthermore, we explore the relationship between the discrimination metric and the discriminability of the original features. Experimental results confirm the effectiveness of the proposed discrimination metric and reveal there exists a trade-off between the discrimination metric and the discriminability of the original features.

cs.CV↗

arXiv2024

HiLo: Detailed and Robust 3D Clothed Human Reconstruction with High-and Low-Frequency Information of Parametric Models

Reconstructing 3D clothed human involves creating a detailed geometry of individuals in clothing, with applications ranging from virtual try-on, movies, to games. To enable practical and widespread applications, recent advances propose to generate a clothed human from an RGB image. However, they struggle to reconstruct detailed and robust avatars simultaneously. We empirically find that the high-frequency (HF) and low-frequency (LF) information from a parametric model has the potential to enhance geometry details and improve robustness to noise, respectively. Based on this, we propose HiLo, namely clothed human reconstruction with high- and low-frequency information, which contains two components. 1) To recover detailed geometry using HF information, we propose a progressive HF Signed Distance Function to enhance the detailed 3D geometry of a clothed human. We analyze that our progressive learning manner alleviates large gradients that hinder model convergence. 2) To achieve robust reconstruction against inaccurate estimation of the parametric model by using LF information, we propose a spatial interaction implicit function. This function effectively exploits the complementary spatial information from a low-resolution voxel grid of the parametric model. Experimental results demonstrate that HiLo outperforms the state-of-the-art methods by 10.43% and 9.54% in terms of Chamfer distance on the Thuman2.0 and CAPE datasets, respectively. Additionally, HiLo demonstrates robustness to noise from the parametric model, challenging poses, and various clothing styles.

cs.CV↗