arXiv subjects

Yixuan Wu

Publications and source records attributed to Yixuan Wu.

45 records · Page 3Linked to original sources

arXiv2023

MolHF: A Hierarchical Normalizing Flow for Molecular Graph Generation

Molecular de novo design is a critical yet challenging task in scientific fields, aiming to design novel molecular structures with desired property profiles. Significant progress has been made by resorting to generative models for graphs. However, limited attention is paid to hierarchical generative models, which can exploit the inherent hierarchical structure (with rich semantic information) of the molecular graphs and generate complex molecules of larger size that we shall demonstrate to be difficult for most existing models. The primary challenge to hierarchical generation is the non-differentiable issue caused by the generation of intermediate discrete coarsened graph structures. To sidestep this issue, we cast the tricky hierarchical generation problem over discrete spaces as the reverse process of hierarchical representation learning and propose MolHF, a new hierarchical flow-based model that generates molecular graphs in a coarse-to-fine manner. Specifically, MolHF first generates bonds through a multi-scale architecture, then generates atoms based on the coarsened graph structure at each scale. We demonstrate that MolHF achieves state-of-the-art performance in random generation and property optimization, implying its high capacity to model data distribution. Furthermore, MolHF is the first flow-based model that can be applied to model larger molecules (polymer) with more than 100 heavy atoms. The code and models are available at https://github.com/violet-sto/MolHF.

cs.LG↗

arXiv2023

Gaussian Max-Value Entropy Search for Multi-Agent Bayesian Optimization

We study the multi-agent Bayesian optimization (BO) problem, where multiple agents maximize a black-box function via iterative queries. We focus on Entropy Search (ES), a sample-efficient BO algorithm that selects queries to maximize the mutual information about the maximum of the black-box function. One of the main challenges of ES is that calculating the mutual information requires computationally-costly approximation techniques. For multi-agent BO problems, the computational cost of ES is exponential in the number of agents. To address this challenge, we propose the Gaussian Max-value Entropy Search, a multi-agent BO algorithm with favorable sample and computational efficiency. The key to our idea is to use a normal distribution to approximate the function maximum and calculate its mutual information accordingly. The resulting approximation allows queries to be cast as the solution of a closed-form optimization problem which, in turn, can be solved via a modified gradient ascent algorithm and scaled to a large number of agents. We demonstrate the effectiveness of Gaussian max-value Entropy Search through numerical experiments on standard test functions and real-robot experiments on the source-seeking problem. Results show that the proposed algorithm outperforms the multi-agent BO baselines in the numerical experiments and can stably seek the source with a limited number of noisy observations on real robots.

cs.LG↗

arXiv2023

T2G-Former: Organizing Tabular Features into Relation Graphs Promotes Heterogeneous Feature Interaction

Recent development of deep neural networks (DNNs) for tabular learning has largely benefited from the capability of DNNs for automatic feature interaction. However, the heterogeneity nature of tabular features makes such features relatively independent, and developing effective methods to promote tabular feature interaction still remains an open problem. In this paper, we propose a novel Graph Estimator, which automatically estimates the relations among tabular features and builds graphs by assigning edges between related features. Such relation graphs organize independent tabular features into a kind of graph data such that interaction of nodes (tabular features) can be conducted in an orderly fashion. Based on our proposed Graph Estimator, we present a bespoke Transformer network tailored for tabular learning, called T2G-Former, which processes tabular data by performing tabular feature interaction guided by the relation graphs. A specific Cross-level Readout collects salient features predicted by the layers in T2G-Former across different levels, and attains global semantics for final prediction. Comprehensive experiments show that our T2G-Former achieves superior performance among DNNs and is competitive with non-deep Gradient Boosted Decision Tree models.

cs.LG↗

arXiv2022

Learning Detail-Structure Alternative Optimization for Blind Super-Resolution

Existing convolutional neural networks (CNN) based image super-resolution (SR) methods have achieved impressive performance on bicubic kernel, which is not valid to handle unknown degradations in real-world applications. Recent blind SR methods suggest to reconstruct SR images relying on blur kernel estimation. However, their results still remain visible artifacts and detail distortion due to the estimation errors. To alleviate these problems, in this paper, we propose an effective and kernel-free network, namely DSSR, which enables recurrent detail-structure alternative optimization without blur kernel prior incorporation for blind SR. Specifically, in our DSSR, a detail-structure modulation module (DSMM) is built to exploit the interaction and collaboration of image details and structures. The DSMM consists of two components: a detail restoration unit (DRU) and a structure modulation unit (SMU). The former aims at regressing the intermediate HR detail reconstruction from LR structural contexts, and the latter performs structural contexts modulation conditioned on the learned detail maps at both HR and LR spaces. Besides, we use the output of DSMM as the hidden state and design our DSSR architecture from a recurrent convolutional neural network (RCNN) view. In this way, the network can alternatively optimize the image details and structural contexts, achieving co-optimization across time. Moreover, equipped with the recurrent connection, our DSSR allows low- and high-level feature representations complementary by observing previous HR details and contexts at every unrolling time. Extensive experiments on synthetic datasets and real-world images demonstrate that our method achieves the state-of-the-art against existing methods. The source code can be found at https://github.com/Arcananana/DSSR.

cs.CV↗

arXiv2022

Bridging Component Learning with Degradation Modelling for Blind Image Super-Resolution

Convolutional Neural Network (CNN)-based image super-resolution (SR) has exhibited impressive success on known degraded low-resolution (LR) images. However, this type of approach is hard to hold its performance in practical scenarios when the degradation process is unknown. Despite existing blind SR methods proposed to solve this problem using blur kernel estimation, the perceptual quality and reconstruction accuracy are still unsatisfactory. In this paper, we analyze the degradation of a high-resolution (HR) image from image intrinsic components according to a degradation-based formulation model. We propose a components decomposition and co-optimization network (CDCN) for blind SR. Firstly, CDCN decomposes the input LR image into structure and detail components in feature space. Then, the mutual collaboration block (MCB) is presented to exploit the relationship between both two components. In this way, the detail component can provide informative features to enrich the structural context and the structure component can carry structural context for better detail revealing via a mutual complementary manner. After that, we present a degradation-driven learning strategy to jointly supervise the HR image detail and structure restoration process. Finally, a multi-scale fusion module followed by an upsampling layer is designed to fuse the structure and detail features and perform SR reconstruction. Empowered by such degradation-based components decomposition, collaboration, and mutual optimization, we can bridge the correlation between component learning and degradation modelling for blind SR, thereby producing SR results with more accurate textures. Extensive experiments on both synthetic SR datasets and real-world images show that the proposed method achieves the state-of-the-art performance compared to existing methods.

cs.CV↗

arXiv2022

D-Former: A U-shaped Dilated Transformer for 3D Medical Image Segmentation

Computer-aided medical image segmentation has been applied widely in diagnosis and treatment to obtain clinically useful information of shapes and volumes of target organs and tissues. In the past several years, convolutional neural network (CNN) based methods (e.g., U-Net) have dominated this area, but still suffered from inadequate long-range information capturing. Hence, recent work presented computer vision Transformer variants for medical image segmentation tasks and obtained promising performances. Such Transformers model long-range dependency by computing pair-wise patch relations. However, they incur prohibitive computational costs, especially on 3D medical images (e.g., CT and MRI). In this paper, we propose a new method called Dilated Transformer, which conducts self-attention for pair-wise patch relations captured alternately in local and global scopes. Inspired by dilated convolution kernels, we conduct the global self-attention in a dilated manner, enlarging receptive fields without increasing the patches involved and thus reducing computational costs. Based on this design of Dilated Transformer, we construct a U-shaped encoder-decoder hierarchical architecture called D-Former for 3D medical image segmentation. Experiments on the Synapse and ACDC datasets show that our D-Former model, trained from scratch, outperforms various competitive CNN-based or Transformer-based segmentation models at a low computational cost without time-consuming per-training process.

cs.CV↗

arXiv2021

Highly accurate operator factorization methods for the integral fractional Laplacian and its generalization

In this paper, we propose a new class of operator factorization methods to discretize the integral fractional Laplacian $(-Δ)^\fracα{2}$ for $α\in (0, 2)$. The main advantage of our method is to easily increase numerical accuracy by using high-degree Lagrange basis functions, but remain the scheme structure and computer implementation unchanged. Moreover, our discretization of the fractional Laplacian results in a symmetric (multilevel) Toeplitz differentiation matrix, which not only saves memory cost in simulations but enables efficient computations via the fast Fourier transforms. The performance of our method in both approximating the fractional Laplacian and solving the fractional Poisson problems was detailedly examined. It shows that our method has an optimal accuracy of ${\mathcal O}(h^2)$ for constant or linear basis functions, while ${\mathcal O}(h^4)$ if quadratic basis functions are used, with $h$ a small mesh size. Note that this accuracy holds for any $α\in (0, 2)$ and can be further increased if higher-degree basis functions are used. If the solution of fractional Poisson problem satisfies $u \in C^{m, l}(\barΩ)$ for $m \in {\mathbb N}$ and $0 < l < 1$, then our method has an accuracy of ${\mathcal O}\big(h^{\min\{m+l,\, 2\}}\big)$ for constant and linear basis functions, while ${\mathcal O}\big(h^{\min\{m+l,\, 4\}}\big)$ for quadratic basis functions. Additionally, our method can be readily applied to study generalized fractional Laplacians with a symmetric kernel function, and numerical study on the tempered fractional Poisson problem demonstrates its efficiency.

math.NA↗

arXiv2021

A universal solution scheme for fractional and classical PDEs

We propose a unified meshless method to solve classical and fractional PDE problems with $(-Δ)^{\fracα{2}}$ for $α\in (0, 2]$. The classical ($α= 2$) and fractional ($α< 2$) Laplacians, one local and the other nonlocal, have distinct properties. Therefore, their numerical methods and computer implementations are usually incompatible. We notice that for any $α\ge 0$, the Laplacian $(-Δ)^{\fracα{2}}$ of generalized inverse multiquadric (GIMQ) functions can be analytically written by the Gauss hypergeometric function, and thus propose a GIMQ-based method. Our method unifies the discretization of classical and fractional Laplacians and also bypasses numerical approximation to the hypersingular integral of fractional Laplacian. These two merits distinguish our method from other existing methods for the fractional Laplacian. Extensive numerical experiments are carried out to test the performance of our method. Compared to other methods, our method can achieve high accuracy with fewer number of unknowns, which effectively reduces the storage and computational requirements in simulations of fractional PDEs. Moreover, the meshfree nature makes it free of geometric constraints and enables simple implementation for any dimension $d \ge 1$. Additionally, two approaches of selecting shape parameters, including condition number-indicated method and random-perturbed method, are studied to avoid the ill-conditioning issues when large number of points.

math.NA↗

arXiv2020

A unified meshfree pseudospectral method for solving both classical and fractional PDEs

In this paper, we propose a meshfree method based on the Gaussian radial basis function (RBF) to solve both classical and fractional PDEs. The proposed method takes advantage of the analytical Laplacian of Gaussian functions so as to accommodate the discretization of the classical and fractional Laplacian in a single framework and avoid the large computational cost for numerical evaluation of the fractional derivatives. These important merits distinguish it from other numerical methods for fractional PDEs. Moreover, our method is simple and easy to handle complex geometry and local refinement, and its computer program implementation remains the same for any dimension $d \ge 1$. Extensive numerical experiments are provided to study the performance of our method in both approximating the Dirichlet Laplace operators and solving PDE problems. Compared to the recently proposed Wendland RBF method, our method exactly incorporates the Dirichlet boundary conditions into the scheme and is free of the Gibbs phenomenon as observed in the literature. Our studies suggest that to obtain good accuracy the shape parameter cannot be too small or too big, and the optimal shape parameter might depend on the RBF center points and the solution properties.

math.NA↗