arXiv subjects

Jiahang Li

Publications and source records attributed to Jiahang Li.

29 records · Page 2Linked to original sources

arXiv2024

RoadFormer+: Delivering RGB-X Scene Parsing through Scale-Aware Information Decoupling and Advanced Heterogeneous Feature Fusion

Task-specific data-fusion networks have marked considerable achievements in urban scene parsing. Among these networks, our recently proposed RoadFormer successfully extracts heterogeneous features from RGB images and surface normal maps and fuses these features through attention mechanisms, demonstrating compelling efficacy in RGB-Normal road scene parsing. However, its performance significantly deteriorates when handling other types/sources of data or performing more universal, all-category scene parsing tasks. To overcome these limitations, this study introduces RoadFormer+, an efficient, robust, and adaptable model capable of effectively fusing RGB-X data, where ``X'', represents additional types/modalities of data such as depth, thermal, surface normal, and polarization. Specifically, we propose a novel hybrid feature decoupling encoder to extract heterogeneous features and decouple them into global and local components. These decoupled features are then fused through a dual-branch multi-scale heterogeneous feature fusion block, which employs parallel Transformer attentions and convolutional neural network modules to merge multi-scale features across different scales and receptive fields. The fused features are subsequently fed into a decoder to generate the final semantic predictions. Notably, our proposed RoadFormer+ ranks first on the KITTI Road benchmark and achieves state-of-the-art performance in mean intersection over union on the Cityscapes, MFNet, FMB, and ZJU datasets. Moreover, it reduces the number of learnable parameters by 65\% compared to RoadFormer. Our source code will be publicly available at mias.group/RoadFormerPlus.

cs.CV↗

arXiv2024

RoadFormer: Duplex Transformer for RGB-Normal Semantic Road Scene Parsing

The recent advancements in deep convolutional neural networks have shown significant promise in the domain of road scene parsing. Nevertheless, the existing works focus primarily on freespace detection, with little attention given to hazardous road defects that could compromise both driving safety and comfort. In this paper, we introduce RoadFormer, a novel Transformer-based data-fusion network developed for road scene parsing. RoadFormer utilizes a duplex encoder architecture to extract heterogeneous features from both RGB images and surface normal information. The encoded features are subsequently fed into a novel heterogeneous feature synergy block for effective feature fusion and recalibration. The pixel decoder then learns multi-scale long-range dependencies from the fused and recalibrated heterogeneous features, which are subsequently processed by a Transformer decoder to produce the final semantic prediction. Additionally, we release SYN-UDTIRI, the first large-scale road scene parsing dataset that contains over 10,407 RGB images, dense depth images, and the corresponding pixel-level annotations for both freespace and road defects of different shapes and sizes. Extensive experimental evaluations conducted on our SYN-UDTIRI dataset, as well as on three public datasets, including KITTI road, CityScapes, and ORFD, demonstrate that RoadFormer outperforms all other state-of-the-art networks for road scene parsing. Specifically, RoadFormer ranks first on the KITTI road benchmark. Our source code, created dataset, and demo video are publicly available at mias.group/RoadFormer.

cs.CV↗

arXiv2024

4DBInfer: A 4D Benchmarking Toolbox for Graph-Centric Predictive Modeling on Relational DBs

Although RDBs store vast amounts of rich, informative data spread across interconnected tables, the progress of predictive machine learning models as applied to such tasks arguably falls well behind advances in other domains such as computer vision or natural language processing. This deficit stems, at least in part, from the lack of established/public RDB benchmarks as needed for training and evaluation purposes. As a result, related model development thus far often defaults to tabular approaches trained on ubiquitous single-table benchmarks, or on the relational side, graph-based alternatives such as GNNs applied to a completely different set of graph datasets devoid of tabular characteristics. To more precisely target RDBs lying at the nexus of these two complementary regimes, we explore a broad class of baseline models predicated on: (i) converting multi-table datasets into graphs using various strategies equipped with efficient subsampling, while preserving tabular characteristics; and (ii) trainable models with well-matched inductive biases that output predictions based on these input subgraphs. Then, to address the dearth of suitable public benchmarks and reduce siloed comparisons, we assemble a diverse collection of (i) large-scale RDB datasets and (ii) coincident predictive tasks. From a delivery standpoint, we operationalize the above four dimensions (4D) of exploration within a unified, scalable open-source toolbox called 4DBInfer. We conclude by presenting evaluations using 4DBInfer, the results of which highlight the importance of considering each such dimension in the design of RDB predictive models, as well as the limitations of more naive approaches such as simply joining adjacent tables. Our source code is released at https://github.com/awslabs/multi-table-benchmark .

cs.LG↗

arXiv2024

UDTIRI: An Online Open-Source Intelligent Road Inspection Benchmark Suite

In the nascent domain of urban digital twins (UDT), the prospects for leveraging cutting-edge deep learning techniques are vast and compelling. Particularly within the specialized area of intelligent road inspection (IRI), a noticeable gap exists, underscored by the current dearth of dedicated research efforts and the lack of large-scale well-annotated datasets. To foster advancements in this burgeoning field, we have launched an online open-source benchmark suite, referred to as UDTIRI. Along with this article, we introduce the road pothole detection task, the first online competition published within this benchmark suite. This task provides a well-annotated dataset, comprising 1,000 RGB images and their pixel/instance-level ground-truth annotations, captured in diverse real-world scenarios under different illumination and weather conditions. Our benchmark provides a systematic and thorough evaluation of state-of-the-art object detection, semantic segmentation, and instance segmentation networks, developed based on either convolutional neural networks or Transformers. We anticipate that our benchmark will serve as a catalyst for the integration of advanced UDT techniques into IRI. By providing algorithms with a more comprehensive understanding of diverse road conditions, we seek to unlock their untapped potential and foster innovation in this critical domain.

cs.CV↗

arXiv2023

Trace and Edit Relation Associations in GPT

This study introduces a novel approach for analyzing and modifying entity relationships in GPT models, diverging from ROME's entity-focused methods. We develop a relation tracing technique to understand the influence of language model computations on relationship judgments. Using the FewRel dataset, we identify key roles of MLP modules and attention mechanisms in processing relationship information. Our method, tested against ROME on a new dataset, shows improved balance in specificity and generalization, underscoring the potential of manipulating early-layer modules for enhanced model understanding and accuracy.

cs.CL↗

arXiv2023

On the Initialization of Graph Neural Networks

Graph Neural Networks (GNNs) have displayed considerable promise in graph representation learning across various applications. The core learning process requires the initialization of model weight matrices within each GNN layer, which is typically accomplished via classic initialization methods such as Xavier initialization. However, these methods were originally motivated to stabilize the variance of hidden embeddings and gradients across layers of Feedforward Neural Networks (FNNs) and Convolutional Neural Networks (CNNs) to avoid vanishing gradients and maintain steady information flow. In contrast, within the GNN context classical initializations disregard the impact of the input graph structure and message passing on variance. In this paper, we analyze the variance of forward and backward propagation across GNN layers and show that the variance instability of GNN initializations comes from the combined effect of the activation function, hidden dimension, graph structure and message passing. To better account for these influence factors, we propose a new initialization method for Variance Instability Reduction within GNN Optimization (Virgo), which naturally tends to equate forward and backward variances across successive layers. We conduct comprehensive experiments on 15 datasets to show that Virgo can lead to superior model performance and more stable variance at initialization on node classification, link prediction and graph classification tasks. Codes are in https://github.com/LspongebobJH/virgo_icml2023.

cs.LG↗

arXiv2023

Improving full-waveform inversion based on sparse regularisation for geophysical data

Full Waveform Inversion (FWI) is an advanced geophysical inversion technique. In fields such as oil exploration and geology, FWI is used for providing images of subsurface structures with higher resolution. The conventional algorithm minimises the misfit error by calculating the least squares of the wavefield solutions between observed data and simulated data, followed by gradient direction and model update increment. Since the gradient is calculated by forward and backward wavefields, the high-accuracy model update relies on accurate forward and backward wavefield modelling. However, the quality of wavefield solutions obtained in practical situations could be poor and does not meet the requirements of high-resolution FWI. Specifically, the low-frequency wavefield is easily affected by noise and downsampling, which influences data quality, while the high-frequency wavefield is susceptible to spatial aliasing effects that produce imaging artefacts. Therefore, we propose using an algorithm called Sparse Relaxation Regularised Regression to optimise the wavefield solution in frequency domain FWI, which is the forward and backward wavefield obtained from the Helmholtz equation, and thus improve the accuracy of the FWI. The sparse relaxed regularised regression algorithm combines sparsity and regularisation, allowing the broadband FWI to reduce the effects of noise and outliers, which can provide data supplementation in the low-frequency band and anti-aliasing in the high-frequency band. Our numerical examples demonstrate the wavefield optimization effect of the sparse relaxed regularised regression-based algorithm in various cases. The accuracy and stability of the improved algorithm are verified in comparison to the Tikhonov regularisation algorithm.

physics.geo-ph↗

arXiv2023

Inexact Augmented Lagrangian Method-Based Full-waveform Inversion with Randomized Singular Value Decomposition

Full Waveform Inversion (FWI) is a modeling algorithm used for seismic data processing and subsurface structure inversion. Theoretically, the main advantage of FWI is its ability to obtain useful subsurface structure information, such as velocity and density, from complex seismic data through inversion simulation. However, under complex conditions, FWI is difficult to achieve high-resolution imaging results, and most of the cases are due to random noise, initial model, or inversion parameters and so on. Therefore, we consider an effective image processing and dimension reduction tool, randomized singular value decomposition (rSVD) - weighted truncated nuclear norm regularization (WTNNR), for embedding FWI to achieve high-resolution imaging results. This algorithm obtains a truncated matrix approximating the original matrix by reducing the rank of the velocity increment matrix, thus achieving the truncation of noisy data, with the truncation range controlled by WTNNR. Subsequently, we employ an inexact augmented Lagrangian method (iALM) algorithm in the optimization to compress the solution space range, thus relaxing the dependence of FWI and rSVD-WTNNR on the initial model and accelerating the convergence rate of the objective function. We tested on two sets of synthetic data, and the results show that compared with traditional FWI, our method can more effectively suppress the impact of random noise, thus obtaining higher resolution and more accurate subsurface model information. Meanwhile, due to the introduction of iALM, our method also significantly improves the convergence rate. This work indicates that the combination of rSVD-WTNNR and FWI is an effective imaging strategy which can help to solve the challenges faced by traditional FWI.

physics.geo-ph↗

arXiv2023

Improved full-waveform inversion for seismic data in the presence of noise based on the K-support norm

Full-waveform inversion (FWI) is known as a seismic data processing method that achieves high-resolution imaging. In the inversion part of the method that brings high resolution in finding a convergence point in the model space, a local numerical optimization algorithm minimizes the objective function based on the norm using the least-square form. Since the norm is sensitive to outliers and noise, the method may often lead to inaccurate imaging results. Thus, a new regulation form with a more practical relaxation form is proposed to solve the overfitting drawback caused by the use of the norm,, namely the K-support norm, which has the form of more reasonable and tighter constraints. In contrast to the least-square method that minimizes the norm, our K-support constraints combine the and the norms. Then, a quadratic penalty method is adopted to linearize the non-linear problem to lighten the computational load. This paper introduces the concept of the K-support norm and integrates this scheme with the quadratic penalty problem to improve the convergence and robustness against background noise. In the numerical example, two synthetic models are tested to clarify the effectiveness of the K-support norm by comparison to the conventional norm with noisy data set. Experimental results indicate that the modified FWI based on the new regularization form effectively improves inversion accuracy and stability, which significantly enhances the lateral resolution of depth inversion even with data with a low signal-to-noise ratio (SNR).

physics.geo-ph↗

arXiv2023

COVRECON: Combining Genome-scale Metabolic Network Reconstruction and Data-driven Inverse Modeling to Reveal Changes in Metabolic Interaction Networks

One central goal of systems biology is to infer biochemical regulations from large-scale OMICS data. Many aspects of cellular physiology and organism phenotypes could be understood as a result of the metabolic interaction network dynamics. Previously, we have derived a mathematical method addressing this problem using metabolomics data for the inverse calculation of a biochemical Jacobian network. However, these algorithms for this inference are limited by two issues: they rely on structural network information that needs to be assembled manually, and they are numerically unstable due to ill-conditioned regression problems, which makes them inadequate for dealing with large-scale metabolic networks. In this work, we present a novel regression-loss based inverse Jacobian algorithm and related workflow COVRECON. It consists of two parts: a, Sim-Network and b, Inverse differential Jacobian evaluation. Sim-Network automatically generates an organism-specific enzyme and reaction dataset from Bigg and KEGG databases, which is then used to reconstruct the Jacobian's structure for a specific metabolomics dataset. Instead of directly solving a regression problem, the new inverse differential Jacobian part is based on a more robust approach and rates the biochemical interactions according to their relevance from large-scale metabolomics data. This approach is illustrated by in silico stochastic analysis with different-sized metabolic networks from the BioModels database. The advantages of COVRECON are that 1) it automatically reconstructs a data-driven superpathway metabolic interaction model; 2) more general network structures can be considered; 3) the new inverse algorithms improve stability, decrease computation time, and extend to large-scale models

q-bio.MN↗

arXiv2021

Network In Graph Neural Network

Graph Neural Networks (GNNs) have shown success in learning from graph structured data containing node/edge feature information, with application to social networks, recommendation, fraud detection and knowledge graph reasoning. In this regard, various strategies have been proposed in the past to improve the expressiveness of GNNs. For example, one straightforward option is to simply increase the parameter size by either expanding the hid-den dimension or increasing the number of GNN layers. However, wider hidden layers can easily lead to overfitting, and incrementally adding more GNN layers can potentially result in over-smoothing.In this paper, we present a model-agnostic methodology, namely Network In Graph Neural Network (NGNN ), that allows arbitrary GNN models to increase their model capacity by making the model deeper. However, instead of adding or widening GNN layers, NGNN deepens a GNN model by inserting non-linear feedforward neural network layer(s) within each GNN layer. An analysis of NGNN as applied to a GraphSage base GNN on ogbn-products data demonstrate that it can keep the model stable against either node feature or graph structure perturbations. Furthermore, wide-ranging evaluation results on both node classification and link prediction tasks show that NGNN works reliably across diverse GNN architectures.For instance, it improves the test accuracy of GraphSage on the ogbn-products by 1.6% and improves the hits@100 score of SEAL on ogbl-ppa by 7.08% and the hits@20 score of GraphSage+Edge-Attr on ogbl-ppi by 6.22%. And at the time of this submission, it achieved two first places on the OGB link prediction leaderboard.

cs.LG↗