Searcharxiv⌕ Search

arXiv subjects

Yan Xu

Publications and source records attributed to Yan Xu.

At least 163 records · Page 9Linked to original sources
arXiv2023

Partitioned Saliency Ranking with Dense Pyramid Transformers

In recent years, saliency ranking has emerged as a challenging task focusing on assessing the degree of saliency at instance-level. Being subjective, even humans struggle to identify the precise order of all salient instances. Previous approaches undertake the saliency ranking by directly sorting the rank scores of salient instances, which have not explicitly resolved the inherent ambiguities. To overcome this limitation, we propose the ranking by partition paradigm, which segments unordered salient instances into partitions and then ranks them based on the correlations among these partitions. The ranking by partition paradigm alleviates ranking ambiguities in a general sense, as it consistently improves the performance of other saliency ranking models. Additionally, we introduce the Dense Pyramid Transformer (DPT) to enable global cross-scale interactions, which significantly enhances feature interactions with reduced computational burden. Extensive experiments demonstrate that our approach outperforms all existing methods. The code for our method is available at \url{https://github.com/ssecv/PSR}.

cs.CV↗
arXiv2023

NusaCrowd: Open Source Initiative for Indonesian NLP Resources

We present NusaCrowd, a collaborative initiative to collect and unify existing resources for Indonesian languages, including opening access to previously non-public resources. Through this initiative, we have brought together 137 datasets and 118 standardized data loaders. The quality of the datasets has been assessed manually and automatically, and their value is demonstrated through multiple experiments. NusaCrowd's data collection enables the creation of the first zero-shot benchmarks for natural language understanding and generation in Indonesian and the local languages of Indonesia. Furthermore, NusaCrowd brings the creation of the first multilingual automatic speech recognition benchmark in Indonesian and the local languages of Indonesia. Our work strives to advance natural language processing (NLP) research for languages that are under-represented despite being widely spoken.

cs.CL↗
arXiv2023

Urban Radiance Field Representation with Deformable Neural Mesh Primitives

Neural Radiance Fields (NeRFs) have achieved great success in the past few years. However, most current methods still require intensive resources due to ray marching-based rendering. To construct urban-level radiance fields efficiently, we design Deformable Neural Mesh Primitive~(DNMP), and propose to parameterize the entire scene with such primitives. The DNMP is a flexible and compact neural variant of classic mesh representation, which enjoys both the efficiency of rasterization-based rendering and the powerful neural representation capability for photo-realistic image synthesis. Specifically, a DNMP consists of a set of connected deformable mesh vertices with paired vertex features to parameterize the geometry and radiance information of a local area. To constrain the degree of freedom for optimization and lower the storage budgets, we enforce the shape of each primitive to be decoded from a relatively low-dimensional latent space. The rendering colors are decoded from the vertex features (interpolated with rasterization) by a view-dependent MLP. The DNMP provides a new paradigm for urban-level scene representation with appealing properties: $(1)$ High-quality rendering. Our method achieves leading performance for novel view synthesis in urban scenarios. $(2)$ Low computational costs. Our representation enables fast rendering (2.07ms/1k pixels) and low peak memory usage (110MB/1k pixels). We also present a lightweight version that can run 33$\times$ faster than vanilla NeRFs, and comparable to the highly-optimized Instant-NGP (0.61 vs 0.71ms/1k pixels). Project page: \href{https://dnmp.github.io/}{https://dnmp.github.io/}.

cs.CV↗
arXiv2023

NeRF-Loc: Transformer-Based Object Localization Within Neural Radiance Fields

Neural Radiance Fields (NeRFs) have become a widely-applied scene representation technique in recent years, showing advantages for robot navigation and manipulation tasks. To further advance the utility of NeRFs for robotics, we propose a transformer-based framework, NeRF-Loc, to extract 3D bounding boxes of objects in NeRF scenes. NeRF-Loc takes a pre-trained NeRF model and camera view as input and produces labeled, oriented 3D bounding boxes of objects as output. Using current NeRF training tools, a robot can train a NeRF environment model in real-time and, using our algorithm, identify 3D bounding boxes of objects of interest within the NeRF for downstream navigation or manipulation tasks. Concretely, we design a pair of paralleled transformer encoder branches, namely the coarse stream and the fine stream, to encode both the context and details of target objects. The encoded features are then fused together with attention layers to alleviate ambiguities for accurate object localization. We have compared our method with conventional RGB(-D) based methods that take rendered RGB images and depths from NeRFs as inputs. Our method is better than the baselines.

cs.CV↗
arXiv2023

DRMC: A Generalist Model with Dynamic Routing for Multi-Center PET Image Synthesis

Multi-center positron emission tomography (PET) image synthesis aims at recovering low-dose PET images from multiple different centers. The generalizability of existing methods can still be suboptimal for a multi-center study due to domain shifts, which result from non-identical data distribution among centers with different imaging systems/protocols. While some approaches address domain shifts by training specialized models for each center, they are parameter inefficient and do not well exploit the shared knowledge across centers. To address this, we develop a generalist model that shares architecture and parameters across centers to utilize the shared knowledge. However, the generalist model can suffer from the center interference issue, \textit{i.e.} the gradient directions of different centers can be inconsistent or even opposite owing to the non-identical data distribution. To mitigate such interference, we introduce a novel dynamic routing strategy with cross-layer connections that routes data from different centers to different experts. Experiments show that our generalist model with dynamic routing (DRMC) exhibits excellent generalizability across centers. Code and data are available at: https://github.com/Yaziwel/Multi-Center-PET-Image-Synthesis.

eess.IV↗
arXiv2023

Zero-shot Nuclei Detection via Visual-Language Pre-trained Models

Large-scale visual-language pre-trained models (VLPM) have proven their excellent performance in downstream object detection for natural scenes. However, zero-shot nuclei detection on H\&E images via VLPMs remains underexplored. The large gap between medical images and the web-originated text-image pairs used for pre-training makes it a challenging task. In this paper, we attempt to explore the potential of the object-level VLPM, Grounded Language-Image Pre-training (GLIP) model, for zero-shot nuclei detection. Concretely, an automatic prompts design pipeline is devised based on the association binding trait of VLPM and the image-to-text VLPM BLIP, avoiding empirical manual prompts engineering. We further establish a self-training framework, using the automatically designed prompts to generate the preliminary results as pseudo labels from GLIP and refine the predicted boxes in an iterative manner. Our method achieves a remarkable performance for label-free nuclei detection, surpassing other comparison methods. Foremost, our work demonstrates that the VLPM pre-trained on natural image-text pairs exhibits astonishing potential for downstream tasks in the medical field as well. Code will be released at https://github.com/wuyongjianCODE/VLPMNuD.

cs.CV↗
arXiv2023

A Framework for Operational Volume Generation for Urban Air Mobility Strategic Deconfliction

Strategic pre-flight functions focus on the planning and deconfliction of routes for aircraft systems. The urban air mobility concept calls for higher levels of autonomy with onboard and en route functions but also strategic and pre-flight systems. Existing endeavours into strategic pre-flight functions focus on improving the route generation and strategic deconfliction of these routes. Introduced with the urban air mobility concept is the premise of operational volumes. These 4D regions of airspace, describe the intended operational region for an aircraft for finite time. Chaining these together forms a contract of finite operational volumes over a given route. It is no longer enough to only deconflict routes within the airspace, but to now consider these 4D operational volumes. To provide an effective all-in-one approach, we propose a novel framework for generating routes and accompanying contracts of operational volumes, along with deconfliction focused around 4D operational volumes. Experimental results show efficiency of operational volume generation utilising reachability analysis and demonstrate sufficient success in deconfliction of operational volumes.

cs.OH↗
arXiv2023

A Novel Dual-pooling Attention Module for UAV Vehicle Re-identification

Vehicle re-identification (Re-ID) involves identifying the same vehicle captured by other cameras, given a vehicle image. It plays a crucial role in the development of safe cities and smart cities. With the rapid growth and implementation of unmanned aerial vehicles (UAVs) technology, vehicle Re-ID in UAV aerial photography scenes has garnered significant attention from researchers. However, due to the high altitude of UAVs, the shooting angle of vehicle images sometimes approximates vertical, resulting in fewer local features for Re-ID. Therefore, this paper proposes a novel dual-pooling attention (DpA) module, which achieves the extraction and enhancement of locally important information about vehicles from both channel and spatial dimensions by constructing two branches of channel-pooling attention (CpA) and spatial-pooling attention (SpA), and employing multiple pooling operations to enhance the attention to fine-grained information of vehicles. Specifically, the CpA module operates between the channels of the feature map and splices features by combining four pooling operations so that vehicle regions containing discriminative information are given greater attention. The SpA module uses the same pooling operations strategy to identify discriminative representations and merge vehicle features in image regions in a weighted manner. The feature information of both dimensions is finally fused and trained jointly using label smoothing cross-entropy loss and hard mining triplet loss, thus solving the problem of missing detail information due to the high height of UAV shots. The proposed method's effectiveness is demonstrated through extensive experiments on the UAV-based vehicle datasets VeRi-UAV and VRU.

cs.CV↗
arXiv2023

Cyclic Learning: Bridging Image-level Labels and Nuclei Instance Segmentation

Nuclei instance segmentation on histopathology images is of great clinical value for disease analysis. Generally, fully-supervised algorithms for this task require pixel-wise manual annotations, which is especially time-consuming and laborious for the high nuclei density. To alleviate the annotation burden, we seek to solve the problem through image-level weakly supervised learning, which is underexplored for nuclei instance segmentation. Compared with most existing methods using other weak annotations (scribble, point, etc.) for nuclei instance segmentation, our method is more labor-saving. The obstacle to using image-level annotations in nuclei instance segmentation is the lack of adequate location information, leading to severe nuclei omission or overlaps. In this paper, we propose a novel image-level weakly supervised method, called cyclic learning, to solve this problem. Cyclic learning comprises a front-end classification task and a back-end semi-supervised instance segmentation task to benefit from multi-task learning (MTL). We utilize a deep learning classifier with interpretability as the front-end to convert image-level labels to sets of high-confidence pseudo masks and establish a semi-supervised architecture as the back-end to conduct nuclei instance segmentation under the supervision of these pseudo masks. Most importantly, cyclic learning is designed to circularly share knowledge between the front-end classifier and the back-end semi-supervised part, which allows the whole system to fully extract the underlying information from image-level labels and converge to a better optimum. Experiments on three datasets demonstrate the good generality of our method, which outperforms other image-level weakly supervised methods for nuclei instance segmentation, and achieves comparable performance to fully-supervised methods.

cs.CV↗
arXiv2023

Port-Hamiltonian formulations of the incompressible Euler equations with a free surface

In this paper, we present port-Hamiltonian formulations of the incompressible Euler equations with a free surface governed by surface tension and gravity forces, modelling e.g. capillary and gravity waves and the evolution of droplets in air. Three sets of variables are considered, namely $(v,Σ)$, $(η,ϕ_{\partial},Σ)$ and $(ω,ϕ_{\partial},Σ)$, with $v$ the velocity, $η$ the solenoidal velocity, $ϕ_{\partial}$ a potential, $ω$ the vorticity, and $Σ$ the free surface, resulting in the incompressible Euler equations in primitive variables and the vorticity equation. First, the Hamiltonian formulation for the incompressible Euler equations in a domain with a free surface combined with a fixed boundary surface with a homogeneous boundary condition will be derived in the proper Sobolev spaces of differential forms. Next, these results will be extended to port-Hamiltonian formulations allowing inhomogeneous boundary conditions and a non-zero energy flow through the boundaries. Our main results are the construction and proof of Dirac structures in suitable Sobolev spaces of differential forms for each variable set, which provides the core of any port-Hamiltonian formulation. Finally, it is proven that the state dependent Dirac structures are related to Poisson brackets that are linear, skew-symmetric and satisfy the Jacobi identity.

math.AP↗
arXiv2023

Embodied Concept Learner: Self-supervised Learning of Concepts and Mapping through Instruction Following

Humans, even at a very early age, can learn visual concepts and understand geometry and layout through active interaction with the environment, and generalize their compositions to complete tasks described by natural languages in novel scenes. To mimic such capability, we propose Embodied Concept Learner (ECL) in an interactive 3D environment. Specifically, a robot agent can ground visual concepts, build semantic maps and plan actions to complete tasks by learning purely from human demonstrations and language instructions, without access to ground-truth semantic and depth supervisions from simulations. ECL consists of: (i) an instruction parser that translates the natural languages into executable programs; (ii) an embodied concept learner that grounds visual concepts based on language descriptions; (iii) a map constructor that estimates depth and constructs semantic maps by leveraging the learned concepts; and (iv) a program executor with deterministic policies to execute each program. ECL has several appealing benefits thanks to its modularized design. Firstly, it enables the robotic agent to learn semantics and depth unsupervisedly acting like babies, e.g., ground concepts through active interaction and perceive depth by disparities when moving forward. Secondly, ECL is fully transparent and step-by-step interpretable in long-term planning. Thirdly, ECL could be beneficial for the embodied instruction following (EIF), outperforming previous works on the ALFRED benchmark when the semantic label is not provided. Also, the learned concept can be reused for other downstream tasks, such as reasoning of object states. Project page: http://ecl.csail.mit.edu/

cs.CV↗
arXiv2023

Probing into the Possible Range of the U Bosonic Coupling Constants in Neutron Stars Containing Hyperons

The range of the U bosonic coupling constants in neutron star matter is a very interesting but still unsolved problem which has multifaceted influences in nuclear physics, particle physics, astrophysics and cosmology. The combination of the theoretical numerical simulation and the recent observations provides a very good opportunity to solve this problem. In the present work, the range of the U bosonic coupling constants is inferred based on the three relations of the mass-radius, mass-frequency and mass-tidal deformability in neutron star containing hyperons using the GM1, TM1 and NL3 parameter sets under the two flavor symmetries of the SU(6) and SU(3) in the framework of the relativistic mean field theory. Combined with observations from PSRs J1614-2230, J0348+0432, J2215-5135, J0952-0607, J0740+6620, J0030-0451, J1748-2446ad, XTE J1739-285, GW170817 and GW190814 events, our numerical results show that the U bosonic coupling constants may tend to be within the range from 0 to 20 GeV$^{-2}$ in neutron star containing hyperons. Moreover, the numerical results of the three relations obtained by the SU(3) symmetry are better in accordance with observation data than those obtained by the SU(6) symmetry. The results will help us to improve the strict constraints of the equation of state for neutron stars containing hyperons.

nucl-th↗
arXiv2023

KILM: Knowledge Injection into Encoder-Decoder Language Models

Large pre-trained language models (PLMs) have been shown to retain implicit knowledge within their parameters. To enhance this implicit knowledge, we propose Knowledge Injection into Language Models (KILM), a novel approach that injects entity-related knowledge into encoder-decoder PLMs, via a generative knowledge infilling objective through continued pre-training. This is done without architectural modifications to the PLMs or adding additional parameters. Experimental results over a suite of knowledge-intensive tasks spanning numerous datasets show that KILM enables models to retain more knowledge and hallucinate less, while preserving their original performance on general NLU and NLG tasks. KILM also demonstrates improved zero-shot performances on tasks such as entity disambiguation, outperforming state-of-the-art models having 30x more parameters.

cs.CL↗
arXiv2023

Multi-instrument Comparative Study of Temperature, Number Density, and Emission Measure during the Precursor Phase of a Solar Flare

We present a multi-instrument study of the two precursor brightenings prior to the M6.5 flare (SOL2015-06-22T18:23) in the NOAA Active Region 12371, with a focus on the temperature (T), electron number density (n), and emission measure (EM). The data used in this study were obtained from four instruments with a variety of wavelengths, i.e., the Solar Dynamics Observatory's Atmospheric Imaging Assembly (AIA), in six extreme ultraviolet (EUV) passbands; the Expanded Owens Valley Solar Array (EOVSA) in microwave (MW); the Reuven Ramaty High Energy Solar Spectroscopic Imager (RHESSI) in hard X-rays (HXR); and the Geostationary Operational Environmental Satellite (GOES) in soft X-rays (SXR). We compare the temporal variations of T, n, and EM derived from the different data sets. Here are the key results. (1) GOES SXR and AIA EUV have almost identical EM variations (1.5-3x10^48 per cm^3) and very similar T variations, from 8 to 15 million Kelvin (MK). (2) Listed from highest to lowest, EOVSA MW provides the highest temperature variations (15-60 MK), followed by RHESSI HXR (10-24 MK), then GOES SXR and AIA EUV (8-15 MK). (3) The EM variation from the RHESSI HXR measurements is always less than the values from AIA EUV and GOES SXR by at most 20 times. The number density variation from EOVSA MW is greater than the value from AIA EUV by at most 100 times. The results quantitatively describe the differences in the thermal parameters at the precursor phase, as measured by different instruments operating at different wavelength regimes and for different emission mechanisms.

astro-ph.SR↗
arXiv2023

Optimization of Topology-Aware Job Allocation on a High-Performance Computing Cluster by Neural Simulated Annealing

Jobs on high-performance computing (HPC) clusters can suffer significant performance degradation due to inter-job network interference. Topology-aware job allocation problem (TJAP) is such a problem that decides how to dedicate nodes to specific applications to mitigate inter-job network interference. In this paper, we study the window-based TJAP on a fat-tree network aiming at minimizing the cost of communication hop, a defined inter-job interference metric. The window-based approach for scheduling repeats periodically taking the jobs in the queue and solving an assignment problem that maps jobs to the available nodes. Two special allocation strategies are considered, i.e., static continuity assignment strategy (SCAS) and dynamic continuity assignment strategy (DCAS). For the SCAS, a 0-1 integer programming is developed. For the DCAS, an approach called neural simulated algorithm (NSA), which is an extension to simulated algorithm (SA) that learns a repair operator and employs them in a guided heuristic search, is proposed. The efficacy of NSA is demonstrated with a computational study against SA and SCIP. The results of numerical experiments indicate that both the model and algorithm proposed in this paper are effective.

cs.DC↗
arXiv2023

Weakly-Supervised 3D Medical Image Segmentation using Geometric Prior and Contrastive Similarity

Medical image segmentation is almost the most important pre-processing procedure in computer-aided diagnosis but is also a very challenging task due to the complex shapes of segments and various artifacts caused by medical imaging, (i.e., low-contrast tissues, and non-homogenous textures). In this paper, we propose a simple yet effective segmentation framework that incorporates the geometric prior and contrastive similarity into the weakly-supervised segmentation framework in a loss-based fashion. The proposed geometric prior built on point cloud provides meticulous geometry to the weakly-supervised segmentation proposal, which serves as better supervision than the inherent property of the bounding-box annotation (i.e., height and width). Furthermore, we propose contrastive similarity to encourage organ pixels to gather around in the contrastive embedding space, which helps better distinguish low-contrast tissues. The proposed contrastive embedding space can make up for the poor representation of the conventionally-used gray space. Extensive experiments are conducted to verify the effectiveness and the robustness of the proposed weakly-supervised segmentation framework. The proposed framework is superior to state-of-the-art weakly-supervised methods on the following publicly accessible datasets: LiTS 2017 Challenge, KiTS 2021 Challenge, and LPBA40. We also dissect our method and evaluate the performance of each component.

eess.IV↗