Searcharxiv⌕ Search

arXiv subjects

Yan Xu

Publications and source records attributed to Yan Xu.

At least 127 records · Page 7Linked to original sources
arXiv2024

All-In-One Medical Image Restoration via Task-Adaptive Routing

Although single-task medical image restoration (MedIR) has witnessed remarkable success, the limited generalizability of these methods poses a substantial obstacle to wider application. In this paper, we focus on the task of all-in-one medical image restoration, aiming to address multiple distinct MedIR tasks with a single universal model. Nonetheless, due to significant differences between different MedIR tasks, training a universal model often encounters task interference issues, where different tasks with shared parameters may conflict with each other in the gradient update direction. This task interference leads to deviation of the model update direction from the optimal path, thereby affecting the model's performance. To tackle this issue, we propose a task-adaptive routing strategy, allowing conflicting tasks to select different network paths in spatial and channel dimensions, thereby mitigating task interference. Experimental results demonstrate that our proposed \textbf{A}ll-in-one \textbf{M}edical \textbf{I}mage \textbf{R}estoration (\textbf{AMIR}) network achieves state-of-the-art performance in three MedIR tasks: MRI super-resolution, CT denoising, and PET synthesis, both in single-task and all-in-one settings. The code and data will be available at \href{https://github.com/Yaziwel/All-In-One-Medical-Image-Restoration-via-Task-Adaptive-Routing.git}{https://github.com/Yaziwel/AMIR}.

cs.CV↗
arXiv2024

An interpretable imbalanced semi-supervised deep learning framework for improving differential diagnosis of skin diseases

Dermatological diseases are among the most common disorders worldwide. This paper presents the first study of the interpretability and imbalanced semi-supervised learning of the multiclass intelligent skin diagnosis framework (ISDL) using 58,457 skin images with 10,857 unlabeled samples. Pseudo-labelled samples from minority classes have a higher probability at each iteration of class-rebalancing self-training, thereby promoting the utilization of unlabeled samples to solve the class imbalance problem. Our ISDL achieved a promising performance with an accuracy of 0.979, sensitivity of 0.975, specificity of 0.973, macro-F1 score of 0.974 and area under the receiver operating characteristic curve (AUC) of 0.999 for multi-label skin disease classification. The Shapley Additive explanation (SHAP) method is combined with our ISDL to explain how the deep learning model makes predictions. This finding is consistent with the clinical diagnosis. We also proposed a sampling distribution optimisation strategy to select pseudo-labelled samples in a more effective manner using ISDLplus. Furthermore, it has the potential to relieve the pressure placed on professional doctors, as well as help with practical issues associated with a shortage of such doctors in rural areas.

cs.CV↗
arXiv2024

Preserving Tumor Volumes for Unsupervised Medical Image Registration

Medical image registration is a critical task that estimates the spatial correspondence between pairs of images. However, current traditional and deep-learning-based methods rely on similarity measures to generate a deforming field, which often results in disproportionate volume changes in dissimilar regions, especially in tumor regions. These changes can significantly alter the tumor size and underlying anatomy, which limits the practical use of image registration in clinical diagnosis. To address this issue, we have formulated image registration with tumors as a constraint problem that preserves tumor volumes while maximizing image similarity in other normal regions. Our proposed strategy involves a two-stage process. In the first stage, we use similarity-based registration to identify potential tumor regions by their volume change, generating a soft tumor mask accordingly. In the second stage, we propose a volume-preserving registration with a novel adaptive volume-preserving loss that penalizes the change in size adaptively based on the masks calculated from the previous stage. Our approach balances image similarity and volume preservation in different regions, i.e., normal and tumor regions, by using soft tumor masks to adjust the imposition of volume-preserving loss on each one. This ensures that the tumor volume is preserved during the registration process. We have evaluated our strategy on various datasets and network architectures, demonstrating that our method successfully preserves the tumor volume while achieving comparable registration results with state-of-the-art methods. Our codes is available at: \url{https://dddraxxx.github.io/Volume-Preserving-Registration/}.

eess.IV↗
arXiv2024

OmniActions: Predicting Digital Actions in Response to Real-World Multimodal Sensory Inputs with LLMs

The progression to "Pervasive Augmented Reality" envisions easy access to multimodal information continuously. However, in many everyday scenarios, users are occupied physically, cognitively or socially. This may increase the friction to act upon the multimodal information that users encounter in the world. To reduce such friction, future interactive interfaces should intelligently provide quick access to digital actions based on users' context. To explore the range of possible digital actions, we conducted a diary study that required participants to capture and share the media that they intended to perform actions on (e.g., images or audio), along with their desired actions and other contextual information. Using this data, we generated a holistic design space of digital follow-up actions that could be performed in response to different types of multimodal sensory inputs. We then designed OmniActions, a pipeline powered by large language models (LLMs) that processes multimodal sensory inputs and predicts follow-up actions on the target information grounded in the derived design space. Using the empirical data collected in the diary study, we performed quantitative evaluations on three variations of LLM techniques (intent classification, in-context learning and finetuning) and identified the most effective technique for our task. Additionally, as an instantiation of the pipeline, we developed an interactive prototype and reported preliminary user feedback about how people perceive and react to the action predictions and its errors.

cs.HC↗
arXiv2024

Solar Flare Ribbon Fronts. II. Evolution of heating rates in individual flare footpoints

Solar flare ribbon fronts appear ahead of the bright structures that normally characterise solar flares, and can persist for an extended period of time in spatially localised patches before transitioning to `regular' bright ribbons. They likely represent the initial onset of flare energy deposition into the chromosphere. Chromospheric spectra (e.g. He I 10830A and the Mg II near-UV lines) from ribbon fronts exhibit properties rather different to typical flare behaviour. In prior numerical modelling efforts we were unable to reproduce the long lifetime of ribbon fronts. Here we present a series of numerical experiments that are rather simple but which have important implications. We inject a very low flux of nonthermal electrons ($F = 5\times10^{8}$ erg s$^{-1}$ cm$^{-2}$) into the chromosphere for 100 s before ramping up to standard flare energy fluxes $(F = 10^{10-11}$ erg s$^{-1}$ cm$^{-2}$). Synthetic spectra not only sustained their ribbon front-like properties for significantly longer, in the case of harder nonthermal electron spectra the ribbon front behaviour persisted for the entirety of this weak-heating phase. Lengthening or shortening the duration of the weak-heating phase commensurately lengthened or shortened the ribbon front lifetimes. Ribbon fronts transitioned to regular bright ribbons when the upper chromosphere became sufficiently hot and dense, which happened faster for softer nonthermal electron spectra. Thus, the lifetime of flare ribbon fronts are a direct measure of the duration over which a relatively low flux of high energy electrons precipitates to the chromosphere prior to the bombardment of a much larger energy flux.

astro-ph.SR↗
arXiv2024

Lightweight Facial Attractiveness Prediction Using Dual Label Distribution

Facial attractiveness prediction (FAP) aims to assess facial attractiveness automatically based on human aesthetic perception. Previous methods using deep convolutional neural networks have improved the performance, but their large-scale models have led to a deficiency in flexibility. In addition, most methods fail to take full advantage of the dataset. In this paper, we present a novel end-to-end FAP approach that integrates dual label distribution and lightweight design. The manual ratings, attractiveness score, and standard deviation are aggregated explicitly to construct a dual-label distribution to make the best use of the dataset, including the attractiveness distribution and the rating distribution. Such distributions, as well as the attractiveness score, are optimized under a joint learning framework based on the label distribution learning (LDL) paradigm. The data processing is simplified to a minimum for a lightweight design, and MobileNetV2 is selected as our backbone. Extensive experiments are conducted on two benchmark datasets, where our approach achieves promising results and succeeds in balancing performance and efficiency. Ablation studies demonstrate that our delicately designed learning modules are indispensable and correlated. Additionally, the visualization indicates that our approach can perceive facial attractiveness and capture attractive facial regions to facilitate semantic predictions. The code is available at https://github.com/enquan/2D_FAP.

cs.CV↗
arXiv2024

Ada-DF: An Adaptive Label Distribution Fusion Network For Facial Expression Recognition

Facial expression recognition (FER) plays a significant role in our daily life. However, annotation ambiguity in the datasets could greatly hinder the performance. In this paper, we address FER task via label distribution learning paradigm, and develop a dual-branch Adaptive Distribution Fusion (Ada-DF) framework. One auxiliary branch is constructed to obtain the label distributions of samples. The class distributions of emotions are then computed through the label distributions of each emotion. Finally, those two distributions are adaptively fused according to the attention weights to train the target branch. Extensive experiments are conducted on three real-world datasets, RAF-DB, AffectNet and SFEW, where our Ada-DF shows advantages over the state-of-the-art works.

cs.CV↗
arXiv2024

Urban Architect: Steerable 3D Urban Scene Generation with Layout Prior

Text-to-3D generation has achieved remarkable success via large-scale text-to-image diffusion models. Nevertheless, there is no paradigm for scaling up the methodology to urban scale. Urban scenes, characterized by numerous elements, intricate arrangement relationships, and vast scale, present a formidable barrier to the interpretability of ambiguous textual descriptions for effective model optimization. In this work, we surmount the limitations by introducing a compositional 3D layout representation into text-to-3D paradigm, serving as an additional prior. It comprises a set of semantic primitives with simple geometric structures and explicit arrangement relationships, complementing textual descriptions and enabling steerable generation. Upon this, we propose two modifications -- (1) We introduce Layout-Guided Variational Score Distillation to address model optimization inadequacies. It conditions the score distillation sampling process with geometric and semantic constraints of 3D layouts. (2) To handle the unbounded nature of urban scenes, we represent 3D scene with a Scalable Hash Grid structure, incrementally adapting to the growing scale of urban scenes. Extensive experiments substantiate the capability of our framework to scale text-to-3D generation to large-scale urban scenes that cover over 1000m driving distance for the first time. We also present various scene editing demonstrations, showing the powers of steerable urban scene generation. Website: https://urbanarchitect.github.io.

cs.CV↗
arXiv2024

Super-Resolution of SOHO/MDI Magnetograms of Solar Active Regions Using SDO/HMI Data and an Attention-Aided Convolutional Neural Network

Image super-resolution has been an important subject in image processing and recognition. Here, we present an attention-aided convolutional neural network (CNN) for solar image super-resolution. Our method, named SolarCNN, aims to enhance the quality of line-of-sight (LOS) magnetograms of solar active regions (ARs) collected by the Michelson Doppler Imager (MDI) on board the Solar and Heliospheric Observatory (SOHO). The ground-truth labels used for training SolarCNN are the LOS magnetograms collected by the Helioseismic and Magnetic Imager (HMI) on board the Solar Dynamics Observatory (SDO). Solar ARs consist of strong magnetic fields in which magnetic energy can suddenly be released to produce extreme space weather events, such as solar flares, coronal mass ejections, and solar energetic particles. SOHO/MDI covers Solar Cycle 23, which is stronger with more eruptive events than Cycle 24. Enhanced SOHO/MDI magnetograms allow for better understanding and forecasting of violent events of space weather. Experimental results show that SolarCNN improves the quality of SOHO/MDI magnetograms in terms of the structural similarity index measure (SSIM), Pearson's correlation coefficient (PCC), and the peak signal-to-noise ratio (PSNR).

astro-ph.SR↗
arXiv2024

Well-balanced path-conservative discontinuous Galerkin methods with equilibrium preserving space for two-layer shallow water equations

This paper introduces well-balanced path-conservative discontinuous Galerkin (DG) methods for two-layer shallow water equations, ensuring exactness for both still water and moving water equilibrium steady states. The approach involves approximating the equilibrium variables within the DG piecewise polynomial space, while expressing the DG scheme in the form of path-conservative schemes. To robustly handle the nonconservative products governing momentum exchange between the layers, we incorporate the theory of Dal Maso, LeFloch, and Murat (DLM) within the DG method. Additionally, linear segment paths connecting the equilibrium functions are chosen to guarantee the well-balanced property of the resulting scheme. The simple ``lake-at-rest" steady state is naturally satisfied without any modification, while a specialized treatment of the numerical flux is crucial for preserving the moving water steady state. Extensive numerical examples in one and two dimensions validate the exact equilibrium preservation of the steady state solutions and demonstrate its high-order accuracy. The performance of the method and high-resolution results further underscore its potential as a robust approach for nonconservative hyperbolic balance laws.

math.NA↗
arXiv2024

Seeking Soulmate via Voice: Understanding Promises and Challenges of Online Synchronized Voice-Based Mobile Dating

Online dating has become a popular way for individuals to connect with potential romantic partners. Many dating apps use personal profiles that include a headshot and self-description, allowing users to present themselves and search for compatible matches. However, this traditional model often has limitations. In this study, we explore a non-traditional voice-based dating app called "Soul". Unlike traditional platforms that rely heavily on profile information, Soul facilitates user interactions through voice-based communication. We conducted semi-structured interviews with 18 dedicated Soul users to investigate how they engage with the platform and perceive themselves and others in this unique dating environment. Our findings indicate that the role of voice as a moderator influences impression management and shapes perceptions between the sender and the receiver of the voice. Additionally, the synchronous voice-based and community-based dating model offers benefits to users in the Chinese cultural context. Our study contributes to understanding the affordances introduced by voice-based interactions in online dating in China.

cs.HC↗
arXiv2024

LAVE: LLM-Powered Agent Assistance and Language Augmentation for Video Editing

Video creation has become increasingly popular, yet the expertise and effort required for editing often pose barriers to beginners. In this paper, we explore the integration of large language models (LLMs) into the video editing workflow to reduce these barriers. Our design vision is embodied in LAVE, a novel system that provides LLM-powered agent assistance and language-augmented editing features. LAVE automatically generates language descriptions for the user's footage, serving as the foundation for enabling the LLM to process videos and assist in editing tasks. When the user provides editing objectives, the agent plans and executes relevant actions to fulfill them. Moreover, LAVE allows users to edit videos through either the agent or direct UI manipulation, providing flexibility and enabling manual refinement of agent actions. Our user study, which included eight participants ranging from novices to proficient editors, demonstrated LAVE's effectiveness. The results also shed light on user perceptions of the proposed LLM-assisted editing paradigm and its impact on users' creativity and sense of co-creation. Based on these findings, we propose design implications to inform the future development of agent-assisted content editing.

cs.HC↗
arXiv2024

High Resolution Imaging Spectroscopy of a Tiny Sigmoidal Mini-filament Eruption

Minifilament (MF) eruption producing small jets and micro-flares is regarded as an important source for coronal heating and the solar wind transients through studies mostly based on coronal observations in the extreme ultraviolet (EUV) and X-ray wavelengths. In this study, we focus on the chromospheric plasma diagnostics of a tiny minifilament in quiet Sun located at [71'', 450''] on 2021--08--07 at 19:11 UT observed as part of the ninth encounter of the PSP campaign. Main data obtained are the high cadence, high resolution spectroscopy from the Fast Imaging Solar Spectrograph (FISS) and high-resolution magnetograms from the Near InfraRed Imaging Spectropolarimeter (NIRIS) on the 1.6~m Goode Solar Telescope (GST) at Big Bear Solar Observatory (BBSO). The mini-filament with size $\sim$1''$\times$5'' and a micro-flare are detected in both the H$α$ line center and SDO/AIA 193, 304~Å images. On the NIRIS magnetogram, we found that the cancellation of a magnetic bipole in the footpoints of the minifilament triggered its eruption in a sigmoidal shape. By inversion of the \ha\ and Ca {\sc ii} spectra under the embedded cloud model, we found a temperature increase of 3,800 K in the brightening region, associated with rising speed average of MF increased by 18~$km~s^{-1}$. This cool plasma is also found in the EUV images. We estimate the kinetic energy change of the rising filament as 1.5$\times$$10^{25}$~ergs, and thermal energy accumulation in the MF, 1.4$\times$$10^{25}$~ergs. From the photospheric magnetograms, we find the magnetic energy change is 1.6$\times$$10^{26}$~ergs across the PIL of converging opposite magnetic elements, which amounts to the energy release in the chromosphere in this smallest two-ribbon flare ever observed.

astro-ph.SR↗
arXiv2024

Equilibrium preserving space in discontinuous Galerkin methods for hyperbolic balance laws

In this paper, we develop a general framework for the design of the arbitrary high-order well-balanced discontinuous Galerkin (DG) method for hyperbolic balance laws, including the compressible Euler equations with gravitation and the shallow water equations with horizontal temperature gradients (referred to as the Ripa model). Not only the hydrostatic equilibrium including the more complicated isobaric steady state in Ripa system, but our scheme is also well-balanced for the exact preservation of the moving equilibrium state. The strategy adopted is to approximate the equilibrium variables in the DG piecewise polynomial space, rather than the conservative variables, which is pivotal in the well-balanced property. Our approach provides flexibility in combination with any consistent numerical flux, and it is free of the reference equilibrium state recovery and the special source term treatment. This approach enables the construction of a well-balanced method for non-hydrostatic equilibria in Euler systems. Extensive numerical examples such as moving or isobaric equilibria validate the high order accuracy and exact equilibrium preservation for various flows given by hyperbolic balance laws. With a relatively coarse mesh, it is also possible to capture small perturbations at or close to steady flow without numerical oscillations.

math.NA↗
arXiv2024

Multi-Person 3D Pose Estimation from Multi-View Uncalibrated Depth Cameras

We tackle the task of multi-view, multi-person 3D human pose estimation from a limited number of uncalibrated depth cameras. Recently, many approaches have been proposed for 3D human pose estimation from multi-view RGB cameras. However, these works (1) assume the number of RGB camera views is large enough for 3D reconstruction, (2) the cameras are calibrated, and (3) rely on ground truth 3D poses for training their regression model. In this work, we propose to leverage sparse, uncalibrated depth cameras providing RGBD video streams for 3D human pose estimation. We present a simple pipeline for Multi-View Depth Human Pose Estimation (MVD-HPE) for jointly predicting the camera poses and 3D human poses without training a deep 3D human pose regression model. This framework utilizes 3D Re-ID appearance features from RGBD images to formulate more accurate correspondences (for deriving camera positions) compared to using RGB-only features. We further propose (1) depth-guided camera-pose estimation by leveraging 3D rigid transformations as guidance and (2) depth-constrained 3D human pose estimation by utilizing depth-projected 3D points as an alternative objective for optimization. In order to evaluate our proposed pipeline, we collect three video sets of RGBD videos recorded from multiple sparse-view depth cameras and ground truth 3D poses are manually annotated. Experiments show that our proposed method outperforms the current 3D human pose regression-free pipelines in terms of both camera pose estimation and 3D human pose estimation.

cs.CV↗
arXiv2023

Joint Planning of Active Distribution Network and EV Charging Stations Considering Vehicle-to-Grid Functionality and Reactive Power Support

This paper proposes a collaborative planning model for the active distribution network (ADN) and electric vehicle (EV) charging stations that fully considers the vehicle-to-grid (V2G) function and reactive power support of EVs in different regions. This paper employs a sequential decomposition method based on the physical characteristics of the problem, breaking down the holistic problem into two sub-problems for solution. Subproblem I optimizes the charging and discharging behavior of the autopilot electric vehicles (AEVs) using a mixed-integer linear programming (MILP) model. Subproblem II uses a mixed-integer second-order cone programming (MISOCP) model to plan the ADN and retrofit or construct V2G charging stations (V2GCS), as well as multiple distributed generation resources (DGRs). The paper also analyzes the impact of the bi-directional active-reactive power interaction of V2GCS on ADN planning. The presented model was tested in the 47 nodes ADN in Longgang District, Shenzhen, China, and the IEEE 33 nodes ADN, demonstrating that decomposition can significantly improve the speed of solving large-scale problems while maintaining accuracy with low AEV penetration.

eess.SY↗