arXiv subjects

Jun Li

Publications and source records attributed to Jun Li.

At least 325 records · Page 18Linked to original sources

arXiv2024

Design as Desired: Utilizing Visual Question Answering for Multimodal Pre-training

Multimodal pre-training demonstrates its potential in the medical domain, which learns medical visual representations from paired medical reports. However, many pre-training tasks require extra annotations from clinicians, and most of them fail to explicitly guide the model to learn the desired features of different pathologies. In this paper, we utilize Visual Question Answering (VQA) for multimodal pre-training to guide the framework focusing on targeted pathological features. We leverage descriptions in medical reports to design multi-granular question-answer pairs associated with different diseases, which assist the framework in pre-training without requiring extra annotations from experts. We also propose a novel pre-training framework with a quasi-textual feature transformer, a module designed to transform visual features into a quasi-textual space closer to the textual domain via a contrastive learning strategy. This narrows the vision-language gap and facilitates modality alignment. Our framework is applied to four downstream tasks: report generation, classification, segmentation, and detection across five datasets. Extensive experiments demonstrate the superiority of our framework compared to other state-of-the-art methods. Our code is available at https://github.com/MoramiSu/QFT-MICCAI2024.

cs.CV↗

arXiv2024

3DGR-CAR: Coronary artery reconstruction from ultra-sparse 2D X-ray views with a 3D Gaussians representation

Reconstructing 3D coronary arteries is important for coronary artery disease diagnosis, treatment planning and operation navigation. Traditional reconstruction techniques often require many projections, while reconstruction from sparse-view X-ray projections is a potential way of reducing radiation dose. However, the extreme sparsity of coronary arteries in a 3D volume and ultra-limited number of projections pose significant challenges for efficient and accurate 3D reconstruction. To this end, we propose 3DGR-CAR, a 3D Gaussian Representation for Coronary Artery Reconstruction from ultra-sparse X-ray projections. We leverage 3D Gaussian representation to avoid the inefficiency caused by the extreme sparsity of coronary artery data and propose a Gaussian center predictor to overcome the noisy Gaussian initialization from ultra-sparse view projections. The proposed scheme enables fast and accurate 3D coronary artery reconstruction with only 2 views. Experimental results on two datasets indicate that the proposed approach significantly outperforms other methods in terms of voxel accuracy and visual quality of coronary arteries. The code will be available in https://github.com/windrise/3DGR-CAR.

eess.IV↗

arXiv2024

FMBench: Benchmarking Fairness in Multimodal Large Language Models on Medical Tasks

Advancements in Multimodal Large Language Models (MLLMs) have significantly improved medical task performance, such as Visual Question Answering (VQA) and Report Generation (RG). However, the fairness of these models across diverse demographic groups remains underexplored, despite its importance in healthcare. This oversight is partly due to the lack of demographic diversity in existing medical multimodal datasets, which complicates the evaluation of fairness. In response, we propose FMBench, the first benchmark designed to evaluate the fairness of MLLMs performance across diverse demographic attributes. FMBench has the following key features: 1: It includes four demographic attributes: race, ethnicity, language, and gender, across two tasks, VQA and RG, under zero-shot settings. 2: Our VQA task is free-form, enhancing real-world applicability and mitigating the biases associated with predefined choices. 3: We utilize both lexical metrics and LLM-based metrics, aligned with clinical evaluations, to assess models not only for linguistic accuracy but also from a clinical perspective. Furthermore, we introduce a new metric, Fairness-Aware Performance (FAP), to evaluate how fairly MLLMs perform across various demographic attributes. We thoroughly evaluate the performance and fairness of eight state-of-the-art open-source MLLMs, including both general and medical MLLMs, ranging from 7B to 26B parameters on the proposed benchmark. We aim for FMBench to assist the research community in refining model evaluation and driving future advancements in the field. All data and code will be released upon acceptance.

cs.CV↗

arXiv2024

Variational Quantum Imaginary Time Evolution for Matrix Product State Ansatz with Tests on Transcorrelated Hamiltonians

The matrix product state (MPS) ansatz offers a promising approach for finding the ground state of molecular Hamiltonians and solving quantum chemistry problems. Building on this concept, the proposed technique of quantum circuit MPS (QCMPS) enables the simulation of chemical systems using a relatively small number of qubits. In this study, we enhance the optimization performance of the QCMPS ansatz by employing the variational quantum imaginary time evolution (VarQITE) approach. Guided by McLachlan's variational principle, the VarQITE method provides analytical metrics and gradients, resulting in improved convergence efficiency and robustness of the QCMPS. We validate these improvements numerically through simulations of $\rm H_2$, $\rm H_4$, and $\rm LiH$ molecules. Additionally, given that VarQITE is applicable to non-Hermitian Hamiltonians, we evaluate its effectiveness in preparing the ground state of transcorrelated (TC) Hamiltonians. This approach yields energy estimates comparable to the complete basis set (CBS) limit while using even fewer qubits. Specifically, we perform simulations of the beryllium atom and $\rm LiH$ molecule using only three qubits, maintaining high fidelity with the CBS ground state energy of these systems. This qubit reduction is achieved through the combined advantages of both the QCMPS ansatz and transcorrelation. Our findings demonstrate the potential practicality of this quantum chemistry algorithm on near-term quantum devices.

quant-ph↗

arXiv2024

XWSB: A Blend System Utilizing XLS-R and WavLM with SLS Classifier detection system for SVDD 2024 Challenge

This paper introduces the model structure used in the SVDD 2024 Challenge. The SVDD 2024 challenge has been introduced this year for the first time. Singing voice deepfake detection (SVDD) which faces complexities due to informal speech intonations and varying speech rates. In this paper, we propose the XWSB system, which achieved SOTA per-formance in the SVDD challenge. XWSB stands for XLS-R, WavLM, and SLS Blend, representing the integration of these technologies for the purpose of SVDD. Specifically, we used the best performing model structure XLS-R&SLS from the ASVspoof DF dataset, and applied SLS to WavLM to form the WavLM&SLS structure. Finally, we integrated two models to form the XWSB system. Experimental results show that our system demonstrates advanced recognition capabilities in the SVDD challenge, specifically achieving an EER of 2.32% in the CtrSVDD track. The code and data can be found at https://github.com/QiShanZhang/XWSB_for_ SVDD2024.

cs.SD↗

arXiv2024

Gate-controlled superconducting switch in GaSe/NbSe$_2$ van der Waals heterostructure

The demand for low-power devices is on the rise as semiconductor engineering approaches the quantum limit and quantum computing continues to advance. Two-dimensional (2D) superconductors, thanks to their rich physical properties, hold significant promise for both fundamental physics and potential applications in superconducting integrated circuits and quantum computation. Here, we report a gate-controlled superconducting switch in GaSe/NbSe$_2$ van der Waals (vdW) heterostructure. By injecting high-energy electrons into NbSe$_2$ under an electric field, a non-equilibrium state is induced, resulting in significant modulation of the superconducting properties. Owing to the intrinsic polarization of ferroelectric GaSe, a much steeper subthreshold slope and asymmetric modulation are achieved, which is beneficial to the device performance. Based on these results, a superconducting switch is realized that can reversibly and controllably switch between the superconducting and normal state under an electric field. Our findings highlight a significant high-energy injection effect from band engineering in 2D vdW heterostructures combining superconductors and ferroelectric semiconductors, and demonstrate the potential applications for superconducting integrated circuits.

cond-mat.supr-con↗

arXiv2024

Time-correlation functions of stochastic three-sphere micromachines

We discuss and compare the statistical properties of two stochastic three-sphere micromachines, i.e., odd micromachine and thermal micromachine. We calculate the steady state time-correlation functions for these micromachines and decompose them into the symmetric and antisymmetric parts. In both cases, the cross-correlation between the two spring extensions has an antisymmetric part, which is a direct consequence of the broken time-reversal symmetry. For the odd micromachine, the antisymmetric part of the correlation function is proportional to the odd elasticity, whereas it is proportional to the temperature difference between the two edge spheres for the thermal micromachine. The entropy production rate and the Green-Kubo relations for the two micromachines are also obtained. Comparing the results of the two models, we argue an effective odd elastic constant of the thermal micromachine. The effective odd elasticity of the thermal micromachine is proportional to the temperature difference among the spheres, which causes an internal heat flow and leads to directional locomotion in the presence of hydrodynamic interactions.

cond-mat.stat-mech↗

arXiv2024

Superband: an Electronic-band and Fermi surface structure database of superconductors

In comparison to simpler data such as chemical formulas and lattice structures, electronic band structure data provide a more fundamental and intuitive insight into superconducting phenomena. In this work, we generate superconductor's lattice structure files optimized for density functional theory (DFT) calculations. Through DFT, we obtain electronic band superconductors, including band structures, density of states (DOS), and Fermi surface data. Additionally, we outline efficient methodologies for acquiring structure data, establish high-throughput DFT computational protocols, and introduce tools for extracting this data from large-scale DFT calculations. As an example, we have curated a dataset containing information on 2474 superconductors along with their experimentally determined superconducting transition temperatures, which is well-suited for machine learning applications. This work also provides guidelines for accessing and utilizing this dataset. Furthermore, we present a neural network model designed for training with this data. All the aforementioned data and code are publicly available at http://www.superband.work.

cond-mat.supr-con↗

arXiv2024

Unique quantum metallic state in the titanium sesquioxide heterointerface superconductor

The emergence of quantum metallic state marked by a saturating finite electrical resistance in the zero-temperature limit in a variety of two-dimensional superconductors injects an exciting momentum to the realm of heterostructure superconductivity. Despite much research efforts over last few decades, there is not yet a general consensus on the nature of this unexpected quantum metal. Here, we report the observation of a unique quantum metallic state within the hallmark of Bose-metal in the titanium sesquioxide heterointerface superconductor Ti$_2$O$_3$/GaN. Remarkably, the quantum bosonic metallic state continuously tuned by a magnetic field in the vicinity of the two-dimensional superconductivity-metal transition persists in the normal phase, indicating the existence of composite bosons formed by electron Cooper pairs even in the normal phase. Our findings provide a distinct evidence for electron pairing in the normal phase of heterointerface superconductors, and shed fresh light on the pairing nature underlying heterointerface superconductivity.

cond-mat.supr-con↗

arXiv2024

A deep learning approach to search for superconductors from electronic bands

Energy band theory is a foundational framework in condensed matter physics. In this work, we employ a deep learning method, BNAS, to find a direct correlation between electronic band structure and superconducting transition temperature. Our findings suggest that electronic band structures can act as primary indicators of superconductivity. To avoid overfitting, we utilize a relatively simple deep learning neural network model, which, despite its simplicity, demonstrates predictive capabilities for superconducting properties. By leveraging the attention mechanism within deep learning, we are able to identify specific regions of the electronic band structure most correlated with superconductivity. This novel approach provides new insights into the mechanisms driving superconductivity from an alternative perspective. Moreover, we predict several potential superconductors that may serve as candidates for future experimental synthesis.

cond-mat.supr-con↗

arXiv2024

Observation of superconducting diode effect in antiferromagnetic Mott insulator $α$-RuCl$_3$

Nonreciprocal superconductivity, also called as superconducting diode effect that spontaneously breaks time-reversal symmetry, is characterized by asymmetric critical currents under opposite applied current directions. This distinct state unveils a rich ore of intriguing physical properties, particularly in the realm of nanoscience application of superconductors. Towards the experimental realization of superconducting diode effect, the construction of two-dimensional heterostructures of magnets and $s$-wave superconductors is considered to be a promising pathway. In this study, we present our findings of superconducting diode effect manifested in the magnetic Mott insulator $α$-RuCl$_3$. This phenomenon is induced by the proximity effect within a van der Waals heterostructure, consisting of thin $α$-RuCl$_3$/NbSe$_2$ flakes. Through transport property measurements, we have confirmed a weak superconducting gap of 0.2 meV, which is significantly lower than the intrinsic gap of NbSe$_2$(1.2 meV). Upon the application of a weak magnetic field below 70 mT, we observed an asymmetry in the critical currents under positive and negative applied currents. This observation demonstrates a typical superconducting diode effect in the superconducting $α$-RuCl$_3$. The superconducting diode effect and nonreciprocal resistance are observed exclusively when the magnetic field is aligned out-of-plane. This suggests that an Ising-type spin-orbit coupling in the superconducting $α$-RuCl$_3$ may be responsible for the mechanism. Our findings furnish a platform for the exploration of superconducting diode effect via the artificial construction of heterostructures.

cond-mat.supr-con↗

arXiv2024

GMFL-Net: A Global Multi-geometric Feature Learning Network for Repetitive Action Counting

With the continuous development of deep learning, the field of repetitive action counting is gradually gaining notice from many researchers. Extraction of pose keypoints using human pose estimation networks is proven to be an effective pose-level method. However, existing pose-level methods suffer from the shortcomings that the single coordinate is not stable enough to handle action distortions due to changes in camera viewpoints, thus failing to accurately identify salient poses, and is vulnerable to misdetection during the transition from the exception to the actual action. To overcome these problems, we propose a simple but efficient Global Multi-geometric Feature Learning Network (GMFL-Net). Specifically, we design a MIA-Module that aims to improve information representation by fusing multi-geometric features, and learning the semantic similarity among the input multi-geometric features. Then, to improve the feature representation from a global perspective, we also design a GBFL-Module that enhances the inter-dependencies between point-wise and channel-wise elements and combines them with the rich local information generated by the MIA-Module to synthesise a comprehensive and most representative global feature representation. In addition, considering the insufficient existing dataset, we collect a new dataset called Countix-Fitness-pose (https://github.com/Wantong66/Countix-Fitness) which contains different cycle lengths and exceptions, a test set with longer duration, and annotate it with fine-grained annotations at the pose-level. We also add two new action classes, namely lunge and rope push-down. Finally, extensive experiments on the challenging RepCount-pose, UCFRep-pose, and Countix-Fitness-pose benchmarks show that our proposed GMFL-Net achieves state-of-the-art performance.

cs.CV↗

arXiv2024

Resource Efficient Asynchronous Federated Learning for Digital Twin Empowered IoT Network

As an emerging technology, digital twin (DT) can provide real-time status and dynamic topology mapping for Internet of Things (IoT) devices. However, DT and its implementation within industrial IoT networks necessitates substantial, distributed data support, which often leads to ``data silos'' and raises privacy concerns. To address these issues, we develop a dynamic resource scheduling algorithm tailored for the asynchronous federated learning (FL)-based lightweight DT empowered IoT network. Specifically, our approach aims to minimize a multi-objective function that encompasses both energy consumption and latency by optimizing IoT device selection and transmit power control, subject to FL model performance constraints. We utilize the Lyapunov method to decouple the formulated problem into a series of one-slot optimization problems and develop a two-stage optimization algorithm to achieve the optimal transmission power control and IoT device scheduling strategies. In the first stage, we derive closed-form solutions for optimal transmit power on the IoT device side. In the second stage, since partial state information is unknown, e.g., the transmitting power and computational frequency of IoT device, the edge server employs a multi-armed bandit (MAB) framework to model the IoT device selection problem and utilizes an efficient online algorithm, namely the client utility-based upper confidence bound (CU-UCB), to address it. Numerical results validate our algorithm's superiority over benchmark schemes, and simulations demonstrate that our algorithm achieves faster training speeds on the Fashion-MNIST and CIFAR-10 datasets within the same training duration.

cs.DC↗

arXiv2024

Transmissive RIS Enabled Transceiver Systems:Architecture, Design Issues and Opportunities

Reconfigurable intelligent surface (RIS) is anticipated to augment the performance of beyond fifth-generation (B5G) and sixth-generation (6G) networks by intelligently manipulating the state of its components. Rather than employing reflective RIS for aided communications, this paper proposes an innovative transmissive RIS-enabled transceiver (TRTC) architecture that can accomplish the functions of traditional multi-antenna systems in a cost-effective and energy-efficient manner. First, the proposed network architecture and its corresponding transmission scheme are elaborated from the perspectives of downlink (DL) and uplink (UL) transmissions. Then, we illustrate several significant advantages and differences of TRTC compared to other multiantenna systems. Furthermore, the downlink modulation and extraction principle based on time-modulation array (TMA) is introduced in detail to tackle the multi-stream communications. Moreover, a near-far field channel model appropriate for this architecture is proposed. Based on the channel model, we summarize some state-of-the-art channel estimation schemes, and the channel estimation scheme of TRTC is also provided. Considering the optimization for DL and UL communications, we present numerical simulations that confirm the superiority of the proposed optimization algorithm. Lastly, numerous prospective research avenues for TRTC systems are delineated to inspire further exploration.

eess.SP↗

arXiv2024

A Physics-Informed Indirect Method for Trajectory Optimization

This work presents a Physics-Informed Indirect Method (PIIM) that propagates the dynamics of both states and co-states backward in time for trajectory optimization problems. In the case of a Time-Optimal Soft Landing Problem (TOSLP), based on the initial co-state vector normalization technique, we show that the initial guess of the mass co-state and the numerical factor can be eliminated from the shooting procedure. As a result, the initial guess of the unknown co-states can be constrained to lie on a unit 3-D hypersphere. Then, using the PIIM allows one to exploit the physical significance of the optimal control law, which further narrows down the solution space to a unit 3-D octant sphere. Meanwhile, the analytical estimations of the fuel consumption and final time are provided. Additionally, a usually overlooked issue that results in an infeasible solution with a negative final time, is fixed by a simple remedy strategy. Consequently, the reduced solution space becomes sufficiently small to ensure fast, robust, and guaranteed convergence for the TOSLP. Then, we extend the PIIM to solve the Fuel-Optimal Soft Landing Problem (FOSLP) with a homotopy approach. The numerical simulations show that compared with the conventional indirect method with a success rate of 89.35%, it takes a shorter time for the proposed method to find the feasible solution to the FOSLP with a success rate of 100%.

math.OC↗

arXiv2024

Flatten: Video Action Recognition is an Image Classification task

In recent years, video action recognition, as a fundamental task in the field of video understanding, has been deeply explored by numerous researchers.Most traditional video action recognition methods typically involve converting videos into three-dimensional data that encapsulates both spatial and temporal information, subsequently leveraging prevalent image understanding models to model and analyze these data. However,these methods have significant drawbacks. Firstly, when delving into video action recognition tasks, image understanding models often need to be adapted accordingly in terms of model architecture and preprocessing for these spatiotemporal tasks; Secondly, dealing with high-dimensional data often poses greater challenges and incurs higher time costs compared to its lower-dimensional counterparts.To bridge the gap between image-understanding and video-understanding tasks while simplifying the complexity of video comprehension, we introduce a novel video representation architecture, Flatten, which serves as a plug-and-play module that can be seamlessly integrated into any image-understanding network for efficient and effective 3D temporal data modeling.Specifically, by applying specific flattening operations (e.g., row-major transform), 3D spatiotemporal data is transformed into 2D spatial information, and then ordinary image understanding models are used to capture temporal dynamic and spatial semantic information, which in turn accomplishes effective and efficient video action recognition. Extensive experiments on commonly used datasets (Kinetics-400, Something-Something v2, and HMDB-51) and three classical image classification models (Uniformer, SwinV2, and ResNet), have demonstrated that embedding Flatten provides a significant performance improvements over original model.

cs.CV↗

arXiv2024

Roles of the Narrow Electronic Band near the Fermi Level in 1T-TaS$_2$-Related Layered Materials

Here we use low-temperature scanning tunneling microscopy and spectroscopy (STM/STS) to reveal the roles of the narrow electronic band in two 1T-TaS$_2$-related materials (bulk 1T-TaS$_2$ and 4Hb-TaS$_2$). 4Hb-TaS$_2$ is a superconducting compound with alternating 1T-TaS$_2$ and 1H-TaS$_2$ layers, where the 1H-TaS$_2$ layer has weak charge density wave (CDW) pattern and reduces the CDW coupling between the adjacent 1T-TaS$_2$ layers. In the 1T-TaS$_2$ layer of 4Hb-TaS$_2$, we observe a narrow electronic band located near Fermi level, and its spatial distribution is consistent with the tight-binding calculations for two-dimensional 1T-TaS$_2$ layers. The weak electronic hybridization between the 1T-TaS$_2$ and 1H-TaS$_2$ layers in 4Hb-TaS$_2$ shifts the narrow electronic band to be slightly above the Fermi level, which suppresses the electronic correlation induced band splitting. In contrast, in bulk 1T-TaS$_2$, there is an interlayer CDW coupling induced insulating gap. In comparison with the spatial distributions of the electronic states in bulk 1T-TaS$_2$ and 4Hb-TaS$_2$, the insulating gap in bulk 1T-TaS$_2$ results from the formation of a bonding band and an antibonding band due to the overlap of the narrow electronic bands in the dimerized 1T-TaS$_2$ layers.

cond-mat.mes-hall↗

arXiv2024

Utilizing Navigation Paths to Generate Target Points for Enhanced End-to-End Autonomous Driving Planning

In recent years, end-to-end autonomous driving frameworks have been shown to not only enhance perception performance but also improve planning capabilities. However, most previous end-to-end autonomous driving frameworks have focused primarily on enhancing environmental perception while neglecting the learning of autonomous vehicle driving intent, which refers to the vehicle's intended direction of travel. In planning, the autonomous vehicle's direction is clear and well-defined, yet this crucial aspect has often been overlooked. This paper introduces NTT (Navigation to Target for Trajectory planning), a method within an end-to-end framework for autonomous driving. NTT generates the planned trajectory in two steps. First, it generates the future target point for the autonomous vehicle on the basis of the navigation path. Then, it produces the complete planned trajectory on the basis of this target point. On the one hand, generating the target point for the autonomous vehicle from the navigation path enables the vehicle to learn a clear driving intent. On the other hand, generating the trajectory on the basis of the target point allows for a flexible planned trajectory that can adapt to complex environmental changes, thereby enhancing the safety of the planning process. Our method achieved excellent planning performance on the widely used nuScenes dataset and its effectiveness was validated through ablation experiments.

cs.RO↗