Searcharxiv⌕ Search

arXiv subjects

Shu Zhang

Publications and source records attributed to Shu Zhang.

At least 163 records · Page 9Linked to original sources
arXiv2023

HIC-YOLOv5: Improved YOLOv5 For Small Object Detection

Small object detection has been a challenging problem in the field of object detection. There has been some works that proposes improvements for this task, such as adding several attention blocks or changing the whole structure of feature fusion networks. However, the computation cost of these models is large, which makes deploying a real-time object detection system unfeasible, while leaving room for improvement. To this end, an improved YOLOv5 model: HIC-YOLOv5 is proposed to address the aforementioned problems. Firstly, an additional prediction head specific to small objects is added to provide a higher-resolution feature map for better prediction. Secondly, an involution block is adopted between the backbone and neck to increase channel information of the feature map. Moreover, an attention mechanism named CBAM is applied at the end of the backbone, thus not only decreasing the computation cost compared with previous works but also emphasizing the important information in both channel and spatial domain. Our result shows that HIC-YOLOv5 has improved mAP@[.5:.95] by 6.42% and mAP@0.5 by 9.38% on VisDrone-2019-DET dataset.

cs.CV↗
arXiv2023

GlueGen: Plug and Play Multi-modal Encoders for X-to-image Generation

Text-to-image (T2I) models based on diffusion processes have achieved remarkable success in controllable image generation using user-provided captions. However, the tight coupling between the current text encoder and image decoder in T2I models makes it challenging to replace or upgrade. Such changes often require massive fine-tuning or even training from scratch with the prohibitive expense. To address this problem, we propose GlueGen, which applies a newly proposed GlueNet model to align features from single-modal or multi-modal encoders with the latent space of an existing T2I model. The approach introduces a new training objective that leverages parallel corpora to align the representation spaces of different encoders. Empirical results show that GlueNet can be trained efficiently and enables various capabilities beyond previous state-of-the-art models: 1) multilingual language models such as XLM-Roberta can be aligned with existing T2I models, allowing for the generation of high-quality images from captions beyond English; 2) GlueNet can align multi-modal encoders such as AudioCLIP with the Stable Diffusion model, enabling sound-to-image generation; 3) it can also upgrade the current text encoder of the latent diffusion model for challenging case generation. By the alignment of various feature representations, the GlueNet allows for flexible and efficient integration of new functionality into existing T2I models and sheds light on X-to-image (X2I) generation.

cs.CV↗
arXiv2023

UniControl: A Unified Diffusion Model for Controllable Visual Generation In the Wild

Achieving machine autonomy and human control often represent divergent objectives in the design of interactive AI systems. Visual generative foundation models such as Stable Diffusion show promise in navigating these goals, especially when prompted with arbitrary languages. However, they often fall short in generating images with spatial, structural, or geometric controls. The integration of such controls, which can accommodate various visual conditions in a single unified model, remains an unaddressed challenge. In response, we introduce UniControl, a new generative foundation model that consolidates a wide array of controllable condition-to-image (C2I) tasks within a singular framework, while still allowing for arbitrary language prompts. UniControl enables pixel-level-precise image generation, where visual conditions primarily influence the generated structures and language prompts guide the style and context. To equip UniControl with the capacity to handle diverse visual conditions, we augment pretrained text-to-image diffusion models and introduce a task-aware HyperNet to modulate the diffusion models, enabling the adaptation to different C2I tasks simultaneously. Trained on nine unique C2I tasks, UniControl demonstrates impressive zero-shot generation abilities with unseen visual conditions. Experimental results show that UniControl often surpasses the performance of single-task-controlled methods of comparable model sizes. This control versatility positions UniControl as a significant advancement in the realm of controllable visual generation.

cs.CV↗
arXiv2023

ChatRadio-Valuer: A Chat Large Language Model for Generalizable Radiology Report Generation Based on Multi-institution and Multi-system Data

Radiology report generation, as a key step in medical image analysis, is critical to the quantitative analysis of clinically informed decision-making levels. However, complex and diverse radiology reports with cross-source heterogeneity pose a huge generalizability challenge to the current methods under massive data volume, mainly because the style and normativity of radiology reports are obviously distinctive among institutions, body regions inspected and radiologists. Recently, the advent of large language models (LLM) offers great potential for recognizing signs of health conditions. To resolve the above problem, we collaborate with the Second Xiangya Hospital in China and propose ChatRadio-Valuer based on the LLM, a tailored model for automatic radiology report generation that learns generalizable representations and provides a basis pattern for model adaptation in sophisticated analysts' cases. Specifically, ChatRadio-Valuer is trained based on the radiology reports from a single institution by means of supervised fine-tuning, and then adapted to disease diagnosis tasks for human multi-system evaluation (i.e., chest, abdomen, muscle-skeleton, head, and maxillofacial $\&$ neck) from six different institutions in clinical-level events. The clinical dataset utilized in this study encompasses a remarkable total of \textbf{332,673} observations. From the comprehensive results on engineering indicators, clinical efficacy and deployment cost metrics, it can be shown that ChatRadio-Valuer consistently outperforms state-of-the-art models, especially ChatGPT (GPT-3.5-Turbo) and GPT-4 et al., in terms of the diseases diagnosis from radiology reports. ChatRadio-Valuer provides an effective avenue to boost model generalization performance and alleviate the annotation workload of experts to enable the promotion of clinical AI applications in radiology reports.

cs.CL↗
arXiv2023

Data-driven mode shape selection and model-based vibration suppression of 3-RRR parallel manipulator with flexible actuation links

The mode shape function is difficult to determine in modeling manipulators with flexible links using the assumed mode method. In this paper, for a planar 3-RRR parallel manipulator with flexible actuation links, we provide a data-driven method to identify the mode shape of the flexible links and propose a model-based controller for the vibration suppression. By deriving the inverse kinematics of the studied mechanism in analytical form, the dynamic model is established by using the assumed mode method. To select the mode shape function, the software of multi-body system dynamics is used to simulate the dynamic behavior of the mechanism, and then the data-driven method which combines the DMD and SINDy algorithms is employed to identify the reasonable mode shape functions for the flexible links. To suppress the vibration of the flexible links, a state observer for the end-effector is constructed by a neural network, and the model-based control law is designed on this basis. In comparison with the model-free controller, the proposed controller with developed dynamic model has promising performance in terms of tracking accuracy and vibration suppression.

cs.RO↗
arXiv2023

Type-A quasi-periodic oscillation in the black hole transient MAXI J1348-630

We present a detailed analysis of the spectral and timing characteristics of a 7-Hz type-A quasi-periodic oscillation (QPO) detected in NICER observations of the black hole X-ray binary MAXI J1348-630 during its high-soft state. The QPO is broad and weak, with an integrated fractional rms amplitude of 0.9 per cent in the 0.5-10 keV band. Thanks to the large effective area of NICER, combined with the high flux of the source and a relatively long accumulative exposure time, we construct the first rms and phase-lag spectra for a type-A QPO. Our analysis reveals that the fractional rms amplitude of the QPO increases with energy from below 1 per cent at 1 keV to 3 per cent at 6 keV. The shape of the QPO spectrum is similar to that of the Comptonised component, suggesting that the Comptonised region is driving the variability. The phase lags at the QPO frequency are always soft taking the lowest energy as reference. By jointly fitting the time-averaged spectrum of the source and the rms and phase-lag spectra of the QPO with the time-dependent Comptonisation model vkompthdk, we find that the radiative properties of the type-A QPO can be explained by a vertically extended Comptonised region with a size of 2300 km.

astro-ph.HE↗
arXiv2023

Recovering lossless propagation of polaritons with synthesized complex frequency excitation

Surface plasmon polaritons and phonon polaritons offer a means of surpassing the diffraction limit of conventional optics and facilitate efficient energy storage, local field enhancement, high sensitivities, benefitting from their subwavelength confinement of light. Unfortunately, losses severely limit the propagation decay length, thus restricting the practical use of polaritons. While optimizing the fabrication technique can help circumvent the scattering loss of imperfect structures, the intrinsic absorption channel leading to heat production cannot be eliminated. Here, we utilize synthetic optical excitation of complex frequency with virtual gain, synthesized by combining the measurements taken at multiple real frequencies, to restore the lossless propagations of phonon polaritons with significantly reduced intrinsic losses. The concept of synthetic complex frequency excitation represents a viable solution to compensate for loss and would benefit applications including photonic circuits, waveguiding and plasmonic/phononic structured illumination microscopy.

physics.optics↗
arXiv2023

Stepwise Model Reconstruction of Robotic Manipulator Based on Data-Driven Method

Research on dynamics of robotic manipulators provides promising support for model-based control. In general, rigorous first-principles-based dynamics modeling and accurate identification of mechanism parameters are critical to achieving high precision in model-based control, while data-driven model reconstruction provides alternative approaches of the above process. Taking the level of activation of data as an indicator, this paper classifies the collected robotic manipulator data by means of K-means clustering algorithm. With the fundamental prior knowledge, we find the corresponding dynamical properties behind the classified data separately. Afterwards, the sparse identification of nonlinear dynamics (SINDy) method is used to reconstruct the dynamics model of the robotic manipulator step by step according to the activation level of the classified data. The simulation results show that the proposed method not only reduces the complexity of the basis function library, enabling the application of SINDy method to multi-degree-of-freedom robotic manipulators, but also decreases the influence of data noise on the regression results. Finally, the dynamic control based on the reconfigured model is deployed on the experimental platform, and the experimental results prove the effectiveness of the proposed method.

cs.RO↗
arXiv2023

Search for light dark matter from atmosphere in PandaX-4T

We report a search for light dark matter produced through the cascading decay of $η$ mesons, which are created as a result of inelastic collisions between cosmic rays and Earth's atmosphere. We introduce a new and general framework, publicly accessible, designed to address boosted dark matter specifically, with which a full and dedicated simulation including both elastic and quasi-elastic processes of Earth attenuation effect on the dark matter particles arriving at the detector is performed. In the PandaX-4T commissioning data of 0.63 tonne$\cdot$year exposure, no significant excess over background is observed. The first constraints on the interaction between light dark matter generated in the atmosphere and nucleus through a light scalar mediator are obtained. The lowest excluded cross-section is set at $5.9 \times 10^{-37}{\rm cm^2}$ for dark matter mass of $0.1$ MeV$/c^2$ and mediator mass of 300 MeV$/c^2$. The lowest upper limit of $η$ to dark matter decay branching ratio is $1.6 \times 10^{-7}$.

hep-ex↗
arXiv2023

Correlation-Aware Mutual Learning for Semi-supervised Medical Image Segmentation

Semi-supervised learning has become increasingly popular in medical image segmentation due to its ability to leverage large amounts of unlabeled data to extract additional information. However, most existing semi-supervised segmentation methods only focus on extracting information from unlabeled data, disregarding the potential of labeled data to further improve the performance of the model. In this paper, we propose a novel Correlation Aware Mutual Learning (CAML) framework that leverages labeled data to guide the extraction of information from unlabeled data. Our approach is based on a mutual learning strategy that incorporates two modules: the Cross-sample Mutual Attention Module (CMA) and the Omni-Correlation Consistency Module (OCC). The CMA module establishes dense cross-sample correlations among a group of samples, enabling the transfer of label prior knowledge to unlabeled data. The OCC module constructs omni-correlations between the unlabeled and labeled datasets and regularizes dual models by constraining the omni-correlation matrix of each sub-model to be consistent. Experiments on the Atrial Segmentation Challenge dataset demonstrate that our proposed approach outperforms state-of-the-art methods, highlighting the effectiveness of our framework in medical image segmentation tasks. The codes, pre-trained weights, and data are publicly available.

cs.CV↗
arXiv2023

A Search for Light Fermionic Dark Matter Absorption on Electrons in PandaX-4T

We report a search on a sub-MeV fermionic dark matter absorbed by electrons with an outgoing active neutrino using the 0.63 tonne-year exposure collected by PandaX-4T liquid xenon experiment. No significant signals are observed over the expected background. The data are interpreted into limits to the effective couplings between such dark matter and electrons. For axial-vector or vector interactions, our sensitivity is competitive in comparison to existing astrophysical bounds on the decay of such dark matter into photon final states. In particular, we present the first direct detection limits for an axial-vector (vector) interaction which are the strongest in the mass range from 25 to 45 (35 to 50) keV/c$^2$.

hep-ex↗
arXiv2023

A spectral-timing study of the inner flow geometry in MAXI J1535--571 with $Insight$-HXMT and NICER

We have performed a spectral-timing analysis on the black hole X-ray binary MAXI J1535--571 during its 2017 outburst, with the aim of exploring the evolution of the inner accretion flow geometry. X-ray reverberation lags are observed in the hard-intermediate state (HIMS) and soft-intermediate state (SIMS) of the outburst. During the HIMS, the characteristic frequency of the reverberation lags $ν_0$ (the frequency at which the soft lag turns to zero in the lag-frequency spectra) increases when the spectrum softens. This reflects a reduction of the spatial distance between the corona and accretion disc, when assuming the measured time lags are associated with the light travel time. We also find a strong correlation between $ν_0$ and type-C Quasi Periodic Oscillation (QPO) centroid frequency $ν_{QPO}$, which can be well explained by the Lense-Thirring (L-T) precession model under a truncated disk geometry. Despite the degeneracy in the spectral modellings, our results suggest that the accretion disc is largely truncated in the low hard state (LHS), and moves inward as the spectrum softens. Combine the spectral modelling results with the $ν_0$ - $ν_{QPO}$ evolution, we are inclined to believe that this source probably have a truncated disk geometry in the hard state.

astro-ph.HE↗
arXiv2023

Review of Large Vision Models and Visual Prompt Engineering

Visual prompt engineering is a fundamental technology in the field of visual and image Artificial General Intelligence, serving as a key component for achieving zero-shot capabilities. As the development of large vision models progresses, the importance of prompt engineering becomes increasingly evident. Designing suitable prompts for specific visual tasks has emerged as a meaningful research direction. This review aims to summarize the methods employed in the computer vision domain for large vision models and visual prompt engineering, exploring the latest advancements in visual prompt engineering. We present influential large models in the visual domain and a range of prompt engineering methods employed on these models. It is our hope that this review provides a comprehensive and systematic description of prompt engineering methods based on large visual models, offering valuable insights for future researchers in their exploration of this field.

cs.CV↗
arXiv2023

The mHz quasi-regular modulations of 4U 1630--47 during its 1998 outburst

We present the results of a detailed timing and spectral analysis of the quasi-regular modulation (QRM) phenomenon in the black hole X-ray binary 4U 1630--47 during its 1998 outburst observed by Rossi X-ray Timing Explore (RXTE). We find that the $\sim$ 50-110 mHz QRM is flux dependent, and the QRM is detected with simultaneous low frequency quasi-periodic oscillations (LFQPOs). According to the behavior of the power density spectrum, we divide the observations into four groups. In the first group, namely behavior A, LFQPOs are detected, but no mHz QRM. The second group, namely behavior B, a QRM with frequency above $\sim$ 88 mHz is detected and the $\sim$ 5 Hz and $\sim$ 7 Hz LFQPOs are almost overlapping. In the third group, namely behavior C, the QRM frequency below $\sim$ 88 mHz is detected and the LFQPOs are significantly separated. In the forth group, namely behavior D, neither QRM nor LFQPOs are detected. We study the energy-dependence of the fractional rms, centroid frequency, and phase-lag of QRM and LFQPOs for behavior B and C. We then study the evolution of QRM and find that the frequency of QRM increases with hardness, while its rms decreases with hardness. We also analyze the spectra of each observation, and find that the QRM rms of behavior B has a positive correlation with $\rm F_{\rm powerlaw}$ / $\rm F_{\rm total}$. Finally, we give our understanding for this mHz QRM phenomena.

astro-ph.HE↗
arXiv2023

The hard to soft transition of GX 339-4 as seen by Insight-HXMT

We present an analysis of the relativistic reflection spectra of GX 339-4 during the hard-to-soft transition of its 2021 outburst observed by Insight-HXMT. The strong relativistic reflection signatures in the data suggest a high black hole spin ($a_*>0.86$) and an intermediate disk inclination angle (35-43 deg) of the system. The transition is accompanied by an increasing temperature of the disk and a softening of the corona emission while the inner disk radius remains stable. Assuming a lamppost geometry, the corona height is also found to stay close to the black hole across the state transition. If we include the Comptonization of the reflection spectrum, the scattering fraction parameter is found to decrease during the state transition. We also perform an analysis with a reflection model designed for hot accretion disks of stellar mass black holes where the surface of the innermost accretion disk is illuminated by emission from the corona and the thermal disk below. Our results support the scenario in which the state transition is associated with variations in the corona properties.

astro-ph.HE↗
arXiv2023

A Transformer-based representation-learning model with unified processing of multimodal input for clinical diagnostics

During the diagnostic process, clinicians leverage multimodal information, such as chief complaints, medical images, and laboratory-test results. Deep-learning models for aiding diagnosis have yet to meet this requirement. Here we report a Transformer-based representation-learning model as a clinical diagnostic aid that processes multimodal input in a unified manner. Rather than learning modality-specific features, the model uses embedding layers to convert images and unstructured and structured text into visual tokens and text tokens, and bidirectional blocks with intramodal and intermodal attention to learn a holistic representation of radiographs, the unstructured chief complaint and clinical history, structured clinical information such as laboratory-test results and patient demographic information. The unified model outperformed an image-only model and non-unified multimodal diagnosis models in the identification of pulmonary diseases (by 12% and 9%, respectively) and in the prediction of adverse clinical outcomes in patients with COVID-19 (by 29% and 7%, respectively). Leveraging unified multimodal Transformer-based models may help streamline triage of patients and facilitate the clinical decision process.

cs.CV↗
arXiv2023

Attention Paper: How Generative AI Reshapes Digital Shadow Industry?

The rapid development of digital economy has led to the emergence of various black and shadow internet industries, which pose potential risks that can be identified and managed through digital risk management (DRM) that uses different techniques such as machine learning and deep learning. The evolution of DRM architecture has been driven by changes in data forms. However, the development of AI-generated content (AIGC) technology, such as ChatGPT and Stable Diffusion, has given black and shadow industries powerful tools to personalize data and generate realistic images and conversations for fraudulent activities. This poses a challenge for DRM systems to control risks from the source of data generation and to respond quickly to the fast-changing risk environment. This paper aims to provide a technical analysis of the challenges and opportunities of AIGC from upstream, midstream, and downstream paths of black/shadow industries and suggest future directions for improving existing risk control systems. The paper will explore the new black and shadow techniques triggered by generative AI technology and provide insights for building the next-generation DRM system.

cs.CY↗
arXiv2023

Hilbert-Huang Transform analysis of quasi-periodic oscillations in MAXI J1820+070

We present time-frequency analysis, based on the Hilbert-Huang transform (HHT), of the evolution on the low-frequency quasi-periodic oscillations (LFQPOs) observed in the black hole X-ray binary MAXI J1820+070. Through the empirical mode decomposition (EMD) method, we decompose the light curve of the QPO component and measure its intrinsic phase lag between photons from different energy bands. We find that the QPO phase lag is negative (low energy photons lag behind high energy photons), meanwhile the absolute value of the lag increases with energy. By applying the Hilbert transform to the light curve of the QPO, we further extract the instantaneous frequency and amplitude of the QPO. Compared these results with those from the Fourier analysis, we find that the broadening of the QPO peak is mainly caused by the frequency modulation. Through further analysis, we find that these modulations could share a common physical origin with the broad-band noise, and can be well explained by the internal shock model of the jet.

astro-ph.HE↗