arXiv subjects

Zhang Li

Publications and source records attributed to Zhang Li.

At least 37 records · Page 2Linked to original sources

arXiv2025

Developing a Reliable, Fast, General-Purpose Hallucination Detection and Mitigation Service

Hallucination, a phenomenon where large language models (LLMs) produce output that is factually incorrect or unrelated to the input, is a major challenge for LLM applications that require accuracy and dependability. In this paper, we introduce a reliable and high-speed production system aimed at detecting and rectifying the hallucination issue within LLMs. Our system encompasses named entity recognition (NER), natural language inference (NLI), span-based detection (SBD), and an intricate decision tree-based process to reliably detect a wide range of hallucinations in LLM responses. Furthermore, we have crafted a rewriting mechanism that maintains an optimal mix of precision, response time, and cost-effectiveness. We detail the core elements of our framework and underscore the paramount challenges tied to response time, availability, and performance metrics, which are crucial for real-world deployment of these technologies. Our extensive evaluation, utilizing offline data and live production traffic, confirms the efficacy of our proposed framework and service.

cs.CL↗

arXiv2025

High-precision visual navigation device calibration method based on collimator

Visual navigation devices require precise calibration to achieve high-precision localization and navigation, which includes camera and attitude calibration. To address the limitations of time-consuming camera calibration and complex attitude adjustment processes, this study presents a collimator-based calibration method and system. Based on the optical characteristics of the collimator, a single-image camera calibration algorithm is introduced. In addition, integrated with the precision adjustment mechanism of the calibration frame, a rotation transfer model between coordinate systems enables efficient attitude calibration. Experimental results demonstrate that the proposed method achieves accuracy and stability comparable to traditional multi-image calibration techniques. Specifically, the re-projection errors are less than 0.1463 pixels, and average attitude angle errors are less than 0.0586 degrees with a standard deviation less than 0.0257 degrees, demonstrating high precision and robustness.

cs.CV↗

arXiv2025

Multiband Nonthermal Radiative Properties of the Pulsar Wind Nebula CTB 87

The pulsar wind nebula CTB 87 (G74.9+1.2) is one of the sources emitting $γ$-rays with energies higher than 10 TeV, as measured by the Very Energetic Radiation Imaging Telescope Array System telescope (VERITAS). In this study, we undertake a reanalysis of the GeV emission from the CTB 87 region, utilising $\sim$16 years of high-energy $γ$-ray data collected with the Fermi Large Area Telescope. In the energy range of 0.03--1 TeV, the spectrum can be adequately described by a power-law model with an index of 1.34 $\pm$ 0.18, and the integral energy flux is calculated to be (7.25 $\pm$ 1.36) $\times$ 10$^{-13}$ erg cm$^{-2}$ s$^{-1}$. Based on the multiband data, we have employed a time-dependent model to investigate the non-thermal emission properties of CTB 87. In the model, it is assumed that particles with broken power-law energy distributions are continuously injected into the nebula. This results in multiband non-thermal emission being produced by relativistic leptons via synchrotron radiation and inverse Compton processes. Furthermore, the model suggests an energy of approximately 2.4 PeV for the most energetic particle in the nebula.

astro-ph.HE↗

arXiv2025

Detection of the Extended $γ$-ray Emission around TeV source 1LHAASO J0249+6022 with Fermi-LAT

1LHAASO J0249+6022 is an extended very-high-energy gamma-ray source discovered by the Large High-Altitude Air Shower Observatory. Based on nearly 16.1 years of data from the Fermi Large Area Telescope, we report the probable gamma-ray emission from 1LHAASO J0249+6022 in the 0.03-1 TeV energy range. The results show that its gamma-ray spectrum can be well fitted by a single power law with an index of 1.54 $\pm$ 0.17, and integral photon flux is (4.28 $\pm$ 1.03) $\times$ 10$^{-11}$ photons cm$^{-2}$ s$^{-1}$. We also considered theoretically whether the non-thermal emission could originate from a pulsar wind nebula (PWN) scenario. Assuming that the particles injected into the nebula have a power-law distribution, the resulting spectrum from the inverse Compton scattering is consistent with the detected GeV and TeV gamma-ray fluxes. Our study shows that the PWN scenario is reasonable for 1LHAASO J0249+6022.

astro-ph.HE↗

arXiv2024

Supervised cooperation on interdependent public goods games

It is a challenging task to reach global cooperation among self-interested agents, which often requires sophisticated design or usage of incentives. For example, we may apply supervisors or referees who are able to detect and punish selfishness. As a response, defectors may offer bribes for corrupt referees to remain hidden, hence generating a new conflict among supervisors. By using the interdependent network approach, we model the key element of the coevolution between strategy and judgment. In a game layer, agents play public goods game by using one of the two major strategies of a social dilemma. In a monitoring layer, supervisors follow the strategy change and may alter the income of competitors. Fair referees punish defectors while corrupt referees remain silent for a bribe. Importantly, there is a learning process not only among players but also among referees. Our results suggest that large fines and bribes boost the emergence of cooperation by significantly reducing the phase transition threshold between the pure defection state and the mixed solution where competing strategies coexist. Interestingly, the presence of bribes could be as harmful for defectors as the usage of harsh fines. The explanation of this system behavior is based on a strong correlation between cooperators and fair referees, which is cemented via overlapping clusters in both layers.

physics.soc-ph↗

arXiv2024

Monkey: Image Resolution and Text Label Are Important Things for Large Multi-modal Models

Large Multimodal Models (LMMs) have shown promise in vision-language tasks but struggle with high-resolution input and detailed scene understanding. Addressing these challenges, we introduce Monkey to enhance LMM capabilities. Firstly, Monkey processes input images by dividing them into uniform patches, each matching the size (e.g., 448x448) used in the original training of the well-trained vision encoder. Equipped with individual adapter for each patch, Monkey can handle higher resolutions up to 1344x896 pixels, enabling the detailed capture of complex visual information. Secondly, it employs a multi-level description generation method, enriching the context for scene-object associations. This two-part strategy ensures more effective learning from generated data: the higher resolution allows for a more detailed capture of visuals, which in turn enhances the effectiveness of comprehensive descriptions. Extensive ablative results validate the effectiveness of our designs. Additionally, experiments on 18 datasets further demonstrate that Monkey surpasses existing LMMs in many tasks like Image Captioning and various Visual Question Answering formats. Specially, in qualitative tests focused on dense text question answering, Monkey has exhibited encouraging results compared with GPT4V. Code is available at https://github.com/Yuliang-Liu/Monkey.

cs.CV↗

arXiv2024

OCRBench: On the Hidden Mystery of OCR in Large Multimodal Models

Large models have recently played a dominant role in natural language processing and multimodal vision-language learning. However, their effectiveness in text-related visual tasks remains relatively unexplored. In this paper, we conducted a comprehensive evaluation of Large Multimodal Models, such as GPT4V and Gemini, in various text-related visual tasks including Text Recognition, Scene Text-Centric Visual Question Answering (VQA), Document-Oriented VQA, Key Information Extraction (KIE), and Handwritten Mathematical Expression Recognition (HMER). To facilitate the assessment of Optical Character Recognition (OCR) capabilities in Large Multimodal Models, we propose OCRBench, a comprehensive evaluation benchmark. OCRBench contains 29 datasets, making it the most comprehensive OCR evaluation benchmark available. Furthermore, our study reveals both the strengths and weaknesses of these models, particularly in handling multilingual text, handwritten text, non-semantic text, and mathematical expression recognition. Most importantly, the baseline results presented in this study could provide a foundational framework for the conception and assessment of innovative strategies targeted at enhancing zero-shot multimodal techniques. The evaluation pipeline and benchmark are available at https://github.com/Yuliang-Liu/MultimodalOCR.

cs.CV↗

arXiv2024

Exploring the Capabilities of Large Multimodal Models on Dense Text

While large multi-modal models (LMM) have shown notable progress in multi-modal tasks, their capabilities in tasks involving dense textual content remains to be fully explored. Dense text, which carries important information, is often found in documents, tables, and product descriptions. Understanding dense text enables us to obtain more accurate information, assisting in making better decisions. To further explore the capabilities of LMM in complex text tasks, we propose the DT-VQA dataset, with 170k question-answer pairs. In this paper, we conduct a comprehensive evaluation of GPT4V, Gemini, and various open-source LMMs on our dataset, revealing their strengths and weaknesses. Furthermore, we evaluate the effectiveness of two strategies for LMM: prompt engineering and downstream fine-tuning. We find that even with automatically labeled training datasets, significant improvements in model performance can be achieved. We hope that this research will promote the study of LMM in dense text tasks. Code will be released at https://github.com/Yuliang-Liu/MultimodalOCR.

cs.CL↗

arXiv2024

3MOS: Multi-sources, Multi-resolutions, and Multi-scenes dataset for Optical-SAR image matching

Optical-SAR image matching is a fundamental task for image fusion and visual navigation. However, all large-scale open SAR dataset for methods development are collected from single platform, resulting in limited satellite types and spatial resolutions. Since images captured by different sensors vary significantly in both geometric and radiometric appearance, existing methods may fail to match corresponding regions containing the same content. Besides, most of existing datasets have not been categorized based on the characteristics of different scenes. To encourage the design of more general multi-modal image matching methods, we introduce a large-scale Multi-sources,Multi-resolutions, and Multi-scenes dataset for Optical-SAR image matching(3MOS). It consists of 155K optical-SAR image pairs, including SAR data from six commercial satellites, with resolutions ranging from 1.25m to 12.5m. The data has been classified into eight scenes including urban, rural, plains, hills, mountains, water, desert, and frozen earth. Extensively experiments show that none of state-of-the-art methods achieve consistently superior performance across different sources, resolutions and scenes. In addition, the distribution of data has a substantial impact on the matching capability of deep learning models, this proposes the domain adaptation challenge in optical-SAR image matching. Our data and code will be available at:https://github.com/3M-OS/3MOS.

cs.CV↗

arXiv2024

TextMonkey: An OCR-Free Large Multimodal Model for Understanding Document

We present TextMonkey, a large multimodal model (LMM) tailored for text-centric tasks. Our approach introduces enhancement across several dimensions: By adopting Shifted Window Attention with zero-initialization, we achieve cross-window connectivity at higher input resolutions and stabilize early training; We hypothesize that images may contain redundant tokens, and by using similarity to filter out significant tokens, we can not only streamline the token length but also enhance the model's performance. Moreover, by expanding our model's capabilities to encompass text spotting and grounding, and incorporating positional information into responses, we enhance interpretability. It also learns to perform screenshot tasks through finetuning. Evaluation on 12 benchmarks shows notable improvements: 5.2% in Scene Text-Centric tasks (including STVQA, TextVQA, and OCRVQA), 6.9% in Document-Oriented tasks (such as DocVQA, InfoVQA, ChartVQA, DeepForm, Kleister Charity, and WikiTableQuestions), and 2.8% in Key Information Extraction tasks (comprising FUNSD, SROIE, and POIE). It outperforms in scene text spotting with a 10.9\% increase and sets a new standard on OCRBench, a comprehensive benchmark consisting of 29 OCR-related assessments, with a score of 561, surpassing previous open-sourced large multimodal models for document understanding. Code will be released at https://github.com/Yuliang-Liu/Monkey.

cs.CV↗

arXiv2023

LYSTO: The Lymphocyte Assessment Hackathon and Benchmark Dataset

We introduce LYSTO, the Lymphocyte Assessment Hackathon, which was held in conjunction with the MICCAI 2019 Conference in Shenzen (China). The competition required participants to automatically assess the number of lymphocytes, in particular T-cells, in histopathological images of colon, breast, and prostate cancer stained with CD3 and CD8 immunohistochemistry. Differently from other challenges setup in medical image analysis, LYSTO participants were solely given a few hours to address this problem. In this paper, we describe the goal and the multi-phase organization of the hackathon; we describe the proposed methods and the on-site results. Additionally, we present post-competition results where we show how the presented methods perform on an independent set of lung cancer slides, which was not part of the initial competition, as well as a comparison on lymphocyte assessment between presented methods and a panel of pathologists. We show that some of the participants were capable to achieve pathologist-level performance at lymphocyte assessment. After the hackathon, LYSTO was left as a lightweight plug-and-play benchmark dataset on grand-challenge website, together with an automatic evaluation platform. LYSTO has supported a number of research in lymphocyte assessment in oncology. LYSTO will be a long-lasting educational challenge for deep learning and digital pathology, it is available at https://lysto.grand-challenge.org/.

eess.IV↗

arXiv2022

Bridging the Domain Gap in Satellite Pose Estimation: a Self-Training Approach based on Geometrical Constraints

Recently, unsupervised domain adaptation in satellite pose estimation has gained increasing attention, aiming at alleviating the annotation cost for training deep models. To this end, we propose a self-training framework based on the domain-agnostic geometrical constraints. Specifically, we train a neural network to predict the 2D keypoints of a satellite and then use PnP to estimate the pose. The poses of target samples are regarded as latent variables to formulate the task as a minimization problem. Furthermore, we leverage fine-grained segmentation to tackle the information loss issue caused by abstracting the satellite as sparse keypoints. Finally, we iteratively solve the minimization problem in two steps: pseudo-label generation and network training. Experimental results show that our method adapts well to the target domain. Moreover, our method won the 1st place on the sunlamp task of the second international Satellite Pose Estimation Competition.

cs.CV↗

arXiv2020

Deep Learning Methods for Lung Cancer Segmentation in Whole-slide Histopathology Images -- the ACDC@LungHP Challenge 2019

Accurate segmentation of lung cancer in pathology slides is a critical step in improving patient care. We proposed the ACDC@LungHP (Automatic Cancer Detection and Classification in Whole-slide Lung Histopathology) challenge for evaluating different computer-aided diagnosis (CADs) methods on the automatic diagnosis of lung cancer. The ACDC@LungHP 2019 focused on segmentation (pixel-wise detection) of cancer tissue in whole slide imaging (WSI), using an annotated dataset of 150 training images and 50 test images from 200 patients. This paper reviews this challenge and summarizes the top 10 submitted methods for lung cancer segmentation. All methods were evaluated using the false positive rate, false negative rate, and DICE coefficient (DC). The DC ranged from 0.7354$\pm$0.1149 to 0.8372$\pm$0.0858. The DC of the best method was close to the inter-observer agreement (0.8398$\pm$0.0890). All methods were based on deep learning and categorized into two groups: multi-model method and single model method. In general, multi-model methods were significantly better ($\textit{p}$<$0.01$) than single model methods, with mean DC of 0.7966 and 0.7544, respectively. Deep learning based methods could potentially help pathologists find suspicious regions for further analysis of lung cancer in WSI.

eess.IV↗

arXiv2020

Minimal Solutions for Relative Pose with a Single Affine Correspondence

In this paper we present four cases of minimal solutions for two-view relative pose estimation by exploiting the affine transformation between feature points and we demonstrate efficient solvers for these cases. It is shown, that under the planar motion assumption or with knowledge of a vertical direction, a single affine correspondence is sufficient to recover the relative camera pose. The four cases considered are two-view planar relative motion for calibrated cameras as a closed-form and a least-squares solution, a closed-form solution for unknown focal length and the case of a known vertical direction. These algorithms can be used efficiently for outlier detection within a RANSAC loop and for initial motion estimation. All the methods are evaluated on both synthetic data and real-world datasets from the KITTI benchmark. The experimental results demonstrate that our methods outperform comparable state-of-the-art methods in accuracy with the benefit of a reduced number of needed RANSAC iterations.

cs.CV↗

arXiv2018

Computer-aided diagnosis of lung carcinoma using deep learning - a pilot study

Aim: Early detection and correct diagnosis of lung cancer are the most important steps in improving patient outcome. This study aims to assess which deep learning models perform best in lung cancer diagnosis. Methods: Non-small cell lung carcinoma and small cell lung carcinoma biopsy specimens were consecutively obtained and stained. The specimen slides were diagnosed by two experienced pathologists (over 20 years). Several deep learning models were trained to discriminate cancer and non-cancer biopsies. Result: Deep learning models give reasonable AUC from 0.8810 to 0.9119. Conclusion: The deep learning analysis could help to speed up the detection process for the whole-slide image (WSI) and keep the comparable detection rate with human observer.

cs.CV↗

arXiv2018

Optimize transfer learning for lung diseases in bronchoscopy using a new concept: sequential fine-tuning

Bronchoscopy inspection as a follow-up procedure from the radiological imaging plays a key role in lung disease diagnosis and determining treatment plans for the patients. Doctors needs to make a decision whether to biopsy the patients timely when performing bronchoscopy. However, the doctors also needs to be very selective with biopsies as biopsies may cause uncontrollable bleeding of the lung tissue which is life-threaten. To help doctors to be more selective on biopsies and provide a second opinion on diagnosis, in this work, we propose a computer-aided diagnosis (CAD) system for lung diseases including cancers and tuberculosis (TB). The system is developed based on transfer learning. We propose a novel transfer learning method: sentential fine-tuning . Compared to traditional fine-tuning methods, our methods achieves the best performance. We obtained a overall accuracy of 77.0% a dataset of 81 normal cases, 76 tuberculosis cases and 277 lung cancer cases while the other traditional transfer learning methods achieve an accuracy of 73% and 68%. . The detection accuracy of our method for cancers, TB and normal cases are 87%, 54% and 91% respectively. This indicates that the CAD system has potential to improve lung disease diagnosis accuracy in bronchoscopy and it also might be used to be more selective with biopsies.

cs.CV↗

arXiv2017

Technical Report for "User-Centric Participatory Sensing: A Game Theoretic Analysis"

Participatory sensing (PS) is a novel and promising sensing network paradigm for achieving a flexible and scalable sensing coverage with a low deploying cost, by encouraging mobile users to participate and contribute their smartphones as sensors. In this work, we consider a general PS system model with location-dependent and time-sensitive tasks, which generalizes the existing models in the literature. We focus on the task scheduling in the user-centric PS system, where each participating user will make his individual task scheduling decision (including both the task selection and the task execution order) distributively. Specifically, we formulate the interaction of users as a strategic game called Task Scheduling Game (TSG) and perform a comprehensive game-theoretic analysis. First, we prove that the proposed TSG game is a potential game, which guarantees the existence of Nash equilibrium (NE). Then, we analyze the efficiency loss and the fairness index at the NE. Our analysis shows the efficiency at NE may increase or decrease with the number of users, depending on the level of competition. This implies that it is not always better to employ more users in the user-centric PS system, which is important for the system designer to determine the optimal number of users to be employed in a practical system.

cs.GT↗

arXiv2017

A Double Auction Mechanism for Mobile Crowd Sensing with Data Reuse

Mobile Crowd Sensing (MCS) is a new paradigm of sensing, which can achieve a flexible and scalable sensing coverage with a low deployment cost, by employing mobile users/devices to perform sensing tasks. In this work, we propose a novel MCS framework with data reuse, where multiple tasks with common data requirement can share (reuse) the common data with each other through an MCS platform. We study the optimal assignment of mobile users and tasks (with data reuse) systematically, under both information symmetry and asymmetry, depending on whether the user cost and the task valuation are public information. In the former case, we formulate the assignment problem as a generalized Knapsack problem and solve the problem by using classic algorithms. In the latter case, we propose a truthful and optimal double auction mechanism, built upon the above Knapsack assignment problem, to elicit the private information of both users and tasks and meanwhile achieve the same optimal assignment as under information symmetry. Simulation results show by allowing data reuse among tasks, the social welfare can be increased up to 100~380%, comparing with those without data reuse. We further show that the proposed double auction is not budget balance for the auctioneer, mainly due to the data reuse among tasks. To this end, we further introduce a reserve price into the double auction (for each data item) to achieve a desired tradeoff between the budget balance and the social efficiency.

cs.GT↗