Searcharxiv⌕ Search

arXiv subjects

Dong Zhang

Publications and source records attributed to Dong Zhang.

At least 145 records · Page 8Linked to original sources
arXiv2024

BoNuS: Boundary Mining for Nuclei Segmentation with Partial Point Labels

Nuclei segmentation is a fundamental prerequisite in the digital pathology workflow. The development of automated methods for nuclei segmentation enables quantitative analysis of the wide existence and large variances in nuclei morphometry in histopathology images. However, manual annotation of tens of thousands of nuclei is tedious and time-consuming, which requires significant amount of human effort and domain-specific expertise. To alleviate this problem, in this paper, we propose a weakly-supervised nuclei segmentation method that only requires partial point labels of nuclei. Specifically, we propose a novel boundary mining framework for nuclei segmentation, named BoNuS, which simultaneously learns nuclei interior and boundary information from the point labels. To achieve this goal, we propose a novel boundary mining loss, which guides the model to learn the boundary information by exploring the pairwise pixel affinity in a multiple-instance learning manner. Then, we consider a more challenging problem, i.e., partial point label, where we propose a nuclei detection module with curriculum learning to detect the missing nuclei with prior morphological knowledge. The proposed method is validated on three public datasets, MoNuSeg, CPM, and CoNIC datasets. Experimental results demonstrate the superior performance of our method to the state-of-the-art weakly-supervised nuclei segmentation methods. Code: https://github.com/hust-linyi/bonus.

cs.CV↗
arXiv2024

SpeechAgents: Human-Communication Simulation with Multi-Modal Multi-Agent Systems

Human communication is a complex and diverse process that not only involves multiple factors such as language, commonsense, and cultural backgrounds but also requires the participation of multimodal information, such as speech. Large Language Model (LLM)-based multi-agent systems have demonstrated promising performance in simulating human society. Can we leverage LLM-based multi-agent systems to simulate human communication? However, current LLM-based multi-agent systems mainly rely on text as the primary medium. In this paper, we propose SpeechAgents, a multi-modal LLM based multi-agent system designed for simulating human communication. SpeechAgents utilizes multi-modal LLM as the control center for individual agent and employes multi-modal signals as the medium for exchanged messages among agents. Additionally, we propose Multi-Agent Tuning to enhance the multi-agent capabilities of LLM without compromising general abilities. To strengthen and evaluate the effectiveness of human communication simulation, we build the Human-Communication Simulation Benchmark. Experimental results demonstrate that SpeechAgents can simulate human communication dialogues with consistent content, authentic rhythm, and rich emotions and demonstrate excellent scalability even with up to 25 agents, which can apply to tasks such as drama creation and audio novels generation. Code and models will be open-sourced at https://github. com/0nutation/SpeechAgents

cs.CL↗
arXiv2023

Bridging the Gap: Generalising State-of-the-Art U-Net Models to Sub-Saharan African Populations

A critical challenge for tumour segmentation models is the ability to adapt to diverse clinical settings, particularly when applied to poor-quality neuroimaging data. The uncertainty surrounding this adaptation stems from the lack of representative datasets, leaving top-performing models without exposure to common artifacts found in MRI data throughout Sub-Saharan Africa (SSA). We replicated a framework that secured the 2nd position in the 2022 BraTS competition to investigate the impact of dataset composition on model performance and pursued four distinct approaches through training a model with: 1) BraTS-Africa data only (train_SSA, N=60), 2) BraTS-Adult Glioma data only (train_GLI, N=1251), 3) both datasets together (train_ALL, N=1311), and 4) through further training the train_GLI model with BraTS-Africa data (train_ftSSA). Notably, training on a smaller low-quality dataset alone (train_SSA) yielded subpar results, and training on a larger high-quality dataset alone (train_GLI) struggled to delineate oedematous tissue in the low-quality validation set. The most promising approach (train_ftSSA) involved pre-training a model on high-quality neuroimages and then fine-tuning it on the smaller, low-quality dataset. This approach outperformed the others, ranking second in the MICCAI BraTS Africa global challenge external testing phase. These findings underscore the significance of larger sample sizes and broad exposure to data in improving segmentation performance. Furthermore, we demonstrated that there is potential for improving such models by fine-tuning them with a wider range of data locally.

cs.CV↗
arXiv2023

Towards SAMBA: Segment Anything Model for Brain Tumor Segmentation in Sub-Sharan African Populations

Gliomas, the most prevalent primary brain tumors, require precise segmentation for diagnosis and treatment planning. However, this task poses significant challenges, particularly in the African population, were limited access to high-quality imaging data hampers algorithm performance. In this study, we propose an innovative approach combining the Segment Anything Model (SAM) and a voting network for multi-modal glioma segmentation. By fine-tuning SAM with bounding box-guided prompts (SAMBA), we adapt the model to the complexities of African datasets. Our ensemble strategy, utilizing multiple modalities and views, produces a robust consensus segmentation, addressing intra-tumoral heterogeneity. Although the low quality of scans presents difficulties, our methodology has the potential to profoundly impact clinical practice in resource-limited settings such as Africa, improving treatment decisions and advancing neuro-oncology research. Furthermore, successful application to other brain tumor types and lesions in the future holds promise for a broader transformation in neurological imaging, improving healthcare outcomes across all settings. This study was conducted on the Brain Tumor Segmentation (BraTS) Challenge Africa (BraTS-Africa) dataset, which provides a valuable resource for addressing challenges specific to resource-limited settings, particularly the African population, and facilitating the development of effective and more generalizable segmentation algorithms. To illustrate our approach's potential, our experiments on the BraTS-Africa dataset yielded compelling results, with SAM attaining a Dice coefficient of 86.6 for binary segmentation and 60.4 for multi-class segmentation.

eess.IV↗
arXiv2023

SeqXGPT: Sentence-Level AI-Generated Text Detection

Widely applied large language models (LLMs) can generate human-like content, raising concerns about the abuse of LLMs. Therefore, it is important to build strong AI-generated text (AIGT) detectors. Current works only consider document-level AIGT detection, therefore, in this paper, we first introduce a sentence-level detection challenge by synthesizing a dataset that contains documents that are polished with LLMs, that is, the documents contain sentences written by humans and sentences modified by LLMs. Then we propose \textbf{Seq}uence \textbf{X} (Check) \textbf{GPT}, a novel method that utilizes log probability lists from white-box LLMs as features for sentence-level AIGT detection. These features are composed like \textit{waves} in speech processing and cannot be studied by LLMs. Therefore, we build SeqXGPT based on convolution and self-attention networks. We test it in both sentence and document-level detection challenges. Experimental results show that previous methods struggle in solving sentence-level AIGT detection, while our method not only significantly surpasses baseline methods in both sentence and document-level detection challenges but also exhibits strong generalization capabilities.

cs.CL↗
arXiv2023

Physics-Informed Neural Network for Discovering Systems with Unmeasurable States with Application to Lithium-Ion Batteries

Combining machine learning with physics is a trending approach for discovering unknown dynamics, and one of the most intensively studied frameworks is the physics-informed neural network (PINN). However, PINN often fails to optimize the network due to its difficulty in concurrently minimizing multiple losses originating from the system's governing equations. This problem can be more serious when the system's states are unmeasurable, like lithium-ion batteries (LiBs). In this work, we introduce a robust method for training PINN that uses fewer loss terms and thus constructs a less complex landscape for optimization. In particular, instead of having loss terms from each differential equation, this method embeds the dynamics into a loss function that quantifies the error between observed and predicted system outputs. This is accomplished by numerically integrating the predicted states from the neural network(NN) using known dynamics and transforming them to obtain a sequence of predicted outputs. Minimizing such a loss optimizes the NN to predict states consistent with observations given the physics. Further, the system's parameters can be added to the optimization targets. To demonstrate the ability of this method to perform various modeling and control tasks, we apply it to a battery model to concurrently estimate its states and parameters.

cs.LG↗
arXiv2023

Homological eigenvalues of graph $p$-Laplacians

Inspired by persistent homology in topological data analysis, we introduce the homological eigenvalues of the graph $p$-Laplacian $Δ_p$, which allows us to analyse and classify non-variational eigenvalues. We show the stability of homological eigenvalues, and we prove that for any homological eigenvalue $λ(Δ_p)$, the function $p\mapsto p(2λ(Δ_p))^{\frac1p}$ is locally increasing, while the function $p\mapsto 2^{-p}λ(Δ_p)$ is locally decreasing. As a special class of homological eigenvalues, the min-max eigenvalues $λ_1(Δ_p)$, $\cdots$, $λ_k(Δ_p)$, $\cdots$, are locally Lipschitz continuous with respect to $p\in[1,+\infty)$. We also establish the monotonicity of $p(2λ_k(Δ_p))^{\frac1p}$ and $2^{-p}λ_k(Δ_p)$ with respect to $p\in[1,+\infty)$. These results systematically establish a refined analysis of $Δ_p$-eigenvalues for varying $p$, which lead to several applications, including: (1) settle an open problem by Amghibech on the monotonicity of some function involving eigenvalues of $p$-Laplacian with respect to $p$; (2) resolve a question asking whether the third eigenvalue of graph $p$-Laplacian is of min-max form; (3) refine the higher order Cheeger inequalities for graph $p$-Laplacians by Tudisco and Hein, and extend the multi-way Cheeger inequality by Lee, Oveis Gharan and Trevisan to the $p$-Laplacian case. Furthermore, for the 1-Laplacian case, we characterize the homological eigenvalues and min-max eigenvalues from the perspective of topological combinatorics, where our idea is similar to the authors' work on discrete Morse theory.

math.SP↗
arXiv2023

Boosting Weakly-Supervised Image Segmentation via Representation, Transform, and Compensator

Weakly-supervised image segmentation (WSIS) is a critical task in computer vision that relies on image-level class labels. Multi-stage training procedures have been widely used in existing WSIS approaches to obtain high-quality pseudo-masks as ground-truth, resulting in significant progress. However, single-stage WSIS methods have recently gained attention due to their potential for simplifying training procedures, despite often suffering from low-quality pseudo-masks that limit their practical applications. To address this issue, we propose a novel single-stage WSIS method that utilizes a siamese network with contrastive learning to improve the quality of class activation maps (CAMs) and achieve a self-refinement process. Our approach employs a cross-representation refinement method that expands reliable object regions by utilizing different feature representations from the backbone. Additionally, we introduce a cross-transform regularization module that learns robust class prototypes for contrastive learning and captures global context information to feed back rough CAMs, thereby improving the quality of CAMs. Our final high-quality CAMs are used as pseudo-masks to supervise the segmentation result. Experimental results on the PASCAL VOC 2012 dataset demonstrate that our method significantly outperforms other state-of-the-art methods, achieving 67.2% and 68.76% mIoU on PASCAL VOC 2012 val set and test set, respectively. Furthermore, our method has been extended to weakly supervised object localization task, and experimental results demonstrate that our method continues to achieve very competitive results.

cs.CV↗
arXiv2023

Quantum-enhanced Electrometer based on Microwave-dressed Rydberg Atoms

Rydberg atoms have been shown remarkable performance in sensing microwave field. The sensitivity of such an electrometer based on optical readout of atomic ensemble has been demonstrated to approach the photon-shot-noise limit. However, the sensitivity can not be promoted infinitely by increasing the power of probe light due to the increased collision rates and power broadening. Compared with classical light, the use of quantum light may lead to a better sensitivity with lower number of photons. In this paper, we exploit entanglement in a microwave-dressed Rydberg electrometer to suppress the fluctuation of noise. The results show a sensitivity enhancement beating the shot noise limit in both cold and hot atom schemes. Through optimizing the transmission of optical readout, our quantum advantage can be maintained with different absorptive index of atomic vapor, which makes it possible to apply quantum light source in the absorptive electrometer.

quant-ph↗
arXiv2023

Improving Reference-based Distinctive Image Captioning with Contrastive Rewards

Distinctive Image Captioning (DIC) -- generating distinctive captions that describe the unique details of a target image -- has received considerable attention over the last few years. A recent DIC method proposes to generate distinctive captions by comparing the target image with a set of semantic-similar reference images, i.e., reference-based DIC (Ref-DIC). It aims to force the generated captions to distinguish between the target image and the reference image. To ensure Ref-DIC models really perceive the unique objects (or attributes) in target images, we propose two new Ref-DIC benchmarks and develop a Transformer-based Ref-DIC baseline TransDIC. The model only extracts visual features from the target image, but also encodes the differences between objects in the target and reference images. Taking one step further, we propose a stronger TransDIC++, which consists of an extra contrastive learning module to make full use of the reference images. This new module is model-agnostic, which can be easily incorporated into various Ref-DIC architectures. Finally, for more trustworthy benchmarking, we propose a new evaluation metric named DisCIDEr for Ref-DIC, which evaluates both the accuracy and distinctiveness of the generated captions. Experimental results demonstrate that our TransDIC++ can generate distinctive captions. Besides, it outperforms several state-of-the-art models on the two new benchmarks over different metrics.

cs.CV↗
arXiv2023

Should ChatGPT and Bard Share Revenue with Their Data Providers? A New Business Model for the AI Era

With various AI tools such as ChatGPT becoming increasingly popular, we are entering a true AI era. We can foresee that exceptional AI tools will soon reap considerable profits. A crucial question arise: should AI tools share revenue with their training data providers in additional to traditional stakeholders and shareholders? The answer is Yes. Large AI tools, such as large language models, always require more and better quality data to continuously improve, but current copyright laws limit their access to various types of data. Sharing revenue between AI tools and their data providers could transform the current hostile zero-sum game relationship between AI tools and a majority of copyrighted data owners into a collaborative and mutually beneficial one, which is necessary to facilitate the development of a virtuous cycle among AI tools, their users and data providers that drives forward AI technology and builds a healthy AI ecosystem. However, current revenue-sharing business models do not work for AI tools in the forthcoming AI era, since the most widely used metrics for website-based traffic and action, such as clicks, will be replaced by new metrics such as prompts and cost per prompt for generative AI tools. A completely new revenue-sharing business model, which must be almost independent of AI tools and be easily explained to data providers, needs to establish a prompt-based scoring system to measure data engagement of each data provider. This paper systematically discusses how to build such a scoring system for all data providers for AI tools based on classification and content similarity models, and outlines the requirements for AI tools or third parties to build it. Sharing revenue with data providers using such a scoring system would encourage more data owners to participate in the revenue-sharing program. This will be a utilitarian AI era where all parties benefit.

cs.LG↗
arXiv2023

Petals and Books: The largest Laplacian spectral gap from 1

We prove that, for any connected graph on $N\geq 3$ vertices, the spectral gap from the value $1$ with respect to the normalized Laplacian is at most $1/2$. Moreover, we show that equality is achieved if and only if the graph is either a petal graph (for $N$ odd) or a book graph (for $N$ even). This implies that $(\frac12,\frac32)$ is a maximal gap interval for the normalized Laplacian on connected graphs. This is closely related to the Alon-Boppana bound on regular graphs and a recent result by Kollár and Sarnak on cubic graphs. Our result also provides a sharp bound for the convergence rate of some eigenvalues of the Laplacian on neighborhood graphs.

math.CO↗
arXiv2023

Nodal domain theorems for $p$-Laplacians on signed graphs

We establish various nodal domain theorems for $p$-Laplacians on signed graphs, which unify most of the existing results on nodal domains of graph $p$-Laplacians and arbitrary symmetric matrices. Based on our nodal domain estimates, we obtain a higher order Cheeger inequality that relates the variational eigenvalues of $p$-Laplacians and Atay-Liu's multi-way Cheeger constants on signed graphs. In the particular case of $p=1$, this leads to several identities relating variational eigenvalues and multi-way Cheeger constants. Intriguingly, our approach also leads to new results on usual graphs, including a weak version of Sturm's oscillation theorem for graph $1$-Laplacians and nonexistence of eigenvalues between the largest and second largest variational eigenvalues of $p$-Laplacians with $p>1$ on connected bipartite graphs.

math.SP↗
arXiv2023

The Brain Tumor Segmentation (BraTS) Challenge 2023: Glioma Segmentation in Sub-Saharan Africa Patient Population (BraTS-Africa)

Gliomas are the most common type of primary brain tumors. Although gliomas are relatively rare, they are among the deadliest types of cancer, with a survival rate of less than 2 years after diagnosis. Gliomas are challenging to diagnose, hard to treat and inherently resistant to conventional therapy. Years of extensive research to improve diagnosis and treatment of gliomas have decreased mortality rates across the Global North, while chances of survival among individuals in low- and middle-income countries (LMICs) remain unchanged and are significantly worse in Sub-Saharan Africa (SSA) populations. Long-term survival with glioma is associated with the identification of appropriate pathological features on brain MRI and confirmation by histopathology. Since 2012, the Brain Tumor Segmentation (BraTS) Challenge have evaluated state-of-the-art machine learning methods to detect, characterize, and classify gliomas. However, it is unclear if the state-of-the-art methods can be widely implemented in SSA given the extensive use of lower-quality MRI technology, which produces poor image contrast and resolution and more importantly, the propensity for late presentation of disease at advanced stages as well as the unique characteristics of gliomas in SSA (i.e., suspected higher rates of gliomatosis cerebri). Thus, the BraTS-Africa Challenge provides a unique opportunity to include brain MRI glioma cases from SSA in global efforts through the BraTS Challenge to develop and evaluate computer-aided-diagnostic (CAD) methods for the detection and characterization of glioma in resource-limited settings, where the potential for CAD tools to transform healthcare are more likely.

eess.IV↗
arXiv2023

There is no going back: Properties of the non-backtracking Laplacian

We prove new properties of the non-backtracking graph and the non-backtracking Laplacian for graphs. In particular, among other results, we prove that two simple graphs are isomorphic if and only if their corresponding non-backtracking graphs are isomorphic, and we investigate properties of various classes of non-backtracking Laplacian eigenfunctions, such as symmetric and antisymmetric eigenfunctions. Moreover, we introduce and study circularly partite graphs as a generalization of bipartite graphs, and we use this notion to state a sharp upper bound for the spectral gap from $1$. We also investigate the singular values of the non-backtracking Laplacian in relation to independence numbers, and we use them to bound the moduli of the eigenvalues.

math.CO↗
arXiv2023

SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilities

Multi-modal large language models are regarded as a crucial step towards Artificial General Intelligence (AGI) and have garnered significant interest with the emergence of ChatGPT. However, current speech-language models typically adopt the cascade paradigm, preventing inter-modal knowledge transfer. In this paper, we propose SpeechGPT, a large language model with intrinsic cross-modal conversational abilities, capable of perceiving and generating multi-model content. With discrete speech representations, we first construct SpeechInstruct, a large-scale cross-modal speech instruction dataset. Additionally, we employ a three-stage training strategy that includes modality-adaptation pre-training, cross-modal instruction fine-tuning, and chain-of-modality instruction fine-tuning. The experimental results demonstrate that SpeechGPT has an impressive capacity to follow multi-modal human instructions and highlight the potential of handling multiple modalities with one model. Demos are shown in https://0nutation.github.io/SpeechGPT.github.io/.

cs.CL↗
arXiv2023

DUB: Discrete Unit Back-translation for Speech Translation

How can speech-to-text translation (ST) perform as well as machine translation (MT)? The key point is to bridge the modality gap between speech and text so that useful MT techniques can be applied to ST. Recently, the approach of representing speech with unsupervised discrete units yields a new way to ease the modality problem. This motivates us to propose Discrete Unit Back-translation (DUB) to answer two questions: (1) Is it better to represent speech with discrete units than with continuous features in direct ST? (2) How much benefit can useful MT techniques bring to ST? With DUB, the back-translation technique can successfully be applied on direct ST and obtains an average boost of 5.5 BLEU on MuST-C En-De/Fr/Es. In the low-resource language scenario, our method achieves comparable performance to existing methods that rely on large-scale external data. Code and models are available at https://github.com/0nutation/DUB.

cs.CL↗
arXiv2023

Discrete-to-Continuous Extensions: Lovász extension, optimizations and eigenvalue problems

In this paper, we use various versions of Lovász extension to systematically derive continuous formulations of problems from discrete mathematics. This will take place in the following context: (1) For combinatorial optimization problems in quotient form, we systematically develop equivalent continuous versions, thereby making tools from convex optimization, fractional programming and more general continuous algorithms like the stochastic subgradient method available for such optimization problems. (2) For some combinatorial quantities like Cheeger-type constants, we suggest a nonlinear eigenvalue problem for a pair of Lovász extensions of certain functions, which encodes certain combinatorial structures. This theory has several applications to quantitative and combinatorial problems, including: (a) The equivalent continuous representations for the max $k$-cut problem, various Cheeger sets and isoperimetric constants are constructed. This also initiates a study of Dirichlet and Neumann 1-Laplacians on graphs, in which the nodal domain property and Cheeger-type equalities are presented. (b) Also, we derive a new equivalent continuous representation of the graph independence number, which can be compared with the Motzkin-Straus theorem. More importantly, an equivalent continuous optimization for the chromatic number is provided, which seems to be the first continuous representation of the graph vertex coloring number. We provide the first continuous reformulation of the frustration index in signed networks, and we find a connection to the so-called modularity measure. Graph matching numbers, submodular vertex covers and multiway partition problems can also be studied in this framework.

math.CO↗