arXiv subjects

Bo Tong

Publications and source records attributed to Bo Tong.

7 recordsLinked to original sources

arXiv2024

FlashSloth: Lightning Multimodal Large Language Models via Embedded Visual Compression

Despite a big leap forward in capability, multimodal large language models (MLLMs) tend to behave like a sloth in practical use, i.e., slow response and large latency. Recent efforts are devoted to building tiny MLLMs for better efficiency, but the plethora of visual tokens still used limit their actual speedup. In this paper, we propose a powerful and fast tiny MLLM called FlashSloth. Different from previous efforts, FlashSloth focuses on improving the descriptive power of visual tokens in the process of compressing their redundant semantics. In particular, FlashSloth introduces embedded visual compression designs to capture both visually salient and instruction-related image information, so as to achieving superior multimodal performance with fewer visual tokens. Extensive experiments are conducted to validate the proposed FlashSloth, and a bunch of tiny but strong MLLMs are also comprehensively compared, e.g., InternVL2, MiniCPM-V2 and Qwen2-VL. The experimental results show that compared with these advanced tiny MLLMs, our FlashSloth can greatly reduce the number of visual tokens, training memory and computation complexity while retaining high performance on various VL tasks.

cs.CV↗

arXiv2024

Deep Instruction Tuning for Segment Anything Model

Recently, Segment Anything Model (SAM) has become a research hotspot in the fields of multimedia and computer vision, which exhibits powerful yet versatile capabilities on various (un) conditional image segmentation tasks. Although SAM can support different types of segmentation prompts, we note that, compared to point- and box-guided segmentations, it performs much worse on text-instructed tasks, e.g., referring image segmentation (RIS). In this paper, we argue that deep text instruction tuning is key to mitigate such shortcoming caused by the shallow fusion scheme in its default light-weight mask decoder. To address this issue, we propose two simple yet effective deep instruction tuning (DIT) methods for SAM, one is end-to-end and the other is layer-wise. With minimal modifications, DITs can directly transform the image encoder of SAM as a stand-alone vision-language learner in contrast to building another deep fusion branch, maximizing the benefit of its superior segmentation capability. Extensive experiments on three highly competitive benchmark datasets of RIS show that a simple end-to-end DIT can improve SAM by a large margin, while the layer-wise DIT can further boost the performance to state-of-the-art with much less data and training expenditures. Our code is released at: https://github.com/wysnzzzz/DIT.

cs.CV↗

arXiv2023

Bottomonium transport in p-Pb and Pb-Pb collisions at the LHC

We employ the Boltzmann transport model to study the sequential suppression pattern of $\Upsilon(1S,2S,3S)$ states in both small (p-Pb) and the large (Pb-Pb) collision systems at $\sqrt{s_{NN}}=5.02$ TeV. The cold nuclear matter effects happen before the formation of bottomonium, which is the same for different bottomonium states $\Upsilon(1S,2S,3S)$. The sequential suppression pattern of bottomonium states is treated as a signal of the hot medium effects, where bottomonium states suffer different magnitudes of the color screening and parton inelastic scatterings due to their different geometry sizes and binding energies. Bottomonium excited states are more easily dissociated with smaller binding energy via the final-state interactions. Including both cold and hot medium effects, transport model consistently explains the experimental data about bottomonium in both small and large collision systems.

nucl-th↗

arXiv2023

PK-Chat: Pointer Network Guided Knowledge Driven Generative Dialogue Model

In the research of end-to-end dialogue systems, using real-world knowledge to generate natural, fluent, and human-like utterances with correct answers is crucial. However, domain-specific conversational dialogue systems may be incoherent and introduce erroneous external information to answer questions due to the out-of-vocabulary issue or the wrong knowledge from the parameters of the neural network. In this work, we propose PK-Chat, a Pointer network guided Knowledge-driven generative dialogue model, incorporating a unified pretrained language model and a pointer network over knowledge graphs. The words generated by PK-Chat in the dialogue are derived from the prediction of word lists and the direct prediction of the external knowledge graph knowledge. Moreover, based on the PK-Chat, a dialogue system is built for academic scenarios in the case of geosciences. Finally, an academic dialogue benchmark is constructed to evaluate the quality of dialogue systems in academic scenarios and the source code is available online.

cs.CL↗

arXiv2023

Bottom energy loss and non-prompt $J/\psi$ production in relativistic heavy ion collisions

We study the momentum and centrality dependence of the non-prompt $J/\psi$ nuclear modification factors ($R_{AA}$), which comes from the $B$ hadrons decay, in Pb-Pb collisions at the Large Hadron Collider. Bottom quarks are produced in the parton hard scatterings and suffer energy loss in the quark-gluon plasma and the hadronic gas, where the spatial and time evolution of the medium is described with the hydrodynamic equations. Medium-induced elastic scatterings and the radiation in bottom quarks are included in the energy loss of bottom quarks. The hadronization process of bottom quarks is described with the instantaneous coalescence model. After considering both cold and hot nuclear matter effects in Pb-Pb collisions at $\sqrt{s_{NN}}=5.02$ TeV, we calculated the $R_{AA}$ and also the elliptic flows of non-prompt $J/\psi$ from the decay of $B$ mesons at different centralities and transverse momentum bins. The $R_{AA}$ and $v_2$ of non-prompt $J/\psi$ sensitive to the hot medium reflect and centrality supply an opportunity to study the bottom quarks energy loss in the hot medium.

nucl-th↗

arXiv2022

Charmonium dissociation at high baryon chemical potential

We study the charmonium dissociation in the hot medium with finite baryon chemical potential $\mu_B$. Charmonium bound states are dissociated in the medium by the color screening effect and the random scatterings with thermal partons, which are included in the real and imaginary parts of the potential respectively. $J/\psi$ fraction in the $c\bar c$ pair defined to be the quantum overlap between the wave package and the wave function of $J/\psi$ eigenstate decreases with time due to the complex potentials. When $\mu_B$ is large compared with the medium temperature, the Deybe mass is increased evidently. We consider $\mu_B$-dependent Deybe mass in both real and imaginary parts of the potential to calculate the $J/\psi$ survival probability in the static medium and the Bjorken medium. $J/\psi$ survival probability is reduced evidently by the $\mu_B$ effect at low temperatures available in the medium produced in Beam Energy Scan experiments, while this effect becomes not apparent at high temperatures.

nucl-th↗

arXiv2014

Existence and stability of nonconstant positive steady states of morphogenesis models

In this paper, We study an one--dimensional morphogenesis model considered by C. Stinner et al. in (Math. Meth. Appl. Sci. 2012,35 (445-465). Under homogeneous boundary conditions, we prove the existence of nonconstant positive steady states through local bifurcation theories. We also rigorously study the stability of the nonconstant solutions when the sensitivity function are chosen to be linear and logarithmic function respectively. Finally, we present numerical solutions to illustrate the formation of stable spatially inhomogeneous patterns. Our numerical simulations suggests that this model can develop very complicated and interesting structures even over one--dimensional finite domains.

math.AP↗