arXiv subjects

Bin Wang

Publications and source records attributed to Bin Wang.

At least 433 records · Page 24Linked to original sources

arXiv2024

ToolRerank: Adaptive and Hierarchy-Aware Reranking for Tool Retrieval

Tool learning aims to extend the capabilities of large language models (LLMs) with external tools. A major challenge in tool learning is how to support a large number of tools, including unseen tools. To address this challenge, previous studies have proposed retrieving suitable tools for the LLM based on the user query. However, previously proposed methods do not consider the differences between seen and unseen tools, nor do they take the hierarchy of the tool library into account, which may lead to suboptimal performance for tool retrieval. Therefore, to address the aforementioned issues, we propose ToolRerank, an adaptive and hierarchy-aware reranking method for tool retrieval to further refine the retrieval results. Specifically, our proposed ToolRerank includes Adaptive Truncation, which truncates the retrieval results related to seen and unseen tools at different positions, and Hierarchy-Aware Reranking, which makes retrieval results more concentrated for single-tool queries and more diverse for multi-tool queries. Experimental results show that ToolRerank can improve the quality of the retrieval results, leading to better execution results generated by the LLM.

cs.IR↗

arXiv2024

Eckart streaming with nonlinear high-order harmonics: an example at gigahertz

Acoustic streaming shows great potential in applications such as bubble dynamics, cell aggregation, and nano-sized particle isolation in the biomedical and drug industries. As the acoustic shock distance decreases with the increase of incident frequency, the nonlinear propagation effect will play a role in acoustic streaming, e.g., Eckart (bulk) streaming at a few gigahertz (GHz). However, the theory of source terms of bulk streaming is still missing at this stage when high-order acoustic harmonics play a role. In this paper, we derive the source term including the contribution of higher-order harmonics. The streaming-induced hydrodynamic flow is assumed to be incompressible and no shock wave occurs during the nonlinear acoustic propagation as restricted by the traditional Goldberg number Γ< 1 or Γ\approx 1 which indicates the importance of nonlinearity relative to dissipation. The derived force terms allow evaluating bulk streaming with high-order harmonics at GHz and provide an exact expression compared to the existing empirical formulas. Numerical results show that the contribution of higher-order harmonics increases the streaming flow velocity by more than 20%. We show that the expression introduced by Nyborg should be avoided in numerical computations as it includes part of the acoustic radiation force that does not lead to acoustic streaming.

physics.flu-dyn↗

arXiv2024

Communication Efficient ConFederated Learning: An Event-Triggered SAGA Approach

Federated learning (FL) is a machine learning paradigm that targets model training without gathering the local data dispersed over various data sources. Standard FL, which employs a single server, can only support a limited number of users, leading to degraded learning capability. In this work, we consider a multi-server FL framework, referred to as \emph{Confederated Learning} (CFL), in order to accommodate a larger number of users. A CFL system is composed of multiple networked edge servers, with each server connected to an individual set of users. Decentralized collaboration among servers is leveraged to harness all users' data for model training. Due to the potentially massive number of users involved, it is crucial to reduce the communication overhead of the CFL system. We propose a stochastic gradient method for distributed learning in the CFL framework. The proposed method incorporates a conditionally-triggered user selection (CTUS) mechanism as the central component to effectively reduce communication overhead. Relying on a delicately designed triggering condition, the CTUS mechanism allows each server to select only a small number of users to upload their gradients, without significantly jeopardizing the convergence performance of the algorithm. Our theoretical analysis reveals that the proposed algorithm enjoys a linear convergence rate. Simulation results show that it achieves substantial improvement over state-of-the-art algorithms in terms of communication efficiency.

cs.LG↗

arXiv2024

Hitchin systems: some recent advances

A survey of some recent advances in parabolic Hitchin systems (parabolic Bouville--Narasimhan--Ramanan correspondence, mirror symmetry for parabolic Hitchin systems), and in exact methods of solving the non-parabolic Hitchin systems.

math-ph↗

arXiv2024

Asymptotical quasinormal mode spectrum for piecewise approximate effective potential

It was pointed out that the black hole quasinormal modes resulting from a piecewise approximate potential are drastically distinct from those pertaining to the original black hole metric. In particular, instead of lining up parallel to the imaginary axis, the spectrum is found to stretch out along the real axis. In this work, we prove that if there is a single discontinuity in the effective potential, no matter how insignificant it is, the asymptotical behavior of the quasinormal modes will be appreciably modified. Besides showing numerical evidence, we give analytical derivations to support the above assertion even when the discontinuity is located significantly further away from the maximum of the potential and/or the size of the step is arbitrarily small. Moreover, we discuss the astrophysical significance of the potential implications in terms of the present findings.

gr-qc↗

arXiv2024

Linear displacement current solely driven by the quantum metric

Quantum metric and Berry curvature are the real part and imaginary part of the quantum geometric tensor, respectively. The T-odd (T: time-reversal) nonlinear Hall effect driven by the quantum metric dipole, recently confirmed in Science 381, 181 (2023) and Nature 621, 487 (2023), established the geometric duality to the T-even nonlinear Hall effect that driven by the Berry curvature dipole. Interestingly, a similar geometric duality between the quantum metric and the Berry curvature, particularly for the linear response of Bloch electrons, has not been established, although the T-odd linear intrinsic anomalous Hall effect (IAHE) solely driven by the Berry curvature has been known for a long time. Herein, we develop the quantum theory for displacement current under an AC electric field. Particularly, we show that the T-even component of the linear displacement current conductivity (LDCC) is solely determined by the quantum metric, by both the response theory and the semiclassical theory. Notably, with symmetry analysis we find that the T-even LDCC can contribute a Hall current in T-invariant systems but with low symmetry, while its longitudinal component is immune to symmetry. Furthermore, employing the Dirac Hamiltonian, we arrive at a $1/μ$ ($μ$: chemical potential) experimental observable enhancement of the displacement current owing to the divergent behavior of quantum metric near Dirac point, similar to the IAHE at Weyl point. Our work reveals the band geometric origin of the linear displacement current and establishes, together with the IAHE, the geometric duality for the linear response of Bloch electrons. Additionally, our work offers the very first intrinsic Hall effect in T-invariant materials, which can not be envisioned in DC transport in both linear and nonlinear regimes.

cond-mat.mes-hall↗

arXiv2024

Beyond Hallucinations: Enhancing LVLMs through Hallucination-Aware Direct Preference Optimization

Multimodal large language models have made significant advancements in recent years, yet they still suffer from a common issue known as the "hallucination problem", in which the models generate textual descriptions that inaccurately depict or entirely fabricate content from associated images. This paper introduces a novel solution, Hallucination-Aware Direct Preference Optimization (HA-DPO), which reframes the hallucination problem as a preference selection task. The model is trained to favor the non-hallucinating response when presented with two responses of the same image (one accurate and one hallucinatory). Furthermore, this paper proposes an efficient pipeline for constructing positive~(non-hallucinatory) and negative~(hallucinatory) sample pairs, ensuring a high-quality, style-consistent dataset for robust preference learning. When applied to three mainstream multimodal models, HA-DPO significantly reduced hallucination issues and amplified the models' generalization capabilities. Notably, the MiniGPT-4 model, when enhanced with HA-DPO, demonstrated a substantial improvement: POPE accuracy rose from 51.13% to 86.13% (an absolute improvement of 35%), and the MME score surged from 932.00 to 1326.46 (a relative improvement of 42.32%). The codes, models, and datasets are made accessible at https://opendatalab.github.io/HA-DPO.

cs.CV↗

arXiv2024

VIGC: Visual Instruction Generation and Correction

The integration of visual encoders and large language models (LLMs) has driven recent progress in multimodal large language models (MLLMs). However, the scarcity of high-quality instruction-tuning data for vision-language tasks remains a challenge. The current leading paradigm, such as LLaVA, relies on language-only GPT-4 to generate data, which requires pre-annotated image captions and detection bounding boxes, suffering from understanding image details. A practical solution to this problem would be to utilize the available multimodal large language models (MLLMs) to generate instruction data for vision-language tasks. However, it's worth noting that the currently accessible MLLMs are not as powerful as their LLM counterparts, as they tend to produce inadequate responses and generate false information. As a solution for addressing the current issue, this paper proposes the Visual Instruction Generation and Correction (VIGC) framework that enables multimodal large language models to generate instruction-tuning data and progressively enhance its quality on-the-fly. Specifically, Visual Instruction Generation (VIG) guides the vision-language model to generate diverse instruction-tuning data. To ensure generation quality, Visual Instruction Correction (VIC) adopts an iterative update mechanism to correct any inaccuracies in data produced by VIG, effectively reducing the risk of hallucination. Leveraging the diverse, high-quality data generated by VIGC, we finetune mainstream models and validate data quality based on various evaluations. Experimental results demonstrate that VIGC not only compensates for the shortcomings of language-only data generation methods, but also effectively enhances the benchmark performance. The models, datasets, and code are available at https://opendatalab.github.io/VIGC.

cs.CV↗

arXiv2024

In-Context Learning for Few-Shot Nested Named Entity Recognition

In nested Named entity recognition (NER), entities are nested with each other, and thus requiring more data annotations to address. This leads to the development of few-shot nested NER, where the prevalence of pretrained language models with in-context learning (ICL) offers promising solutions. In this work, we introduce an effective and innovative ICL framework for the setting of few-shot nested NER. We improve the ICL prompt by devising a novel example demonstration selection mechanism, EnDe retriever. In EnDe retriever, we employ contrastive learning to perform three types of representation learning, in terms of semantic similarity, boundary similarity, and label similarity, to generate high-quality demonstration examples. Extensive experiments over three nested NER and four flat NER datasets demonstrate the efficacy of our system.

cs.CL↗

arXiv2024

Active Support of Inverters for Improving Short-Term Voltage Security in 100% IBRsPenetrated Power Systems

Due to the energy crisis and environmental pollution, the installed capacity of inverter-based resources (IBRs) in power grids is rapidly increasing, and grid-following control (GFL) is the most prevalent at present. Meanwhile, grid-forming control-based (GFM) devices have been installed in the grid to provide active support for frequency and voltage. In the future GFL devices combined with GFM will be promising, especially in power systems with high penetration or 100% IBRs. When a short-circuit fault occurs in the grid, the controlled current source characteristic of the GFL devices leads to insufficient dynamic voltage support (DVS), while the GFM devices usually reduce the internal voltage to limit the current. Thus, deep voltage sags and undesired disconnections of IBRs may occur. Moreover, due to the dispersed locations and the control strategies' diversity of IBRs, the voltage support of different devices may not be fully coordinated, which is not conducive to short-term voltage security (STVS). To address this issue, a control scheme based on the simulation of transient characteristics of synchronous machines (SMs) is proposed. Then, a new fault ride-through strategy (FRT) is proposed based on the characteristic differences between GFL and GFM devices, and an optimization model of multi-device control parameters is formulated to meet the short-term voltage security constraints (SVSCs) and device capacity constraints. Finally, a fast solution method based on analytical modeling is proposed for the model. Test results based on the doublegenerator-one-load system, the IEEE 14-bus system, and other systems of different sizes show that the proposed method can effectively enhance the active support capability of GFL and GFM to the grid voltage, and avoid the large-scale disconnection of IBRs

eess.SY↗

arXiv2024

Parrot Captions Teach CLIP to Spot Text

Despite CLIP being the foundation model in numerous vision-language applications, the CLIP suffers from a severe text spotting bias. Such bias causes CLIP models to `Parrot' the visual text embedded within images while disregarding the authentic visual semantics. We uncover that in the most popular image-text dataset LAION-2B, the captions also densely parrot (spell) the text embedded in images. Our analysis shows that around 50% of images are embedded with visual text content, and around 30% of captions words are in these embedded visual content. Based on such observation, we thoroughly inspect the different released versions of CLIP models and verify that the visual text is the dominant factor in measuring the LAION-style image-text similarity for these models. To examine whether these parrot captions shape the text spotting bias, we train a series of CLIP models with LAION subsets curated by different parrot-caption-oriented criteria. We show that training with parrot captions easily shapes such bias but harms the expected visual-language representation learning in CLIP models. This suggests that it is urgent to revisit either the design of CLIP-like models or the existing image-text dataset curation pipeline built on CLIP score filtering.

cs.CV↗

arXiv2024

InternLM-XComposer2: Mastering Free-form Text-Image Composition and Comprehension in Vision-Language Large Model

We introduce InternLM-XComposer2, a cutting-edge vision-language model excelling in free-form text-image composition and comprehension. This model goes beyond conventional vision-language understanding, adeptly crafting interleaved text-image content from diverse inputs like outlines, detailed textual specifications, and reference images, enabling highly customizable content creation. InternLM-XComposer2 proposes a Partial LoRA (PLoRA) approach that applies additional LoRA parameters exclusively to image tokens to preserve the integrity of pre-trained language knowledge, striking a balance between precise vision understanding and text composition with literary talent. Experimental results demonstrate the superiority of InternLM-XComposer2 based on InternLM2-7B in producing high-quality long-text multi-modal content and its exceptional vision-language understanding performance across various benchmarks, where it not only significantly outperforms existing multimodal models but also matches or even surpasses GPT-4V and Gemini Pro in certain assessments. This highlights its remarkable proficiency in the realm of multimodal understanding. The InternLM-XComposer2 model series with 7B parameters are publicly available at https://github.com/InternLM/InternLM-XComposer.

cs.CV↗

arXiv2024

Impact of Flexible and Bidirectional Charging in Medium- and Heavy-Duty Trucks on California's Decarbonization Pathway

California has committed to ambitious decarbonization targets across multiple sectors, including decarbonizing the electrical grid by 2045. In addition, the medium- and heavy-duty truck fleets are expected to see rapid electrification over the next two decades. Considering these two pathways in tandem is critical for ensuring cost optimality and reliable power system operation. In particular, we examine the potential cost savings of electrical generation infrastructure by enabling flexible charging and bidirectional charging for these trucks. We also examine costs adjacent to enabling these services, such as charger upgrades and battery degradation. We deploy a large mixed-integer decarbonization planning model to quantify the costs associated with the electric generation decarbonization pathway. Example scenarios governing truck driving and charging behaviors are implemented to reveal the sensitivity of temporal driving patterns. Our experiments show that cost savings on the order of multiple billions of dollars are possible by enabling flexible and bidirectional charging in medium- and heavy-duty trucks in California.

eess.SY↗

arXiv2024

Black holes with scalar hair: Extending from and beyond the Schwarzschild solution

We construct novel scalarized black hole (BH) solutions beyond the general relativity (GR) framework. These scalarized BH solutions are extended from the Schwarzschild one and the non-Schwarzschild one in the pure Einstein-Weyl gravity. By studying the BH entropy and free energy, we demonstrate that the scalarized BH extending from the Schwarzschild one exhibits thermodynamically preferred. We obtain these novel solutions by directly solving the full fourth-order equations of motion. This narrows the problematic solution space obtained by commonly adopted second-order reduction to physically valid spaces. Our findings also unveil the evasion of the no-hair theorem within the realm of higher-derivative gravity.

gr-qc↗

arXiv2024

eCIL-MU: Embedding based Class Incremental Learning and Machine Unlearning

New categories may be introduced over time, or existing categories may need to be reclassified. Class incremental learning (CIL) is employed for the gradual acquisition of knowledge about new categories while preserving information about previously learned ones in such dynamic environments. It might also be necessary to also eliminate the influence of related categories on the model to adapt to reclassification. We thus introduce class-level machine unlearning (MU) within CIL. Typically, MU methods tend to be time-consuming and can potentially harm the model's performance. A continuous stream of unlearning requests could lead to catastrophic forgetting. To address these issues, we propose a non-destructive eCIL-MU framework based on embedding techniques to map data into vectors and then be stored in vector databases. Our approach exploits the overlap between CIL and MU tasks for acceleration. Experiments demonstrate the capability of achieving unlearning effectiveness and orders of magnitude (upto $\sim 278\times$) of acceleration.

cs.LG↗

arXiv2023

Simulating Parametric Thin Shells by Bicubic Hermite Elements

In this study, we present the bicubic Hermite element method (BHEM), a new computational framework devised for the elastodynamic simulation of parametric thin-shell structures. The BHEM is constructed based on parametric quadrilateral Hermite patches, which serve as a unified representation for shell geometry, simulation, collision avoidance, as well as rendering. Compared with the commonly utilized linear FEM, the BHEM offers higher-order solution spaces, enabling the capture of more intricate and smoother geometries while employing significantly fewer finite elements. In comparison to other high-order methods, the BHEM achieves conforming $\mathcal{C}^1$ continuity for Kirchhoff-Love (KL) shells with minimal complexity. Furthermore, by leveraging the subdivision and convex hull properties of Hermite patches, we develop an efficient algorithm for ray-patch intersections, facilitating collision handling in simulations and ray tracing in rendering. This eliminates the need for laborious remodeling of the pre-existing parametric surface as the conventional approaches do. We substantiate our claims with comprehensive experiments, which demonstrate the high accuracy and versatility of the proposed method.

cs.GR↗

arXiv2023

Curvature estimates for hypersurfaces of constant curvature in hyperbolic space

In this note, we prove that for every $0<σ<1$, there exists a smooth complete hypersurface $Σ$ in $\mathbb{H}^{n+1}$ with prescribed asymptotic boundary $\partial Σ=Γ$ at infinity, whose principal curvatures $κ=(κ_1,\ldots,κ_n)$ lie in a general cone $K$ and satisfy $f(κ)=σ$ at each point of $Σ$. Previously, the problem has been studied by Guan-Spruck in [J. Eur. Math. Soc. (JEMS) 12 (2010), no. 3, 797-817], and they proved the existence result for $σ\in (σ_0,1)$, where $σ_0>0$. A major ingredient of our proof is a refined curvature estimate of theirs that is applicable when the curvature function $f(κ)$ has controllable partial derivatives, but it is adequate for our purpose; specifically, we solve the problem for $f=H_{k}/H_{k-1}$ in the $k$-th Garding cone where $H_k$ is the normalized $k$-th elementary symmetric polynomial and $1 \leq k \leq n$.

math.DG↗

arXiv2023

InternLM-XComposer: A Vision-Language Large Model for Advanced Text-image Comprehension and Composition

We propose InternLM-XComposer, a vision-language large model that enables advanced image-text comprehension and composition. The innovative nature of our model is highlighted by three appealing properties: 1) Interleaved Text-Image Composition: InternLM-XComposer can effortlessly generate coherent and contextual articles that seamlessly integrate images, providing a more engaging and immersive reading experience. Simply provide a writing instruction, and our system will generate the corresponding manuscript. It can intelligently identify the areas in the text where images would enhance the content and automatically insert the most appropriate visual candidates. 2) Comprehension with Rich Multilingual Knowledge: The text-image comprehension is empowered by training on an extensive multi-modal multilingual database with carefully crafted strategies, resulting in a deep understanding of visual content. 3) State-of-the-art Performance: Our model consistently achieves state-of-the-art results across various mainstream benchmarks for vision-language foundational models, including MME Benchmark, MMBench, MMBench-CN, Seed-Bench, CCBench (Chinese Cultural Benchmark), QBench and Tiny LVLM. Owing to the absence of established metrics for quantitatively assessing text-image composition, we have devised a robust evaluation procedure that comprises both human and GPT4-Vision (GPT4-V) to ensure reliability. Notably, our InternLM-XComposer achieves competitive text-image composition scores compared to public solutions, including GPT4-V and GPT3.5. Collectively, InternLM-XComposer seamlessly blends advanced text-image comprehension and composition, revolutionizing vision-language interaction and offering new insights and opportunities. The InternLM-XComposer model series are publicly available at https://github.com/InternLM/InternLM-XComposer.

cs.CV↗