arXiv subjects

Bin Wang

Publications and source records attributed to Bin Wang.

At least 361 records · Page 20Linked to original sources

arXiv2024

MinerU: An Open-Source Solution for Precise Document Content Extraction

Document content analysis has been a crucial research area in computer vision. Despite significant advancements in methods such as OCR, layout detection, and formula recognition, existing open-source solutions struggle to consistently deliver high-quality content extraction due to the diversity in document types and content. To address these challenges, we present MinerU, an open-source solution for high-precision document content extraction. MinerU leverages the sophisticated PDF-Extract-Kit models to extract content from diverse documents effectively and employs finely-tuned preprocessing and postprocessing rules to ensure the accuracy of the final results. Experimental results demonstrate that MinerU consistently achieves high performance across various document types, significantly enhancing the quality and consistency of content extraction. The MinerU open-source project is available at https://github.com/opendatalab/MinerU.

cs.CV↗

arXiv2024

Dark QCD perspective inspired by strong CP problem at QCD scale

We discuss a QCD-scale composite axion model arising from dark QCD coupled to QCD. The presently proposed scenario not only solves the strong CP problem, but also is compatible with the preheating setup for the QCD baryogenesis. The composite axion is phenomenologically required to mimic the QCD pion, but can generically be flavorful, which could be testable via the induced flavor changing processes at experiments. Another axionlike particle (ALP) is predicted to achieve the axion relaxation mechanism, which can phenomenologically act as the conventional QCD axion. This ALP can be ultralight, having the mass less than 1 eV, to be a dark matter candidate. The QCD $\times$ dark QCD symmetry structure constrains dark QCD meson spectra, so that the dark $η'$-like meson would only be accessible at the collider experiments. Still, the Belle II and Electron ion collider experiments can have a high enough sensitivity to probe the dark $η'$-like meson in the diphoton channel, which dominantly arises from the mixing with the QCD $η'$ and the pionic composite axion. We also briefly address nontrivial cosmological aspects, such as those related to the dark-chiral phase transition, the dark matter production, and an ultraviolet completion related to the ultralight ALP.

hep-ph↗

arXiv2024

PMSS: Pretrained Matrices Skeleton Selection for LLM Fine-tuning

Low-rank adaptation (LoRA) and its variants have recently gained much interest due to their ability to avoid excessive inference costs. However, LoRA still encounters the following challenges: (1) Limitation of low-rank assumption; and (2) Its initialization method may be suboptimal. To this end, we propose PMSS(Pre-trained Matrices Skeleton Selection), which enables high-rank updates with low costs while leveraging semantic and linguistic information inherent in pre-trained weight. It achieves this by selecting skeletons from the pre-trained weight matrix and only learning a small matrix instead. Experiments demonstrate that PMSS outperforms LoRA and other fine-tuning methods across tasks with much less trainable parameters. We demonstrate its effectiveness, especially in handling complex tasks such as DROP benchmark(+3.4%/+5.9% on LLaMA2-7B/13B) and math reasoning(+12.89%/+5.61%/+3.11% on LLaMA2-7B, Mistral-7B and Gemma-7B of GSM8K). The code and model will be released soon.

cs.CL↗

arXiv2024

Starshaped compact hypersurfaces in warped product manifolds II: a class of Hessian type equations

In this note, we prove the existence of one particular class of starshaped compact hypersurfaces, by deriving global curvature estimates for such hypersurfaces; this generalizes the main result in [Hypersurfaces of prescribed mixed Weingarten curvature. J. Geom. Anal. $\textbf{34}$ (2024).] from the Euclidean space to Riemannian warped products. Moreover, we show that interior second order a priori estimates for admissible solutions to the associated fully nonlinear elliptic partial differential equations can be readily established by similar arguments.

math.DG↗

arXiv2024

Gauge invariant quantum transport theory for non-Hermitian systems

Gauge invariance is a fundamental principle that must be preserved in quantum transport. However, when a complex potential is incorporated into the Hamiltonian, we find that the current described by the well-established Landauer-B$\ddot{u}$ttiker formula no longer satisfies gauge invariance. Using the non-equilibrium Green's function (NEGF) method, we derive a current expression for a multi-probe system that includes a complex potential in the scattering region. We observe that an additional current term arises compared to the Landauer-B$\ddot{u}$ttiker formula, which leads to a violation of gauge invariance. To address this, we propose two phenomenological methods for redistributing the conductance to restore gauge invariance in non-Hermitian systems. These methods are applied to various trivial and nontrivial non-Hermitian quantum states, confirming the necessity of gauge-invariant treatments in non-Hermitian systems.

cond-mat.mes-hall↗

arXiv2024

Tunable Anomalous Hall Effect in a Kagome Ferromagnetic Weyl Semimetal

Emerging from the intricate interplay of topology and magnetism, the giant anomalous Hall effect (AHE) is the most known topological property of the recently discovered kagome ferromagnetic Weyl semimetal Co_3Sn_2S_2 with the magnetic Co atoms arranged on a kagome lattice. Here we report that the AHE in Co_3Sn_2S_2 can be fine-tuned by an applied magnetic field orientated within ~2 degrees of the kagome plane, while beyond this regime, it stays unchanged. Particularly, it can vanish in magnetic fields parallel to the kagome plane and even decrease in magnetic fields collinear with the spin direction. This tunable AHE can be attributed to local spin switching enabled by the geometrical frustration of the magnetic kagome lattice, revealing that spins in a kagome ferromagnet change their switching behavior as the magnetic field approaches the kagome plane. Our results also suggest a versatile way to tune the properties of a kagome magnet.

cond-mat.mtrl-sci↗

arXiv2024

Two-scale exponential integrators with uniform accuracy for three-dimensional charged-particle dynamics under strong magnetic field

The numerical simulation of three-dimensional charged-particle dynamics (CPD) under strong magnetic field is a basic and challenging algorithmic task in plasma physics. In this paper, we introduce a new methodology to design two-scale exponential integrators for three-dimensional CPD whose magnetic field's strength is inversely proportional to a dimensionless and small parameter $0<\varepsilon \ll 1$. By dealing with the transformed form of three-dimensional CPD, we linearize the magnetic field and put the residual component in a new nonlinear function which is shown to be uniformly bounded. Based on this foundation and the proposed two-scale exponential integrators, a class of novel integrators is formulated and studied. The corresponding uniform accuracy of the proposed $r$-th order integrator is shown to be $\mathcal{O}(h^r)$, where $r=1,2,3,4$ and the constant symbolized by $\mathcal{O}$, the time stepsize $h$ and the computation cost are all independent of $\varepsilon$. Moreover, in the case of maximal ordering strong magnetic field, improved error bound $\mathcal{O}(\varepsilon^r h^r)$ is obtained for the proposed $r$-th order integrator. A rigorous proof of these uniform and improved error bounds is presented, and a numerical test is performed to illustrate the error and efficiency behaviour of the proposed integrators.

math.NA↗

arXiv2024

Probing Quantum Gravity Effects with Eccentric Extreme Mass-Ratio Inspirals

In this paper, we investigate the impact of loop quantum gravity (LQG) on extreme mass-ratio inspirals (EMRIs), and the results indicate that LQG effects cause the orbital decay to occur faster compared to the Schwarzschild case. Furthermore, we use the augmented analytic kludge approach to generate EMRI waveforms and study the LISA's capability to detect the LQG effect with faithfulness. Additionally, employing the Fisher information matrix method for parameter estimation, we estimate that after one year of observation, the uncertainty in $r_0$ reduces to approximately $6.59\times 10^{-4}$ with a signal-to-noise ratio of $49$.

gr-qc↗

arXiv2024

Mixture of Diverse Size Experts

The Sparsely-Activated Mixture-of-Experts (MoE) has gained increasing popularity for scaling up large language models (LLMs) without exploding computational costs. Despite its success, the current design faces a challenge where all experts have the same size, limiting the ability of tokens to choose the experts with the most appropriate size for generating the next token. In this paper, we propose the Mixture of Diverse Size Experts (MoDSE), a new MoE architecture with layers designed to have experts of different sizes. Our analysis of difficult token generation tasks shows that experts of various sizes achieve better predictions, and the routing path of the experts tends to be stable after a training period. However, having experts of diverse sizes can lead to uneven workload distribution. To tackle this limitation, we introduce an expert-pair allocation strategy to evenly distribute the workload across multiple GPUs. Comprehensive evaluations across multiple benchmarks demonstrate the effectiveness of MoDSE, as it outperforms existing MoEs by allocating the parameter budget to experts adaptively while maintaining the same total parameter size and the number of experts.

cs.LG↗

arXiv2024

Picard Groups of Spectral Varieties and Moduli of Higgs Sheaves

We study moduli spaces of Higgs sheaves valued in line bundles and the associated Hitchin maps on surfaces. We first work out Picard groups of generic (very general) spectral varieties which holds for dimension of at least 2, i.e., a Noether--Lefschetz type theorem for spectral varieties. We then apply this to obtain a necessary and sufficient condition for the non-emptyness of generic Hitchin fibers for surfaces cases. Then we move on to detect the geometry of the moduli spaces of Higgs sheaves as the second Chern class varies.

math.AG↗

arXiv2024

MATTopo: Topology-preserving Medial Axis Transform with Restricted Power Diagram

We present a novel topology-preserving 3D medial axis computation framework based on volumetric restricted power diagram (RPD), while preserving the medial features and geometric convergence simultaneously, for both 3D CAD and organic shapes. The volumetric RPD discretizes the input 3D volume into sub-regions given a set of medial spheres. With this intermediate structure, we convert the homotopy equivalency between the generated medial mesh and the input 3D shape into a localized contractibility checking for each restricted element (power cell, power face, power edge), by checking their connected components and Euler characteristics. We further propose a fractional Euler characteristic algorithm for efficient GPU-based computation of Euler characteristic for each restricted element on the fly while computing the volumetric RPD. Compared with existing voxel-based or point-cloud-based methods, our approach is the first to adaptively and directly revise the medial mesh without globally modifying the dependent structure, such as voxel size or sampling density, while preserving its topology and medial features. In comparison with the feature preservation method MATFP, our method provides geometrically comparable results with fewer spheres and more robustly captures the topology of the input 3D shape.

cs.GR↗

arXiv2024

PuYun: Medium-Range Global Weather Forecasting Using Large Kernel Attention Convolutional Networks

Accurate weather forecasting is essential for understanding and mitigating weather-related impacts. In this paper, we present PuYun, an autoregressive cascade model that leverages large kernel attention convolutional networks. The model's design inherently supports extended weather prediction horizons while broadening the effective receptive field. The integration of large kernel attention mechanisms within the convolutional layers enhances the model's capacity to capture fine-grained spatial details, thereby improving its predictive accuracy for meteorological phenomena. We introduce PuYun, comprising PuYun-Short for 0-5 day forecasts and PuYun-Medium for 5-10 day predictions. This approach enhances the accuracy of 10-day weather forecasting. Through evaluation, we demonstrate that PuYun-Short alone surpasses the performance of both GraphCast and FuXi-Short in generating accurate 10-day forecasts. Specifically, on the 10th day, PuYun-Short reduces the RMSE for Z500 to 720 $m^2/s^2$, compared to 732 $m^2/s^2$ for GraphCast and 740 $m^2/s^2$ for FuXi-Short. Additionally, the RMSE for T2M is reduced to 2.60 K, compared to 2.63 K for GraphCast and 2.65 K for FuXi-Short. Furthermore, when employing a cascaded approach by integrating PuYun-Short and PuYun-Medium, our method achieves superior results compared to the combined performance of FuXi-Short and FuXi-Medium. On the 10th day, the RMSE for Z500 is further reduced to 638 $m^2/s^2$, compared to 641 $m^2/s^2$ for FuXi. These findings underscore the effectiveness of our model ensemble in advancing medium-range weather prediction. Our training code and model will be open-sourced.

cs.LG↗

arXiv2024

DocGenome: An Open Large-scale Scientific Document Benchmark for Training and Testing Multi-modal Large Language Models

Scientific documents record research findings and valuable human knowledge, comprising a vast corpus of high-quality data. Leveraging multi-modality data extracted from these documents and assessing large models' abilities to handle scientific document-oriented tasks is therefore meaningful. Despite promising advancements, large models still perform poorly on multi-page scientific document extraction and understanding tasks, and their capacity to process within-document data formats such as charts and equations remains under-explored. To address these issues, we present DocGenome, a structured document benchmark constructed by annotating 500K scientific documents from 153 disciplines in the arXiv open-access community, using our custom auto-labeling pipeline. DocGenome features four key characteristics: 1) Completeness: It is the first dataset to structure data from all modalities including 13 layout attributes along with their LaTeX source codes. 2) Logicality: It provides 6 logical relationships between different entities within each scientific document. 3) Diversity: It covers various document-oriented tasks, including document classification, visual grounding, document layout detection, document transformation, open-ended single-page QA and multi-page QA. 4) Correctness: It undergoes rigorous quality control checks conducted by a specialized team. We conduct extensive experiments to demonstrate the advantages of DocGenome and objectively evaluate the performance of large models on our benchmark.

cs.CV↗

arXiv2024

Understanding Literary Texts by LLMs: A Case Study of Ancient Chinese Poetry

The birth and rapid development of large language models (LLMs) have caused quite a stir in the field of literature. Once considered unattainable, AI's role in literary creation is increasingly becoming a reality. In genres such as poetry, jokes, and short stories, numerous AI tools have emerged, offering refreshing new perspectives. However, it's difficult to further improve the quality of these works. This is primarily because understanding and appreciating a good literary work involves a considerable threshold, such as knowledge of literary theory, aesthetic sensibility, interdisciplinary knowledge. Therefore, authoritative data in this area is quite lacking. Additionally, evaluating literary works is often complex and hard to fully quantify, which directly hinders the further development of AI creation. To address this issue, this paper attempts to explore the mysteries of literary texts from the perspective of LLMs, using ancient Chinese poetry as an example for experimentation. First, we collected a variety of ancient poems from different sources and had experts annotate a small portion of them. Then, we designed a range of comprehension metrics based on LLMs to evaluate all these poems. Finally, we analyzed the correlations and differences between various poem collections to identify literary patterns. Through our experiments, we observed a series of enlightening phenomena that provide technical support for the future development of high-level literary creation based on LLMs.

cs.CL↗

arXiv2024

UMOD: A Novel and Effective Urban Metro Origin-Destination Flow Prediction Method

Accurate prediction of metro Origin-Destination (OD) flow is essential for the development of intelligent transportation systems and effective urban traffic management. Existing approaches typically either predict passenger outflow of departure stations or inflow of destination stations. However, we argue that travelers generally have clearly defined departure and arrival stations, making these OD pairs inherently interconnected. Consequently, considering OD pairs as a unified entity more accurately reflects actual metro travel patterns and allows for analyzing potential spatio-temporal correlations between different OD pairs. To address these challenges, we propose a novel and effective urban metro OD flow prediction method (UMOD), comprising three core modules: a data embedding module, a temporal relation module, and a spatial relation module. The data embedding module projects raw OD pair inputs into hidden space representations, which are subsequently processed by the temporal and spatial relation modules to capture both inter-pair and intra-pair spatio-temporal dependencies. Experimental results on two real-world urban metro OD flow datasets demonstrate that adopting the OD pairs perspective is critical for accurate metro OD flow prediction. Our method outperforms existing approaches, delivering superior predictive performance.

cs.LG↗

arXiv2024

UniMERNet: A Universal Network for Real-World Mathematical Expression Recognition

The paper introduces the UniMER dataset, marking the first study on Mathematical Expression Recognition (MER) targeting complex real-world scenarios. The UniMER dataset includes a large-scale training set, UniMER-1M, which offers unprecedented scale and diversity with one million training instances to train high-quality, robust models. Additionally, UniMER features a meticulously designed, diverse test set, UniMER-Test, which covers a variety of formula distributions found in real-world scenarios, providing a more comprehensive and fair evaluation. To better utilize the UniMER dataset, the paper proposes a Universal Mathematical Expression Recognition Network (UniMERNet), tailored to the characteristics of formula recognition. UniMERNet consists of a carefully designed encoder that incorporates detail-aware and local context features, and an optimized decoder for accelerated performance. Extensive experiments conducted using the UniMER-1M dataset and UniMERNet demonstrate that training on the large-scale UniMER-1M dataset can produce a more generalizable formula recognition model, significantly outperforming all previous datasets. Furthermore, the introduction of UniMERNet enhances the model's performance in formula recognition, achieving higher accuracy and speeds. All data, models, and code are available at https://github.com/opendatalab/UniMERNet.

cs.CV↗

arXiv2024

Segmentation-guided Layer-wise Image Vectorization with Gradient Fills

The widespread use of vector graphics creates a significant demand for vectorization methods. While recent learning-based techniques have shown their capability to create vector images of clear topology, filling these primitives with gradients remains a challenge. In this paper, we propose a segmentation-guided vectorization framework to convert raster images into concise vector graphics with radial gradient fills. With the guidance of an embedded gradient-aware segmentation subroutine, our approach progressively appends gradient-filled Bézier paths to the output, where primitive parameters are initiated with our newly designed initialization technique and are optimized to minimize our novel loss function. We build our method on a differentiable renderer with traditional segmentation algorithms to develop it as a model-free tool for raster-to-vector conversion. It is tested on various inputs to demonstrate its feasibility, independent of datasets, to synthesize vector graphics with improved visual quality and layer-wise topology compared to prior work.

cs.CV↗

arXiv2024

SUBLLM: A Novel Efficient Architecture with Token Sequence Subsampling for LLM

While Large Language Models (LLMs) have achieved remarkable success in various fields, the efficiency of training and inference remains a major challenge. To address this issue, we propose SUBLLM, short for Subsampling-Upsampling-Bypass Large Language Model, an innovative architecture that extends the core decoder-only framework by incorporating subsampling, upsampling, and bypass modules. The subsampling modules are responsible for shortening the sequence, while the upsampling modules restore the sequence length, and the bypass modules enhance convergence. In comparison to LLaMA, the proposed SUBLLM exhibits significant enhancements in both training and inference speeds as well as memory usage, while maintaining competitive few-shot performance. During training, SUBLLM increases speeds by 26% and cuts memory by 10GB per GPU. In inference, it boosts speeds by up to 37% and reduces memory by 1GB per GPU. The training and inference speeds can be enhanced by 34% and 52% respectively when the context window is expanded to 8192. Our code is available at https://github.com/XiaoMi/subllm.

cs.CL↗