Searcharxiv⌕ Search

arXiv subjects

Wei Li

Publications and source records attributed to Wei Li.

At least 721 records · Page 40Linked to original sources
arXiv2023

WanJuan: A Comprehensive Multimodal Dataset for Advancing English and Chinese Large Models

The rise in popularity of ChatGPT and GPT-4 has significantly accelerated the development of large models, leading to the creation of numerous impressive large language models(LLMs) and multimodal large language models (MLLMs). These cutting-edge models owe their remarkable performance to high-quality data. However, the details of the training data used in leading paradigms are often kept confidential. This lack of transparency, coupled with the scarcity of open-source data, impedes further developments within the community. As a response, this paper presents "Wan Juan", a large-scale multimodal dataset composed of both Chinese and English data, collected from a wide range of web sources. The dataset incorporates text, image-text, and video modalities, with a total volume exceeding 2TB. It was utilized in the training of InternLM, a model that demonstrated significant advantages in multi-dimensional evaluations when compared to models of a similar scale. All data can be accessed at https://opendatalab.org.cn/WanJuan1.0.

cs.CL↗
arXiv2023

Multiply robust estimation of causal effects using linked data

Unmeasured confounding presents a common challenge in observational studies, potentially making standard causal parameters unidentifiable without additional assumptions. Given the increasing availability of diverse data sources, exploiting data linkage offers a potential solution to mitigate unmeasured confounding within a primary study of interest. However, this approach often introduces selection bias, as data linkage is feasible only for a subset of the study population. To address this concern, we explore three nonparametric identification strategies under the assumption that a unit' s inclusion in the linked cohort is determined solely by the observed confounders, while acknowledging that the ignorability assumption may depend on some partially unobserved covariates. The existence of multiple identification strategies motivates the development of estimators that effectively capture distinct components of the observed data distribution. Appropriately combining these estimators yields triply robust estimators for the average treatment effect. These estimators remain consistent if at least one of the three distinct parts of the observed data law is correct. Moreover, they are locally efficient if all the models are correctly specified. We evaluate the proposed estimators using simulation studies and real data analysis.

stat.ME↗
arXiv2023

Plaquette Singlet Transition, Magnetic Barocaloric Effect, and Spin Supersolidity in the Shastry-Sutherland Model

Inspired by recent experimental measurements [Guo \textit{et al.}, Phys. Rev. Lett.~\textbf{124}, 206602 (2020); Jiménez \textit{et al.}, Nature \textbf{592}, 370 (2021)] on frustrated quantum magnet SrCu$_2$(BO$_3$)$_2$ under combined pressure and magnetic fields, we study the related spin-$1/2$ Shastry-Sutherland (SS) model using state-of-the-art tensor network methods. By calculating thermodynamics, correlations and susceptibilities, we find, in zero magnetic field, not only a line of first-order plaquette-singlet (PS) to dimer-singlet phase transition ending with a critical point, but also signatures of the ordered PS transition with its critical endpoint terminating on this first-order line. Moreover, we uncover prominent magnetic barocaloric responses, a novel type of quantum correlation induced cooling effect, in the strongly fluctuating supercritical regime. Under finite fields, we identify a quantum phase transition from the PS phase to the spin supersolid phase that breaks simultaneously lattice translational and spin rotational symmetries. The present findings on the SS model are accessible in current experiments and would shed new light on exotic critical and supercritical phenomena in archetypal frustrated quantum magnets.

cond-mat.str-el↗
arXiv2023

PaLM 2 Technical Report

We introduce PaLM 2, a new state-of-the-art language model that has better multilingual and reasoning capabilities and is more compute-efficient than its predecessor PaLM. PaLM 2 is a Transformer-based model trained using a mixture of objectives. Through extensive evaluations on English and multilingual language, and reasoning tasks, we demonstrate that PaLM 2 has significantly improved quality on downstream tasks across different model sizes, while simultaneously exhibiting faster and more efficient inference compared to PaLM. This improved efficiency enables broader deployment while also allowing the model to respond faster, for a more natural pace of interaction. PaLM 2 demonstrates robust reasoning capabilities exemplified by large improvements over PaLM on BIG-Bench and other reasoning tasks. PaLM 2 exhibits stable performance on a suite of responsible AI evaluations, and enables inference-time control over toxicity without additional overhead or impact on other capabilities. Overall, PaLM 2 achieves state-of-the-art performance across a diverse set of tasks and capabilities. When discussing the PaLM 2 family, it is important to distinguish between pre-trained models (of various sizes), fine-tuned variants of these models, and the user-facing products that use these models. In particular, user-facing products typically include additional pre- and post-processing steps. Additionally, the underlying models may evolve over time. Therefore, one should not expect the performance of user-facing products to exactly match the results reported in this report.

cs.CL↗
arXiv2023

Possible Intermediate Quantum Spin Liquid Phase in $α$-RuCl$_3$ under High Magnetic Fields up to 100 T

Pursuing the exotic quantum spin liquid (QSL) state in the Kitaev material $α$-RuCl$_3$ has intrigued great research interest recently. A fascinating question is on the possible existence of field-induced QSL phase in this compound. Here we perform a high-field measurement of the magnetization process of $α$-RuCl$_3$ up to 102 T employing the non-destructive and destructive pulsed magnets. Under the out-of-plane field along the c$^*$ axis (i.e., perpendicular to the honeycomb plane), two quantum phase transitions are uncovered at respectively 36 T and about 83 T, between which there lies an intermediate phase as the predicted QSL. By measuring the magnetization data with fields tilted from the c$^*$ axis up to $90^\circ$ (i.e., in-plane direction), we obtain the field-angle phase diagram that contains the zigzag, paramagnetic, and QSL phases. Based on the accurate $K$-$J$-$Γ$-$Γ^{\prime}$ model of $α$-RuCl$_3$, we perform density matrix renormalization group simulations and reproduce the quantum phase diagram in excellent agreement with experiments.

cond-mat.str-el↗
arXiv2023

SoccerNet 2023 Challenges Results

The SoccerNet 2023 challenges were the third annual video understanding challenges organized by the SoccerNet team. For this third edition, the challenges were composed of seven vision-based tasks split into three main themes. The first theme, broadcast video understanding, is composed of three high-level tasks related to describing events occurring in the video broadcasts: (1) action spotting, focusing on retrieving all timestamps related to global actions in soccer, (2) ball action spotting, focusing on retrieving all timestamps related to the soccer ball change of state, and (3) dense video captioning, focusing on describing the broadcast with natural language and anchored timestamps. The second theme, field understanding, relates to the single task of (4) camera calibration, focusing on retrieving the intrinsic and extrinsic camera parameters from images. The third and last theme, player understanding, is composed of three low-level tasks related to extracting information about the players: (5) re-identification, focusing on retrieving the same players across multiple views, (6) multiple object tracking, focusing on tracking players and the ball through unedited video streams, and (7) jersey number recognition, focusing on recognizing the jersey number of players from tracklets. Compared to the previous editions of the SoccerNet challenges, tasks (2-3-7) are novel, including new annotations and data, task (4) was enhanced with more data and annotations, and task (6) now focuses on end-to-end approaches. More information on the tasks, challenges, and leaderboards are available on https://www.soccer-net.org. Baselines and development kits can be found on https://github.com/SoccerNet.

cs.CV↗
arXiv2023

Crossover from viscous fingering to fracturing in cohesive wet granular media: a photoporomechanics study

We study fluid-induced deformation and fracture of cohesive granular media, and apply photoporomechanics to uncover the underpinning grain-scale mechanics. We fabricate photoelastic spherical particles of diameter d=2mm, and make a monolayer granular pack with tunable intergranular cohesion in a circular Hele-Shaw cell that is initially filled with viscous silicone oil. We inject water into the oil-filled photoelastic granular pack, varying the injection flow rate, defending-fluid viscosity, and intergranular cohesion. We find two different modes of fluid invasion: viscous fingering, and fracturing with leak-off of the injection fluid. We directly visualize the evolving effective stress field through the particles' photoelastic response, and discover a hoop effective stress region behind the water invasion front, where we observe tensile force chains in the circumferential direction. Outside the invasion front, we observe compressive force chains aligning in the radial direction. We conceptualize the system's behavior by means of a two-phase poroelastic continuum model. The model captures granular pack dilation and compaction with the boundary delineated by the invasion front, which explains the observed distinct alignments of the force chains. Finally, we rationalize the crossover from viscous fingering to fracturing by comparing the competing forces behind the process: viscous force from fluid injection that drives fractures, and intergranular cohesion and friction that resist fractures.

cond-mat.soft↗
arXiv2023

Visual-Guided Mesh Repair

Mesh repair is a long-standing challenge in computer graphics and related fields. Converting defective meshes into watertight manifold meshes can greatly benefit downstream applications such as geometric processing, simulation, fabrication, learning, and synthesis. In this work, we first introduce three visual measures for visibility, orientation, and openness, based on ray-tracing. We then present a novel mesh repair framework that incorporates visual measures with several critical steps, i.e., open surface closing, face reorientation, and global optimization, to effectively repair defective meshes, including gaps, holes, self-intersections, degenerate elements, and inconsistent orientations. Our method reduces unnecessary mesh complexity without compromising geometric accuracy or visual quality while preserving input attributes such as UV coordinates for rendering. We evaluate our approach on hundreds of models randomly selected from ShapeNet and Thingi10K, demonstrating its effectiveness and robustness compared to existing approaches.

cs.GR↗
arXiv2023

Stripe charge order driven manipulation of Majorana bound states in 2M-WS2 topological superconductor

Majorana bound states (MBSs) are building blocks for topological quantum computing. They can be generated via the combination of electronic topology and superconductivity. To achieve logic operations via Majorana braiding, positional control of the MBS must be established. To this end, exotic co-existing phases or collective modes in an intrinsic topological superconductor can provide a tuning knob to manipulate the MBS. Here we report the observation of a striped surface charge order coexisting with superconductivity and its controllable tuning of the MBS in the topological superconductor 2M-WS2 using low-temperature scanning tunneling microscopy. By applying an out-of-plane magnetic field, we observe that MBS is absent in vortices in the region with strong stripe order. This is in contrast to adjacent underlaying layers without charge order where vortex-bound MBSs are observed. Via theoretical simulations, we show that the surface stripe order does not destroy the bulk topology, but it can effectively modify the spatial distribution of MBS, i.e., it pushes them downward away from the 2M-WS2 surface. Our findings demonstrate that the interplay of charge order and topological superconductivity can be used to manipulate the positions of the MBS, and to explore of new states of matter.

cond-mat.supr-con↗
arXiv2023

SAIPy: A Python Package for single station Earthquake Monitoring using Deep Learning

Seismology has witnessed significant advancements in recent years with the application of deep learning methods to address a broad range of problems. These techniques have demonstrated their remarkable ability to effectively extract statistical properties from extensive datasets, surpassing the capabilities of traditional approaches to an extent. In this study, we present SAIPy, an open source Python package specifically developed for fast data processing by implementing deep learning. SAIPy offers solutions for multiple seismological tasks, including earthquake detection, magnitude estimation, seismic phase picking, and polarity identification. We introduce upgraded versions of previously published models such as CREIMERT capable of identifying earthquakes with an accuracy above 99.8 percent and a root mean squared error of 0.38 unit in magnitude estimation. These upgraded models outperform state of the art approaches like the Vision Transformer network. SAIPy provides an API that simplifies the integration of these advanced models, including CREIMERT, DynaPickerv2, and PolarCAP, along with benchmark datasets. The package has the potential to be used for real time earthquake monitoring to enable timely actions to mitigate the impact of seismic events. Ongoing development efforts aim to enhance the performance of SAIPy and incorporate additional features that enhance exploration efforts, and it also would be interesting to approach the retraining of the whole package as a multi-task learning problem.

physics.geo-ph↗
arXiv2023

Modular factorization of superconformal indices

Superconformal indices of four-dimensional $\mathcal{N}=1$ gauge theories factorize into holomorphic blocks. We interpret this as a modular property resulting from the combined action of an $SL(3,\mathbb{Z})$ and $SL(2,\mathbb{Z})\ltimes \mathbb{Z}^2$ transformation. The former corresponds to a gluing transformation and the latter to an overall large diffeomorphism, both associated with a Heegaard splitting of the underlying geometry. The extension to more general transformations leads us to argue that a given index can be factorized in terms of a family of holomorphic blocks parametrized by modular (congruence sub)groups. We find precise agreement between this proposal and new modular properties of the elliptic $Γ$ function. This leads to our conjecture for the ``modular factorization'' of superconformal lens indices of general $\mathcal{N}=1$ gauge theories. We provide evidence for the conjecture in the context of the free chiral multiplet and SQED, and sketch the extension of our arguments to more general gauge theories. Based on this result, we systematically prove that a normalized part of superconformal lens indices defines a non-trivial first cohomology class associated with $SL(3,\mathbb{Z})$. Finally, we use this framework to propose a formula for the general lens space index.

hep-th↗
arXiv2023

Partial wave effects in the heavy quarkonium radiative electromagnetic decays

In a previous paper \cite{Bc}, it was pointed out that the wave functions of all particles are not pure waves, besides the main partial waves, they all contain {other partial waves}. It is very interesting to know what role these different partial waves play in particle transitions. Therefore, by using the Bethe-Salpeter equation method, we study the radiative electromagnetic decays $ψ\rightarrowγχ_{_{cJ}}$ and $Υ\rightarrowγχ_{_{bJ}}$ ($J=0,1,2$). We find that for the $S$ and $P$ wave dominated states, like the $ψ(2S)$, $Υ(2S)$, $χ_{_{cJ}}(1P)$, and $χ_{_{bJ}}(1P)$ etc., the dominant $S$ and $P$ waves provide main and nonrelativistic contrition to the decays; other partial waves mainly contribute to the relativistic correction. For the states like the $ψ(1D)$, $Υ(2D)$, $χ_{c2}(1F)$, and $χ_{b2}(1F)$ etc., they are the $S-P-D$ mixing state dominated by $D$ wave or the $P-D-F$ mixing state dominated by $F$ wave. Large decay widths are found in the transitions $ψ(2D)\to χ_{c2}(1F)$, $Υ(1D)\to χ_{bJ}(1P)$, and $Υ(2D)\to χ_{bJ}(2P)$ etc., which may be helpful to study the missing states $χ_{c2}(1F)$, $Υ(1D)$, and $Υ(2D)$.

hep-ph↗
arXiv2023

Improving Modular Bootstrap Bounds with Integrality

We implement methods that efficiently impose integrality -- i.e., the condition that the coefficients of characters in the partition function must be integers -- into numerical modular bootstrap. We demonstrate the method with a number of examples where it can be used to strengthen modular bootstrap results. First, we show that, with a mild extra assumption, imposing integrality improves the bound on the maximal allowed gap in dimensions of operators in theories with a $U(1)^c$ symmetry at $c=3$, and reduces it to the value saturated by the $SU(4)_1$ WZW model point of $c=3$ Narain lattices moduli space. Second, we show that our method can be used to eliminate all but a discrete set of points saturating the bound from previous Virasoro modular bootstrap results. Finally, when central charge is close to $1$, we can slightly improve the upper bound on the scaling dimension gap.

hep-th↗
arXiv2023

DEEPCHORUS: A Hybrid Model of Multi-scale Convolution and Self-attention for Chorus Detection

Chorus detection is a challenging problem in musical signal processing as the chorus often repeats more than once in popular songs, usually with rich instruments and complex rhythm forms. Most of the existing works focus on the receptiveness of chorus sections based on some explicit features such as loudness and occurrence frequency. These pre-assumptions for chorus limit the generalization capacity of these methods, causing misdetection on other repeated sections such as verse. To solve the problem, in this paper we propose an end-to-end chorus detection model DeepChorus, reducing the engineering effort and the need for prior knowledge. The proposed model includes two main structures: i) a Multi-Scale Network to derive preliminary representations of chorus segments, and ii) a Self-Attention Convolution Network to further process the features into probability curves representing chorus presence. To obtain the final results, we apply an adaptive threshold to binarize the original curve. The experimental results show that DeepChorus outperforms existing state-of-the-art methods in most cases.

eess.AS↗
arXiv2023

MaMMUT: A Simple Architecture for Joint Learning for MultiModal Tasks

The development of language models have moved from encoder-decoder to decoder-only designs. In addition, we observe that the two most popular multimodal tasks, the generative and contrastive tasks, are nontrivial to accommodate in one architecture, and further need adaptations for downstream tasks. We propose a novel paradigm of training with a decoder-only model for multimodal tasks, which is surprisingly effective in jointly learning of these disparate vision-language tasks. This is done with a simple model, called MaMMUT. It consists of a single vision encoder and a text decoder, and is able to accommodate contrastive and generative learning by a novel two-pass approach on the text decoder. We demonstrate that joint learning of these diverse objectives is simple, effective, and maximizes the weight-sharing of the model across these tasks. Furthermore, the same architecture enables straightforward extensions to open-vocabulary object detection and video-language tasks. The model tackles a diverse range of tasks, while being modest in capacity. Our model achieves the state of the art on image-text and text-image retrieval, video question answering and open-vocabulary detection tasks, outperforming much larger and more extensively trained foundational models. It shows very competitive results on VQA and Video Captioning, especially considering its capacity. Ablations confirm the flexibility and advantages of our approach.

cs.CV↗
arXiv2023

Differentiable programming tensor networks for Kitaev magnets

We present a general computational framework to investigate ground state properties of quantum spin models on infinite two-dimensional lattices using automatic differentiation-based gradient optimization of infinite projected entangled-pair states. The approach exploits the variational uniform matrix product states to contract infinite tensor networks with unit-cell structure and incorporates automatic differentiation to optimize the local tensors. We applied this framework to the Kitaev-type model, which involves complex interactions and competing ground states. To evaluate the accuracy of this method, we compared the results with exact solutions for the Kitaev model and found that it has a better agreement for various observables compared to previous tensor network calculations based on imaginary-time projection. Additionally, by finding out the ground state with lower variational energy compared to previous studies, we provided convincing evidence for the existence of nematic paramagnetic phases and 18-site configuration in the phase diagram of the $K$-$Γ$ model. Furthermore, in the case of the realistic $K$-$J$-$Γ$-$Γ'$ model for the Kitaev material $α$-RuCl$_3$, we discovered a non-colinear zigzag ground state. Lastly, we also find that the strength of the critical out-of-plane magnetic field that suppresses such a zigzag state has a lower transition field value than the previous finite-cylinder calculations. The framework is versatile and will be useful for a quick scan of phase diagrams for a broad class of quantum spin models.

physics.comp-ph↗
arXiv2023

Mlinear: Rethink the Linear Model for Time-series Forecasting

Recently, significant advancements have been made in time-series forecasting research, with an increasing focus on analyzing the nature of time-series data, e.g, channel-independence (CI) and channel-dependence (CD), rather than solely focusing on designing sophisticated forecasting models. However, current research has primarily focused on either CI or CD in isolation, and the challenge of effectively combining these two opposing properties to achieve a synergistic effect remains an unresolved issue. In this paper, we carefully examine the opposing properties of CI and CD, and raise a practical question that has not been effectively answered, e.g.,"How to effectively mix the CI and CD properties of time series to achieve better predictive performance?" To answer this question, we propose Mlinear (MIX-Linear), a simple yet effective method based mainly on linear layers. The design philosophy of Mlinear mainly includes two aspects:(1) dynamically tuning the CI and CD properties based on the time semantics of different input time series, and (2) providing deep supervision to adjust the individual performance of the "CI predictor" and "CD predictor". In addition, empirically, we introduce a new loss function that significantly outperforms the widely used mean squared error (MSE) on multiple datasets. Experiments on time-series datasets covering multiple fields and widely used have demonstrated the superiority of our method over PatchTST which is the lateset Transformer-based method in terms of the MSE and MAE metrics on 7 datasets with identical sequence inputs (336 or 512). Specifically, our method significantly outperforms PatchTST with a ratio of 21:3 at 336 sequence length input and 29:10 at 512 sequence length input. Additionally, our approach has a 10 $\times$ efficiency advantage at the unit level, taking into account both training and inference times.

cs.LG↗