Searcharxiv⌕ Search

arXiv subjects

Yun Wang

Publications and source records attributed to Yun Wang.

At least 163 records · Page 9Linked to original sources
arXiv2024

Tree-Regularized Tabular Embeddings

Tabular neural network (NN) has attracted remarkable attentions and its recent advances have gradually narrowed the performance gap with respect to tree-based models on many public datasets. While the mainstreams focus on calibrating NN to fit tabular data, we emphasize the importance of homogeneous embeddings and alternately concentrate on regularizing tabular inputs through supervised pretraining. Specifically, we extend a recent work (DeepTLF) and utilize the structure of pretrained tree ensembles to transform raw variables into a single vector (T2V), or an array of tokens (T2T). Without loss of space efficiency, these binarized embeddings can be consumed by canonical tabular NN with fully-connected or attention-based building blocks. Through quantitative experiments on 88 OpenML datasets with binary classification task, we validated that the proposed tree-regularized representation not only tapers the difference with respect to tree-based models, but also achieves on-par and better performance when compared with advanced NN models. Most importantly, it possesses better robustness and can be easily scaled and generalized as standalone encoder for tabular modality. Codes: https://github.com/milanlx/tree-regularized-embedding.

cs.LG↗
arXiv2024

A Literature Review on Fetus Brain Motion Correction in MRI

This paper provides a comprehensive review of the latest advancements in fetal motion correction in MRI. We delve into various contemporary methodologies and technological advancements aimed at overcoming these challenges. It includes traditional 3D fetal MRI correction methods like Slice to Volume Registration (SVR), deep learning-based techniques such as Convolutional Neural Networks (CNNs), Long Short-Term Memory (LSTM) Networks, Transformers, Generative Adversarial Networks (GANs) and most recent advancements of Diffusion Models. The insights derived from this literature review reflect a thorough understanding of both the technical intricacies and practical implications of fetal motion in MRI studies, offering a reasoned perspective on potential solutions and future improvements in this field.

eess.IV↗
arXiv2024

The Activity of Black Hole Imprints on the Internal Plateau and the Subsequent Sharp Decay

A stellar-mass black hole (BH) or a millisecond magnetar is believed to be born as the central engine of Gamma-ray bursts (GRBs). The presence of plateaus in the X-ray extended emission or afterglow of GRBs is widely accepted as an indicator of magnetar central engine, particularly those with a sharp decay (faster than $t^{-3}$), so-called internal plateau. However, an alternative model, by taking the evolution of the magnetic flux at the BH horizon into account, suggests that an internal plateau can also arise from a Blandford-Znajek (BZ) mechanism powered jet (hereafter referred to as the BZ jet). In this study, we propose that a precessional BZ jet would manifest a Quasi-Periodic Oscillation (QPO) signature on the internal plateau and the subsequent sharp decay. Such lightcurves cannot be readily explained by the activity of a short-lived, supermassive magnetar, thus favoring a Kerr BH as the central engine. The X-ray afterglow of GRB 050904, comprising nine flares, is characterized by a QPO-modulated plateau and sharp decay, which can be well reproduced by a precessional BZ jet model. Therefore, one potential clue for distinguishing between these two engines lies in whether QPO signature throughout the entire plateau and the subsequent sharp decay, as the magnetar scenario suggests a collapse at the end of the plateau.

astro-ph.HE↗
arXiv2024

How do baryonic effects on the cosmic matter distribution vary with scale and local density environment?

In this study, we investigate how the baryonic effects vary with scale and local density environment mainly by utilizing a novel statistic, the environment-dependent wavelet power spectrum (env-WPS). With four state-of-the-art cosmological simulation suites, EAGLE, SIMBA, Illustris, and IllustrisTNG, we compare the env-WPS of the total matter density field between the hydrodynamic and dark matter-only (DMO) runs at $z=0$. We find that the clustering is most strongly suppressed in the emptiest environment of $ρ_\mathrm{m}/\barρ_\mathrm{m}<0.1$ with maximum amplitudes $\sim67-89$ per cent on scales $\sim1.86-10.96\ h\mathrm{Mpc}^{-1}$, and less suppressed in higher density environments on small scales (except Illustris). In the environments of $ρ_\mathrm{m}/\barρ_\mathrm{m}\geqslant0.316$ ($\geqslant10$ in EAGLE), the feedbacks also lead to enhancement features at intermediate and large scales, which is most pronounced in the densest environment of $ρ_\mathrm{m}/\barρ_\mathrm{m}\geqslant100$ and reaches a maximum $\sim 7-15$ per cent on scales $\sim0.87-2.62\ h\mathrm{Mpc}^{-1}$ (except Illustris). The baryon fraction of the local environment decreases with increasing density, denoting the feedback strength, and potentially explaining some differences between simulations. We also measure the volume and mass fractions of local environments, which are affected by $\gtrsim 1$ per cent due to baryon physics. In conclusion, our results show that the baryonic processes can strongly modify the overall cosmic structure on the scales of $k>0.1\ h\mathrm{Mpc}^{-1}$, which encourages further research in this direction.

astro-ph.CO↗
arXiv2024

Multi-Task Learning for Front-End Text Processing in TTS

We propose a multi-task learning (MTL) model for jointly performing three tasks that are commonly solved in a text-to-speech (TTS) front-end: text normalization (TN), part-of-speech (POS) tagging, and homograph disambiguation (HD). Our framework utilizes a tree-like structure with a trunk that learns shared representations, followed by separate task-specific heads. We further incorporate a pre-trained language model to utilize its built-in lexical and contextual knowledge, and study how to best use its embeddings so as to most effectively benefit our multi-task model. Through task-wise ablations, we show that our full model trained on all three tasks achieves the strongest overall performance compared to models trained on individual or sub-combinations of tasks, confirming the advantages of our MTL framework. Finally, we introduce a new HD dataset containing a balanced number of sentences in diverse contexts for a variety of homographs and their pronunciations. We demonstrate that incorporating this dataset into training significantly improves HD performance over only using a commonly used, but imbalanced, pre-existing dataset.

cs.CL↗
arXiv2023

PanGu-$π$: Enhancing Language Model Architectures via Nonlinearity Compensation

The recent trend of large language models (LLMs) is to increase the scale of both model size (\aka the number of parameters) and dataset to achieve better generative ability, which is definitely proved by a lot of work such as the famous GPT and Llama. However, large models often involve massive computational costs, and practical applications cannot afford such high prices. However, the method of constructing a strong model architecture for LLMs is rarely discussed. We first analyze the state-of-the-art language model architectures and observe the feature collapse problem. Based on the theoretical analysis, we propose that the nonlinearity is also very important for language models, which is usually studied in convolutional neural networks for vision tasks. The series informed activation function is then introduced with tiny calculations that can be ignored, and an augmented shortcut is further used to enhance the model nonlinearity. We then demonstrate that the proposed approach is significantly effective for enhancing the model nonlinearity through carefully designed ablations; thus, we present a new efficient model architecture for establishing modern, namely, PanGu-$π$. Experiments are then conducted using the same dataset and training strategy to compare PanGu-$π$ with state-of-the-art LLMs. The results show that PanGu-$π$-7B can achieve a comparable performance to that of benchmarks with about 10\% inference speed-up, and PanGu-$π$-1B can achieve state-of-the-art performance in terms of accuracy and efficiency. In addition, we have deployed PanGu-$π$-7B in the high-value domains of finance and law, developing an LLM named YunShan for practical application. The results show that YunShan can surpass other models with similar scales on benchmarks.

cs.CL↗
arXiv2023

Liouville-type theorems for steady Navier-Stokes system under helical symmetry or Navier boundary conditions

In this paper, the Liouville-type theorems for the steady Navier-Stokes system are investigated. First, we prove that any bounded smooth helically symmetric solution in $\mathbb{R}^3$ must be a constant vector. Second, for steady Navier-Stokes system in a slab supplemented with Navier boundary conditions, we prove that any bounded smooth solution must be zero if either the swirl or radial velocity is axisymmetric, or $ru^{r}$ decays to zero as $r$ tends to infinity. Finally, when the velocity is not big in $L^{\infty}$-space, the general three-dimensional steady Navier-Stokes flow in a slab with the Navier boundary conditions must be a Poiseuille type flow. The key idea of the proof is to establish Saint-Venant type estimates that characterize the growth of Dirichlet integral of nontrivial solutions.

math.AP↗
arXiv2023

User-guided Page Merging for Memory Deduplication in Serverless Systems

Serverless computing is an emerging cloud paradigm that offers an elastic and scalable allocation of computing resources with pay-as-you-go billing. In the Function-as-a-Service (FaaS) programming model, applications comprise short-lived and stateless serverless functions executed in isolated containers or microVMs, which can quickly scale to thousands of instances and process terabytes of data. This flexibility comes at the cost of duplicated runtimes, libraries, and user data spread across many function instances, and cloud providers do not utilize this redundancy. The memory footprint of serverless forces removing idle containers to make space for new ones, which decreases performance through more cold starts and fewer data caching opportunities. We address this issue by proposing deduplicating memory pages of serverless workers with identical content, based on the content-based page-sharing concept of Linux Kernel Same-page Merging (KSM). We replace the background memory scanning process of KSM, as it is too slow to locate sharing candidates in short-lived functions. Instead, we design User-Guided Page Merging (UPM), a built-in Linux kernel module that leverages the madvise system call: we enable users to advise the kernel of memory areas that can be shared with others. We show that UPM reduces memory consumption by up to 55% on 16 concurrent containers executing a typical image recognition function, more than doubling the density for containers of the same function that can run on a system.

cs.DC↗
arXiv2023

Accurate and Efficient Stereo Matching via Attention Concatenation Volume

Stereo matching is a fundamental building block for many vision and robotics applications. An informative and concise cost volume representation is vital for stereo matching of high accuracy and efficiency. In this paper, we present a novel cost volume construction method, named attention concatenation volume (ACV), which generates attention weights from correlation clues to suppress redundant information and enhance matching-related information in the concatenation volume. The ACV can be seamlessly embedded into most stereo matching networks, the resulting networks can use a more lightweight aggregation network and meanwhile achieve higher accuracy. We further design a fast version of ACV to enable real-time performance, named Fast-ACV, which generates high likelihood disparity hypotheses and the corresponding attention weights from low-resolution correlation clues to significantly reduce computational and memory cost and meanwhile maintain a satisfactory accuracy. The core idea of our Fast-ACV is volume attention propagation (VAP) which can automatically select accurate correlation values from an upsampled correlation volume and propagate these accurate values to the surroundings pixels with ambiguous correlation clues. Furthermore, we design a highly accurate network ACVNet and a real-time network Fast-ACVNet based on our ACV and Fast-ACV respectively, which achieve the state-of-the-art performance on several benchmarks (i.e., our ACVNet ranks the 2nd on KITTI 2015 and Scene Flow, and the 3rd on KITTI 2012 and ETH3D among all the published methods; our Fast-ACVNet outperforms almost all state-of-the-art real-time methods on Scene Flow, KITTI 2012 and 2015 and meanwhile has better generalization ability)

cs.CV↗
arXiv2023

Human Still Wins over LLM: An Empirical Study of Active Learning on Domain-Specific Annotation Tasks

Large Language Models (LLMs) have demonstrated considerable advances, and several claims have been made about their exceeding human performance. However, in real-world tasks, domain knowledge is often required. Low-resource learning methods like Active Learning (AL) have been proposed to tackle the cost of domain expert annotation, raising this question: Can LLMs surpass compact models trained with expert annotations in domain-specific tasks? In this work, we conduct an empirical experiment on four datasets from three different domains comparing SOTA LLMs with small models trained on expert annotations with AL. We found that small models can outperform GPT-3.5 with a few hundreds of labeled data, and they achieve higher or similar performance with GPT-4 despite that they are hundreds time smaller. Based on these findings, we posit that LLM predictions can be used as a warmup method in real-world applications and human experts remain indispensable in tasks involving data annotation driven by domain-specific knowledge.

cs.CL↗
arXiv2023

Depth and Breadth of Research Area Coverage and Its Impact on Publication Citation: An Analysis of Bibliometric Papers

Many other factors affecting citation of publications, except for research area coverage, have been studied. This study aims to investigate impact of research area coverage. Bibliometric papers and their related papers (referred papers, citing papers and first author's papers) were screened and matched by Python program. Papers' research areas were classified according to Web of Science. Bibliometric parameters of the most cited 5% and the least cited 5% papers were compared. Firstly, coverage of related papers' research areas impacts the citation of their original papers. The impact of references and citing papers are positive and negative, separately, while the first author's papers have no influence. Secondly, high-influence papers tend to cite references from a wider area and are cited by followers from a wider area. Additionally, the pattern of knowledge flow differs significantly between high- and low-influence papers. Low-influence papers narrow knowledge flow, whereas high-influence papers broaden it. This study has shown that both depth and breadth of research area coverage can influence citations. It is recommended that authors should extensively cite high-influence publications, both within and beyond their own area.

cs.DL↗
arXiv2023

fMRI-PTE: A Large-scale fMRI Pretrained Transformer Encoder for Multi-Subject Brain Activity Decoding

The exploration of brain activity and its decoding from fMRI data has been a longstanding pursuit, driven by its potential applications in brain-computer interfaces, medical diagnostics, and virtual reality. Previous approaches have primarily focused on individual subject analysis, highlighting the need for a more universal and adaptable framework, which is the core motivation behind our work. In this work, we propose fMRI-PTE, an innovative auto-encoder approach for fMRI pre-training, with a focus on addressing the challenges of varying fMRI data dimensions due to individual brain differences. Our approach involves transforming fMRI signals into unified 2D representations, ensuring consistency in dimensions and preserving distinct brain activity patterns. We introduce a novel learning strategy tailored for pre-training 2D fMRI images, enhancing the quality of reconstruction. fMRI-PTE's adaptability with image generators enables the generation of well-represented fMRI features, facilitating various downstream tasks, including within-subject and cross-subject brain activity decoding. Our contributions encompass introducing fMRI-PTE, innovative data transformation, efficient training, a novel learning strategy, and the universal applicability of our approach. Extensive experiments validate and support our claims, offering a promising foundation for further research in this domain.

cs.CV↗
arXiv2023

A two-component jet model for the optical plateau in the afterglow of GRB 191221B

The long gamma-ray burst GRB 191221B has abundant observations in X-ray, optical and radio bands. In the literature, the observed optical light curve of GRB 191221B displays a plateau around 0.1-day, which is rather peculiar in gamma-ray bursts. Here we performed detailed analysis of the observational data from Swift/UVOT, VLT and LCO, obtained the light curve of the multi-band afterglow of GRB 191221B. By examining optical, ultraviolet, X-ray, and radio data for this event, we demonstrate that an on-axis two-component jet model can explain the observations. Our analysis suggests that the narrow component has an initial Lorentz factor of 400 and a jet opening half-angle of $1.4^{\circ}$, while the wide component has an initial Lorentz factor of 25 and a jet opening half-angle of $2.8^{\circ}$. The narrow jet dominates the early decay, whereas the wider jet causes the optical plateau and dominates late decay. According to this model, the reason for the absence of the X-ray plateau is due to the steeper spectral index of the wide component, resulting in a less significant flux contribution from the wide jet in the X-ray bands than in the optical bands. Moreover, we have explained the inconsistency in the decay indices of the UVOT and Rc-band data around 2000 seconds using reverse shock emission.

astro-ph.HE↗
arXiv2023

LLM4Vis: Explainable Visualization Recommendation using ChatGPT

Data visualization is a powerful tool for exploring and communicating insights in various domains. To automate visualization choice for datasets, a task known as visualization recommendation has been proposed. Various machine-learning-based approaches have been developed for this purpose, but they often require a large corpus of dataset-visualization pairs for training and lack natural explanations for their results. To address this research gap, we propose LLM4Vis, a novel ChatGPT-based prompting approach to perform visualization recommendation and return human-like explanations using very few demonstration examples. Our approach involves feature description, demonstration example selection, explanation generation, demonstration example construction, and inference steps. To obtain demonstration examples with high-quality explanations, we propose a new explanation generation bootstrapping to iteratively refine generated explanations by considering the previous generation and template-based hint. Evaluations on the VizML dataset show that LLM4Vis outperforms or performs similarly to supervised learning models like Random Forest, Decision Tree, and MLP in both few-shot and zero-shot settings. The qualitative evaluation also shows the effectiveness of explanations generated by LLM4Vis. We make our code publicly available at \href{https://github.com/demoleiwang/LLM4Vis}{https://github.com/demoleiwang/LLM4Vis}.

cs.HC↗
arXiv2023

Comparisons between fast algorithms for the continuous wavelet transform and applications in cosmology: the 1D case

The continuous wavelet transform (CWT) is very useful for processing signals with intricate and irregular structures in astrophysics and cosmology. It is crucial to propose precise and fast algorithms for the CWT. In this work, we review and compare four different fast CWT algorithms for the 1D signals, including the FFTCWT, the V97CWT, the M02CWT, and the A19CWT. The FFTCWT algorithm implements the CWT using the Fast Fourier Transform (FFT) with a computational complexity of $\mathcal{O}(N\log_2N)$ per scale. The rest algorithms achieve the complexity of $\mathcal{O}(N)$ per scale by simplifying the CWT into some smaller convolutions. We illustrate explicitly how to set the parameters as well as the boundary conditions for them. To examine the actual performance of these algorithms, we use them to perform the CWT of signals with different wavelets. From the aspect of accuracy, we find that the FFTCWT is the most accurate algorithm, though its accuracy degrades a lot when processing the non-periodic signal with zero boundaries. The accuracy of $\mathcal{O}(N)$ algorithms is robust to signals with different boundaries, and the M02CWT is more accurate than the V97CWT and A19CWT. From the aspect of speed, the $\mathcal{O}(N)$ algorithms do not show an overall speed superiority over the FFTCWT at sampling numbers of $N\lesssim10^6$, which is due to their large leading constants. Only the speed of the V97CWT with real wavelets is comparable to that of the FFTCWT. However, both the FFTCWT and V97CWT are substantially less efficient in processing the non-periodic signal because of zero padding. Finally, we conduct wavelet analysis of the 1D density fields, which demonstrate the convenience and power of techniques based on the CWT. We publicly release our CWT codes as resources for the community.

astro-ph.IM↗
arXiv2023

Towards Automated Infographic Design: Deep Learning-based Auto-Extraction of Extensible Timeline

Designers need to consider not only perceptual effectiveness but also visual styles when creating an infographic. This process can be difficult and time consuming for professional designers, not to mention non-expert users, leading to the demand for automated infographics design. As a first step, we focus on timeline infographics, which have been widely used for centuries. We contribute an end-to-end approach that automatically extracts an extensible timeline template from a bitmap image. Our approach adopts a deconstruction and reconstruction paradigm. At the deconstruction stage, we propose a multi-task deep neural network that simultaneously parses two kinds of information from a bitmap timeline: 1) the global information, i.e., the representation, scale, layout, and orientation of the timeline, and 2) the local information, i.e., the location, category, and pixels of each visual element on the timeline. At the reconstruction stage, we propose a pipeline with three techniques, i.e., Non-Maximum Merging, Redundancy Recover, and DL GrabCut, to extract an extensible template from the infographic, by utilizing the deconstruction results. To evaluate the effectiveness of our approach, we synthesize a timeline dataset (4296 images) and collect a real-world timeline dataset (393 images) from the Internet. We first report quantitative evaluation results of our approach over the two datasets. Then, we present examples of automatically extracted templates and timelines automatically generated based on these templates to qualitatively demonstrate the performance. The results confirm that our approach can effectively extract extensible templates from real-world timeline infographics.

cs.HC↗
arXiv2023

Revisiting the Electrified Pt(111)/Water Interfaces through an Affordable Double-Reference ab-initio Approach

The electrified solid-liquid interface plays an essential role in many renewable energy-related applications, including hydrogen production and utilization. Limitations in computational modelling of the electrified solid-liquid interface have held back the understanding of its properties at the atomic-scale level. In this study, we applied the grand canonical density functional theory (GC-DFT) combined with a hybrid implicit/explicit solvation model to reinvestigate the widely studied electrified platinum-water interface affordably. The calculated double layer capacitances of the Pt(111)-water interface over the applied bias potential closely match the experimental and previous theoretical data from expensive ab-initio molecular dynamics simulations. The structural analysis of the interface models reveals that the applied bias potential can significantly affect the Pt(111)-water atomic interface configurations. The orientation of the water molecules next to the Pt(111) surface is vital for correctly describing the potential of zero charge (PZC) and capacitance. Additionally, the GC-DFT results confirm that the absorption of the hydrogen atom under applied bias potential can significantly affect the electrified interfacial properties. The presented affordable GC-DFT approach, therefore, offers an efficient and accurate means to enhance the understanding of electrified solid-liquid interfaces.

cond-mat.mtrl-sci↗
arXiv2023

Strong arc decompositions of split digraphs

A {\bf strong arc decomposition} of a digraph $D=(V,A)$ is a partition of its arc set $A$ into two sets $A_1,A_2$ such that the digraph $D_i=(V,A_i)$ is strong for $i=1,2$. Bang-Jensen and Yeo (2004) conjectured that there is some $K$ such that every $K$-arc-strong digraph has a strong arc decomposition. They also proved that with one exception on 4 vertices every 2-arc-strong semicomplete digraph has a strong arc decomposition. Bang-Jensen and Huang (2010) extended this result to locally semicomplete digraphs by proving that every 2-arc-strong locally semicomplete digraph which is not the square of an even cycle has a strong arc decomposition. This implies that every 3-arc-strong locally semicomplete digraph has a strong arc decomposition. A {\bf split digraph} is a digraph whose underlying undirected graph is a split graph, meaning that its vertices can be partioned into a clique and an independent set. Equivalently, a split digraph is any digraph which can be obtained from a semicomplete digraph $D=(V,A)$ by adding a new set $V'$ of vertices and some arcs between $V'$ and $V$. In this paper we prove that every 3-arc-strong split digraph has a strong arc decomposition which can be found in polynomial time and we provide infinite classes of 2-strong split digraphs with no strong arc decomposition. We also pose a number of open problems on split digraphs.

math.CO↗