Searcharxiv⌕ Search

arXiv subjects

Yan Gong

Publications and source records attributed to Yan Gong.

At least 109 records · Page 6Linked to original sources
arXiv2023

Boon: A Neural Search Engine for Cross-Modal Information Retrieval

Visual-Semantic Embedding (VSE) networks can help search engines better understand the meaning behind visual content and associate it with relevant textual information, leading to more accurate search results. VSE networks can be used in cross-modal search engines to embed image and textual descriptions in a shared space, enabling image-to-text and text-to-image retrieval tasks. However, the full potential of VSE networks for search engines has yet to be fully explored. This paper presents Boon, a novel cross-modal search engine that combines two state-of-the-art networks: the GPT-3.5-turbo large language model, and the VSE network VITR (VIsion Transformers with Relation-focused learning) to enhance the engine's capabilities in extracting and reasoning with regional relationships in images. VITR employs encoders from CLIP that were trained with 400 million image-description pairs and it was fine-turned on the RefCOCOg dataset. Boon's neural-based components serve as its main functionalities: 1) a 'cross-modal search engine' that enables end-users to perform image-to-text and text-to-image retrieval. 2) a 'multi-lingual conversational AI' component that enables the end-user to converse about one or more images selected by the end-user. Such a feature makes the search engine accessible to a wide audience, including those with visual impairments. 3) Boon is multi-lingual and can take queries and handle conversations about images in multiple languages. Boon was implemented using the Django and PyTorch frameworks. The interface and capabilities of the Boon search engine are demonstrated using the RefCOCOg dataset, and the engine's ability to search for multimedia through the web is facilitated by Google's API.

cs.MM↗
arXiv2023

Implications of the Stellar Mass Density of High-$z$ Massive Galaxies from JWST on Warm Dark Matter

A significant excess of the stellar mass density at high redshift has been discovered from the early data release of James Webb Space Telescope ($\it{JWST}$), and it may require a high star formation efficiency. However, this will lead to large number density of ionizing photons in the epoch of reionization (EoR), so that the reionization history will be changed, which can arise tension with the current EoR observations. Warm dark matter (WDM), via the free streaming effect, can suppress the formation of small-scale structure as well as low-mass galaxies. This provides an effective way to decrease the ionizing photons when considering a large star formation efficiency in high-$z$ massive galaxies without altering the cosmic reionization history. On the other hand, the constraints on the properties of warm dark matter can be derived from the $\it JWST$ observations. In this work, we study WDM as a possible solution to reconcile the $\it JWST$ stellar mass density of high-$z$ massive galaxies and reionization history. We find that, the $\it JWST$ high-$z$ comoving cumulative stellar mass density alone has no significant preference for either CDM or WDM model. But using the observational data of other stellar mass density measurements and reionization history, we obtain that the WDM particle mass with $m_{\text{W}} = 0.51^{+0.22}_{-0.12}$ keV and star formation efficiency parameter $f_{*}^0>0.39$ in 2$σ$ confidence level can match both the $\it JWST$ high-$z$ comoving cumulative stellar mass density and the reionization history.

astro-ph.CO↗
arXiv2023

Maser Investigation toward Off-Plane Stars (MIOPS): detection of SiO masers in the Galactic thick disk and halo

Studying stars that are located off the Galactic plane is important for understanding the formation history of the Milky Way. We searched for SiO masers toward off-plane O-rich asymptotic giant branch (AGB) stars from the catalog presented by Mauron et al. (2019) in order to shed light on the origin of these objects. A total of 102 stars were observed in the SiO $J$=1-0, $v=1$ and 2 transitions with the Effelsberg-100 m and Tianma-65 m telescopes. SiO masers were discovered in eight stars, all first detections. The measured maser velocities allow the first estimates of the host AGB stars' radial velocities. We find that the radial velocities of three stars (namely G068.881-24.615, G070.384-24.886, and G084.453-21.863) significantly deviate from the values expected from Galactic circular motion. The updated distances and 3D motions indicate that G068.881$-$24.615 is likely located in the Galactic halo, while G160.648-08.846 is probably located in the Galactic thin disk, and the other six stars are probably part of the Galactic thick disk.

astro-ph.GA↗
arXiv2023

Mapping the Milky Way's stellar halo with 2D data

We propose a new method for measuring the spatial density distribution of the stellar halo of the Milky Way. Our method is based on a pairwise statistic of the distribution of stars on the sky, the angular two-point correlation function (ATPCF). The ATPCF utilizes two dimensional data of stars only and is therefore immune to the large uncertainties in the determination of distances to stars. We test our method using mock stellar data coming from various models including the single power-law (SPL) and the broken power-law (BPL) density profiles. We also test the influence of axisymmetric flattening factors using both constant and varying values. We find that the ATPCF is a powerful tool for recovering the spatial distributions of the stellar halos in our models. We apply our method to observational data from the type ab RR Lyrae catalog in the Catalina Survey Data Release 1. In the 3-parameter BPL model, we find that $s_{1}=2.46_{-0.20}^{+0.18}, s_{2}=3.99_{-1.33}^{+0.75}$ and $r_{0}=31.11_{-5.88}^{+7.61}$, which are in good agreement with previous results. We also find that introducing an extra parameter, the radially varying flattening factor, greatly improves our ability to model accurately the observed data distribution. This implies perhaps that the stellar halo of the Milky Way should be regarded as oblate.

astro-ph.GA↗
arXiv2023

VITR: Augmenting Vision Transformers with Relation-Focused Learning for Cross-Modal Information Retrieval

The relations expressed in user queries are vital for cross-modal information retrieval. Relation-focused cross-modal retrieval aims to retrieve information that corresponds to these relations, enabling effective retrieval across different modalities. Pre-trained networks, such as Contrastive Language-Image Pre-training (CLIP), have gained significant attention and acclaim for their exceptional performance in various cross-modal learning tasks. However, the Vision Transformer (ViT) used in these networks is limited in its ability to focus on image region relations. Specifically, ViT is trained to match images with relevant descriptions at the global level, without considering the alignment between image regions and descriptions. This paper introduces VITR, a novel network that enhances ViT by extracting and reasoning about image region relations based on a local encoder. VITR is comprised of two key components. Firstly, it extends the capabilities of ViT-based cross-modal networks by enabling them to extract and reason with region relations present in images. Secondly, VITR incorporates a fusion module that combines the reasoned results with global knowledge to predict similarity scores between images and descriptions. The proposed VITR network was evaluated through experiments on the tasks of relation-focused cross-modal information retrieval. The results derived from the analysis of the RefCOCOg, CLEVR, and Flickr30K datasets demonstrated that the proposed VITR network consistently outperforms state-of-the-art networks in image-to-text and text-to-image retrieval.

cs.CV↗
arXiv2023

Neural-based Cross-modal Search and Retrieval of Artwork

Creating an intelligent search and retrieval system for artwork images, particularly paintings, is crucial for documenting cultural heritage, fostering wider public engagement, and advancing artistic analysis and interpretation. Visual-Semantic Embedding (VSE) networks are deep learning models used for information retrieval, which learn joint representations of textual and visual data, enabling 1) cross-modal search and retrieval tasks, such as image-to-text and text-to-image retrieval; and 2) relation-focused retrieval to capture entity relationships and provide more contextually relevant search results. Although VSE networks have played a significant role in cross-modal information retrieval, their application to painting datasets, such as ArtUK, remains unexplored. This paper introduces BoonArt, a VSE-based cross-modal search engine that allows users to search for images using textual queries, and to obtain textual descriptions along with the corresponding images when using image queries. The performance of BoonArt was evaluated using the ArtUK dataset. Experimental evaluations revealed that BoonArt achieved 97% Recall@10 for image-to-text retrieval, and 97.4% Recall@10 for text-to-image Retrieval. By bridging the gap between textual and visual modalities, BoonArt provides a much-improved search performance compared to traditional search engines, such as the one provided by the ArtUK website. BoonArt can be utilised to work with other artwork datasets.

cs.MM↗
arXiv2023

Cross-Correlation Forecast of CSST Spectroscopic Galaxy and MeerKAT Neutral Hydrogen Intensity Mapping Surveys

Cross-correlating the data of neutral hydrogen (HI) 21cm intensity mapping with galaxy surveys is an effective method to extract astrophysical and cosmological information. In this work, we investigate the cross-correlation of MeerKAT single-dish mode HI intensity mapping and China Space Station Telescope (CSST) spectroscopic galaxy surveys. We simulate a survey area of $\sim 300$ $\mathrm{deg}^2$ of MeerKAT and CSST surveys at $z=0.5$ using Multi-Dark N-body simulation. The PCA algorithm is applied to remove the foregrounds of HI intensity mapping, and signal compensation is considered to solve the signal loss problem in the HI-galaxy cross power spectrum caused by the foreground removal process. We find that from CSST galaxy auto and MeerKAT-CSST cross power spectra, the constraint accuracy of the parameter product $Ω_{\rm HI}b_{\rm HI}r_{{\rm HI},g}$ can reach to $\sim1\%$, which is about one order of magnitude higher than the current results. After performing the full MeerKAT HI intensity mapping survey with 5000 deg$^2$ survey area, the accuracy can be enhanced to $<0.3\%$. This implies that the MeerKAT-CSST cross-correlation can be a powerful tool to probe the cosmic HI property and the evolution of galaxies and the Universe.

astro-ph.CO↗
arXiv2023

The distance to the Serpens South Cluster from H2O masers

In this Letter, we report Very Long Baseline Array observations of 22 GHz water masers toward the protostar CARMA-6, located at the center of the Serpens South young cluster. From the astrometric fits to maser spots, we derive a distance of 440.7+/-3.5 pc for the protostar (1% error). This represents the best direct distance determination obtained so far for an object this young and deeply embedded in this highly obscured region. Taking into account depth effects, we obtain a distance to the cluster of 440.7+/-4.6 pc. Stars visible in the optical that have astrometric solutions in the Gaia Data Release 3 are, on the other hand, all located in the periphery of the cluster. Their mean distance of 437 (+51, -41) pc is consistent within 1-sigma with the value derived from maser astrometry. As the maser source is just at the center of Serpens South, we finally solve the ambiguity of the distance to this region that has prevailed over the years.

astro-ph.GA↗
arXiv2023

Towards Better Instruction Following Language Models for Chinese: Investigating the Impact of Training Data and Evaluation

Recently, significant public efforts have been directed towards developing low-cost models with capabilities akin to ChatGPT, thereby fostering the growth of open-source conversational models. However, there remains a scarcity of comprehensive and in-depth evaluations of these models' performance. In this study, we examine the influence of training data factors, including quantity, quality, and linguistic distribution, on model performance. Our analysis is grounded in several publicly accessible, high-quality instruction datasets, as well as our own Chinese multi-turn conversations. We assess various models using a evaluation set of 1,000 samples, encompassing nine real-world scenarios. Our goal is to supplement manual evaluations with quantitative analyses, offering valuable insights for the continued advancement of open-source chat models. Furthermore, to enhance the performance and training and inference efficiency of models in the Chinese domain, we extend the vocabulary of LLaMA - the model with the closest open-source performance to proprietary language models like GPT-3 - and conduct secondary pre-training on 3.4B Chinese words. We make our model, data, as well as code publicly available.

cs.CL↗
arXiv2023

Exploring the Impact of Instruction Data Scaling on Large Language Models: An Empirical Study on Real-World Use Cases

The success of ChatGPT has recently attracted numerous efforts to replicate it, with instruction-tuning strategies being a key factor in achieving remarkable results. Instruction-tuning not only significantly enhances the model's performance and generalization but also makes the model's generated results more consistent with human speech patterns. However current research rarely studies the impact of different amounts of instruction data on model performance, especially in the real-world use cases. In this paper we explore the performance of large language models based on instruction tuning across different scales of instruction data. An evaluation dataset consisting of 12 major online use cases is constructed in the experiment. With Bloomz-7B1-mt as the base model, the results show that 1) merely increasing the amount of instruction data leads to continuous improvement in tasks such as open-ended generation, 2) in tasks such as math and code, the model performance curve remains quite flat while increasing data size. We further analyze the possible causes of these phenomena and propose potential future research directions such as effectively selecting high-quality training data, scaling base models and training methods specialized for hard tasks. We will release our training and evaluation datasets, as well as model checkpoints.

cs.CL↗
arXiv2023

Exploring ChatGPT's Ability to Rank Content: A Preliminary Study on Consistency with Human Preferences

As a natural language assistant, ChatGPT is capable of performing various tasks, including but not limited to article generation, code completion, and data analysis. Furthermore, ChatGPT has consistently demonstrated a remarkable level of accuracy and reliability in terms of content evaluation, exhibiting the capability of mimicking human preferences. To further explore ChatGPT's potential in this regard, a study is conducted to assess its ability to rank content. In order to do so, a test set consisting of prompts is created, covering a wide range of use cases, and five models are utilized to generate corresponding responses. ChatGPT is then instructed to rank the responses generated by these models. The results on the test set show that ChatGPT's ranking preferences are consistent with human to a certain extent. This preliminary experimental finding implies that ChatGPT's zero-shot ranking capability could be used to reduce annotation pressure in a number of ranking tasks.

cs.CL↗
arXiv2023

Fuzzy Dark Matter as a Solution to Reconcile the Stellar Mass Density of High-z Massive Galaxies and Reionization History

The JWST early release data show unexpected high stellar mass densities of massive galaxies at $7<z<11$, a high star formation efficiency is probably needed to explain this. However, such a high star formation efficiency would greatly increase the number of ionizing photons, which would be in serious conflict with the current cosmic microwave background (CMB) and other measurements of cosmic reionization history. To solve this problem, we explore the fuzzy dark matter (FDM), which is composed of ultra-light scalar particles, e.g. ultra-light axions, and calculate its halo mass function and stellar mass density for different axion masses. We find that the FDM model with $m_a\simeq 5\times10^{-23} \rm eV$ and a possible uncertainty range $\sim3\times10^{-23}-10^{-22}\, \rm eV$ can effectively suppress the formation of small halos and galaxies, so that with higher star formation efficiency, both the JWST data at $z\sim8$ and the reionization history measurements from the optical depth of CMB scattering and ionization fraction can be simultaneously matched. We also find that the JWST data at $z\sim10$ are still too high to fit in this scenario. We note that the estimated mean redshift of the sample may have large uncertainty, that it can be as low as $z\sim9$ depending on adopted spectral energy distribution (SED) templates and photometric-redshift code. Besides, the warm dark matter with $\sim$keV mass can also be an alternative choice, since it should have similar effects on halo formation as the FDM.

astro-ph.CO↗
arXiv2023

Foreground Removal of CO Intensity Mapping Using Deep Learning

Line intensity mapping (LIM) is a promising probe to study star formation, the large-scale structure of the Universe, and the epoch of reionization (EoR). Since carbon monoxide (CO) is the second most abundant molecule in the Universe except for molecular hydrogen ${\rm H}_2$, it is suitable as a tracer for LIM surveys. However, just like other LIM surveys, CO intensity mapping also suffers strong foreground contamination that needs to be eliminated for extracting valuable astrophysical and cosmological information. In this work, we take $^{12}$CO($\it J$=1-0) emission line as an example to investigate whether deep learning method can effectively recover the signal by removing the foregrounds. The CO(1-0) intensity maps are generated by N-body simulations considering CO luminosity and halo mass relation, and we discuss two cases with median and low CO signals by comparing different relations. We add foregrounds generated from real observations, including thermal dust, spinning dust, free-free, synchrotron emission and CMB anisotropy. The beam with sidelobe effect is also considered. Our deep learning model is built upon ResUNet, which combines image generation algorithm UNet with the state-of-the-art architecture of deep learning, ResNet. The principal component analysis (PCA) method is employed to preprocess data before feeding it to the ResUNet. We find that, in the case of low instrumental noise, our UNet can efficiently reconstruct the CO signal map with correct line power spectrum by removing the foregrounds and recovering PCA signal loss and beam effects. Our method also can be applied to other intensity mappings like neutral hydrogen 21cm surveys.

astro-ph.CO↗
arXiv2023

Improving Visual-Semantic Embeddings by Learning Semantically-Enhanced Hard Negatives for Cross-modal Information Retrieval

Visual Semantic Embedding (VSE) aims to extract the semantics of images and their descriptions, and embed them into the same latent space for cross-modal information retrieval. Most existing VSE networks are trained by adopting a hard negatives loss function which learns an objective margin between the similarity of relevant and irrelevant image-description embedding pairs. However, the objective margin in the hard negatives loss function is set as a fixed hyperparameter that ignores the semantic differences of the irrelevant image-description pairs. To address the challenge of measuring the optimal similarities between image-description pairs before obtaining the trained VSE networks, this paper presents a novel approach that comprises two main parts: (1) finds the underlying semantics of image descriptions; and (2) proposes a novel semantically enhanced hard negatives loss function, where the learning objective is dynamically determined based on the optimal similarity scores between irrelevant image-description pairs. Extensive experiments were carried out by integrating the proposed methods into five state-of-the-art VSE networks that were applied to three benchmark datasets for cross-modal information retrieval tasks. The results revealed that the proposed methods achieved the best performance and can also be adopted by existing and future VSE networks.

cs.CV↗
arXiv2022

Cosmological Constraint Precision of the Photometric and Spectroscopic Multi-probe Surveys of China Space Station Telescope (CSST)

As one of Stage IV space-based telescopes, China Space Station Telescope (CSST) can perform photometric and spectroscopic surveys simultaneously to efficiently explore the Universe in extreme precision. In this work, we investigate several powerful CSST cosmological probes, including cosmic shear, galaxy-galaxy lensing, photometric and spectroscopic galaxy clustering, and number counts of galaxy clusters, and study the capability of these probes by forecasting the results of joint constraints on the cosmological parameters. By referring to real observational results, we generate mock data and estimate the measured errors based on CSST observational and instrumental designs. To study the systematical effects on the results, we also consider a number of systematics in CSST photometric and spectroscopic surveys, such as the intrinsic alignment, shear calibration uncertainties, photometric redshift uncertainties, galaxy bias, non-linear effects, instrumental effects, etc. The Fisher matrix method is used to derive the constraint results from individual or joint surveys on the cosmological and systematical parameters. We find that the joint constraints by including all these CSST cosmological probes can significantly improve the results from current observations by one order of magnitude at least, which gives $Ω_m$ and $σ_8$ $<$1% accuracy, and $w_0$ and $w_a$ $<$5% and 20% accuracies, respectively. This indicates that the CSST photometric and spectroscopic multi-probe surveys could provide powerful tools to explore the Universe and greatly improve the studies of relevant cosmological problems.

astro-ph.CO↗
arXiv2022

Partial Tail-Correlation Coefficient Applied to Extremal-Network Learning

We propose a novel extremal dependence measure called the partial tail-correlation coefficient (PTCC), in analogy to the partial correlation coefficient in classical multivariate analysis. The construction of our new coefficient is based on the framework of multivariate regular variation and transformed-linear algebra operations. We show how this coefficient allows identifying pairs of variables that have partially uncorrelated tails given the other variables in a random vector. Unlike other recently introduced conditional independence frameworks for extremes, our approach requires minimal modeling assumptions and can thus be used in exploratory analyses to learn the structure of extremal graphical models. Similarly to traditional Gaussian graphical models where edges correspond to the non-zero entries of the precision matrix, we can exploit classical inference methods for high-dimensional data, such as the graphical LASSO with Laplacian spectral constraints, to efficiently learn the extremal network structure via the PTCC. We apply our new method to study extreme risk networks in two different datasets (extreme river discharges and historical global currency exchange data) and show that we can extract meaningful extremal structures with meaningful domain-specific interpretations.

stat.ME↗
arXiv2022

Multi-modal Fusion Technology based on Vehicle Information: A Survey

Multi-modal fusion is a basic task of autonomous driving system perception, which has attracted many scholars' interest in recent years. The current multi-modal fusion methods mainly focus on camera data and LiDAR data, but pay little attention to the kinematic information provided by the bottom sensors of the vehicle, such as acceleration, vehicle speed, angle of rotation. These information are not affected by complex external scenes, so it is more robust and reliable. In this paper, we introduce the existing application fields of vehicle bottom information and the research progress of related methods, as well as the multi-modal fusion methods based on bottom information. We also introduced the relevant information of the vehicle bottom information data set in detail to facilitate the research as soon as possible. In addition, new future ideas of multi-modal fusion technology for autonomous driving tasks are proposed to promote the further utilization of vehicle bottom information.

cs.RO↗
arXiv2022

Photometric redshift estimates using Bayesian neural networks in the CSST survey

Galaxy photometric redshift (photo-$z$) is crucial in cosmological studies, such as weak gravitational lensing and galaxy angular clustering measurements. In this work, we try to extract photo-$z$ information and construct its probability distribution function (PDF) using the Bayesian neural networks (BNN) from both galaxy flux and image data expected to be obtained by the China Space Station Telescope (CSST). The mock galaxy images are generated from the Advanced Camera for Surveys of Hubble Space Telescope ($HST$-ACS) and COSMOS catalog, in which the CSST instrumental effects are carefully considered. And the galaxy flux data are measured from galaxy images using aperture photometry. We construct Bayesian multilayer perceptron (B-MLP) and Bayesian convolutional neural network (B-CNN) to predict photo-$z$ along with the PDFs from fluxes and images, respectively. We combine the B-MLP and B-CNN together, and construct a hybrid network and employ the transfer learning techniques to investigate the improvement of including both flux and image data. For galaxy samples with SNR$>$10 in $g$ or $i$ band, we find the accuracy and outlier fraction of photo-$z$ can achieve $σ_{\rm NMAD}=0.022$ and $η=2.35\%$ for the B-MLP using flux data only, and $σ_{\rm NMAD}=0.022$ and $η=1.32\%$ for the B-CNN using image data only. The Bayesian hybrid network can achieve $σ_{\rm NMAD}=0.021$ and $η=1.23\%$, and utilizing transfer learning technique can improve results to $σ_{\rm NMAD}=0.019$ and $η=1.17\%$, which can provide the most confident predictions with the lowest average uncertainty.

astro-ph.CO↗