arXiv subjects

Sheng-Chieh Lin

Publications and source records attributed to Sheng-Chieh Lin.

34 records · Page 2Linked to original sources

arXiv2022

Estimating Cluster Masses from SDSS Multi-band Images with Transfer Learning

The total masses of galaxy clusters characterize many aspects of astrophysics and the underlying cosmology. It is crucial to obtain reliable and accurate mass estimates for numerous galaxy clusters over a wide range of redshifts and mass scales. We present a transfer-learning approach to estimate cluster masses using the ugriz-band images in the SDSS Data Release 12. The target masses are derived from X-ray or SZ measurements that are only available for a small subset of the clusters. We designed a semi-supervised deep learning model consisting of two convolutional neural networks. In the first network, a feature extractor is trained to classify the SDSS photometric bands. The second network takes the previously trained features as inputs to estimate their total masses. The training and testing processes in this work depend purely on real observational data. Our algorithm reaches a mean absolute error (MAE) of 0.232 dex on average and 0.214 dex for the best fold. The performance is comparable to that given by redMaPPer, 0.192 dex. We have further applied a joint integrated gradient and class activation mapping method to interpret such a two-step neural network. The performance of our algorithm is likely to improve as the size of training dataset increases. This proof-of-concept experiment demonstrates the potential of deep learning in maximizing the scientific return of the current and future large cluster surveys.

astro-ph.IM↗

arXiv2021

Densifying Sparse Representations for Passage Retrieval by Representational Slicing

Learned sparse and dense representations capture different successful approaches to text retrieval and the fusion of their results has proven to be more effective and robust. Prior work combines dense and sparse retrievers by fusing their model scores. As an alternative, this paper presents a simple approach to densifying sparse representations for text retrieval that does not involve any training. Our densified sparse representations (DSRs) are interpretable and can be easily combined with dense representations for end-to-end retrieval. We demonstrate that our approach can jointly learn sparse and dense representations within a single model and then combine them for dense retrieval. Experimental results suggest that combining our DSRs and dense representations yields a balanced tradeoff between effectiveness and efficiency.

cs.IR↗

arXiv2021

Contextualized Query Embeddings for Conversational Search

This paper describes a compact and effective model for low-latency passage retrieval in conversational search based on learned dense representations. Prior to our work, the state-of-the-art approach uses a multi-stage pipeline comprising conversational query reformulation and information retrieval modules. Despite its effectiveness, such a pipeline often includes multiple neural models that require long inference times. In addition, independently optimizing each module ignores dependencies among them. To address these shortcomings, we propose to integrate conversational query reformulation directly into a dense retrieval model. To aid in this goal, we create a dataset with pseudo-relevance labels for conversational search to overcome the lack of training data and to explore different training strategies. We demonstrate that our model effectively rewrites conversational queries as dense representations in conversational search and open-domain question answering datasets. Finally, after observing that our model learns to adjust the $L_2$ norm of query token embeddings, we leverage this property for hybrid retrieval and to support error analysis.

cs.IR↗

arXiv2021

Halo-model analysis of the clustering of photometric luminous red galaxies at $0.10 \leq z \leq 1.05$ from the Subaru Hyper Suprime-Cam Survey

We present the clustering analysis of photometric luminous red galaxies (LRGs) at a redshift range of $0.1\leq z \leq 1.05$ using $615,317$ photometric LRGs selected from the Hyper Suprime-Cam Subaru Strategic Program covering $\sim124$ deg$^{2}$. Our sample covers a broad range of stellar masses and photometric redshifts and enables a halo occupation distribution analysis to study the redshift and stellar-mass dependence of dark halo properties of LRGs. We find a tight correlation between the characteristic dark halo mass to host central LRGs, $M_{\min}$, and the number density of LRGs independently of redshifts, indicating that the formation of LRGs is associated with the global environment. The $M_{\min}$ of LRGs depends only weakly on the stellar mass $M_{\star}$ at $M_{\star} \lesssim 10^{10.75}h^{-2} M_{\odot}$ at $0.3<z<1.05$, in contrast to the case for all photometrically selected galaxies for which $M_{\min}$ shows significant dependence on $M_{\star}$ even at low $M_{\star}$. The weak stellar mass dependence is indicative of the dark halo mass being the key parameter for the formation of LRGs rather than the stellar mass. Our result suggests that the halo mass of $\sim 10^{12.5 \pm 0.2}h^{-1} M_{\odot}$ is the critical mass for an efficient halo quenching due to the halo environment. We compare our result with the result of the hydrodynamical simulation to find that low-mass LRGs at $z \sim 1$ will increase their stellar masses by an order magnitude from $z=1$ to $0$ through mergers and satellite accretions, and a large fraction of massive LRGs at $z<0.9$ consist of LRGs that are recently migrated from massive green valley galaxies or those evolved from less massive LRGs through mergers and satellite accretions.

astro-ph.GA↗

arXiv2021

Angular clustering and host halo properties of [OII] emitters at $z >1$ in the Subaru HSC survey

We study the angular correlation function of star-forming galaxies and properties of their host dark matter halos at z>1 using the Hyper-Suprime Cam (HSC) SSP survey. We use [OII] emitters identified using two narrow-band (NB) filters, NB816 and NB921, in the Deep/UltraDeep layers, which respectively cover large angular areas of 16.3 deg^2 and 16.9 deg^2. Our sample contains 8302 and 9578 [OII] emitters at z=1.19 (NB816) and z=1.47 (NB921), respectively. We detect a strong clustering signal over a wide angular range, 0.001 < θ< 1 [deg], with the bias $b=1.61^{+0.13}_{-0.11}$ (z=1.19) and $b=2.09^{+0.17}_{-0.15}$ (z=1.47). We also find a clear deviation of the correlation from a simple power-law form. To interpret the measured clustering signal, we adopt a halo occupation distribution (HOD) model that is constructed to explain the spatial distribution of galaxies selected by a star formation rate. The observed correlation function and number density are simultaneously explained by the best-fitting HOD model. From the constrained HOD model, the average mass of halos hosting the [OII] emitters is derived to be $\log{M_{eff}/(h^{-1}M_\odot)}=12.70^{+0.09}_{-0.07}$ and $12.61^{+0.09}_{-0.05}$ at z=1.19 and 1.47, respectively, which will become halos with the present-day mass, $M\sim 1.5 \times 10^{13}h^{-1}M_\odot$. The satellite fraction of the [OII] emitter sample is found to be $f_{sat}\sim 0.15$. All these values are consistent with the previous studies of similar samples, but we obtain tighter constraints even in a larger parameter space due to the larger sample size from the HSC. The results obtained for host halos of [OII] emitters in this paper enable the construction of mock galaxy catalogs and the systematic forecast study of cosmological constraints from upcoming emission line galaxy surveys such as the Subaru PFS survey.

astro-ph.GA↗

arXiv2021

Efficiently Teaching an Effective Dense Retriever with Balanced Topic Aware Sampling

A vital step towards the widespread adoption of neural retrieval models is their resource efficiency throughout the training, indexing and query workflows. The neural IR community made great advancements in training effective dual-encoder dense retrieval (DR) models recently. A dense text retrieval model uses a single vector representation per query and passage to score a match, which enables low-latency first stage retrieval with a nearest neighbor search. Increasingly common, training approaches require enormous compute power, as they either conduct negative passage sampling out of a continuously updating refreshing index or require very large batch sizes for in-batch negative sampling. Instead of relying on more compute capability, we introduce an efficient topic-aware query and balanced margin sampling technique, called TAS-Balanced. We cluster queries once before training and sample queries out of a cluster per batch. We train our lightweight 6-layer DR model with a novel dual-teacher supervision that combines pairwise and in-batch negative teachers. Our method is trainable on a single consumer-grade GPU in under 48 hours (as opposed to a common configuration of 8x V100s). We show that our TAS-Balanced training method achieves state-of-the-art low-latency (64ms per query) results on two TREC Deep Learning Track query sets. Evaluated on NDCG@10, we outperform BM25 by 44%, a plainly trained DR by 19%, docT5query by 11%, and the previous best DR model by 5%. Additionally, TAS-Balanced produces the first dense retriever that outperforms every other method on recall at any cutoff on TREC-DL and allows more resource intensive re-ranking models to operate on fewer passages to improve results further.

cs.IR↗

arXiv2021

Multi-Stage Conversational Passage Retrieval: An Approach to Fusing Term Importance Estimation and Neural Query Rewriting

Conversational search plays a vital role in conversational information seeking. As queries in information seeking dialogues are ambiguous for traditional ad-hoc information retrieval (IR) systems due to the coreference and omission resolution problems inherent in natural language dialogue, resolving these ambiguities is crucial. In this paper, we tackle conversational passage retrieval (ConvPR), an important component of conversational search, by addressing query ambiguities with query reformulation integrated into a multi-stage ad-hoc IR system. Specifically, we propose two conversational query reformulation (CQR) methods: (1) term importance estimation and (2) neural query rewriting. For the former, we expand conversational queries using important terms extracted from the conversational context with frequency-based signals. For the latter, we reformulate conversational queries into natural, standalone, human-understandable queries with a pretrained sequence-tosequence model. Detailed analyses of the two CQR methods are provided quantitatively and qualitatively, explaining their advantages, disadvantages, and distinct behaviors. Moreover, to leverage the strengths of both CQR methods, we propose combining their output with reciprocal rank fusion, yielding state-of-the-art retrieval effectiveness, 30% improvement in terms of NDCG@3 compared to the best submission of TREC CAsT 2019.

cs.CL↗

arXiv2021

Pyserini: An Easy-to-Use Python Toolkit to Support Replicable IR Research with Sparse and Dense Representations

Pyserini is an easy-to-use Python toolkit that supports replicable IR research by providing effective first-stage retrieval in a multi-stage ranking architecture. Our toolkit is self-contained as a standard Python package and comes with queries, relevance judgments, pre-built indexes, and evaluation scripts for many commonly used IR test collections. We aim to support, out of the box, the entire research lifecycle of efforts aimed at improving ranking with modern neural approaches. In particular, Pyserini supports sparse retrieval (e.g., BM25 scoring using bag-of-words representations), dense retrieval (e.g., nearest-neighbor search on transformer-encoded representations), as well as hybrid retrieval that integrates both approaches. This paper provides an overview of toolkit features and presents empirical results that illustrate its effectiveness on two popular ranking tasks. We also describe how our group has built a culture of replicability through shared norms and tools that enable rigorous automated testing.

cs.IR↗

arXiv2020

Optical Wavelength Guided Self-Supervised Feature Learning For Galaxy Cluster Richness Estimate

Most galaxies in the nearby Universe are gravitationally bound to a cluster or group of galaxies. Their optical contents, such as optical richness, are crucial for understanding the co-evolution of galaxies and large-scale structures in modern astronomy and cosmology. The determination of optical richness can be challenging. We propose a self-supervised approach for estimating optical richness from multi-band optical images. The method uses the data properties of the multi-band optical images for pre-training, which enables learning feature representations from a large but unlabeled dataset. We apply the proposed method to the Sloan Digital Sky Survey. The result shows our estimate of optical richness lowers the mean absolute error and intrinsic scatter by 11.84% and 20.78%, respectively, while reducing the need for labeled training data by up to 60%. We believe the proposed method will benefit astronomy and cosmology, where a large number of unlabeled multi-band images are available, but acquiring image labels is costly.

cs.CV↗

arXiv2020

Distilling Dense Representations for Ranking using Tightly-Coupled Teachers

We present an approach to ranking with dense representations that applies knowledge distillation to improve the recently proposed late-interaction ColBERT model. Specifically, we distill the knowledge from ColBERT's expressive MaxSim operator for computing relevance scores into a simple dot product, thus enabling single-step ANN search. Our key insight is that during distillation, tight coupling between the teacher model and the student model enables more flexible distillation strategies and yields better learned representations. We empirically show that our approach improves query latency and greatly reduces the onerous storage requirements of ColBERT, while only making modest sacrifices in terms of effectiveness. By combining our dense representations with sparse representations derived from document expansion, we are able to approach the effectiveness of a standard cross-encoder reranker using BERT that is orders of magnitude slower.

cs.IR↗

arXiv2020

Personalized TV Recommendation: Fusing User Behavior and Preferences

In this paper, we propose a two-stage ranking approach for recommending linear TV programs. The proposed approach first leverages user viewing patterns regarding time and TV channels to identify potential candidates for recommendation and then further leverages user preferences to rank these candidates given textual information about programs. To evaluate the method, we conduct empirical studies on a real-world TV dataset, the results of which demonstrate the superior performance of our model in terms of both recommendation accuracy and time efficiency.

cs.IR↗

arXiv2020

Conversational Question Reformulation via Sequence-to-Sequence Architectures and Pretrained Language Models

This paper presents an empirical study of conversational question reformulation (CQR) with sequence-to-sequence architectures and pretrained language models (PLMs). We leverage PLMs to address the strong token-to-token independence assumption made in the common objective, maximum likelihood estimation, for the CQR task. In CQR benchmarks of task-oriented dialogue systems, we evaluate fine-tuned PLMs on the recently-introduced CANARD dataset as an in-domain task and validate the models using data from the TREC 2019 CAsT Track as an out-domain task. Examining a variety of architectures with different numbers of parameters, we demonstrate that the recent text-to-text transfer transformer (T5) achieves the best results both on CANARD and CAsT with fewer parameters, compared to similar transformer architectures.

cs.CL↗

arXiv2020

TTTTTackling WinoGrande Schemas

We applied the T5 sequence-to-sequence model to tackle the AI2 WinoGrande Challenge by decomposing each example into two input text strings, each containing a hypothesis, and using the probabilities assigned to the "entailment" token as a score of the hypothesis. Our first (and only) submission to the official leaderboard yielded 0.7673 AUC on March 13, 2020, which is the best known result at this time and beats the previous state of the art by over five points.

cs.CL↗

arXiv2017

First results on the cluster galaxy population from the Subaru Hyper Suprime-Cam survey. III. Brightest cluster galaxies, stellar mass distribution, and active galaxies

The unprecedented depth and area surveyed by the Subaru Strategic Program with the Hyper Suprime-Cam (HSC-SSP) have enabled us to construct and publish the largest distant cluster sample out to $z\sim 1$ to date. In this exploratory study of cluster galaxy evolution from $z=1$ to $z=0.3$, we investigate the stellar mass assembly history of brightest cluster galaxies (BCGs), evolution of stellar mass and luminosity distributions, stellar mass surface density profile, as well as the population of radio galaxies. Our analysis is the first high redshift application of the top N richest cluster selection, which is shown to allow us to trace the cluster galaxy evolution faithfully. Over the 230 deg$^2$ area of the current HSC-SSP footprint, selecting the top 100 clusters in each of the 4 redshift bins allows us to observe the buildup of galaxy population in descendants of clusters whose $z\approx 1$ mass is about $2\times 10^{14}\,M_\odot$. Our stellar mass is derived from a machine-learning algorithm, which is found to be unbiased and accurate with respect to the COSMOS data. We find very mild stellar mass growth in BCGs (about 35% between $z=1$ and 0.3), and no evidence for evolution in both the total stellar mass-cluster mass correlation and the shape of the stellar mass surface density profile. We also present the first measurement of the radio luminosity distribution in clusters out to $z\sim 1$, and show hints of changes in the dominant accretion mode powering the cluster radio galaxies at $z\sim 0.8$.

astro-ph.GA↗

arXiv2017

GOLDRUSH. II. Clustering of Galaxies at $z\sim 4-6$ Revealed with the Half-Million Dropouts Over the 100 deg$^2$ Area Corresponding to 1 Gpc$^3$

We present clustering properties from 579,492 Lyman break galaxies (LBGs) at z~4-6 over the 100 deg^2 sky (corresponding to a 1.4 Gpc^3 volume) identified in early data of the Hyper Suprime-Cam (HSC) Subaru strategic program survey. We derive angular correlation functions (ACFs) of the HSC LBGs with unprecedentedly high statistical accuracies at z~4-6, and compare them with the halo occupation distribution (HOD) models. We clearly identify significant ACF excesses in 10"<$θ$<90", the transition scale between 1- and 2-halo terms, suggestive of the existence of the non-linear halo bias effect. Combining the HOD models and previous clustering measurements of faint LBGs at z~4-7, we investigate dark-matter halo mass (Mh) of the z~4-7 LBGs and its correlation with various physical properties including the star-formation rate (SFR), the stellar-to-halo mass ratio (SHMR), and the dark matter accretion rate (dotMh) over a wide-mass range of Mh/M$_\odot$=4x10^10-4x10^12. We find that the SHMR increases from z~4 to 7 by a factor of ~4 at Mh~1x10^11 M$_\odot$, while the SHMR shows no strong evolution in the similar redshift range at Mh~1x10^12 M$_\odot$. Interestingly, we identify a tight relation of SFR/dotMh-Mh showing no significant evolution beyond 0.15 dex in this wide-mass range over z~4-7. This weak evolution suggests that the SFR/dotMh-Mh relation is a fundamental relation in high-redshift galaxy formation whose star formation activities are regulated by the dark matter mass assembly. Assuming this fundamental relation, we calculate the cosmic SFR densities (SFRDs) over z=0-10 (a.k.a. Madau-Lilly plot). The cosmic SFRD evolution based on the fundamental relation agrees with the one obtained by observations, suggesting that the cosmic SFRD increase from z~10 to 4-2 (decrease from z~4-2 to 0) is mainly driven by the increase of the halo abundance (the decrease of the accretion rate).

astro-ph.GA↗

arXiv2017

An optically-selected cluster catalog at redshift 0.1<z<1.1 from the Hyper Suprime-Cam Subaru Strategic Program S16A data

We present an optically-selected cluster catalog from the Hyper Suprime-Cam (HSC) Subaru Strategic Program. The HSC images are sufficiently deep to detect cluster member galaxies down to $M_*\sim 10^{10.2}M_\odot$ even at $z\sim 1$, allowing a reliable cluster detection at such high redshifts. We apply the CAMIRA algorithm to the HSC Wide S16A dataset covering $\sim 232$ deg$^2$ to construct a catalog of 1921 clusters at redshift $0.1 15$ that roughly corresponds to $M_{\rm 200m}\gtrsim 10^{14}h^{-1}M_\odot$. We confirm good cluster photometric redshift performance, with the bias and scatter in $Δz/(1+z)$ being better than 0.005 and 0.01 over most of the redshift range, respectively. We compare our cluster catalog with large X-ray cluster catalogs from XXL and XMM-LSS surveys and find good correlation between richness and X-ray properties. We also study the miscentering effect from the distribution of offsets between optical and X-ray cluster centers. We confirm the high ($>0.9$) completeness and purity for high mass clusters by analyzing mock galaxy catalogs.

astro-ph.CO↗