Searcharxiv⌕ Search

arXiv subjects

Chen Zhang

Publications and source records attributed to Chen Zhang.

At least 397 records · Page 22Linked to original sources
arXiv2023

Sparse Contrastive Learning of Sentence Embeddings

Recently, SimCSE has shown the feasibility of contrastive learning in training sentence embeddings and illustrates its expressiveness in spanning an aligned and uniform embedding space. However, prior studies have shown that dense models could contain harmful parameters that affect the model performance, and it is no wonder that SimCSE can as well be invented with such parameters. Driven by this, parameter sparsification is applied, where alignment and uniformity scores are used to measure the contribution of each parameter to the overall quality of sentence embeddings. Drawing from a preliminary study, we consider parameters with minimal contributions to be detrimental, as their sparsification results in improved model performance. To discuss the ubiquity of detrimental parameters and remove them, more experiments on the standard semantic textual similarity (STS) tasks and transfer learning tasks are conducted, and the results show that the proposed sparsified SimCSE (SparseCSE) has excellent performance in comparison with SimCSE. Furthermore, through in-depth analysis, we establish the validity and stability of our sparsification method, showcasing that the embedding space generated by SparseCSE exhibits improved alignment compared to that produced by SimCSE. Importantly, the uniformity yet remains uncompromised.

cs.CL↗
arXiv2023

Reconciling results of 2019 and 2020 stellar occultations on Pluto's atmosphere. New constraints from both the 5 September 2019 event and consistency analysis

A stellar occultation by Pluto on 5 September 2019 yielded positive detections at two separate stations. Using an approach consistent with comparable studies, we derived a surface pressure of $11.478 \pm 0.55~\mathrm{μbar}$ for Pluto's atmosphere from the observations of this event. In addition, to avoid potential method inconsistancies highlighted by Sicardy et al. when comparing with historical pressure measurements, we reanalyzed the data by 15 August 2018 and 17 July 2019 events, respectively. All the new measurements provide a bridge between the two different perspectives on the pressure variation since 2015: a rapid pressure drop from previous studies of the 15 August 2018 and 17 July 2019 events and a plateau phase from that of the 6 June 2020 event. The pressure measurement from the 5 September 2019 event aligns with those from 2016, 2018, and 2020, supporting the latter perspective. While the measurements from the 4 June 2011 and 17 July 2019 events suggest probable V-shaped pressure variations unaccounted for by the volatile transport model (VTM) from Meza et al., the VTM remains applicable on average. And, the validity of the V-shaped variations is debatable due to the stellar faintness of the 4 June 2011 event and the grazing single-chord geometry of the 17 July 2019 event. To reveal and understand all significant pressure variations of Pluto's atmosphere, it is essential to provide constraints on both short-term and long-term evolutions of the interacting atmosphere and surface by continuous pressure monitoring through occultation observations, whenever possible, complemented by frequent spectroscopy and photometry of the surface.

astro-ph.EP↗
arXiv2023

Choose A Table: Tensor Dirichlet Process Multinomial Mixture Model with Graphs for Passenger Trajectory Clustering

Passenger clustering based on trajectory records is essential for transportation operators. However, existing methods cannot easily cluster the passengers due to the hierarchical structure of the passenger trip information, including multiple trips within each passenger and multi-dimensional information about each trip. Furthermore, existing approaches rely on an accurate specification of the clustering number to start. Finally, existing methods do not consider spatial semantic graphs such as geographical proximity and functional similarity between the locations. In this paper, we propose a novel tensor Dirichlet Process Multinomial Mixture model with graphs, which can preserve the hierarchical structure of the multi-dimensional trip information and cluster them in a unified one-step manner with the ability to determine the number of clusters automatically. The spatial graphs are utilized in community detection to link the semantic neighbors. We further propose a tensor version of Collapsed Gibbs Sampling method with a minimum cluster size requirement. A case study based on Hong Kong metro passenger data is conducted to demonstrate the automatic process of cluster amount evolution and better cluster quality measured by within-cluster compactness and cross-cluster separateness. The code is available at https://github.com/bonaldli/TensorDPMM-G.

stat.ML↗
arXiv2023

STGIC: a graph and image convolution-based method for spatial transcriptomic clustering

Spatial transcriptomic (ST) clustering employs spatial and transcription information to group spots spatially coherent and transcriptionally similar together into the same spatial domain. Graph convolution network (GCN) and graph attention network (GAT), fed with spatial coordinates derived adjacency and transcription profile derived feature matrix are often used to solve the problem. Our proposed method STGIC (spatial transcriptomic clustering with graph and image convolution) utilizes an adaptive graph convolution (AGC) to get high quality pseudo-labels and then resorts to dilated convolution framework (DCF) for virtual image converted from gene expression information and spatial coordinates of spots. The dilation rates and kernel sizes are set appropriately and updating of weight values in the kernels is made to be subject to the spatial distance from the position of corresponding elements to kernel centers so that feature extraction of each spot is better guided by spatial distance to neighbor spots. Self-supervision realized by KL-divergence, spatial continuity loss and cross entropy calculated among spots with high confidence pseudo-labels make up the training objective of DCF. STGIC attains state-of-the-art (SOTA) clustering performance on the benchmark dataset of human dorsolateral prefrontal cortex (DLPFC). Besides, it's capable of depicting fine structures of other tissues from other species as well as guiding the identification of marker genes. Also, STGIC is expandable to Stereo-seq data with high spatial resolution.

q-bio.QM↗
arXiv2023

Enhancing Black-Box Few-Shot Text Classification with Prompt-Based Data Augmentation

Training or finetuning large-scale language models (LLMs) such as GPT-3 requires substantial computation resources, motivating recent efforts to explore parameter-efficient adaptation to downstream tasks. One practical area of research is to treat these models as black boxes and interact with them through their inference APIs. In this paper, we investigate how to optimize few-shot text classification without accessing the gradients of the LLMs. To achieve this, we treat the black-box model as a feature extractor and train a classifier with the augmented text data. Data augmentation is performed using prompt-based finetuning on an auxiliary language model with a much smaller parameter size than the black-box model. Through extensive experiments on eight text classification datasets, we show that our approach, dubbed BT-Classifier, significantly outperforms state-of-the-art black-box few-shot learners and performs on par with methods that rely on full-model tuning.

cs.CL↗
arXiv2023

Variational quantum circuit learning of entanglement purification in multiple degrees of freedom

Entanglement purification is a crucial technique for promising the effective entanglement channel in noisy large-scale quantum networks, yet complicated in designing protocols in multi-degree of freedom (DoF). To execute the above tasks easily and effectively, developing a learning framework for designing the entanglement purification with multi-DoF is a promising way and still an open research question. Inspired by variational quantum circuit (VQC) with remarkable advantage in learning optimal quantum operations with near-term quantum devices, in this paper we propose an effective VQC framework for the entanglement purification in multi-DoF and exploit it to learn the optimal purification protocols of the objective function which are based on postselection. By properly introducing additional circuit lines for representing each of the ancillary DoFs of all the particles, e.g., space and time, the parametrized quantum circuit can effectively simulate scalable entanglement purification. To verify our framework, the well-known protocols in linear optics are learned well with alternative operations in low-depth quantum circuit. Moreover, we simulate the multipair cases with multi-DoF to show the scalability and discover one-round protocols. Our work provides an effective way for exploring the entanglement purification protocols in multi-DoF and multipair with near-term quantum devices.

quant-ph↗
arXiv2023

Underwater and Surface Aquatic Locomotion of Soft Biomimetic Robot Based on Bending Rolled Dielectric Elastomer Actuators

All-around, real-time navigation and sensing across the water environments by miniature soft robotics are promising, for their merits of small size, high agility and good compliance to the unstructured surroundings. In this paper, we propose and demonstrate a mantas-like soft aquatic robot which propels itself by flapping-fins using rolled dielectric elastomer actuators (DEAs) with bending motions. This robot exhibits fast-moving capabilities of swimming at 57mm/s or 1.25 body length per second (BL/s), skating on water surface at 64 mm/s (1.36 BL/s) and vertical ascending at 38mm/s (0.82 BL/s) at 1300 V, 17 Hz of the power supply. These results show the feasibility of adopting rolled DEAs for mesoscale aquatic robots with high motion performance in various water-related scenarios.

cs.RO↗
arXiv2023

Experimental observation of the significant difference between surface and bulk Kondo processes in Kondo lattice YbCu$_2$Si$_2$

Synchrotron-based angle-resolved photoemission spectroscopy was employed to investigate the temperature evolution of the Yb 4f spectral for surface and bulk in the Kondo lattice YbCu$_2$Si$_2$.Our study quantitatively distinguishes between the surface and bulk hybridization processes, revealing that the onset temperatures for both surface and bulk hybridization processes are significantly higher than the Kondo temperature. Additionally, we found that the effective surface Kondo temperature is much lower than that of the bulk. These findings offer valuable insights into the understanding of heavy fermion physics.

cond-mat.str-el↗
arXiv2023

Gravitational capture of magnetic monopoles by primordial black holes in the early universe

It is intriguing to ask whether the existence of primordial black holes (PBHs) in the early universe could significantly reduce the abundance of certain stable massive particles (SMP) via gravitational capture, after which the PBHs evaporate before BBN to avoid conflict with stringent bounds. For example, this mechanism is relevant to an alternative solution of the monopole problem proposed by Stojkovic and Freese, in which magnetic monopoles produced in the early universe are captured by PBHs, thus freeing inflation from having to occur during or after the corresponding phase transitions that produced the monopoles. In this work, we reanalyze the solution by modelling the capture process in the same way as the coexisting monopole annihilation. A subtle issue which is not handled properly in the previous literature is the choice of an effective capture cross section for diffusive capture. We model this aspect properly and justify our treatment. A monochromatic PBH mass function and a radiation-dominated era before PBH evaporation are assumed. We find that for Pati-Salam monopoles corresponding to a symmetry breaking scale between $10^{10}\,\text{GeV}$ and $10^{15}\,\text{GeV}$, the capture rate is many orders of magnitude below what is needed to cause a significant reduction of the monopole density. Within our assumptions, we also find that the magnetic charge that is large enough to make an extremal magnetic black hole cosmologically stable cannot be obtained from magnetic charge fluctuation via monopole capture. The large magnetic charged required by cosmological stability can nevertheless be obtained from magnetic charge fluctuation at PBH formation, and if later the monopole abundance can be reduced significantly by some non-inflationary mechanism, long-lived near-extremal magnetic black holes of observational relevance might result.

hep-ph↗
arXiv2023

Lawyer LLaMA Technical Report

Large Language Models (LLMs), like LLaMA, have exhibited remarkable performance across various tasks. Nevertheless, when deployed to specific domains such as law or medicine, the models still confront the challenge of a deficiency in domain-specific knowledge and an inadequate capability to leverage that knowledge to resolve domain-related problems. In this paper, we propose a new framework to adapt LLMs to specific domains and build Lawyer LLaMA, a legal domain LLM, based on this framework. Specifically, we inject domain knowledge during the continual training stage and teach the model to learn professional skills using properly designed supervised fine-tuning tasks. Moreover, to alleviate the hallucination problem during the model's generation, we add a retrieval module and extract relevant legal articles before the model answers any queries. When learning domain-specific skills, we find that experts' experience is much more useful than experiences distilled from ChatGPT, where hundreds of expert-written data outperform tens of thousands of ChatGPT-generated ones. We will release our model and data.

cs.CL↗
arXiv2023

xDial-Eval: A Multilingual Open-Domain Dialogue Evaluation Benchmark

Recent advancements in reference-free learned metrics for open-domain dialogue evaluation have been driven by the progress in pre-trained language models and the availability of dialogue data with high-quality human annotations. However, current studies predominantly concentrate on English dialogues, and the generalization of these metrics to other languages has not been fully examined. This is largely due to the absence of a multilingual dialogue evaluation benchmark. To address the issue, we introduce xDial-Eval, built on top of open-source English dialogue evaluation datasets. xDial-Eval includes 12 turn-level and 6 dialogue-level English datasets, comprising 14930 annotated turns and 8691 annotated dialogues respectively. The English dialogue data are extended to nine other languages with commercial machine translation systems. On xDial-Eval, we conduct comprehensive analyses of previous BERT-based metrics and the recently-emerged large language models. Lastly, we establish strong self-supervised and multilingual baselines. In terms of average Pearson correlations over all datasets and languages, the best baseline outperforms OpenAI's ChatGPT by absolute improvements of 6.5% and 4.6% at the turn and dialogue levels respectively, albeit with much fewer parameters. The data and code are publicly available at https://github.com/e0397123/xDial-Eval.

cs.CL↗
arXiv2023

Hybrid packet switching assisted by classical frame for entanglement-based quantum networks

One of the first problems of studying the quantum internet is how to realize quantum interconnection between users in a quantum network. To address above problem, by referencing the classical Internet, developing the packet switching of quantum networks is a promising way. In this paper, we propose a new hybrid packet switching for entanglement-based quantum networks assisted by classical frame. Different from the previous packet switching for quantum networks based on single photon, the frame used in our scheme is pure classical rather than the classical-quantum structure, and the transmission of classical and quantum signals over physical channels can be independent, which makes our scheme is also valid for quantum networks with heralded entanglement generation. Using our hybrid packet switching, the process of building entanglement channel between end nodes is analogous to the classical packet-switched networks, which provides an effective way to build large-scale packet-switched entanglement-based quantum internet. To verify the feasibility, we perform end-to-end entanglement distribution using our hybrid packet switching in a quantum network and simulate the fidelities of distributed state with respect to the number of hops.

quant-ph↗
arXiv2023

Rescaling strange-cluster stars and its implications on gravitational-wave echoes

Solid states of strange-cluster matter called strangeon matter can form strangeon stars that are highly compact. We show that strangeon matter and strangeon stars can be recast into dimensionless forms by a simple reparametrization and rescaling, through which we manage to maximally reduce the number of degrees of freedom. With this dimensionless scheme, we find that strangeon stars are generally compact enough to feature a photon sphere that is essential to foster gravitational-wave (GW) echoes. Rescaling the dimension back, we illustrate its implications on the expanded dimensional parameter space, and calculate the GW echo frequencies associated with strangeon stars, showing that the minimum echo frequency is $\sim 8$ kHz for empirical parameter space that satisfies the GW170817 constraint, and can reduce to $\mathcal O(100)$ Hertz at the extended limit.

astro-ph.HE↗
arXiv2023

Magnetic monopole meets primordial black hole: an extended analysis

We investigate gravitational capture of magnetic monopoles by primordial black holes (PBH) that evaporate before Big Bang Nucleosynthesis (BBN), a hypothetical process which was once proposed as an alternative solution to the monopole problem. Magnetic monopoles produced in phase transitions of a grand or partially unified gauge theory are considered. We prove analytically that for all extended PBH mass functions that preserve radiation domination, it is impossible to reduce the monopole abundance via gravitational capture by PBHs to values significantly below the one set by monopole annihilation (or below its initial abundance if it is smaller), regardless of the nature of the capture process (diffusive or non-diffusive). Therefore, the monopole problem cannot be solved by PBH capture in a radiation-dominated era in the early universe.

hep-ph↗
arXiv2023

Influence of mass-ratio corrections in extreme-mass-ratio inspirals for testing general relativity

The popular waveform templates of extreme-mass-ratio inspirals usually omit the mass-ratio corrections in the conservative dynamics, and employ adiabatic approximation to include the radiation reaction. With the help of effective-one-body dynamics, we investigate the influence of mass-ratio corrections in the conservative part on the gravitational waves. We find that for the extra-relativistic orbits, the mass-ratio corrections can produce obvious orbital dephasing even for very small mass-ratio and then affect the waveforms. More importantly, it is noted that omitting the mass-ratio corrections in waveform templates may induce a fake signal of deviation from general relativity.

gr-qc↗
arXiv2023

Overview of Robust and Multilingual Automatic Evaluation Metrics for Open-Domain Dialogue Systems at DSTC 11 Track 4

The advent and fast development of neural networks have revolutionized the research on dialogue systems and subsequently have triggered various challenges regarding their automatic evaluation. Automatic evaluation of open-domain dialogue systems as an open challenge has been the center of the attention of many researchers. Despite the consistent efforts to improve automatic metrics' correlations with human evaluation, there have been very few attempts to assess their robustness over multiple domains and dimensions. Also, their focus is mainly on the English language. All of these challenges prompt the development of automatic evaluation metrics that are reliable in various domains, dimensions, and languages. This track in the 11th Dialogue System Technology Challenge (DSTC11) is part of the ongoing effort to promote robust and multilingual automatic evaluation metrics. This article describes the datasets and baselines provided to participants and discusses the submission and result details of the two proposed subtasks.

cs.CL↗
arXiv2023

Unsupervised Learning of Nanoindentation Data to Infer Microstructural Details of Complex Materials

In this study, Cu-Cr composites were studied by nanoindentation. Arrays of indents were placed over large areas of the samples resulting in datasets consisting of several hundred measurements of Young's modulus and hardness at varying indentation depths. The unsupervised learning technique, Gaussian mixture model, was employed to analyze the data, which helped to determine the number of "mechanical phases" and the respective mechanical properties. Additionally, a cross-validation approach was introduced to infer whether the data quantity was adequate and to suggest the amount of data required for reliable predictions -- one of the often encountered but difficult to resolve issues in machine learning of materials science problems.

cs.LG↗
arXiv2023

Hybrid stars may have an inverted structure

We propose a new stellar structure of compact stars, the ``cross stars" that consist of a hadronic matter core and a quark matter crust, with an inverted structure compared to the conventional hybrid stars. This distinct stellar structure naturally arises from the quark matter to hadronic matter transition associated with the chemical potential crossing, in the context of the quark matter hypothesis that either strange or up-down quark matter is the ground state of baryonic matter at low pressure. We find that the interplay between the hadronic matter and quark matter compositions of cross stars can help to reconcile the small radii constraints indicated by the LIGO/Virgo GW170817 event, the large radii constraints set for massive compact stars by recent NICER X-ray observations, and the recent observation of the most-massive pulsar PSR J0952-0607. This leaves more space open for the equation of states of both hadronic matter and quark matter.

astro-ph.HE↗