arXiv subjects

Jun Gao

Publications and source records attributed to Jun Gao.

At least 163 records · Page 9Linked to original sources

arXiv2024

Novae: An Important Source of Lithium in the Galaxy

The source of the Galactic Lithium (Li) has long been a puzzle. With the discovery of Li in novae, extensive research has been conducted. However, there still exists a significant disparity between the observed abundance of lithium in novae and the existing theoretical predictions. Using the Modules for Experiments in Stellar Astrophysics (MESA), we simulate the evolution of nova with element diffusion and appropriately increased the amount of 3^He in the mixtures. Element diffusion enhances the transport efficiency between the nuclear reaction zone and the convective region on the surface of the white dwarf during nova eruptions, which results in more 7^Be to be transmitted to the white dwarf surface and ultimately ejected. Compared to the previous predictions, the abundance of 7^Be in novae simulated in our model significantly increases. And the result is able to explain almost all observed novae. Using the method of population synthesis, we calculate Li yield in the Galaxy. We find that the Galactic occurrence rate of nova is about 130 yr^{-1}, and about 110M Li produced by nova eruption is ejected into the interstellar medium (ISM). About 73\% of Li in the Galactic ISM originates from novae, and approximately 15\%-20\% of the entire Galaxy. It means that novae are the important source of Li in the Galactic.

astro-ph.SR↗

arXiv2024

SelfCP: Compressing Over-Limit Prompt via the Frozen Large Language Model Itself

Long prompt leads to huge hardware costs when using transformer-based Large Language Models (LLMs). Unfortunately, many tasks, such as summarization, inevitably introduce long documents, and the wide application of in-context learning easily makes the prompt length explode. This paper proposes a Self-Compressor (SelfCP), which employs the target LLM itself to compress over-limit prompts into dense vectors while keeping the allowed prompts unmodified. Dense vectors are then projected into dense tokens via a learnable connector to make the same LLM unburden to understand. The connector is supervised-tuned under the language modeling objective of the LLM on relatively long texts selected from publicly accessed datasets, involving an instruction dataset to make SelfCP respond to various prompts, while the target LLM keeps frozen during training. We build the lightweight SelfCP upon 2 different backbones with merely 17M learnable parameters originating from the connector and a learnable embedding. Evaluation on both English and Chinese benchmarks demonstrate that SelfCP effectively substitutes 12$\times$ over-limit prompts with dense tokens to reduce memory costs and booster inference throughputs, yet improving response quality. The outstanding performance brings an efficient solution for LLMs to tackle long prompts without training LLMs from scratch.

cs.CL↗

arXiv2024

ValueDCG: Measuring Comprehensive Human Value Understanding Ability of Language Models

Personal values are a crucial factor behind human decision-making. Considering that Large Language Models (LLMs) have been shown to impact human decisions significantly, it is essential to make sure they accurately understand human values to ensure their safety. However, evaluating their grasp of these values is complex due to the value's intricate and adaptable nature. We argue that truly understanding values in LLMs requires considering both "know what" and "know why". To this end, we present a comprehensive evaluation metric, ValueDCG (Value Discriminator-Critique Gap), to quantitatively assess the two aspects with an engineering implementation. We assess four representative LLMs and provide compelling evidence that the growth rates of LLM's "know what" and "know why" capabilities do not align with increases in parameter numbers, resulting in a decline in the models' capacity to understand human values as larger amounts of parameters. This may further suggest that LLMs might craft plausible explanations based on the provided context without truly understanding their inherent value, indicating potential risks.

cs.CL↗

arXiv2024

High-energy neutrino deep inelastic scattering cross sections

We present a state-of-the-art prediction for cross sections of neutrino deep inelastic scattering (DIS) from nucleon at high neutrino energies, $E_ν$, from 100 GeV to 1000 EeV ($10^{12}$ GeV). Our calculations are based on the latest CT18 NNLO parton distribution functions (PDFs) and their associated uncertainties. In order to make predictions for the highest energies, we extrapolate the PDFs to small $x$ according to several procedures and assumptions, thus affecting the uncertainties at ultra-high $E_ν$; we quantify the uncertainties corresponding to these choices. Similarly, we quantify the uncertainties introduced by the nuclear corrections which are required to evaluate neutrino-nuclear cross sections for neutrino telescopes. These results can be applied to currently-running astrophysical neutrino observatories, such as IceCube and KM3NeT, as well as various future experiments which have been proposed.

hep-ph↗

arXiv2024

Guiding ChatGPT to Generate Salient Domain Summaries

ChatGPT is instruct-tuned to generate general and human-expected content to align with human preference through Reinforcement Learning from Human Feedback (RLHF), meanwhile resulting in generated responses not salient enough. Therefore, in this case, ChatGPT may fail to satisfy domain requirements in zero-shot settings, leading to poor ROUGE scores. Inspired by the In-Context Learning (ICL) and retelling ability of ChatGPT, this paper proposes PADS, a \textbf{P}ipeline for \textbf{A}ssisting ChatGPT in \textbf{D}omain \textbf{S}ummarization. PADS consists of a retriever to retrieve similar examples from corpora and a rank model to rerank the multiple candidate summaries generated by ChatGPT. Specifically, given an inference document, we first retrieve an in-context demonstration via the retriever. Then, we require ChatGPT to generate $k$ candidate summaries for the inference document at a time under the guidance of the retrieved demonstration. Finally, the rank model independently scores the $k$ candidate summaries according to their quality and selects the optimal one. We extensively explore dense and sparse retrieval methods to select effective demonstrations for reference and efficiently train the rank model to reflect the quality of candidate summaries for each given summarized document. Additionally, PADS contains merely 400M trainable parameters originating from the rank model and we merely collect 2.5k data to train it. We evaluate PADS on five datasets from different domains, and the result indicates that each module in PADS is committed to effectively guiding ChatGPT to generate salient summaries fitting different domain requirements. Specifically, in the popular summarization dataset Gigaword, PADS achieves over +8 gain on ROUGE-L, compared with the naive ChatGPT in the zero-shot setting. \footnote{Our code are available at \url{https://github.com/jungao1106/PADS}}

cs.CL↗

arXiv2024

WebCiteS: Attributed Query-Focused Summarization on Chinese Web Search Results with Citations

Enhancing the attribution in large language models (LLMs) is a crucial task. One feasible approach is to enable LLMs to cite external sources that support their generations. However, existing datasets and evaluation methods in this domain still exhibit notable limitations. In this work, we formulate the task of attributed query-focused summarization (AQFS) and present WebCiteS, a Chinese dataset featuring 7k human-annotated summaries with citations. WebCiteS derives from real-world user queries and web search results, offering a valuable resource for model training and evaluation. Prior works in attribution evaluation do not differentiate between groundedness errors and citation errors. They also fall short in automatically verifying sentences that draw partial support from multiple sources. We tackle these issues by developing detailed metrics and enabling the automatic evaluator to decompose the sentences into sub-claims for fine-grained verification. Our comprehensive evaluation of both open-source and proprietary models on WebCiteS highlights the challenge LLMs face in correctly citing sources, underscoring the necessity for further improvement. The dataset and code will be open-sourced to facilitate further research in this crucial field.

cs.CL↗

arXiv2024

APrompt4EM: Augmented Prompt Tuning for Generalized Entity Matching

Generalized Entity Matching (GEM), which aims at judging whether two records represented in different formats refer to the same real-world entity, is an essential task in data management. The prompt tuning paradigm for pre-trained language models (PLMs), including the recent PromptEM model, effectively addresses the challenges of low-resource GEM in practical applications, offering a robust solution when labeled data is scarce. However, existing prompt tuning models for GEM face the challenges of prompt design and information gap. This paper introduces an augmented prompt tuning framework for the challenges, which consists of two main improvements. The first is an augmented contextualized soft token-based prompt tuning method that extracts a guiding soft token benefit for the PLMs' prompt tuning, and the second is a cost-effective information augmentation strategy leveraging large language models (LLMs). Our approach performs well on the low-resource GEM challenges. Extensive experiments show promising advancements of our basic model without information augmentation over existing methods based on moderate-size PLMs (average 5.24%+), and our model with information augmentation achieves comparable performance compared with fine-tuned LLMs, using less than 14% of the API fee.

cs.CL↗

arXiv2024

WildFusion: Learning 3D-Aware Latent Diffusion Models in View Space

Modern learning-based approaches to 3D-aware image synthesis achieve high photorealism and 3D-consistent viewpoint changes for the generated images. Existing approaches represent instances in a shared canonical space. However, for in-the-wild datasets a shared canonical system can be difficult to define or might not even exist. In this work, we instead model instances in view space, alleviating the need for posed images and learned camera distributions. We find that in this setting, existing GAN-based methods are prone to generating flat geometry and struggle with distribution coverage. We hence propose WildFusion, a new approach to 3D-aware image synthesis based on latent diffusion models (LDMs). We first train an autoencoder that infers a compressed latent representation, which additionally captures the images' underlying 3D structure and enables not only reconstruction but also novel view synthesis. To learn a faithful 3D representation, we leverage cues from monocular depth prediction. Then, we train a diffusion model in the 3D-aware latent space, thereby enabling synthesis of high-quality 3D-consistent image samples, outperforming recent state-of-the-art GAN-based methods. Importantly, our 3D-aware LDM is trained without any direct supervision from multiview images or 3D geometry and does not require posed images or learned pose or camera distributions. It directly learns a 3D representation without relying on canonical camera coordinates. This opens up promising research avenues for scalable 3D-aware image synthesis and 3D content creation from in-the-wild image data. See https://katjaschwarz.github.io/wildfusion for videos of our 3D results.

cs.CV↗

arXiv2024

LATTE3D: Large-scale Amortized Text-To-Enhanced3D Synthesis

Recent text-to-3D generation approaches produce impressive 3D results but require time-consuming optimization that can take up to an hour per prompt. Amortized methods like ATT3D optimize multiple prompts simultaneously to improve efficiency, enabling fast text-to-3D synthesis. However, they cannot capture high-frequency geometry and texture details and struggle to scale to large prompt sets, so they generalize poorly. We introduce LATTE3D, addressing these limitations to achieve fast, high-quality generation on a significantly larger prompt set. Key to our method is 1) building a scalable architecture and 2) leveraging 3D data during optimization through 3D-aware diffusion priors, shape regularization, and model initialization to achieve robustness to diverse and complex training prompts. LATTE3D amortizes both neural field and textured surface generation to produce highly detailed textured meshes in a single forward pass. LATTE3D generates 3D objects in 400ms, and can be further enhanced with fast test-time optimization.

cs.CV↗

arXiv2024

Generalized Ramsey--Turán density for cliques

We study the generalized Ramsey--Turán function $\mathrm{RT}(n,K_s,K_t,o(n))$, which is the maximum possible number of copies of $K_s$ in an $n$-vertex $K_t$-free graph with independence number $o(n)$. The case when $s=2$ was settled by Erd{ő}s, S{ó}s, Bollob{á}s, Hajnal, and Szemerédi in the 1980s. We combinatorially resolve the general case for all $s\ge 3$, showing that the (asymptotic) extremal graphs for this problem have simple (bounded) structures. In particular, it implies that the extremal structures follow a periodic pattern when $t$ is much larger than $s$. Our results disprove a conjecture of Balogh, Liu, and Sharifzadeh and show that a relaxed version does hold.

math.CO↗

arXiv2024

Towards Versatile Graph Learning Approach: from the Perspective of Large Language Models

Graph-structured data are the commonly used and have wide application scenarios in the real world. For these diverse applications, the vast variety of learning tasks, graph domains, and complex graph learning procedures present challenges for human experts when designing versatile graph learning approaches. Facing these challenges, large language models (LLMs) offer a potential solution due to the extensive knowledge and the human-like intelligence. This paper proposes a novel conceptual prototype for designing versatile graph learning methods with LLMs, with a particular focus on the "where" and "how" perspectives. From the "where" perspective, we summarize four key graph learning procedures, including task definition, graph data feature engineering, model selection and optimization, deployment and serving. We then explore the application scenarios of LLMs in these procedures across a wider spectrum. In the "how" perspective, we align the abilities of LLMs with the requirements of each procedure. Finally, we point out the promising directions that could better leverage the strength of LLMs towards versatile graph learning methods.

cs.LG↗

arXiv2024

EventRL: Enhancing Event Extraction with Outcome Supervision for Large Language Models

In this study, we present EventRL, a reinforcement learning approach developed to enhance event extraction for large language models (LLMs). EventRL utilizes outcome supervision with specific reward functions to tackle prevalent challenges in LLMs, such as instruction following and hallucination, manifested as the mismatch of event structure and the generation of undefined event types. We evaluate EventRL against existing methods like Few-Shot Prompting (FSP) (based on GPT4) and Supervised Fine-Tuning (SFT) across various LLMs, including GPT-4, LLaMa, and CodeLLaMa models. Our findings show that EventRL significantly outperforms these conventional approaches by improving the performance in identifying and structuring events, particularly in handling novel event types. The study emphasizes the critical role of reward function selection and demonstrates the benefits of incorporating code data for better event extraction. While increasing model size leads to higher accuracy, maintaining the ability to generalize is essential to avoid overfitting.

cs.CL↗

arXiv2024

Automated calculation of Jet fragmentation at NLO in QCD

We present FMNLO, a framework to combine general-purpose Monte Carlo generators and fragmentation functions (FFs). It is based on a hybrid scheme of phase-space slicing method and local subtraction method, and accurate to next-to-leading order (NLO) in QCD. The new framework has been interfaced to MG5 aMC@NLO and made publicly available in this work. We demonstrate its unique ability by giving theoretical predictions of various fragmentation measurements at the LHC, followed by comparison with the data. With the help of interpolation techniques, FMNLO allows for fast calculation of fragmentation processes for a large number of different FFs, which makes it a promising tool for future fits of FFs. As an example, we perform a NLO fit of parton fragmentation functions to unidentified charged hadrons using measurements at the LHC. We find the ATLAS data from inclusive dijet production show a strong constraining power. Notable disparities are found between our gluon FF and that of BKK, DSS and NNFF, indicating the necessities of additional constraints and data for gluon fragmentation function.

hep-ph↗

arXiv2023

Zero-Shot Digital Rock Image Segmentation with a Fine-Tuned Segment Anything Model

Accurate image segmentation is crucial in reservoir modelling and material characterization, enhancing oil and gas extraction efficiency through detailed reservoir models. This precision offers insights into rock properties, advancing digital rock physics understanding. However, creating pixel-level annotations for complex CT and SEM rock images is challenging due to their size and low contrast, lengthening analysis time. This has spurred interest in advanced semi-supervised and unsupervised segmentation techniques in digital rock image analysis, promising more efficient, accurate, and less labour-intensive methods. Meta AI's Segment Anything Model (SAM) revolutionized image segmentation in 2023, offering interactive and automated segmentation with zero-shot capabilities, essential for digital rock physics with limited training data and complex image features. Despite its advanced features, SAM struggles with rock CT/SEM images due to their absence in its training set and the low-contrast nature of grayscale images. Our research fine-tunes SAM for rock CT/SEM image segmentation, optimizing parameters and handling large-scale images to improve accuracy. Experiments on rock CT and SEM images show that fine-tuning significantly enhances SAM's performance, enabling high-quality mask generation in digital rock image analysis. Our results demonstrate the feasibility and effectiveness of the fine-tuned SAM model (RockSAM) for rock images, offering segmentation without extensive training or complex labelling.

cs.CV↗

arXiv2023

Adaptive Shells for Efficient Neural Radiance Field Rendering

Neural radiance fields achieve unprecedented quality for novel view synthesis, but their volumetric formulation remains expensive, requiring a huge number of samples to render high-resolution images. Volumetric encodings are essential to represent fuzzy geometry such as foliage and hair, and they are well-suited for stochastic optimization. Yet, many scenes ultimately consist largely of solid surfaces which can be accurately rendered by a single sample per pixel. Based on this insight, we propose a neural radiance formulation that smoothly transitions between volumetric- and surface-based rendering, greatly accelerating rendering speed and even improving visual fidelity. Our method constructs an explicit mesh envelope which spatially bounds a neural volumetric representation. In solid regions, the envelope nearly converges to a surface and can often be rendered with a single sample. To this end, we generalize the NeuS formulation with a learned spatially-varying kernel size which encodes the spread of the density, fitting a wide kernel to volume-like regions and a tight kernel to surface-like regions. We then extract an explicit mesh of a narrow band around the surface, with width determined by the kernel size, and fine-tune the radiance field within this band. At inference time, we cast rays against the mesh and evaluate the radiance field only within the enclosed region, greatly reducing the number of samples required. Experiments show that our approach enables efficient rendering at very high fidelity. We also demonstrate that the extracted envelope enables downstream applications such as animation and simulation.

cs.CV↗

arXiv2023

Enhancing Rock Image Segmentation in Digital Rock Physics: A Fusion of Generative AI and State-of-the-Art Neural Networks

In digital rock physics, analysing microstructures from CT and SEM scans is crucial for estimating properties like porosity and pore connectivity. Traditional segmentation methods like thresholding and CNNs often fall short in accurately detailing rock microstructures and are prone to noise. U-Net improved segmentation accuracy but required many expert-annotated samples, a laborious and error-prone process due to complex pore shapes. Our study employed an advanced generative AI model, the diffusion model, to overcome these limitations. This model generated a vast dataset of CT/SEM and binary segmentation pairs from a small initial dataset. We assessed the efficacy of three neural networks: U-Net, Attention-U-net, and TransUNet, for segmenting these enhanced images. The diffusion model proved to be an effective data augmentation technique, improving the generalization and robustness of deep learning models. TransU-Net, incorporating Transformer structures, demonstrated superior segmentation accuracy and IoU metrics, outperforming both U-Net and Attention-U-net. Our research advances rock image segmentation by combining the diffusion model with cutting-edge neural networks, reducing dependency on extensive expert data and boosting segmentation accuracy and robustness. TransU-Net sets a new standard in digital rock physics, paving the way for future geoscience and engineering breakthroughs.

cs.CV↗

arXiv2023

Simulating Photosynthetic Energy Transport on a Photonic Network

Quantum effects in photosynthetic energy transport in nature, especially for the typical Fenna-Matthews-Olson (FMO) complexes, are extensively studied in quantum biology. Such energy transport processes can be investigated as open quantum systems that blend the quantum coherence and environmental noises, and have been experimentally simulated on a few quantum devices. However, the existing experiments always lack a solid quantum simulation for the FMO energy transport due to their constraints to map a variety of issues in actual FMO complexes that have rich biological meanings. Here we successfully map the full coupling profile of the seven-site FMO structure by comprehensive characterization and precise control of the evanescent coupling of the three-dimensional waveguide array. By applying a stochastic dynamical modulation on each waveguide, we introduce the base site energy and the dephasing term in colored noises to faithfully simulate the power spectral density of the FMO complexes. We show our photonic model well interprets the issues including the reorganization energy, vibrational assistance, exciton transfer and energy localization. We further experimentally demonstrate the existence of an optimal transport efficiency at certain dephasing strength, providing a window to closely investigate environment-assisted quantum transport.

quant-ph↗

arXiv2023

Coexistence of extended and localized states in finite-sized mosaic Wannier-Stark lattices

Quantum transport and localization are fundamental concepts in condensed matter physics. It is commonly believed that in one-dimensional systems, the existence of mobility edges is highly dependent on disorder. Recently, there has been a debate over the existence of an exact mobility edge in a modulated mosaic model without quenched disorder, the so-called mosaic Wannier-Stark lattice. Here, we experimentally implement such disorder-free mosaic photonic lattices using a silicon photonics platform. By creating a synthetic electric field, we could observe energy-dependent coexistence of both extended and localized states in a finite number of waveguides. The Wannier-Stark ladder emerges when the resulting potential is strong enough, and can be directly probed by exciting different spatial modes of the lattice. Our studies provide the experimental proof of coexisting sets of strongly localized and conducting (though weakly localized) states in finite-sized mosaic Wannier-Stark lattices, which hold the potential to encode high-dimensional quantum resources with compact and robust structures.

cond-mat.dis-nn↗