arXiv subjects

Jiazheng Li

Publications and source records attributed to Jiazheng Li.

54 records · Page 3Linked to original sources

arXiv2023

OverPrompt: Enhancing ChatGPT through Efficient In-Context Learning

The remarkable performance of pre-trained large language models has revolutionised various natural language processing applications. Due to huge parametersizes and extensive running costs, companies or organisations tend to transfer the models to the target task by zero-shot prompting techniques. However, the prohibitive costs of tokens and time have hindered their adoption in applications. We propose OverPrompt, leveraging the in-context learning capability of LLMs to handle multiple task inputs, thereby reducing token and time costs. This approach could potentially improve task performance during API queries due to better conditional distribution mapping. Evaluated across diverse classification datasets, our experiments show that OverPrompt can achieve cost-efficient zero-shot classification without causing significant detriment to task performance, and in some cases, even improving it. An ablation study conducted on various LLMs, along with an investigation into the robustness of our prompting strategy to different input ordering, offers valuable insights into the broader applicability of our method across diverse tasks. These findings also suggest a more seamless integration of our method with LLMs through an API.

cs.CL↗

arXiv2023

Distilling ChatGPT for Explainable Automated Student Answer Assessment

Providing explainable and faithful feedback is crucial for automated student answer assessment. In this paper, we introduce a novel framework that explores using ChatGPT, a cutting-edge large language model, for the concurrent tasks of student answer scoring and rationale generation. We identify the appropriate instructions by prompting ChatGPT with different templates to collect the rationales, where inconsistent rationales are refined to align with marking standards. The refined ChatGPT outputs enable us to fine-tune a smaller language model that simultaneously assesses student answers and provides rationales. Extensive experiments on the benchmark dataset show that the proposed method improves the overall QWK score by 11% compared to ChatGPT. Furthermore, our thorough analysis and human evaluation demonstrate that the rationales generated by our proposed method are comparable to those of ChatGPT. Our approach provides a viable solution to achieve explainable automated assessment in education. Code available at https://github.com/lijiazheng99/aera.

cs.CL↗

arXiv2023

NarrativePlay: Interactive Narrative Understanding

In this paper, we introduce NarrativePlay, a novel system that allows users to role-play a fictional character and interact with other characters in narratives such as novels in an immersive environment. We leverage Large Language Models (LLMs) to generate human-like responses, guided by personality traits extracted from narratives. The system incorporates auto-generated visual display of narrative settings, character portraits, and character speech, greatly enhancing user experience. Our approach eschews predefined sandboxes, focusing instead on main storyline events extracted from narratives from the perspective of a user-selected character. NarrativePlay has been evaluated on two types of narratives, detective and adventure stories, where users can either explore the world or improve their favorability with the narrative characters through conversations.

cs.CL↗

arXiv2023

CUE: An Uncertainty Interpretation Framework for Text Classifiers Built on Pre-Trained Language Models

Text classifiers built on Pre-trained Language Models (PLMs) have achieved remarkable progress in various tasks including sentiment analysis, natural language inference, and question-answering. However, the occurrence of uncertain predictions by these classifiers poses a challenge to their reliability when deployed in practical applications. Much effort has been devoted to designing various probes in order to understand what PLMs capture. But few studies have delved into factors influencing PLM-based classifiers' predictive uncertainty. In this paper, we propose a novel framework, called CUE, which aims to interpret uncertainties inherent in the predictions of PLM-based models. In particular, we first map PLM-encoded representations to a latent space via a variational auto-encoder. We then generate text representations by perturbing the latent space which causes fluctuation in predictive uncertainty. By comparing the difference in predictive uncertainty between the perturbed and the original text representations, we are able to identify the latent dimensions responsible for uncertainty and subsequently trace back to the input features that contribute to such uncertainty. Our extensive experiments on four benchmark datasets encompassing linguistic acceptability classification, emotion classification, and natural language inference show the feasibility of our proposed framework. Our source code is available at: https://github.com/lijiazheng99/CUE.

cs.CL↗

arXiv2023

propnet: Propagating 2D Annotation to 3D Segmentation for Gastric Tumors on CT Scans

**Background:** Accurate 3D CT scan segmentation of gastric tumors is pivotal for diagnosis and treatment. The challenges lie in the irregular shapes, blurred boundaries of tumors, and the inefficiency of existing methods. **Purpose:** We conducted a study to introduce a model, utilizing human-guided knowledge and unique modules, to address the challenges of 3D tumor segmentation. **Methods:** We developed the PropNet framework, propagating radiologists' knowledge from 2D annotations to the entire 3D space. This model consists of a proposing stage for coarse segmentation and a refining stage for improved segmentation, using two-way branches for enhanced performance and an up-down strategy for efficiency. **Results:** With 98 patient scans for training and 30 for validation, our method achieves a significant agreement with manual annotation (Dice of 0.803) and improves efficiency. The performance is comparable in different scenarios and with various radiologists' annotations (Dice between 0.785 and 0.803). Moreover, the model shows improved prognostic prediction performance (C-index of 0.620 vs. 0.576) on an independent validation set of 42 patients with advanced gastric cancer. **Conclusions:** Our model generates accurate tumor segmentation efficiently and stably, improving prognostic performance and reducing high-throughput image reading workload. This model can accelerate the quantitative analysis of gastric tumors and enhance downstream task performance.

eess.IV↗

arXiv2023

NapSS: Paragraph-level Medical Text Simplification via Narrative Prompting and Sentence-matching Summarization

Accessing medical literature is difficult for laypeople as the content is written for specialists and contains medical jargon. Automated text simplification methods offer a potential means to address this issue. In this work, we propose a summarize-then-simplify two-stage strategy, which we call NapSS, identifying the relevant content to simplify while ensuring that the original narrative flow is preserved. In this approach, we first generate reference summaries via sentence matching between the original and the simplified abstracts. These summaries are then used to train an extractive summarizer, learning the most relevant content to be simplified. Then, to ensure the narrative consistency of the simplified text, we synthesize auxiliary narrative prompts combining key phrases derived from the syntactical analyses of the original text. Our model achieves results significantly better than the seq2seq baseline on an English medical corpus, yielding 3%~4% absolute improvements in terms of lexical similarity, and providing a further 1.1% improvement of SARI score when combined with the baseline. We also highlight shortcomings of existing evaluation methods, and introduce new metrics that take into account both lexical and high-level semantic similarity. A human evaluation conducted on a random sample of the test set further establishes the effectiveness of the proposed approach. Codes and models are released here: https://github.com/LuJunru/NapSS.

cs.CL↗

arXiv2023

Data-Driven Distributionally Robust Scheduling of Community Integrated Energy Systems with Uncertain Renewable Generations Considering Integrated Demand Response

A community integrated energy system (CIES) is an important carrier of the energy internet and smart city in geographical and functional terms. Its emergence provides a new solution to the problems of energy utilization and environmental pollution. To coordinate the integrated demand response and uncertainty of renewable energy generation (RGs), a data-driven two-stage distributionally robust optimization (DRO) model is constructed. A comprehensive norm consisting of the 1-norm and infinity-norm is used as the uncertainty probability distribution information set, thereby avoiding complex probability density information. To address multiple uncertainties of RGs, a generative adversarial network based on the Wasserstein distance with gradient penalty is proposed to generate RG scenarios, which has wide applicability. To further tap the potential of the demand response, we take into account the ambiguity of human thermal comfort and the thermal inertia of buildings. Thus, an integrated demand response mechanism is developed that effectively promotes the consumption of renewable energy. The proposed method is simulated in an actual CIES in North China. In comparison with traditional stochastic programming and robust optimization, it is verified that the proposed DRO model properly balances the relationship between economical operation and robustness while exhibiting stronger adaptability. Furthermore, our approach outperforms other commonly used DRO methods with better operational economy, lower renewable power curtailment rate, and higher computational efficiency.

eess.SY↗

arXiv2022

PHEE: A Dataset for Pharmacovigilance Event Extraction from Text

The primary goal of drug safety researchers and regulators is to promptly identify adverse drug reactions. Doing so may in turn prevent or reduce the harm to patients and ultimately improve public health. Evaluating and monitoring drug safety (i.e., pharmacovigilance) involves analyzing an ever growing collection of spontaneous reports from health professionals, physicians, and pharmacists, and information voluntarily submitted by patients. In this scenario, facilitating analysis of such reports via automation has the potential to rapidly identify safety signals. Unfortunately, public resources for developing natural language models for this task are scant. We present PHEE, a novel dataset for pharmacovigilance comprising over 5000 annotated events from medical case reports and biomedical literature, making it the largest such public dataset to date. We describe the hierarchical event schema designed to provide coarse and fine-grained information about patients' demographics, treatments and (side) effects. Along with the discussion of the dataset, we present a thorough experimental evaluation of current state-of-the-art approaches for biomedical event extraction, point out their limitations, and highlight open challenges to foster future research in this area.

cs.CL↗

arXiv2022

AQP: An Open Modular Python Platform for Objective Speech and Audio Quality Metrics

Audio quality assessment has been widely researched in the signal processing area. Full-reference objective metrics (e.g., POLQA, ViSQOL) have been developed to estimate the audio quality relying only on human rating experiments. To evaluate the audio quality of novel audio processing techniques, researchers constantly need to compare objective quality metrics. Testing different implementations of the same metric and evaluating new datasets are fundamental and ongoing iterative activities. In this paper, we present AQP - an open-source, node-based, light-weight Python pipeline for audio quality assessment. AQP allows researchers to test and compare objective quality metrics helping to improve robustness, reproducibility and development speed. We introduce the platform, explain the motivations, and illustrate with examples how, using AQP, objective quality metrics can be (i) compared and benchmarked; (ii) prototyped and adapted in a modular fashion; (iii) visualised and checked for errors. The code has been shared on GitHub to encourage adoption and contributions from the community.

cs.SD↗

arXiv2022

Exploring the Efficacy of Automatically Generated Counterfactuals for Sentiment Analysis

While state-of-the-art NLP models have been achieving the excellent performance of a wide range of tasks in recent years, important questions are being raised about their robustness and their underlying sensitivity to systematic biases that may exist in their training and test data. Such issues come to be manifest in performance problems when faced with out-of-distribution data in the field. One recent solution has been to use counterfactually augmented datasets in order to reduce any reliance on spurious patterns that may exist in the original data. Producing high-quality augmented data can be costly and time-consuming as it usually needs to involve human feedback and crowdsourcing efforts. In this work, we propose an alternative by describing and evaluating an approach to automatically generating counterfactual data for data augmentation and explanation. A comprehensive evaluation on several different datasets and using a variety of state-of-the-art benchmarks demonstrate how our approach can achieve significant improvements in model performance when compared to models training on the original data and even when compared to models trained with the benefit of human-generated augmented data.

cs.CL↗

arXiv2022

Assessing Planetary Complexity and Potential Agnostic Biosignatures using Epsilon Machines

We present a new approach to exoplanet characterisation using techniques from complexity science, with potential applications to biosignature detection. This agnostic method makes use of the temporal variability of light reflected or emitted from a planet. We use a technique known as epsilon machine reconstruction to compute the statistical complexity, a measure of the minimal model size for time series data. We demonstrate that statistical complexity is an effective measure of the complexity of planetary features. Increasing levels of qualitative planetary complexity correlate with increases in statistical complexity and Shannon entropy, demonstrating that our approach can identify planets with the richest dynamics. We also compare Earth time series with Jupiter data, and find that for the three wavelengths considered, Earth's average complexity and entropy rate are approximately 50% and 43% higher than Jupiter's, respectively. The majority of schemes for the detection of extraterrestrial life rely upon biochemical signatures and planetary context. However, it is increasingly recognised that extraterrestrial life could be very different to life on Earth. Under the hypothesis that there is a correlation between the presence of a biosphere and observable planetary complexity, our technique offers an agnostic and quantitative method for the measurement thereof.

astro-ph.EP↗

arXiv2022

NumHTML: Numeric-Oriented Hierarchical Transformer Model for Multi-task Financial Forecasting

Financial forecasting has been an important and active area of machine learning research because of the challenges it presents and the potential rewards that even minor improvements in prediction accuracy or forecasting may entail. Traditionally, financial forecasting has heavily relied on quantitative indicators and metrics derived from structured financial statements. Earnings conference call data, including text and audio, is an important source of unstructured data that has been used for various prediction tasks using deep earning and related approaches. However, current deep learning-based methods are limited in the way that they deal with numeric data; numbers are typically treated as plain-text tokens without taking advantage of their underlying numeric structure. This paper describes a numeric-oriented hierarchical transformer model to predict stock returns, and financial risk using multi-modal aligned earnings calls data by taking advantage of the different categories of numbers (monetary, temporal, percentages etc.) and their magnitude. We present the results of a comprehensive evaluation of NumHTML against several state-of-the-art baselines using a real-world publicly available dataset. The results indicate that NumHTML significantly outperforms the current state-of-the-art across a variety of evaluation metrics and that it has the potential to offer significant financial gains in a practical trading context.

cs.LG↗

arXiv2021

Hierarchical Stochastic Scheduling of Multi-Community Integrated Energy Systems in Uncertain Environments via Stackelberg Game

An operating entity utilizing community-integrated energy systems with a large number of small-scale distributed energy sources can easily trade with existing distribution markets. To solve the energy management and pricing problem of multi-community integrated energy systems (MCIESs) with multi-energy interaction, this study investigated a hierarchical stochastic optimal scheduling method for uncertain environments. To handle multiple uncertainties, a Wasserstein generative adversarial network with a gradient penalty was used to generate renewable scenarios, and the Kmeans++ clustering algorithm was employed to generate typical scenarios. A Stackelberg-based hierarchical stochastic schedule with an integrated demand response was constructed, where the MCIES operator acted as the leader pursuing the maximum net profit by setting energy prices, while the building users were followers who adjusted their energy consumption plans to minimize their total costs. Finally, a distributed iterative solution method based on a metaheuristic was designed. The effectiveness of the proposed method was verified using practical examples.

eess.SY↗

arXiv2021

Optimal Scheduling of Integrated Demand Response-Enabled Community Integrated Energy Systems in Uncertain Environments

The community integrated energy system (CIES) is an essential energy internet carrier that has recently been the focus of much attention. A scheduling model based on chance-constrained programming is proposed for integrated demand response (IDR)-enabled CIES in uncertain environments to minimize the system operating costs, where an IDR program is used to explore the potential interaction ability of electricity-gas-heat flexible loads and electric vehicles. Moreover, power to gas (P2G) and micro-gas turbine (MT), as links of multi-energy carriers, are adopted to strengthen the coupling of different energy subsystems. Sequence operation theory (SOT) and linearization methods are employed to transform the original model into a solvable mixed-integer linear programming model. Simulation results on a practical CIES in North China demonstrate an improvement in the CIES operational economy via the coordination of IDR and renewable uncertainties, with P2G and MT enhancing the system operational flexibility and user comprehensive satisfaction. The CIES operation is able to achieve a trade-off between economy and system reliability by setting a suitable confidence level for the spinning reserve constraints. Besides, the proposed solution method outperforms the Hybrid Intelligent Algorithm in terms of both optimization results and calculation efficiency.

eess.SY↗

arXiv2021

Privacy-preserving Spatiotemporal Scenario Generation of Renewable Energies: A Federated Deep Generative Learning Approach

Scenario generation is a fundamental and crucial tool for decision-making in power systems with high-penetration renewables. Based on big historical data, a novel federated deep generative learning framework, called Fed-LSGAN, is proposed by integrating federated learning and least square generative adversarial networks (LSGANs) for renewable scenario generation. Specifically, federated learning learns a shared global model in a central server from renewable sites at network edges, which enables the Fed-LSGAN to generate scenarios in a privacy-preserving manner without sacrificing the generation quality by transferring model parameters, rather than all data. Meanwhile, the LSGANs-based deep generative model generates scenarios that conform to the distribution of historical data through fully capturing the spatial-temporal characteristics of renewable powers, which leverages the least squares loss function to improve the training stability and generation quality. The simulation results demonstrate that the proposal manages to generate high-quality renewable scenarios and outperforms the state-of-the-art centralized methods. Besides, an experiment with different federated learning settings is designed and conducted to verify the robustness of our method.

cs.LG↗

arXiv2021

Deep Learning for Short-Term Voltage Stability Assessment of Power Systems

To fully learn the latent temporal dependencies from post-disturbance system dynamic trajectories, deep learning is utilized for short-term voltage stability (STVS) assessment of power systems in this paper. First of all, a semi-supervised cluster algorithm is performed to obtain class labels of STVS instances due to the unavailability of reliable quantitative criteria. Secondly, a long short-term memory (LSTM) based assessment model is built through learning the time dependencies from the post-disturbance system dynamics. Finally, the trained assessment model is employed to determine the systems stability status in real time. The test results on the IEEE 39-bus system suggest that the proposed approach manages to assess the stability status of the system accurately and timely. Furthermore, the superiority of the proposed method over traditional shallow learning-based assessment methods has also been proved.

eess.SP↗

arXiv2020

Earth as a Proxy Exoplanet: Deconstructing and Reconstructing Spectrophotometric Light Curves

Point source spectrophotometric ("single-point") light curves of Earth-like planets contain a surprising amount of information about the spatial features of those worlds. Spatially resolving these light curves is important for assessing time-varying surface features and the existence of an atmosphere, which in turn is critical to life on Earth and significant for determining habitability on exoplanets. Given that Earth is the only celestial body confirmed to harbor life, treating it as a proxy exoplanet by analyzing time-resolved spectral images provides a benchmark in the search for habitable exoplanets. The Earth Polychromatic Imaging Camera (EPIC) on the Deep Space Climate Observatory (DSCOVR) provides such an opportunity, with observations of ~5000 full-disk sunlit Earth images each year at ten wavelengths with high temporal frequency. We disk-integrate these spectral images to create single-point light curves and decompose them into principal components (PCs). Using machine learning techniques to relate the PCs to six preselected spatial features, we find that the first and fourth PCs of the single-point light curves, contributing ~83.23% of the light curve variability, contain information about low and high clouds, respectively. Surface information relevant to the contrast between land and ocean reflectance is contained in the second PC, while individual land sub-types are not easily distinguishable (<0.1% total light curve variation). We build an Earth model by systematically altering the spatial features to derive causal relationships to the PCs. This model can serve as a baseline for analyzing Earth-like exoplanets and guide wavelength selection and sampling strategies for future observations.

astro-ph.EP↗

arXiv2018

Using Deep Space Climate Observatory Measurements to Study the Earth as An Exoplanet

Even though it was not designed as an exoplanetary research mission, the Deep Space Climate Observatory (DSCOVR) has been opportunistically used for a novel experiment, in which Earth serves as a proxy exoplanet. More than two years of DSCOVR Earth images were employed to produce time series of multi-wavelength, single-point light sources, in order to extract information on planetary rotation, cloud patterns, surface type, and orbit around the Sun. In what follows, we assume that these properties of the Earth are unknown, and instead attempt to derive them from first principles. These conclusions are then compared with known data about our planet. We also used the DSCOVR data to simulate phase angle changes, as well as the minimum data collection rate needed to determine the rotation period of an exoplanet. This innovative method of using the time evolution of a multi-wavelength, reflected single-point light source, can be deployed for retrieving a range of intrinsic properties of an exoplanet around a distant star.

astro-ph.EP↗