Searcharxiv⌕ Search

arXiv subjects

Qian Hu

Publications and source records attributed to Qian Hu.

At least 37 records · Page 2Linked to original sources
arXiv2024

FLIRT: Feedback Loop In-context Red Teaming

Warning: this paper contains content that may be inappropriate or offensive. As generative models become available for public use in various applications, testing and analyzing vulnerabilities of these models has become a priority. In this work, we propose an automatic red teaming framework that evaluates a given black-box model and exposes its vulnerabilities against unsafe and inappropriate content generation. Our framework uses in-context learning in a feedback loop to red team models and trigger them into unsafe content generation. In particular, taking text-to-image models as target models, we explore different feedback mechanisms to automatically learn effective and diverse adversarial prompts. Our experiments demonstrate that even with enhanced safety features, Stable Diffusion (SD) models are vulnerable to our adversarial prompts, raising concerns on their robustness in practical uses. Furthermore, we demonstrate that the proposed framework is effective for red teaming text-to-text models.

cs.AI↗
arXiv2024

A Detector-oblivious Multi-arm Network for Keypoint Matching

This paper presents a matching network to establish point correspondence between images. We propose a Multi-Arm Network (MAN) to learn region overlap and depth, which can greatly improve the keypoint matching robustness while bringing little computational cost during the inference stage. Another design that makes this framework different from many existing learning based pipelines that require re-training when a different keypoint detector is adopted, our network can directly work with different keypoint detectors without such a time-consuming re-training process. Comprehensive experiments conducted on outdoor and indoor datasets demonstrated that our proposed MAN outperforms state-of-the-art methods.

cs.CV↗
arXiv2024

GWnext 2024: Meeting Summary

GWnext 2024 was a meeting held in the Kavli Institute for Astronomy and Astrophysics at Peking University in March $4^\text{th} - 8^\text{th}$, 2024. In the meeting researchers at different career stages -- with a particular focus on early career scientists -- working on the different aspects of gravitational wave (GW) astronomy gathered to discuss the current status as well as prospects of the field. The meeting was divided into three core sessions: Astrophysics, GW Theory, and Detection. Each session consisted of introductory talks and extended discussion sessions. Moreover, there was a poster session where students could present their results. In this paper, we summarize the results presented during the meeting and present the most important outcomes.

astro-ph.HE↗
arXiv2024

Toward Informal Language Processing: Knowledge of Slang in Large Language Models

Recent advancement in large language models (LLMs) has offered a strong potential for natural language systems to process informal language. A representative form of informal language is slang, used commonly in daily conversations and online social media. To date, slang has not been comprehensively evaluated in LLMs due partly to the absence of a carefully designed and publicly accessible benchmark. Using movie subtitles, we construct a dataset that supports evaluation on a diverse set of tasks pertaining to automatic processing of slang. For both evaluation and finetuning, we show the effectiveness of our dataset on two core applications: 1) slang detection, and 2) identification of regional and historical sources of slang from natural sentences. We also show how our dataset can be used to probe the output distributions of LLMs for interpretive insights. We find that while LLMs such as GPT-4 achieve good performance in a zero-shot setting, smaller BERT-like models finetuned on our dataset achieve comparable performance. Furthermore, we show that our dataset enables finetuning of LLMs such as GPT-3.5 that achieve substantially better performance than strong zero-shot baselines. Our work offers a comprehensive evaluation and a high-quality benchmark on English slang based on the OpenSubtitles corpus, serving both as a publicly accessible resource and a platform for applying tools for informal language processing.

cs.CL↗
arXiv2024

Can Small Language Models Help Large Language Models Reason Better?: LM-Guided Chain-of-Thought

We introduce a novel framework, LM-Guided CoT, that leverages a lightweight (i.e., <1B) language model (LM) for guiding a black-box large (i.e., >10B) LM in reasoning tasks. Specifically, the lightweight LM first generates a rationale for each input instance. The Frozen large LM is then prompted to predict a task output based on the rationale generated by the lightweight LM. Our approach is resource-efficient in the sense that it only requires training the lightweight LM. We optimize the model through 1) knowledge distillation and 2) reinforcement learning from rationale-oriented and task-oriented reward signals. We assess our method with multi-hop extractive question answering (QA) benchmarks, HotpotQA, and 2WikiMultiHopQA. Experimental results show that our approach outperforms all baselines regarding answer prediction accuracy. We also find that reinforcement learning helps the model to produce higher-quality rationales with improved QA performance.

cs.CL↗
arXiv2023

Faithful Model Evaluation for Model-Based Metrics

Statistical significance testing is used in natural language processing (NLP) to determine whether the results of a study or experiment are likely to be due to chance or if they reflect a genuine relationship. A key step in significance testing is the estimation of confidence interval which is a function of sample variance. Sample variance calculation is straightforward when evaluating against ground truth. However, in many cases, a metric model is often used for evaluation. For example, to compare toxicity of two large language models, a toxicity classifier is used for evaluation. Existing works usually do not consider the variance change due to metric model errors, which can lead to wrong conclusions. In this work, we establish the mathematical foundation of significance testing for model-based metrics. With experiments on public benchmark datasets and a production system, we show that considering metric model errors to calculate sample variances for model-based metrics changes the conclusions in certain experiments.

cs.CL↗
arXiv2023

Rapid pre-merger localization of binary neutron stars in third generation gravitational wave detectors

Pre-merger localization of binary neutron stars (BNSs) is one of the most important scientific goals for the third generation (3G) gravitational wave (GW) detectors. It will enable the electromagnetic observation of the whole process of BNS coalescence, especially for the pre-merger and merger phases which have not been observed yet, opening a window for deeper understandings of compact objects. To reach this goal, we describe a novel combination of multi-band matched filtering and semi-analytical localization algorithms to achieve early-warning localization of long BNS signals in 3G detectors. Using our method we are able to efficiently simulate one month of observations with a three-detector 3G network, and show that it is possible to provide accurate sky localizations more than 30 minutes before the merger. Our simulation shows that there could be ~ 10 (~ 100) BNS events localized within 100 deg2, 20 (6) minutes before merger, per month of observation.

gr-qc↗
arXiv2023

Evaluating Large Language Models on Controlled Generation Tasks

While recent studies have looked into the abilities of large language models in various benchmark tasks, including question generation, reading comprehension, multilingual and etc, there have been few studies looking into the controllability of large language models on generation tasks. We present an extensive analysis of various benchmarks including a sentence planning benchmark with different granularities. After comparing large language models against state-of-the-start finetuned smaller models, we present a spectrum showing large language models falling behind, are comparable, or exceed the ability of smaller models. We conclude that **large language models struggle at meeting fine-grained hard constraints**.

cs.CL↗
arXiv2023

Precessing Binary Black Holes as Better Dark Sirens

Gravitational waves (GWs) from binary black hole mergers provide unique opportunities for cosmological inference such as standard sirens. However, the accurate determination of the luminosity distance of the event is limited by the correlation between the distance and the angle between the binary's orbital angular momentum and the observer's line of sight. In the letter, we investigate the effect of precession on the distance estimation of binary black hole events for the third-generation (3G) GW detectors. We find that the precession can enhance the precision of distance inference by one order of magnitude compared to the scenario where precession is absent. The constraint on the host galaxies can be improved due to the improved distance measurement, therefore the Hubble constant can be measured with higher precision and accuracy. These findings underscore the noteworthy impact of precession on the precision of distance estimation for 3G ground-based GW detectors, which can serve as highly accurate probes of the Universe.

gr-qc↗
arXiv2023

Accumulating errors in tests of general relativity with gravitational waves: overlapping signals and inaccurate waveforms

Observations of gravitational waves (GWs) from compact binary coalescences provide powerful tests of general relativity (GR), but systematic errors in data analysis could lead to incorrect scientific conclusions. This issue is especially serious in the third-generation GW detectors in which the signal-to-noise ratio (SNR) is high and the number of detections is large. In this work, we investigate the impacts of overlapping signals and inaccurate waveform models on tests of GR. We simulate mock catalogs for Einstein Telescope and Cosmic Explorer and perform parametric tests of GR using waveform models with different levels of inaccuracy. We find the systematic error in non-GR parameter estimates could accumulate toward a false deviation from GR when combining results from multiple events, although a bayesian model selection analysis may not favour a deviation. Waveform inaccuracies contribute most to the systematic errors, but multiple overlapping signals could magnify the effects of systematics due to the incorrect removal of signals. We also point out that testing GR using selected ''golden binaries'' with high SNR is even more vulnerable to false deviations from GR. The problem of error accumulation is universal; we emphasize that it must be addressed to fully exploit the data from third-generation GW detectors, and that further investigations, particularly in waveform accuracy, will be essential.

gr-qc↗
arXiv2022

Is the Elephant Flying? Resolving Ambiguities in Text-to-Image Generative Models

Natural language often contains ambiguities that can lead to misinterpretation and miscommunication. While humans can handle ambiguities effectively by asking clarifying questions and/or relying on contextual cues and common-sense knowledge, resolving ambiguities can be notoriously hard for machines. In this work, we study ambiguities that arise in text-to-image generative models. We curate a benchmark dataset covering different types of ambiguities that occur in these systems. We then propose a framework to mitigate ambiguities in the prompts given to the systems by soliciting clarifications from the user. Through automatic and human evaluations, we show the effectiveness of our framework in generating more faithful images aligned with human intention in the presence of ambiguities.

cs.CL↗
arXiv2022

Assessing the model waveform accuracy of gravitational waves

With the improvement in sensitivity of gravitational wave (GW) detectors and the increasing diversity of GW sources, there is a strong need for accurate GW waveform models for data analysis. While the current model accuracy assessments require waveforms generated by numerical relativity (NR) simulations as the "true waveforms", in this paper we propose an assessment approach that does not require NR simulations, which enables us to assess model accuracy everywhere in the parameter space. By measuring the difference between two waveform models, we derive a necessary condition for a pair of waveform models to both be accurate, for a particular set of parameters. We then apply this method to the parameter estimation samples of the Gravitational-Wave Transient Catalogs GWTC-3 and GWTC-2.1, and find that the waveform accuracy for high signal-to-noise ratio events in some cases fails our assessment criterion. Based on analysis of real events' posterior samples, we discuss the correlation between our quantified accuracy assessments and systematic errors in parameter estimation. We find waveform models that perform worse in our assessment are more likely to give inconsistent estimations. We also investigate waveform accuracy in different parameter regions, and find the accuracy degrades as the spin effects go up, the mass ratio deviates from one, or the orbital plane is near-aligned to the line of sight. Furthermore, we make predictions of waveform accuracy requirements for future detectors and find the accuracy of current waveform models should be improved by at least 3 orders of magnitude, which is consistent with previous works.

gr-qc↗
arXiv2022

Balancing the trade-off between cost and reliability for wireless sensor networks: a multi-objective optimized deployment method

The deployment of the sensor nodes (SNs) always plays a decisive role in the system performance of wireless sensor networks (WSNs). In this work, we propose an optimal deployment method for practical heterogeneous WSNs which gives a deep insight into the trade-off between the reliability and deployment cost. Specifically, this work aims to provide the optimal deployment of SNs to maximize the coverage degree and connection degree, and meanwhile minimize the overall deployment cost. In addition, this work fully considers the heterogeneity of SNs (i.e. differentiated sensing range and deployment cost) and three-dimensional (3-D) deployment scenarios. This is a multi-objective optimization problem, non-convex, multimodal and NP-hard. To solve it, we develop a novel swarm-based multi-objective optimization algorithm, known as the competitive multi-objective marine predators algorithm (CMOMPA) whose performance is verified by comprehensive comparative experiments with ten other stateof-the-art multi-objective optimization algorithms. The computational results demonstrate that CMOMPA is superior to others in terms of convergence and accuracy and shows excellent performance on multimodal multiobjective optimization problems. Sufficient simulations are also conducted to evaluate the effectiveness of the CMOMPA based optimal SNs deployment method. The results show that the optimized deployment can balance the trade-off among deployment cost, sensing reliability and network reliability. The source code is available on https://github.com/iNet-WZU/CMOMPA.

cs.NI↗
arXiv2022

Predicting vacant parking space availability zone-wisely: a graph based spatio-temporal prediction approach

Vacant parking space (VPS) prediction is one of the key issues of intelligent parking guidance systems. Accurately predicting VPS information plays a crucial role in intelligent parking guidance systems, which can help drivers find parking space quickly, reducing unnecessary waste of time and excessive environmental pollution. Through the simple analysis of historical data, we found that there not only exists a obvious temporal correlation in each parking lot, but also a clear spatial correlation between different parking lots. In view of this, this paper proposed a graph data-based model ST-GBGRU (Spatial-Temporal Graph Based Gated Recurrent Unit), the number of VPSs can be predicted both in short-term (i.e., within 30 min) and in long-term (i.e., over 30min). On the one hand, the temporal correlation of historical VPS data is extracted by GRU, on the other hand, the spatial correlation of historical VPS data is extracted by GCN inside GRU. Two prediction methods, namely direct prediction and iterative prediction, are combined with the proposed model. Finally, the prediction model is applied to predict the number VPSs of 8 public parking lots in Santa Monica. The results show that in the short-term and long-term prediction tasks, ST-GBGRU model can achieve high accuracy and have good application prospects.

cs.LG↗
arXiv2022

OPP-Miner: Order-preserving sequential pattern mining

A time series is a collection of measurements in chronological order. Discovering patterns from time series is useful in many domains, such as stock analysis, disease detection, and weather forecast. To discover patterns, existing methods often convert time series data into another form, such as nominal/symbolic format, to reduce dimensionality, which inevitably deviates the data values. Moreover, existing methods mainly neglect the order relationships between time series values. To tackle these issues, inspired by order-preserving matching, this paper proposes an Order-Preserving sequential Pattern (OPP) mining method, which represents patterns based on the order relationships of the time series data. An inherent advantage of such representation is that the trend of a time series can be represented by the relative order of the values underneath the time series data. To obtain frequent trends in time series, we propose the OPP-Miner algorithm to mine patterns with the same trend (sub-sequences with the same relative order). OPP-Miner employs the filtration and verification strategies to calculate the support and uses pattern fusion strategy to generate candidate patterns. To compress the result set, we also study finding the maximal OPPs. Experiments validate that OPP-Miner is not only efficient and scalable but can also discover similar sub-sequences in time series. In addition, case studies show that our algorithms have high utility in analyzing the COVID-19 epidemic by identifying critical trends and improve the clustering performance.

cs.DB↗
arXiv2021

Network Effects Research: A Systematic Review of Theoretical Mechanisms and Measures

This article contributes to the network effectiveness literature by identifying the theoretical mechanisms and network measures scholars in public administration and policy use to draw inferences between network structures and network effects. We conducted a systematic review of empirical network effects research in 40 journals of public administration and policy from 1998 to 2019. We reviewed and coded 89 articles and described the main social theories used in the network effectiveness literature and the associated mechanisms that translate network structures to network effects. We also explain how scholars operationalize those theoretical mechanisms through network measures. Overall, our findings reflect that there is a limited use of social theories for the explanation of network effects and in some cases, there is an inconsistent use of network measures associated with theories. Moreover, we identify the main challenges related to network effects, including the difficulty of isolating specific mechanisms related to a particular social theory, the use of network structures both as a mechanism and as a measure, and the lack of data to examine network dynamics and coevolution.

cs.SI↗
arXiv2021

Semianalytical Approach for Sky Localization of Gravitational Waves

Rapid sky localization of gravitational wave sources is crucial to enable prompt electromagnetic follow-ups. In this article, we present a novel semianalytical approach for sky localization of gravitational waves from compact binary coalescences. We use the Bayesian framework with an analytical approximation to the prior distributions for a given astrophysical model. We derive a semianalytical solution to the posterior distribution of source directions. This method only requires one-fold numerical integral that marginalizes over the merger time, compared to the five-fold numerical integration otherwise needed in the Bayesian localization method. The performance of the method is demonstrated using a set of binary neutron stars (BNS) injections on Gaussian noise using LIGO-Virgo's design and O2 sensitivity. We find the median of 90% confidence area in O2 sensitivity to be $\mathcal{O}(10^2) ~\mathrm{deg}^2$, comparable to that of the existing LIGO-Virgo online localization method Bayestar and parameter estimation toolkit LALInference. In the end, we apply this method to localize the BNS event GW170817 and find the 50% (90%) confidence region of 11 $\mathrm{deg}^2$ (50 $\mathrm{deg}^2$). The detected optical counterpart of GW170817 resides within our 50% confidence area.

gr-qc↗