Searcharxiv⌕ Search

arXiv subjects

Anqi Li

Publications and source records attributed to Anqi Li.

At least 91 records · Page 5Linked to original sources
arXiv2023

General Method for Solving Four Types of SAT Problems

Existing methods provide varying algorithms for different types of Boolean satisfiability problems (SAT), lacking a general solution framework. Accordingly, this study proposes a unified framework DCSAT based on integer programming and reinforcement learning (RL) algorithm to solve different types of SAT problems such as MaxSAT, Weighted MaxSAT, PMS, WPMS. Specifically, we first construct a consolidated integer programming representation for four types of SAT problems by adjusting objective function coefficients. Secondly, we construct an appropriate reinforcement learning models based on the 0-1 integer programming for SAT problems. Based on the binary tree search structure, we apply the Monte Carlo tree search (MCTS) method on SAT problems. Finally, we prove that this method can find all optimal Boolean assignments based on Wiener-khinchin law of large Numbers. We experimentally verify that this paradigm can prune the unnecessary search space to find the optimal Boolean assignments for the problem. Furthermore, the proposed method can provide diverse labels for supervised learning methods for SAT problems.

cs.AI↗
arXiv2023

Survival Instinct in Offline Reinforcement Learning

We present a novel observation about the behavior of offline reinforcement learning (RL) algorithms: on many benchmark datasets, offline RL can produce well-performing and safe policies even when trained with "wrong" reward labels, such as those that are zero everywhere or are negatives of the true rewards. This phenomenon cannot be easily explained by offline RL's return maximization objective. Moreover, it gives offline RL a degree of robustness that is uncharacteristic of its online RL counterparts, which are known to be sensitive to reward design. We demonstrate that this surprising robustness property is attributable to an interplay between the notion of pessimism in offline RL algorithms and certain implicit biases in common data collection practices. As we prove in this work, pessimism endows the agent with a "survival instinct", i.e., an incentive to stay within the data support in the long term, while the limited and biased data coverage further constrains the set of survival policies. Formally, given a reward class -- which may not even contain the true reward -- we identify conditions on the training data distribution that enable offline RL to learn a near-optimal and safe policy from any reward within the class. We argue that the survival instinct should be taken into account when interpreting results from existing offline RL benchmarks and when creating future ones. Our empirical and theoretical results suggest a new paradigm for RL, whereby an agent is nudged to learn a desirable behavior with imperfect reward but purposely biased data coverage.

cs.LG↗
arXiv2023

On the optimal pivot path of simplex method for linear programming based on reinforcement learning

Based on the existing pivot rules, the simplex method for linear programming is not polynomial in the worst case. Therefore the optimal pivot of the simplex method is crucial. This study proposes the optimal rule to find all shortest pivot paths of the simplex method for linear programming problems based on Monte Carlo tree search (MCTS). Specifically, we first propose the SimplexPseudoTree to transfer the simplex method into tree search mode while avoiding repeated basis variables. Secondly, we propose four reinforcement learning (RL) models with two actions and two rewards to make the Monte Carlo tree search suitable for the simplex method. Thirdly, we set a new action selection criterion to ameliorate the inaccurate evaluation in the initial exploration. It is proved that when the number of vertices in the feasible region is $C_n^m$, our method can generate all the shortest pivot paths, which is the polynomial of the number of variables. In addition, we experimentally validate that the proposed schedule can avoid unnecessary search and provide the optimal pivot path. Furthermore, this method can provide the best pivot labels for all kinds of supervised learning methods to solve linear programming problems.

math.OC↗
arXiv2023

Latent Jailbreak: A Benchmark for Evaluating Text Safety and Output Robustness of Large Language Models

Considerable research efforts have been devoted to ensuring that large language models (LLMs) align with human values and generate safe text. However, an excessive focus on sensitivity to certain topics can compromise the model's robustness in following instructions, thereby impacting its overall performance in completing tasks. Previous benchmarks for jailbreaking LLMs have primarily focused on evaluating the safety of the models without considering their robustness. In this paper, we propose a benchmark that assesses both the safety and robustness of LLMs, emphasizing the need for a balanced approach. To comprehensively study text safety and output robustness, we introduce a latent jailbreak prompt dataset, each involving malicious instruction embedding. Specifically, we instruct the model to complete a regular task, such as translation, with the text to be translated containing malicious instructions. To further analyze safety and robustness, we design a hierarchical annotation framework. We present a systematic analysis of the safety and robustness of LLMs regarding the position of explicit normal instructions, word replacements (verbs in explicit normal instructions, target groups in malicious instructions, cue words for explicit normal instructions), and instruction replacements (different explicit normal instructions). Our results demonstrate that current LLMs not only prioritize certain instruction verbs but also exhibit varying jailbreak rates for different instruction verbs in explicit normal instructions. Code and data are available at https://github.com/qiuhuachuan/latent-jailbreak.

cs.CL↗
arXiv2023

Quantum Thermodynamics: Inside-Outside Perspective

We introduce an energy-resolved variant of quantum thermodynamics for open systems strongly coupled to their baths. The approach generalizes the Landauer-Buttiker inside-outside duality method [Phys. Rev. Lett. 120, 107701 (2018)] to interacting systems subjected to arbitrary external driving. It is consistent with the underlying dynamical quantum transport description and is capable of overcoming limitations of the only other consistent approach [New J. Phys. 12, 013013 (2010)]. We illustrate viability of the generalized inside-outside method with numerical simulations for generic junction models.

cond-mat.stat-mech↗
arXiv2023

MAHALO: Unifying Offline Reinforcement Learning and Imitation Learning from Observations

We study a new paradigm for sequential decision making, called offline policy learning from observations (PLfO). Offline PLfO aims to learn policies using datasets with substandard qualities: 1) only a subset of trajectories is labeled with rewards, 2) labeled trajectories may not contain actions, 3) labeled trajectories may not be of high quality, and 4) the data may not have full coverage. Such imperfection is common in real-world learning scenarios, and offline PLfO encompasses many existing offline learning setups, including offline imitation learning (IL), offline IL from observations (ILfO), and offline reinforcement learning (RL). In this work, we present a generic approach to offline PLfO, called $\textbf{M}$odality-agnostic $\textbf{A}$dversarial $\textbf{H}$ypothesis $\textbf{A}$daptation for $\textbf{L}$earning from $\textbf{O}$bservations (MAHALO). Built upon the pessimism concept in offline RL, MAHALO optimizes the policy using a performance lower bound that accounts for uncertainty due to the dataset's insufficient coverage. We implement this idea by adversarially training data-consistent critic and reward functions, which forces the learned policy to be robust to data deficiency. We show that MAHALO consistently outperforms or matches specialized algorithms across a variety of offline PLfO tasks in theory and experiments. Our code is available at https://github.com/AnqiLi/mahalo.

cs.LG↗
arXiv2023

A Benchmark for Understanding Dialogue Safety in Mental Health Support

Dialogue safety remains a pervasive challenge in open-domain human-machine interaction. Existing approaches propose distinctive dialogue safety taxonomies and datasets for detecting explicitly harmful responses. However, these taxonomies may not be suitable for analyzing response safety in mental health support. In real-world interactions, a model response deemed acceptable in casual conversations might have a negligible positive impact on users seeking mental health support. To address these limitations, this paper aims to develop a theoretically and factually grounded taxonomy that prioritizes the positive impact on help-seekers. Additionally, we create a benchmark corpus with fine-grained labels for each dialogue session to facilitate further research. We analyze the dataset using popular language models, including BERT-base, RoBERTa-large, and ChatGPT, to detect and understand unsafe responses within the context of mental health support. Our study reveals that ChatGPT struggles to detect safety categories with detailed safety definitions in a zero- and few-shot paradigm, whereas the fine-tuned model proves to be more suitable. The developed dataset and findings serve as valuable benchmarks for advancing research on dialogue safety in mental health support, with significant implications for improving the design and deployment of conversation agents in real-world applications. We release our code and data here: https://github.com/qiuhuachuan/DialogueSafety.

cs.CL↗
arXiv2023

SuperCLUE: A Comprehensive Chinese Large Language Model Benchmark

Large language models (LLMs) have shown the potential to be integrated into human daily lives. Therefore, user preference is the most critical criterion for assessing LLMs' performance in real-world scenarios. However, existing benchmarks mainly focus on measuring models' accuracy using multi-choice questions, which limits the understanding of their capabilities in real applications. We fill this gap by proposing a comprehensive Chinese benchmark SuperCLUE, named after another popular Chinese LLM benchmark CLUE. SuperCLUE encompasses three sub-tasks: actual users' queries and ratings derived from an LLM battle platform (CArena), open-ended questions with single and multiple-turn dialogues (OPEN), and closed-ended questions with the same stems as open-ended single-turn ones (CLOSE). Our study shows that accuracy on closed-ended questions is insufficient to reflect human preferences achieved on open-ended ones. At the same time, they can complement each other to predict actual user preferences. We also demonstrate that GPT-4 is a reliable judge to automatically evaluate human preferences on open-ended questions in a Chinese context. Our benchmark will be released at https://www.CLUEbenchmarks.com

cs.CL↗
arXiv2023

Understanding Client Reactions in Online Mental Health Counseling

Communication success relies heavily on reading participants' reactions. Such feedback is especially important for mental health counselors, who must carefully consider the client's progress and adjust their approach accordingly. However, previous NLP research on counseling has mainly focused on studying counselors' intervention strategies rather than their clients' reactions to the intervention. This work aims to fill this gap by developing a theoretically grounded annotation framework that encompasses counselors' strategies and client reaction behaviors. The framework has been tested against a large-scale, high-quality text-based counseling dataset we collected over the past two years from an online welfare counseling platform. Our study shows how clients react to counselors' strategies, how such reactions affect the final counseling outcomes, and how counselors can adjust their strategies in response to these reactions. We also demonstrate that this study can help counselors automatically predict their clients' states.

cs.CL↗
arXiv2023

You Can Mask More For Extremely Low-Bitrate Image Compression

Learned image compression (LIC) methods have experienced significant progress during recent years. However, these methods are primarily dedicated to optimizing the rate-distortion (R-D) performance at medium and high bitrates (> 0.1 bits per pixel (bpp)), while research on extremely low bitrates is limited. Besides, existing methods fail to explicitly explore the image structure and texture components crucial for image compression, treating them equally alongside uninformative components in networks. This can cause severe perceptual quality degradation, especially under low-bitrate scenarios. In this work, inspired by the success of pre-trained masked autoencoders (MAE) in many downstream tasks, we propose to rethink its mask sampling strategy from structure and texture perspectives for high redundancy reduction and discriminative feature representation, further unleashing the potential of LIC methods. Therefore, we present a dual-adaptive masking approach (DA-Mask) that samples visible patches based on the structure and texture distributions of original images. We combine DA-Mask and pre-trained MAE in masked image modeling (MIM) as an initial compressor that abstracts informative semantic context and texture representations. Such a pipeline can well cooperate with LIC networks to achieve further secondary compression while preserving promising reconstruction quality. Consequently, we propose a simple yet effective masked compression model (MCM), the first framework that unifies MIM and LIC end-to-end for extremely low-bitrate image compression. Extensive experiments have demonstrated that our approach outperforms recent state-of-the-art methods in R-D performance, visual quality, and downstream applications, at very low bitrates. Our code is available at https://github.com/lianqi1008/MCM.git.

cs.CV↗
arXiv2023

NICE-SLAM with Adaptive Feature Grids

NICE-SLAM is a dense visual SLAM system that combines the advantages of neural implicit representations and hierarchical grid-based scene representation. However, the hierarchical grid features are densely stored, leading to memory explosion problems when adapting the framework to large scenes. In our project, we present sparse NICE-SLAM, a sparse SLAM system incorporating the idea of Voxel Hashing into NICE-SLAM framework. Instead of initializing feature grids in the whole space, voxel features near the surface are adaptively added and optimized. Experiments demonstrated that compared to NICE-SLAM algorithm, our approach takes much less memory and achieves comparable reconstruction quality on the same datasets. Our implementation is available at https://github.com/zhangganlin/NICE-SLAM-with-Adaptive-Feature-Grids.

cs.CV↗
arXiv2023

TerrainNet: Visual Modeling of Complex Terrain for High-speed, Off-road Navigation

Effective use of camera-based vision systems is essential for robust performance in autonomous off-road driving, particularly in the high-speed regime. Despite success in structured, on-road settings, current end-to-end approaches for scene prediction have yet to be successfully adapted for complex outdoor terrain. To this end, we present TerrainNet, a vision-based terrain perception system for semantic and geometric terrain prediction for aggressive, off-road navigation. The approach relies on several key insights and practical considerations for achieving reliable terrain modeling. The network includes a multi-headed output representation to capture fine- and coarse-grained terrain features necessary for estimating traversability. Accurate depth estimation is achieved using self-supervised depth completion with multi-view RGB and stereo inputs. Requirements for real-time performance and fast inference speeds are met using efficient, learned image feature projections. Furthermore, the model is trained on a large-scale, real-world off-road dataset collected across a variety of diverse outdoor environments. We show how TerrainNet can also be used for costmap prediction and provide a detailed framework for integration into a planning module. We demonstrate the performance of TerrainNet through extensive comparison to current state-of-the-art baselines for camera-only scene prediction. Finally, we showcase the effectiveness of integrating TerrainNet within a complete autonomous-driving stack by conducting a real-world vehicle test in a challenging off-road scenario.

cs.RO↗
arXiv2023

Exceptional points treatment of cavity spectroscopies

The infrared response of a system of two vibrational modes in a cavity is calculated by an effective non-Hermitian Hamiltonian derived by employing the nonequilibrium Green's functions (NEGF) formalism. Degeneracies of the Hamiltonian (exceptional points, EP) widely employed in theoretical analysis of optical cavity spectroscopies are used in an approximate treatment and compared with the full NEGF. Qualitative limitations of the EP treatment are explained by examining the approximations employed in the calculation.

cond-mat.mes-hall↗
arXiv2023

Electoral Accountability and Selection with Personalized Information Aggregation

We study a model of electoral accountability and selection whereby heterogeneous voters aggregate incumbent politician's performance data into personalized signals through paying limited attention. Extreme voters' signals exhibit an own-party bias, which hampers their ability to discern the good and bad performances of the incumbent. While this effect alone would undermine electoral accountability and selection, there is a countervailing effect stemming from partisan disagreement, which makes the centrist voter more likely to be pivotal. In case the latter's unbiased signal is very informative about the incumbent's performance, the combined effect on electoral accountability and selection can actually be a positive one. For this reason, factors that carry a negative connotation in every political discourse -- such as increasing mass polarization and shrinking attention span -- have ambiguous accountability and selection effects in general. Correlating voters' signals, if done appropriately, unambiguously improves electoral accountability and selection and, hence, voter welfare.

econ.TH↗
arXiv2023

Fountain-driven gas accretion feeding star formation over the disc of NGC 2403

We use a dynamical model of galactic fountain to study the neutral extraplanar gas (EPG) in the nearby spiral galaxy NGC 2403. We have modelled the EPG as a combination of material ejected from the disc by stellar feedback (i.e. galactic fountain) and gas accreting from the inner circumgalactic medium (CGM). This accretion is expected to occur because of cooling/condensation of the hot CGM (corona) triggered by the fountain. Our dynamical model reproduces the distribution and kinematics of the EPG H$\mathrm{\scriptsize{I}}$ emission in NGC 2403 remarkably well and suggests a total EPG mass of $4.7^{+1.2}_{-0.9}\times10^8\mathrm{M}_\odot$, with a typical scale height of around 1 kpc and a vertical gradient of the rotation velocity of $-10.0\pm2.7\,\mathrm{km\,s^{-1}\,kpc^{-1}}$. The best-fitting model requires a characteristic outflow velocity of $50\pm10\,\mathrm{km\,s^{-1}}$. The outflowing gas starts out mostly ionised and only becomes neutral later in the trajectory. The accretion rate from the condensation of the inner hot CGM inferred by the model is 0.8$\,\mathrm{M}_\odot\,\mathrm{yr}^{-1}$, approximately equal to the star formation rate in this galaxy (0.6$\,\mathrm{M}_\odot\,\mathrm{yr}^{-1}$). We show that the accretion profile, which peaks at a radius of about 4.5$\,$kpc, predicts a disc growth rate compatible with the observed value. Our results indicate that fountain-driven corona condensation is a likely mechanism to sustain star formation as well as the disc inside-out growth in local disc galaxies.

astro-ph.GA↗
arXiv2022

Crop mapping in the small sample/no sample case: an approach using a two-level cascade classifier and integrating domain knowledge

Mapping crops using remote sensing technology is important for food security and land management. Machine learning-based methods has become a popular approach for crop mapping in recent years. However, the key to machine learning, acquiring ample and accurate samples, is usually time-consuming and laborious. To solve this problem, a crop mapping method in the small sample/no sample case that integrating domain knowledge and using a cascaded classification framework that combine a weak classifier learned from samples with strong features and a strong classifier trained by samples with weak feature was proposed. First, based on the domain knowledge of various crops, a low-capacity classifier such as decision tree was applied to acquire those pixels with distinctive features and complete observation sequences as "strong feature" samples. Then, to improve the representativeness of these samples, sample augmentation strategy that artificially remove the observations of "strong feature" samples according to the average valid observation proportion in target area was applied. Finally, based on the original samples and augmented samples, a large-capacity classifier such as random forest was trained for crop mapping. The method achieved an overall accuracy of 82% in the MAP crop recognition competition held by Syngenta Group, China in 2021 (third prize, ranked fourth). This method integrates domain knowledge to overcome the difficulties of sample acquisition, providing a convenient, fast and accurate solution for crop mapping.

cs.CV↗
arXiv2022

Generating Linear programming Instances with Controllable Rank and Condition Number

Instances generation is crucial for linear programming algorithms, which is necessary either to find the optimal pivot rules by training learning method or to evaluate and verify corresponding algorithms. This study proposes a general framework for designing linear programming instances based on the preset optimal solution. First, we give a constraint matrix generation method with controllable condition number and rank from the perspective of matrix decomposition. Based on the preset optimal solution, the bounded feasible linear programming instance is generated with the right-hand side and objective coefficient satisfying the original and dual feasibility. In addition, we provide three kind of neighborhood exchange operators and prove that instances generated under this method can fill the whole feasible and bounded case space of linear programming. We experimentally validate that the proposed schedule can generate more controllable linear programming instances, while neighborhood exchange operator can construct more complex instances.

math.OC↗
arXiv2022

Cubic Goldreich-Levin

In this paper, we give a cubic Goldreich-Levin algorithm which makes polynomially-many queries to a function $f \colon \mathbb F_p^n \to \mathbb C$ and produces a decomposition of $f$ as a sum of cubic phases and a small error term. This is a natural higher-order generalization of the classical Goldreich-Levin algorithm. The classical (linear) Goldreich-Levin algorithm has wide-ranging applications in learning theory, coding theory and the construction of pseudorandom generators in cryptography, as well as being closely related to Fourier analysis. Higher-order Goldreich-Levin algorithms on the other hand involve central problems in higher-order Fourier analysis, namely the inverse theory of the Gowers $U^k$ norms, which are well-studied in additive combinatorics. The only known result in this direction prior to this work is the quadratic Goldreich-Levin theorem, proved by Tulsiani and Wolf in 2011. The main step of their result involves an algorithmic version of the $U^3$ inverse theorem. More complications appear in the inverse theory of the $U^4$ and higher norms. Our cubic Goldreich-Levin algorithm is based on algorithmizing recent work by Gowers and Milićević who proved new quantitative bounds for the $U^4$ inverse theorem. Our cubic Goldreich-Levin algorithm is constructed from two main tools: an algorithmic $U^4$ inverse theorem and an arithmetic decomposition result in the style of the Frieze-Kannan graph regularity lemma. As one application of our main theorem we solve the problem of self-correction for cubic Reed-Muller codes beyond the list decoding radius. Additionally we give a purely combinatorial result: an improvement of the quantitative bounds on the $U^4$ inverse theorem.

cs.DS↗