arXiv subjects

Damon Wischik

Publications and source records attributed to Damon Wischik.

6 recordsLinked to original sources

arXiv2025

Your Finetuned Large Language Model is Already a Powerful Out-of-distribution Detector

We revisit the likelihood ratio between a pretrained large language model (LLM) and its finetuned variant as a criterion for out-of-distribution (OOD) detection. The intuition behind such a criterion is that, the pretrained LLM has the prior knowledge about OOD data due to its large amount of training data, and once finetuned with the in-distribution data, the LLM has sufficient knowledge to distinguish their difference. Leveraging the power of LLMs, we show that, the likelihood ratio can serve as an effective OOD detection criterion. Moreover, we apply the proposed LLM-based likelihood ratio to detect OOD questions in question-answering (QA) systems, which can be used to improve the performance of specialized LLMs for general questions. Given that likelihood can be easily obtained by the loss functions within contemporary neural network frameworks, it is straightforward to implement this approach in practice. Since both the pretrained LLMs and its various finetuned models are widely available from online platforms such as Hugging Face, our proposed criterion can be effortlessly incorporated for OOD detection without the need for further training. We conduct comprehensive evaluation across on multiple settings, including far OOD, near OOD, spam detection, and QA scenarios, to demonstrate the effectiveness of the method. Code can be found at https://github.com/andiac/LLMOODratio

cs.CL↗

arXiv2024

Constructing Semantics-Aware Adversarial Examples with a Probabilistic Perspective

We propose a probabilistic perspective on adversarial examples, allowing us to embed subjective understanding of semantics as a distribution into the process of generating adversarial examples, in a principled manner. Despite significant pixel-level modifications compared to traditional adversarial attacks, our method preserves the overall semantics of the image, making the changes difficult for humans to detect. This extensive pixel-level modification enhances our method's ability to deceive classifiers designed to defend against adversarial attacks. Our empirical findings indicate that the proposed methods achieve higher success rates in circumventing adversarial defense mechanisms, while remaining difficult for human observers to detect.

stat.ML↗

arXiv2023

A Combined Convex Model for Travel Demand Forecasting with Hierarchical Extended Logit Model

The travel demand forecasting model plays a crucial role in evaluating large-scale infrastructure projects, such as the construction of new roads or transit lines. While combined modeling approaches have been explored as a solution to overcome the problem of input and output discrepancies in a sequential four-step modeling process, previous attempts at combined models have encountered challenges in real-world applications, primarily due to their limited behavioral richness or computational tractability. In this study, we propose a novel convex programming approach and present a key theorem demonstrating that the optimal solution is the same as the one from solving the hierarchical extended logit model. This model is specifically designed to capture correlations existing in travelers' choices, including similarities among transport modes and route overlaps. The convex property of our model ensures the existence and uniqueness of solutions and offers computational efficiency. The advantages of our proposed model are twofold. Firstly, it provides a single unifying rationale (i.e., utility maximization) that is valid across all steps. Secondly, its combined nature allows a systematic approach to handling observed data, enabling a more realistic representation of reality. By addressing the limited behavior richness of previous combined models, our convex programming-based approach shows promise in enhancing the accuracy and applicability of travel demand forecasting, thereby aiding in the planning and decision-making processes for infrastructure projects.

math.OC↗

arXiv2022

Falsehoods that ML researchers believe about OOD detection

An intuitive way to detect out-of-distribution (OOD) data is via the density function of a fitted probabilistic generative model: points with low density may be classed as OOD. But this approach has been found to fail, in deep learning settings. In this paper, we list some falsehoods that machine learning researchers believe about density-based OOD detection. Many recent works have proposed likelihood-ratio-based methods to `fix' the problem. We propose a framework, the OOD proxy framework, to unify these methods, and we argue that likelihood ratio is a principled method for OOD detection and not a mere `fix'. Finally, we discuss the relationship between domain discrimination and semantics.

stat.ML↗

arXiv2022

Disentangling Domain and Content

Many real-world datasets can be divided into groups according to certain salient features (e.g. grouping images by subject, grouping text by font, etc.). Often, machine learning tasks require that these features be represented separately from those manifesting independently of the grouping. For example, image translation entails changing the style of an image while preserving its content. We formalize these two kinds of attributes as two complementary generative factors called "domain" and "content", and address the problem of disentangling them in a fully unsupervised way. To achieve this, we propose a principled, generalizable probabilistic model inspired by the Variational Autoencoder. Our model exhibits state-of-the-art performance on the composite task of generating images by combining the domain of one input with the content of another. Distinctively, it can perform this task in a few-shot, unsupervised manner, without being provided with explicit labelling for either domain or content. The disentangled representations are learned through the combination of a group-wise encoder and a novel domain-confusion loss.

cs.DC↗

arXiv2012

Switched networks with maximum weight policies: Fluid approximation and multiplicative state space collapse

We consider a queueing network in which there are constraints on which queues may be served simultaneously; such networks may be used to model input-queued switches and wireless networks. The scheduling policy for such a network specifies which queues to serve at any point in time. We consider a family of scheduling policies, related to the maximum-weight policy of Tassiulas and Ephremides [IEEE Trans. Automat. Control 37 (1992) 1936--1948], for single-hop and multihop networks. We specify a fluid model and show that fluid-scaled performance processes can be approximated by fluid model solutions. We study the behavior of fluid model solutions under critical load, and characterize invariant states as those states which solve a certain network-wide optimization problem. We use fluid model results to prove multiplicative state space collapse. A notable feature of our results is that they do not assume complete resource pooling.

math.PR↗