arXiv subjects

Roger Wattenhofer

Publications and source records attributed to Roger Wattenhofer.

At least 199 records · Page 11Linked to original sources

arXiv2022

A Theoretical Comparison of Graph Neural Network Extensions

We study and compare different Graph Neural Network extensions that increase the expressive power of GNNs beyond the Weisfeiler-Leman test. We focus on (i) GNNs based on higher order WL methods, (ii) GNNs that preprocess small substructures in the graph, (iii) GNNs that preprocess the graph up to a small radius, and (iv) GNNs that slightly perturb the graph to compute an embedding. We begin by presenting a simple improvement for this last extension that strictly increases the expressive power of this GNN variant. Then, as our main result, we compare the expressiveness of these extensions to each other through a series of example constructions that can be distinguished by one of the extensions, but not by another one. We also show negative examples that are particularly challenging for each of the extensions, and we prove several claims about the ability of these extensions to count cliques and cycles in the graph.

cs.LG↗

arXiv2022

The Price of Majority Support

We consider the problem of finding a compromise between the opinions of a group of individuals on a number of mutually independent, binary topics. In this paper, we quantify the loss in representativeness that results from requiring the outcome to have majority support, in other words, the "price of majority support". Each individual is assumed to support an outcome if they agree with the outcome on at least as many topics as they disagree on. Our results can also be seen as quantifying Anscombes paradox which states that topic-wise majority outcome may not be supported by a majority. To measure the representativeness of an outcome, we consider two metrics. First, we look for an outcome that agrees with a majority on as many topics as possible. We prove that the maximum number such that there is guaranteed to exist an outcome that agrees with a majority on this number of topics and has majority support, equals $\ceil{(t+1)/2}$ where $t$ is the total number of topics. Second, we count the number of times a voter opinion on a topic matches the outcome on that topic. The goal is to find the outcome with majority support with the largest number of matches. We consider the ratio between this number and the number of matches of the overall best outcome which may not have majority support. We try to find the maximum ratio such that an outcome with majority support and this ratio of matches compared to the overall best is guaranteed to exist. For 3 topics, we show this ratio to be $5/6\approx 0.83$. In general, we prove an upper bound that comes arbitrarily close to $2\sqrt{6}-4\approx 0.90$ as $t$ tends to infinity. Furthermore, we numerically compute a better upper and a non-matching lower bound in the relevant range for $t$.

cs.LG↗

arXiv2022

Cyclic Arbitrage in Decentralized Exchanges

Decentralized Exchanges (DEXes) enable users to create markets for exchanging any pair of cryptocurrencies. The direct exchange rate of two tokens may not match the cross-exchange rate in the market, and such price discrepancies open up arbitrage possibilities with trading through different cryptocurrencies cyclically. In this paper, we conduct a systematic investigation on cyclic arbitrages in DEXes. We propose a theoretical framework for studying cyclic arbitrage. With our framework, we analyze the profitability conditions and optimal trading strategies of cyclic transactions. We further examine exploitable arbitrage opportunities and the market size of cyclic arbitrages with transaction-level data of Uniswap V2. We find that traders have executed 292,606 cyclic arbitrages over eleven months and exploited more than 138 million USD in revenue. However, the revenue of the most profitable unexploited opportunity is persistently higher than 1 ETH (4,000 USD), which indicates that DEX markets may not be efficient enough. By analyzing how traders implement cyclic arbitrages, we find that traders can utilize smart contracts to issue atomic transactions and the atomic implementations could mitigate users' financial loss in cyclic arbitrage from the price impact.

q-fin.TR↗

arXiv2021

3D-RETR: End-to-End Single and Multi-View 3D Reconstruction with Transformers

3D reconstruction aims to reconstruct 3D objects from 2D views. Previous works for 3D reconstruction mainly focus on feature matching between views or using CNNs as backbones. Recently, Transformers have been shown effective in multiple applications of computer vision. However, whether or not Transformers can be used for 3D reconstruction is still unclear. In this paper, we fill this gap by proposing 3D-RETR, which is able to perform end-to-end 3D REconstruction with TRansformers. 3D-RETR first uses a pretrained Transformer to extract visual features from 2D input images. 3D-RETR then uses another Transformer Decoder to obtain the voxel features. A CNN Decoder then takes as input the voxel features to obtain the reconstructed objects. 3D-RETR is capable of 3D reconstruction from a single view or multiple views. Experimental results on two datasets show that 3DRETR reaches state-of-the-art performance on 3D reconstruction. Additional ablation study also demonstrates that 3D-DETR benefits from using Transformers.

cs.CV↗

arXiv2021

DropGNN: Random Dropouts Increase the Expressiveness of Graph Neural Networks

This paper studies Dropout Graph Neural Networks (DropGNNs), a new approach that aims to overcome the limitations of standard GNN frameworks. In DropGNNs, we execute multiple runs of a GNN on the input graph, with some of the nodes randomly and independently dropped in each of these runs. Then, we combine the results of these runs to obtain the final result. We prove that DropGNNs can distinguish various graph neighborhoods that cannot be separated by message passing GNNs. We derive theoretical bounds for the number of runs required to ensure a reliable distribution of dropouts, and we prove several properties regarding the expressive capabilities and limits of DropGNNs. We experimentally validate our theoretical findings on expressiveness. Furthermore, we show that DropGNNs perform competitively on established GNN benchmarks.

cs.LG↗

arXiv2021

EEGEyeNet: a Simultaneous Electroencephalography and Eye-tracking Dataset and Benchmark for Eye Movement Prediction

We present a new dataset and benchmark with the goal of advancing research in the intersection of brain activities and eye movements. Our dataset, EEGEyeNet, consists of simultaneous Electroencephalography (EEG) and Eye-tracking (ET) recordings from 356 different subjects collected from three different experimental paradigms. Using this dataset, we also propose a benchmark to evaluate gaze prediction from EEG measurements. The benchmark consists of three tasks with an increasing level of difficulty: left-right, angle-amplitude and absolute position. We run extensive experiments on this benchmark in order to provide solid baselines, both based on classical machine learning models and on large neural networks. We release our complete code and data and provide a simple and easy-to-use interface to evaluate new methods.

eess.SP↗

arXiv2021

Behavior of Liquidity Providers in Decentralized Exchanges

Decentralized exchanges (DEXes) have introduced an innovative trading mechanism, where it is not necessary to match buy-orders and sell-orders to execute a trade. DEXes execute each trade individually, and the exchange rate is automatically determined by the ratio of assets reserved in the market. Therefore, apart from trading, financial players can also liquidity providers, benefiting from transaction fees from trades executed in DEXes. Although liquidity providers are essential for the functionality of DEXes, it is not clear how liquidity providers behave in such markets. In this paper, we aim to understand how liquidity providers react to market information and how they benefit from providing liquidity in DEXes. We measure the operations of liquidity providers on Uniswap and analyze how they determine their investment strategy based on market changes. We also reveal their returns and risks of investments in different trading pair categories, i.e., stable pairs, normal pairs, and exotic pairs. Further, we investigate the movement of liquidity between trading pools. To the best of our knowledge, this is the first work that systematically studies the behavior of liquidity providers in DEXes.

q-fin.CP↗

arXiv2021

Default Ambiguity: Finding the Best Solution to the Clearing Problem

We study financial networks with debt contracts and credit default swaps between specific pairs of banks. Given such a financial system, we want to decide which of the banks are in default, and how much of their liabilities can these defaulting banks pay. There can easily be multiple different solutions to this problem, leading to a situation of default ambiguity, and a range of possible solutions to implement for a financial authority. In this paper, we study the properties of the solution space of such financial systems, and analyze a wide range of reasonable objective functions for selecting from the set of solutions. Examples of such objective functions include minimizing the number of defaulting banks, minimizing the amount of unpaid debt, maximizing the number of satisfied banks, and many others. We show that for all of these objectives, it is NP-hard to approximate the optimal solution to an $n^{1-ε}$ factor for any $ε>0$, with $n$ denoting the number of banks. Furthermore, we show that this situation is rather difficult to avoid from a financial regulator's perspective: the same hardness results also hold if we apply strong restrictions on the weights of the debts, the structure of the network, or the amount of funds that banks must possess. However, if we restrict both the network structure and the amount of funds simultaneously, then the solution becomes unique, and it can be found efficiently.

cs.CE↗

arXiv2021

On Isotropy Calibration of Transformers

Different studies of the embedding space of transformer models suggest that the distribution of contextual representations is highly anisotropic - the embeddings are distributed in a narrow cone. Meanwhile, static word representations (e.g., Word2Vec or GloVe) have been shown to benefit from isotropic spaces. Therefore, previous work has developed methods to calibrate the embedding space of transformers in order to ensure isotropy. However, a recent study (Cai et al. 2021) shows that the embedding space of transformers is locally isotropic, which suggests that these models are already capable of exploiting the expressive capacity of their embedding space. In this work, we conduct an empirical evaluation of state-of-the-art methods for isotropy calibration on transformers and find that they do not provide consistent improvements across models and tasks. These results support the thesis that, given the local isotropy, transformers do not benefit from additional isotropy calibration.

cs.CL↗

arXiv2021

A Plug-and-Play Method for Controlled Text Generation

Large pre-trained language models have repeatedly shown their ability to produce fluent text. Yet even when starting from a prompt, generation can continue in many plausible directions. Current decoding methods with the goal of controlling generation, e.g., to ensure specific words are included, either require additional models or fine-tuning, or work poorly when the task at hand is semantically unconstrained, e.g., story generation. In this work, we present a plug-and-play decoding method for controlled language generation that is so simple and intuitive, it can be described in a single sentence: given a topic or keyword, we add a shift to the probability distribution over our vocabulary towards semantically similar words. We show how annealing this distribution can be used to impose hard constraints on language generation, something no other plug-and-play method is currently able to do with SOTA language generators. Despite the simplicity of this approach, we see it works incredibly well in practice: decoding from GPT-2 leads to diverse and fluent sentences while guaranteeing the appearance of given guide words. We perform two user studies, revealing that (1) our method outperforms competing methods in human evaluations; and (2) forcing the guide words to appear in the generated text has no impact on the fluency of the generated text.

cs.CL↗

arXiv2021

BERT is Robust! A Case Against Synonym-Based Adversarial Examples in Text Classification

Deep Neural Networks have taken Natural Language Processing by storm. While this led to incredible improvements across many tasks, it also initiated a new research field, questioning the robustness of these neural networks by attacking them. In this paper, we investigate four word substitution-based attacks on BERT. We combine a human evaluation of individual word substitutions and a probabilistic analysis to show that between 96% and 99% of the analyzed attacks do not preserve semantics, indicating that their success is mainly based on feeding poor data to the model. To further confirm that, we introduce an efficient data augmentation procedure and show that many adversarial examples can be prevented by including data similar to the attacks during training. An additional post-processing step reduces the success rates of state-of-the-art attacks below 5%. Finally, by looking at more reasonable thresholds on constraints for word substitutions, we conclude that BERT is a lot more robust than research on attacks suggests.

cs.CL↗

arXiv2021

Online k-Way Matching with Delays and the H-Metric

In this paper, we study $k$-Way Min-cost Perfect Matching with Delays - the $k$-MPMD problem. This problem considers a metric space with $n$ nodes. Requests arrive at these nodes in an online fashion. The task is to match these requests into sets of exactly $k$, such that the space and time cost of all matched requests are minimized. The notion of the space cost requires a definition of an underlying metric space that gives distances of subsets of $k$ elements. For $k>2$, the task of finding a suitable metric space is at the core of our problem: We show that for some known generalizations to $k=3$ points, such as the $2$-metric and the $D$-metric, there exists no competitive randomized algorithm for the $3$-MPMD problem. The $G$-metrics are defined for 3 points and allows for a competitive algorithm for the $3$-MPMD problem. For $k>3$ points, there exist two generalizations of the $G$-metrics known as $n$- and $K$-metrics. We show that neither the $n$-metrics nor the $K$-metrics can be used for the $k$-MPMD problem. On the positive side, we introduce the $H$-metrics, the first metrics to allow for a solution of the $k$-MPMD problem for all $k$. In order to devise an online algorithm for the $k$-MPMD problem on the $H$-metrics, we embed the $H$-metric into trees with an $O(\log n)$ distortion. Based on this embedding result, we extend the algorithm proposed by Azar et al. (2017) and achieve a competitive ratio of $O(\log n)$ for the $k$-MPMD problem.

cs.DS↗

arXiv2021

A Limitlessly Scalable Transaction System

We present Accept, a simple, asynchronous transaction system that achieves perfect horizontal scaling. Usual blockchain-based transaction systems come with a fundamental throughput limitation as they require that all (potentially unrelated) transactions must be totally ordered. Such solutions thus require serious compromises or are outright unsuitable for large-scale applications, such as global retail payments. Accept provides efficient horizontal scaling without any limitation. To that end, Accept satisfies a relaxed form of consensus and does not establish an ordering of unrelated transactions. Furthermore, Accept achieves instant finality and does not depend on a source of randomness.

cs.DC↗

arXiv2021

Two-Class (r,k)-Coloring: Coloring with Service Guarantees

This paper introduces the Two-Class ($r$,$k$)-Coloring problem: Given a fixed number of $k$ colors, such that only $r$ of these $k$ colors allow conflicts, what is the minimal number of conflicts incurred by an optimal coloring of the graph? We establish that the family of Two-Class ($r$,$k$)-Coloring problems is NP-complete for any $k \geq 2$ when $(r, k) \neq (0,2)$. Furthermore, we show that Two-Class ($r$,$k$)-Coloring for $k \geq 2$ colors with one ($r = 1$) relaxed color cannot be approximated to any constant factor ($\notin$ APX). Finally, we show that Two-Class ($r$,$k$)-Coloring with $k \geq r \geq 2$ colors is APX-complete.

cs.CC↗

arXiv2021

KM-BART: Knowledge Enhanced Multimodal BART for Visual Commonsense Generation

We present Knowledge Enhanced Multimodal BART (KM-BART), which is a Transformer-based sequence-to-sequence model capable of reasoning about commonsense knowledge from multimodal inputs of images and texts. We adapt the generative BART architecture to a multimodal model with visual and textual inputs. We further develop novel pretraining tasks to improve the model performance on the Visual Commonsense Generation (VCG) task. In particular, our pretraining task of Knowledge-based Commonsense Generation (KCG) boosts model performance on the VCG task by leveraging commonsense knowledge from a large language model pretrained on external commonsense knowledge graphs. To the best of our knowledge, we are the first to propose a dedicated task for improving model performance on the VCG task. Experimental results show that our model reaches state-of-the-art performance on the VCG task by applying these novel pretraining tasks.

cs.CL↗

arXiv2021

Stabilization Bounds for Influence Propagation from a Random Initial State

We study the stabilization time of two common types of influence propagation. In majority processes, nodes in a graph want to switch to the most frequent state in their neighborhood, while in minority processes, nodes want to switch to the least frequent state in their neighborhood. We consider the sequential model of these processes, and assume that every node starts out from a uniform random state. We first show that if nodes change their state for any small improvement in the process, then stabilization can last for up to $Θ(n^2)$ steps in both cases. Furthermore, we also study the proportional switching case, when nodes only decide to change their state if they are in conflict with a $\frac{1+λ}{2}$ fraction of their neighbors, for some parameter $λ\in (0,1)$. In this case, we show that if $λ< \frac{1}{3}$, then there is a construction where stabilization can indeed last for $Ω(n^{1+c})$ steps for some constant $c>0$. On the other hand, if $λ> \frac{1}{2}$, we prove that the stabilization time of the processes is upper-bounded by $O(n \cdot \log{n})$.

cs.DM↗

arXiv2021

Bailouts in Financial Networks

We consider networks of banks with assets and liabilities. Some banks may be insolvent, and a central bank can decide which insolvent banks, if any, to bail out. We view bailouts as an optimization problem where the central bank has given resources at its disposal and an objective it wants to maximize. We show that under various assumptions and for various natural objectives this optimization problem is NP-hard, and in some cases even hard to approximate. Furthermore, we also show that given a fixed central bank bailout objective, banks in the network can make new debt contracts to increase their own market value in the event of a bailout (at the expense of the central bank).

cs.SI↗

arXiv2021

Debt Swapping for Risk Mitigation in Financial Networks

We study financial networks where banks are connected by debt contracts. We consider the operation of debt swapping when two creditor banks decide to exchange an incoming payment obligation, thus leading to a locally different network structure. We say that a swap is positive if it is beneficial for both of the banks involved; we can interpret this notion either with respect to the amount of assets received by the banks, or their exposure to different shocks that might hit the system. We analyze various properties of these swapping operations in financial networks. We first show that there can be no positive swap for any pair of banks in a static financial system, or when a shock hits each bank in the network proportionally. We then study worst-case shock models, when a shock of given size is distributed in the worst possible way for a specific bank. If the goal of banks is to minimize their losses in such a worst-case setting, then a positive swap can indeed exist. We analyze the effects of such a positive swap on other banks of the system, the computational complexity of finding a swap, and special cases where a swap can be found efficiently. Finally, we also present some results for more complex swapping operations when the banks swap multiple contracts, or when more than two banks participate in the swap.

q-fin.RM↗