arXiv subjects

Mingyu Guo

Publications and source records attributed to Mingyu Guo.

At least 73 records · Page 4Linked to original sources

arXiv2023

Worst-Case VCG Redistribution Mechanism Design Based on the Lottery Ticket Hypothesis

We study worst-case VCG redistribution mechanism design for the public project problem. We use a multilayer perceptron (MLP) with ReLU activation to model the payment function and use mixed integer programming (MIP) to solve for the worst-case type profiles that maximally violate the mechanism design constraints. We collect these worst-case type profiles and use them as training samples to train toward better worst-case mechanisms. In practice, we require a tiny network structure for the above approach to scale. The Lottery Ticket Hypothesis states that a large network is likely to contain a "winning ticket" -- a much smaller subnetwork that "won the initialization lottery", which makes its training particularly effective. Motivated by this hypothesis, we train a large network and prune it into a tiny subnetwork. We run MIP-based worst-case training on the drawn subnetwork and evaluate the resulting mechanism's worst-case performance. If the subnetwork does not achieve good worst-case performance, then we record the type profiles that cause the current draw to be bad. To draw again, we restore the large network to its initial weights and prune using recorded type profiles from earlier draws, therefore avoiding drawing the same ticket twice. We expect to eventually encounter a tiny subnetwork that leads to effective training for our worst-case mechanism design task. Lastly, a by-product of multiple ticket draws is an ensemble of mechanisms with different worst cases, which improves the worst-case performance further. Using our approach, we find previously unknown optimal mechanisms for up to 5 agents. Our results confirm the tightness of existing theoretical upper bounds. For up to 20 agents, we derive significantly improved worst-case mechanisms, surpassing a long list of existing manual results.

cs.GT↗

arXiv2023

METAL: Metamorphic Testing Framework for Analyzing Large-Language Model Qualities

Large-Language Models (LLMs) have shifted the paradigm of natural language data processing. However, their black-boxed and probabilistic characteristics can lead to potential risks in the quality of outputs in diverse LLM applications. Recent studies have tested Quality Attributes (QAs), such as robustness or fairness, of LLMs by generating adversarial input texts. However, existing studies have limited their coverage of QAs and tasks in LLMs and are difficult to extend. Additionally, these studies have only used one evaluation metric, Attack Success Rate (ASR), to assess the effectiveness of their approaches. We propose a MEtamorphic Testing for Analyzing LLMs (METAL) framework to address these issues by applying Metamorphic Testing (MT) techniques. This approach facilitates the systematic testing of LLM qualities by defining Metamorphic Relations (MRs), which serve as modularized evaluation metrics. The METAL framework can automatically generate hundreds of MRs from templates that cover various QAs and tasks. In addition, we introduced novel metrics that integrate the ASR method into the semantic qualities of text to assess the effectiveness of MRs accurately. Through the experiments conducted with three prominent LLMs, we have confirmed that the METAL framework effectively evaluates essential QAs on primary LLM tasks and reveals the quality risks in LLMs. Moreover, the newly proposed metrics can guide the optimal MRs for testing each task and suggest the most effective method for generating MRs.

cs.SE↗

arXiv2023

On the Financial Consequences of Simplified Battery Sizing Models without Considering Operational Details

Optimal battery sizing studies tend to overly simplify the practical aspects of battery operation within the battery sizing framework. Such assumptions may lead to a suboptimal battery capacity, resulting in significant financial losses for a battery project that could last more than a decade. In this paper, we compare the most common existing sizing methods in the literature with a battery sizing model that incorporates the practical operation of a battery, that is, receding horizon operation. Consequently, we quantify the financial losses caused by the suboptimal capacities obtained by these models for a realistic case study related to community battery storage (CBS). We develop the case study by constructing a mathematical framework for the CBS and local end users. Our results show that existing sizing methods can lead to financial losses of up to 22%.

eess.SY↗

arXiv2023

Diverse Approximations for Monotone Submodular Maximization Problems with a Matroid Constraint

Finding diverse solutions to optimization problems has been of practical interest for several decades, and recently enjoyed increasing attention in research. While submodular optimization has been rigorously studied in many fields, its diverse solutions extension has not. In this study, we consider the most basic variants of submodular optimization, and propose two simple greedy algorithms, which are known to be effective at maximizing monotone submodular functions. These are equipped with parameters that control the trade-off between objective and diversity. Our theoretical contribution shows their approximation guarantees in both objective value and diversity, as functions of their respective parameters. Our experimental investigation with maximum vertex coverage instances demonstrates their empirical differences in terms of objective-diversity trade-offs.

cs.DS↗

arXiv2023

Cost Sharing Public Project with Minimum Release Delay

We study the excludable public project model where the decision is binary (build or not build). In a classic excludable and binary public project model, an agent either consumes the project in its whole or is completely excluded. We study a setting where the mechanism can set different project release time for different agents, in the sense that high-paying agents can consume the project earlier than low-paying agents. The release delay, while hurting the social welfare, is implemented to incentivize payments to cover the project cost. The mechanism design objective is to minimize the maximum release delay and the total release delay among all agents. We first consider the setting where we know the prior distribution of the agents' types. Our objectives are minimizing the expected maximum release delay and the expected total release delay. We propose the single deadline mechanisms. We show that the optimal single deadline mechanism is asymptotically optimal for both objectives, regardless of the prior distribution. For small number of agents, we propose the sequential unanimous mechanisms by extending the largest unanimous mechanisms from [Ohseto 2000]. We propose an automated mechanism design approach via evolutionary computation to optimize within the sequential unanimous mechanisms. We next study prior-free mechanism design. We propose the group-based optimal deadline mechanism and show that it is competitive against an undominated mechanism under minor technical assumptions.

cs.GT↗

arXiv2023

Evolving Reinforcement Learning Environment to Minimize Learner's Achievable Reward: An Application on Hardening Active Directory Systems

We study a Stackelberg game between one attacker and one defender in a configurable environment. The defender picks a specific environment configuration. The attacker observes the configuration and attacks via Reinforcement Learning (RL trained against the observed environment). The defender's goal is to find the environment with minimum achievable reward for the attacker. We apply Evolutionary Diversity Optimization (EDO) to generate diverse population of environments for training. Environments with clearly high rewards are killed off and replaced by new offsprings to avoid wasting training time. Diversity not only improves training quality but also fits well with our RL scenario: RL agents tend to improve gradually, so a slightly worse environment earlier on may become better later. We demonstrate the effectiveness of our approach by focusing on a specific application, Active Directory (AD). AD is the default security management system for Windows domain networks. AD environment describes an attack graph, where nodes represent computers/accounts/etc., and edges represent accesses. The attacker aims to find the best attack path to reach the highest-privilege node. The defender can change the graph by removing a limited number of edges (revoke accesses). Our approach generates better defensive plans than the existing approach and scales better.

cs.NE↗

arXiv2023

Defending Active Directory by Combining Neural Network based Dynamic Program and Evolutionary Diversity Optimisation

Active Directory (AD) is the default security management system for Windows domain networks. We study a Stackelberg game model between one attacker and one defender on an AD attack graph. The attacker initially has access to a set of entry nodes. The attacker can expand this set by strategically exploring edges. Every edge has a detection rate and a failure rate. The attacker aims to maximize their chance of successfully reaching the destination before getting detected. The defender's task is to block a constant number of edges to decrease the attacker's chance of success. We show that the problem is #P-hard and, therefore, intractable to solve exactly. We convert the attacker's problem to an exponential sized Dynamic Program that is approximated by a Neural Network (NN). Once trained, the NN provides an efficient fitness function for the defender's Evolutionary Diversity Optimisation (EDO). The diversity emphasis on the defender's solution provides a diverse set of training samples, which improves the training accuracy of our NN for modelling the attacker. We go back and forth between NN training and EDO. Experimental results show that for R500 graph, our proposed EDO based defense is less than 1% away from the optimal defense.

cs.NE↗

arXiv2022

Scalable Edge Blocking Algorithms for Defending Active Directory Style Attack Graphs

Active Directory (AD) is the default security management system for Windows domain networks. An AD environment naturally describes an attack graph where nodes represent computers/accounts/security groups, and edges represent existing accesses/known exploits that allow the attacker to gain access from one node to another. Motivated by practical AD use cases, we study a Stackelberg game between one attacker and one defender. There are multiple entry nodes for the attacker to choose from and there is a single target (Domain Admin). Every edge has a failure rate. The attacker chooses the attack path with the maximum success rate. The defender can block a limited number of edges (i.e., revoke accesses) from a set of blockable edges, limited by budget. The defender's aim is to minimize the attacker's success rate. We exploit the tree-likeness of practical AD graphs to design scalable algorithms. We propose two novel methods that combine theoretical fixed parameter analysis and practical optimisation techniques. For graphs with small tree widths, we propose a tree decomposition based dynamic program. We then propose a general method for converting tree decomposition based dynamic programs to reinforcement learning environments, which leads to an anytime algorithm that scales better, but loses the optimality guarantee. For graphs with small numbers of non-splitting paths (a parameter we invent specifically for AD graphs), we propose a kernelization technique that significantly downsizes the model, which is then solved via mixed-integer programming. Experimentally, our algorithms scale to handle synthetic AD graphs with tens of thousands of nodes.

cs.CR↗

arXiv2022

Analysis of Evolutionary Diversity Optimisation for Permutation Problems

Generating diverse populations of high quality solutions has gained interest as a promising extension to the traditional optimization tasks. This work contributes to this line of research with an investigation on evolutionary diversity optimization for three of the most well-studied permutation problems, namely the Traveling Salesperson Problem (TSP), both symmetric and asymmetric variants, and Quadratic Assignment Problem (QAP). It includes an analysis of the worst-case performance of a simple mutation-only evolutionary algorithm with different mutation operators, using an established diversity measure. Theoretical results show many mutation operators for these problems guarantee convergence to maximally diverse populations of sufficiently small size within cubic to quartic expected run-time. On the other hand, the result on QAP suggests that strong mutations give poor worst-case performance, as mutation strength contributes exponentially to the expected run-time. Additionally, experiments are carried out on QAPLIB and synthetic instances in unconstrained and constrained settings, and reveal much more optimistic practical performances, while corroborating the theoretical finding regarding mutation strength. These results should serve as a baseline for future studies.

cs.NE↗

arXiv2022

Multi-Forgery Detection Challenge 2022: Push the Frontier of Unconstrained and Diverse Forgery Detection

In this paper, we present the Multi-Forgery Detection Challenge held concurrently with the IEEE Computer Society Workshop on Biometrics at CVPR 2022. Our Multi-Forgery Detection Challenge aims to detect automatic image manipulations including but not limited to image editing, image synthesis, image generation, image photoshop, etc. Our challenge has attracted 674 teams from all over the world, with about 2000 valid result submission counts. We invited the Top 10 teams to present their solutions to the challenge, from which three teams are awarded prizes in the grand finale. In this paper, we present the solutions from the Top 3 teams, in order to boost the research work in the field of image forgery detection.

cs.CV↗

arXiv2022

Niching-based Evolutionary Diversity Optimization for the Traveling Salesperson Problem

In this work, we consider the problem of finding a set of tours to a traveling salesperson problem (TSP) instance maximizing diversity, while satisfying a given cost constraint. This study aims to investigate the effectiveness of applying niching to maximize diversity rather than simply maintaining it. To this end, we introduce a 2-stage approach where a simple niching memetic algorithm (NMA), derived from a state-of-the-art for multi-solution TSP, is combined with a baseline diversifying algorithm. The most notable feature of the proposed NMA is the use of randomized improvement-first local search instead of 2-opt. Our experiment on TSPLIB instances shows that while the populations evolved by our NMA tend to contain clusters at tight quality constraints, they frequently occupy distant basins of attraction rather than close-by regions, improving on the baseline diversification in terms of sum-sum diversity. Compared to the original NMA, ours, despite its simplicity, finds more distant solutions of higher quality within less running time, by a large margin.

cs.NE↗

arXiv2022

Redistribution in Public Project Problems via Neural Networks

Many important problems in multiagent systems involve resource allocations. Self-interested agents may lie about their valuations if doing so increases their own utilities. Therefore, it is necessary to design mechanisms (collective decision-making rules) with desired properties and objectives. The VCG redistribution mechanisms are efficient (the agents who value the resources the most will be allocated), strategy-proof (the agents have no incentives to lie about their valuations), and weakly budget-balanced (no deficits). We focus on the VCG redistribution mechanisms for the classic public project problem, where a group of agents needs to decide whether or not to build a non-excludable public project. We design mechanisms via neural networks with two welfare-maximizing objectives: optimal in the worst case and optimal in expectation. Previous studies showed two worst-case optimal mechanisms for 3 agents, but worst-case optimal mechanisms have not been identified for more than 3 agents. For maximizing expected welfare, there are no existing results. We use neural networks to design VCG redistribution mechanisms. Neural networks have been used to design the redistribution mechanisms for multi-unit auctions with unit demand. We show that for the public project problem, the previously proposed neural networks, which led to optimal/near-optimal mechanisms for multi-unit auctions with unit demand, perform abysmally for the public project problem. We significantly improve the existing networks on multiple fronts: We conduct a GAN network to generate worst-case type profiles and feed prior distribution into loss function to provide quality gradients for the optimal-in-expectation objective......

cs.GT↗

arXiv2022

Dependency Structure for News Document Summarization

In this work, we develop a neural network based model which leverages dependency parsing to capture cross-positional dependencies and grammatical structures. With the help of linguistic signals, sentence-level relations can be correctly captured, thus improving news documents summarization performance. Empirical studies demonstrate that this simple but effective method outperforms existing works on the benchmark dataset. Extensive analyses examine different settings and configurations of the proposed model which provide a good reference to the community.

cs.CL↗

arXiv2021

Practical Fixed-Parameter Algorithms for Defending Active Directory Style Attack Graphs

Active Directory is the default security management system for Windows domain networks. We study the shortest path edge interdiction problem for defending Active Directory style attack graphs. The problem is formulated as a Stackelberg game between one defender and one attacker. The attack graph contains one destination node and multiple entry nodes. The attacker's entry node is chosen by nature. The defender chooses to block a set of edges limited by his budget. The attacker then picks the shortest unblocked attack path. The defender aims to maximize the expected shortest path length for the attacker, where the expectation is taken over entry nodes. We observe that practical Active Directory attack graphs have small maximum attack path lengths and are structurally close to trees. We first show that even if the maximum attack path length is a constant, the problem is still $W[1]$-hard with respect to the defender's budget. Having a small maximum attack path length and a small budget is not enough to design fixed-parameter algorithms. If we further assume that the number of entry nodes is small, then we derive a fixed-parameter tractable algorithm. We then propose two other fixed-parameter algorithms by exploiting the tree-like features. One is based on tree decomposition and requires a small tree width. The other assumes a small number of splitting nodes (nodes with multiple out-going edges). Finally, the last algorithm is converted into a graph convolutional neural network based heuristic, which scales to larger graphs with more splitting nodes.

cs.GT↗

arXiv2021

Multi-document Summarization via Deep Learning Techniques: A Survey

Multi-document summarization (MDS) is an effective tool for information aggregation that generates an informative and concise summary from a cluster of topic-related documents. Our survey, the first of its kind, systematically overviews the recent deep learning based MDS models. We propose a novel taxonomy to summarize the design strategies of neural networks and conduct a comprehensive summary of the state-of-the-art. We highlight the differences between various objective functions that are rarely discussed in the existing literature. Finally, we propose several future directions pertaining to this new and exciting field.

cs.CL↗

arXiv2021

CelebA-Spoof Challenge 2020 on Face Anti-Spoofing: Methods and Results

As facial interaction systems are prevalently deployed, security and reliability of these systems become a critical issue, with substantial research efforts devoted. Among them, face anti-spoofing emerges as an important area, whose objective is to identify whether a presented face is live or spoof. Recently, a large-scale face anti-spoofing dataset, CelebA-Spoof which comprised of 625,537 pictures of 10,177 subjects has been released. It is the largest face anti-spoofing dataset in terms of the numbers of the data and the subjects. This paper reports methods and results in the CelebA-Spoof Challenge 2020 on Face AntiSpoofing which employs the CelebA-Spoof dataset. The model evaluation is conducted online on the hidden test set. A total of 134 participants registered for the competition, and 19 teams made valid submissions. We will analyze the top ranked solutions and present some discussion on future work directions.

cs.CV↗

arXiv2020

Cost Sharing Security Information with Minimal Release Delay

We study a cost sharing problem derived from bug bounty programs, where agents gain utility by the amount of time they get to enjoy the cost shared information. Once the information is provided to an agent, it cannot be retracted. The goal, instead of maximizing revenue, is to pick a time as early as possible, so that enough agents are willing to cost share the information and enjoy it for a premium time period, while other agents wait and enjoy the information for free after a certain amount of release delay. We design a series of mechanisms with the goal of minimizing the maximum delay and the total delay. Under prior-free settings, our final mechanism achieves a competitive ratio of $4$ in terms of maximum delay, against an undominated mechanism. Finally, we assume some distributions of the agents' valuations, and investigate our mechanism's performance in terms of expected delays.

cs.GT↗

arXiv2020

Revenue Maximizing Markets for Zero-Day Exploits

Markets for zero-day exploits (software vulnerabilities unknown to the vendor) have a long history and a growing popularity. We study these markets from a revenue-maximizing mechanism design perspective. We first propose a theoretical model for zero-day exploits markets. In our model, one exploit is being sold to multiple buyers. There are two kinds of buyers, which we call the defenders and the offenders. The defenders are buyers who buy vulnerabilities in order to fix them (e.g., software vendors). The offenders, on the other hand, are buyers who intend to utilize the exploits (e.g., national security agencies and police). Our model is more than a single-item auction. First, an exploit is a piece of information, so one exploit can be sold to multiple buyers. Second, buyers have externalities. If one defender wins, then the exploit becomes worthless to the offenders. Third, if we disclose the details of the exploit to the buyers before the auction, then they may leave with the information without paying. On the other hand, if we do not disclose the details, then it is difficult for the buyers to come up with their private valuations. Considering the above, our proposed mechanism discloses the details of the exploit to all offenders before the auction. The offenders then pay to delay the exploit being disclosed to the defenders.

cs.GT↗