Searcharxiv⌕ Search

arXiv subjects

Swati Gupta

Publications and source records attributed to Swati Gupta.

At least 37 records · Page 2Linked to original sources
arXiv2024

TACOS: Topology-Aware Collective Algorithm Synthesizer for Distributed Machine Learning

The surge of artificial intelligence, particularly large language models, has driven the rapid development of large-scale machine learning clusters. Executing distributed models on these clusters is often constrained by communication overhead, making efficient utilization of available network resources crucial. As a result, the routing algorithm employed for collective communications (i.e., collective algorithms) plays a pivotal role in determining overall performance. Unfortunately, existing collective communication libraries for distributed machine learning are limited by a fixed set of basic collective algorithms. This limitation hinders communication optimization, especially in modern clusters with heterogeneous and asymmetric topologies. Furthermore, manually designing collective algorithms for all possible combinations of network topologies and collective patterns requires heavy engineering and validation efforts. To address these challenges, this paper presents TACOS, an autonomous synthesizer capable of automatically generating topology-aware collective algorithms tailored to specific collective patterns and network topologies. TACOS is highly flexible, synthesizing an All-Reduce algorithm for a heterogeneous 128-NPU system in just 1.08 seconds, while achieving up to a 4.27x performance improvement over state-of-the-art synthesizers. Additionally, TACOS demonstrates better scalability with polynomial synthesis times, in contrast to NP-hard approaches which only scale to systems with tens of NPUs. TACOS can synthesize for 40K NPUs in just 2.52 hours.

cs.DC↗
arXiv2024

Balancing Notions of Equity: Trade-offs Between Fair Portfolio Sizes and Achievable Guarantees

Motivated by fairness concerns, we study the `portfolio problem': given an optimization problem with set $D$ of feasible solutions, a class $\mathbf{C}$ of fairness objective functions on $D$, and an approximation factor $α\ge 1$, a set $X \subseteq D$ of feasible solutions is an $α$-approximate portfolio if for each objective $f \in \mathbf{C}$, there is an $α$-approximation for $f$ in $X$. Choosing the classes of top-$k$ norms, ordered norms, and symmetric monotonic norms as our equity objectives, we study the trade-off between the size $|X|$ of the portfolio and its approximation factor $α$ for various combinatorial problems. For the problem of scheduling identical jobs on unidentical machines, we characterize this trade-off for ordered norms and give an exponential improvement in size for symmetric monotonic norms over the general upper bound. We generalize this result as the OrderAndCount framework that obtains an exponential improvement in portfolio sizes for covering polyhedra with a constant number of constraints. Our framework is based on a novel primal-dual counting technique that may be of independent interest. We also introduce a general IterativeOrdering framework for simultaneous approximations or portfolios of size $1$ for symmetric monotonic norms, which generalizes and extends existing results for problems such as scheduling, $k$-clustering, set cover, and routing.

cs.DS↗
arXiv2024

Which $L_p$ norm is the fairest? Approximations for fair facility location across all "$p$"

Fair facility location problems try to balance access costs to open facilities borne by different groups of people by minimizing the $L_p$ norm of these group distances. However, there is no clear choice of "$p$" in the current literature. We present a novel approach to address the challenge of choosing the right notion of fairness. We introduce the concept of portfolios, a set of solutions that contains an approximately optimal solution for each objective in a given class of objectives, such as $L_p$ norms. This concept opens up new possibilities for getting around the "right" notion of fairness for many problems. For $r$ client groups, we demonstrate portfolios of size $Θ(\log r)$ for the facility location and $k$-clustering problems, with an $O(1)$-approximate solution for each $L_p$ norm. Further, motivated by the Justice40 Initiative that provides rolling budget investments, we impose a refinement-like structure on the portfolio. We develop novel approximation algorithms for these structured portfolios and show experimental evidence of their performance in two US counties. We also present a planning tool that provides potential ways to expand access to US healthcare facilities, which might be of independent interest to policymakers.

cs.DS↗
arXiv2023

Hardness and Approximation of Submodular Minimum Linear Ordering Problems

The minimum linear ordering problem (MLOP) generalizes well-known combinatorial optimization problems such as minimum linear arrangement and minimum sum set cover. MLOP seeks to minimize an aggregated cost $f(\cdot)$ due to an ordering $σ$ of the items (say $[n]$), i.e., $\min_σ \sum_{i\in [n]} f(E_{i,σ})$, where $E_{i,σ}$ is the set of items mapped by $σ$ to indices $[i]$. Despite an extensive literature on MLOP variants and approximations for these, it was unclear whether the graphic matroid MLOP was NP-hard. We settle this question through non-trivial reductions from mininimum latency vertex cover and minimum sum vertex cover problems. We further propose a new combinatorial algorithm for approximating monotone submodular MLOP, using the theory of principal partitions. This is in contrast to the rounding algorithm by Iwata, Tetali, and Tripathi [ITT2012], using Lovász extension of submodular functions. We show a $(2-\frac{1+\ell_{f}}{1+|E|})$-approximation for monotone submodular MLOP where $\ell_{f}=\frac{f(E)}{\max_{x\in E}f(\{x\})}$ satisfies $1 \leq \ell_f \leq |E|$. Our theory provides new approximation bounds for special cases of the problem, in particular a $(2-\frac{1+r(E)}{1+|E|})$-approximation for the matroid MLOP, where $f = r$ is the rank function of a matroid. We further show that minimum latency vertex cover (MLVC) is $\frac{4}{3}$-approximable, by which we also lower bound the integrality gap of its natural LP relaxation, which might be of independent interest.

cs.DS↗
arXiv2023

Warm-Started QAOA with Custom Mixers Provably Converges and Computationally Beats Goemans-Williamson's Max-Cut at Low Circuit Depths

We generalize the Quantum Approximate Optimization Algorithm (QAOA) of Farhi et al. (2014) to allow for arbitrary separable initial states with corresponding mixers such that the starting state is the most excited state of the mixing Hamiltonian. We demonstrate this version of QAOA, which we call QAOA-warmest, by simulating Max-Cut on weighted graphs. We initialize the starting state as a warm-start using $2$ and $3$-dimensional approximations obtained using randomized projections of solutions to Max-Cut's semi-definite program, and define a warm-start dependent custom mixer. We show that these warm-starts initialize the QAOA circuit with constant-factor approximations of $0.658$ for $2$-dimensional and $0.585$ for $3$-dimensional warm-starts for graphs with non-negative edge weights, improving upon previously known trivial (i.e., $0.5$ for standard initialization) worst-case bounds at $p=0$. These factors in fact lower bound the approximation achieved for Max-Cut at higher circuit depths, since we also show that QAOA-warmest with any separable initial state converges to Max-Cut under the adiabatic limit as $p\rightarrow \infty$. However, the choice of warm-starts significantly impacts the rate of convergence to Max-Cut, and we show empirically that our warm-starts achieve a faster convergence compared to existing approaches. Additionally, our numerical simulations show higher quality cuts compared to standard QAOA, the classical Goemans-Williamson algorithm, and a warm-started QAOA without custom mixers for an instance library of $1148$ graphs (upto $11$ nodes) and depth $p=8$. We further show that QAOA-warmest outperforms the standard QAOA of Farhi et al. in experiments on current IBM-Q and Quantinuum hardware.

quant-ph↗
arXiv2023

Discovering Opportunities in New York City's Discovery Program: Disadvantaged Students in Highly Competitive Markets

Discovery program (DISC) is a policy used by the New York City Department of Education (NYC DOE) to increase the number of admissions of students from low socio-economic background to specialized high schools. This policy has been instrumental in increasing the number of disadvantaged students attending these schools. However, assuming that students care more about the school they are assigned to rather than the type of seat they occupy (\emph{school-over-seat hypothesis}), our empirical analysis using data from 12 recent academic years shows that DISC creates about 950 in-group blocking pairs each year amongst disadvantaged students, impacting about 650 disadvantaged students every year. Moreover, we find that this program does not respect improvements, thus unintentionally creating an incentive to under-perform. These experimental results are confirmed by our theoretical analysis. In order to alleviate the concerns caused by DISC, we explore two alternative policies: the minority reserve (MR) and the joint-seat allocation (JSA) mechanisms. As our main theoretical contribution, we introduce a feature of markets, that we term high competitiveness, and we show that under this condition, JSA dominates MR for all disadvantaged students. We give sufficient conditions under which high competitiveness is verified. Data from NYC DOE satisfies the high competitiveness condition, and for this dataset our empirical results corroborate our theoretical predictions, showing the superiority of JSA. Given that JSA can be implemented by a simple modification of the classical deferred acceptance algorithm with responsive preference lists, we believe that, when the school-over-seat hypothesis holds, the discovery program can be changed for the better by implementing the JSA mechanism, leading in particular to aligned incentives for the top-performing disadvantaged students.

cs.GT↗
arXiv2023

Artificial Intelligence/Operations Research Workshop 2 Report Out

This workshop Report Out focuses on the foundational elements of trustworthy AI and OR technology, and how to ensure all AI and OR systems implement these elements in their system designs. Four sessions on various topics within Trustworthy AI were held, these being Fairness, Explainable AI/Causality, Robustness/Privacy, and Human Alignment and Human-Computer Interaction. Following discussions of each of these topics, workshop participants also brainstormed challenge problems which require the collaboration of AI and OR researchers and will result in the integration of basic techniques from both fields to eventually benefit societal needs.

cs.AI↗
arXiv2023

Reusing Combinatorial Structure: Faster Iterative Projections over Submodular Base Polytopes

Optimization algorithms such as projected Newton's method, FISTA, mirror descent, and its variants enjoy near-optimal regret bounds and convergence rates, but suffer from a computational bottleneck of computing ``projections'' in potentially each iteration (e.g., $O(T^{1/2})$ regret of online mirror descent). On the other hand, conditional gradient variants solve a linear optimization in each iteration, but result in suboptimal rates (e.g., $O(T^{3/4})$ regret of online Frank-Wolfe). Motivated by this trade-off in runtime v/s convergence rates, we consider iterative projections of close-by points over widely-prevalent submodular base polytopes $B(f)$. We first give necessary and sufficient conditions for when two close points project to the same face of a polytope, and then show that points far away from the polytope project onto its vertices with high probability. We next use this theory and develop a toolkit to speed up the computation of iterative projections over submodular polytopes using both discrete and continuous perspectives. We subsequently adapt the away-step Frank-Wolfe algorithm to use this information and enable early termination. For the special case of cardinality-based submodular polytopes, we improve the runtime of computing certain Bregman projections by a factor of $Ω(n/\log(n))$. Our theoretical results show orders of magnitude reduction in runtime in preliminary computational experiments.

cs.LG↗
arXiv2022

Tree DNN: A Deep Container Network

Multi-Task Learning (MTL) has shown its importance at user products for fast training, data efficiency, reduced overfitting etc. MTL achieves it by sharing the network parameters and training a network for multiple tasks simultaneously. However, MTL does not provide the solution, if each task needs training from a different dataset. In order to solve the stated problem, we have proposed an architecture named TreeDNN along with it's training methodology. TreeDNN helps in training the model with multiple datasets simultaneously, where each branch of the tree may need a different training dataset. We have shown in the results that TreeDNN provides competitive performance with the advantage of reduced ROM requirement for parameter storage and increased responsiveness of the system by loading only specific branch at inference time.

cs.LG↗
arXiv2022

Mathematically Quantifying Non-responsiveness of the 2021 Georgia Congressional Districting Plan

To audit political district maps for partisan gerrymandering, one may determine a baseline for the expected distribution of partisan outcomes by sampling an ensemble of maps. One approach to sampling is to use redistricting policy as a guide to precisely codify preferences between maps. Such preferences give rise to a probability distribution on the space of redistricting plans, and Metropolis-Hastings methods allow one to sample ensembles of maps from the specified distribution. Although these approaches have nice theoretical properties and have successfully detected gerrymandering in legal settings, sampling from commonly-used policy-driven distributions is often computationally difficult. As of yet, there is no algorithm that can be used off-the-shelf for checking maps under generic redistricting criteria. In this work, we mitigate the computational challenges in a Metropolized-sampling technique through a parallel tempering method combined with ReCom[12] and, for the first time, validate that such techniques are effective on these problems at the scale of statewide precinct graphs for more policy informed measures. We develop these improvements through the first case study of district plans in Georgia. Our analysis projects that any election in Georgia will reliably elect 9 Republicans and 5 Democrats under the enacted plan. This result is largely fixed even as public opinion shifts toward either party and the partisan outcome of the enacted plan does not respond to the will of the people. Only 0.12% of the $\sim$160K plans in our ensemble were similarly non-responsive.

cs.CY↗
arXiv2022

Fair and Reliable Reconnections for Temporary Disruptions in Electric Distribution Networks using Submodularity

Increasing reliability and reducing disruptions in supply networks are of increasing importance; for example, power outages in electricity distribution networks cost \$35-50 billion annually in the US. Motivated by the operational constraints of such networks and their rapid adoption of decentralized paradigms and self-healing components, we introduce the "minimum reconnection time" (MRT) problem. MRT seeks to reduce outage time after network disruptions by programming reconnection times of different edges (i.e., switches), while ensuring that the operating network is acyclic. We show that MRT is NP-hard and is a special case of the well-known minimum linear ordering problem (MLOP) in the submodular optimization literature. MLOP is a special case of a broader class of ordering problems that often admit polynomial time approximation algorithms. We develop the theory of kernel-based randomized rounding approaches to give a tight polynomial-time approximation for MRT, improving the state-of-the-art approximation factor for a broad class of MLOP instances. Further, motivated by the reliability incentive structure for utility companies and operational energy losses in distribution networks, we propose local search over spanning trees to balance multiple objectives simultaneously. We computationally validate our reconfiguration methods on the NREL SMART-DS Greensboro synthetic network, and show that this improves service equity by a factor of four, across industrial and residential areas.

math.OC↗
arXiv2022

Bridging Classical and Quantum with SDP initialized warm-starts for QAOA

We study the Quantum Approximate Optimization Algorithm (QAOA) in the context of the Max-Cut problem. Near-term (noisy) quantum devices are only able to (accurately) execute QAOA at low circuit depths while QAOA requires a relatively high circuit-depth in order to "see" the whole graph. We introduce a classical pre-processing step that initializes QAOA with a biased superposition of all possible cuts in the graph, referred to as a warm-start. In particular, our initialization informs QAOA by a solution to a low-rank semidefinite programming relaxation of the Max-Cut problem. Our experimental results show that this variant of QAOA, called QAOA-Warm, is able to outperform standard QAOA on lower circuit depths with less training time (in the optimization stage for QAOA's variational parameters). We provide experimental evidence as well as theoretical intuition on performance of the proposed framework.

quant-ph↗
arXiv2022

Generating Target Graph Couplings for QAOA from Native Quantum Hardware Couplings

We present methods for constructing any target coupling graph using limited global controls in an Ising-like quantum spin system. Our approach is motivated by implementing the quantum approximate optimization algorithm (QAOA) on trapped ion quantum hardware to find approximate solutions to Max-Cut. We present a mathematical description of the problem and provide approximately optimal algorithmic constructions that generate arbitrary unweighted coupling graphs with $n$ nodes in $O(n)$ global entangling operations and weighted graphs with $m$ edges in $O(m)$ operations. These upper bounds are not tight in general, and we formulate a mixed-integer program to solve the graph coupling problem to optimality. We perform numeric experiments on small graphs with $n\le8$ and show that optimal sequences, which use fewer operations, can be found using mixed-integer programs. Noisy simulations of Max-Cut QAOA show that our implementation is less susceptible to noise than the standard gate-based compilation.

quant-ph↗
arXiv2022

Don't let Ricci v. DeStefano Hold You Back: A Bias-Aware Legal Solution to the Hiring Paradox

Companies that try to address inequality in employment face a hiring paradox. Failing to address workforce imbalance can result in legal sanctions and scrutiny, but proactive measures to address these issues might result in the same legal conflict. Recent run-ins of Microsoft and Wells Fargo with the Labor Department's Office of Federal Contract Compliance Programs (OFCCP) are not isolated and are likely to persist. To add to the confusion, existing scholarship on Ricci v. DeStefano often deems solutions to this paradox impossible. Circumventive practices such as the 4/5ths rule further illustrate tensions between too little action and too much action. In this work, we give a powerful way to solve this hiring paradox that tracks both legal and algorithmic challenges. We unpack the nuances of Ricci v. DeStefano and extend the legal literature arguing that certain algorithmic approaches to employment are allowed by introducing the legal practice of banding to evaluate candidates. We thus show that a bias-aware technique can be used to diagnose and mitigate "built-in" headwinds in the employment pipeline. We use the machinery of partially ordered sets to handle the presence of uncertainty in evaluations data. This approach allows us to move away from treating "people as numbers" to treating people as individuals -- a property that is sought after by Title VII in the context of employment.

cs.CY↗
arXiv2021

Balanced Districting on Grid Graphs with Provable Compactness and Contiguity

Given a graph $G = (V,E)$ with vertex weights $w(v)$ and a desired number of parts $k$, the goal in graph partitioning problems is to partition the vertex set V into parts $V_1,\ldots,V_k$. Metrics for compactness, contiguity, and balance of the parts $V_i$ are frequent objectives, with much existing literature focusing on compactness and balance. Revisiting an old method known as striping, we give the first polynomial-time algorithms with guaranteed contiguity and provable bicriteria approximations for compactness and balance for planar grid graphs. We consider several types of graph partitioning, including when vertex weights vary smoothly or are stochastic, reflecting concerns in various real-world instances. We show significant improvements in experiments for balancing workloads for the fire department and reducing over-policing using 911 call data from South Fulton, GA.

cs.DS↗
arXiv2021

Multi-Threshold Attention U-Net (MTAU) based Model for Multimodal Brain Tumor Segmentation in MRI scans

Gliomas are one of the most frequent brain tumors and are classified into high grade and low grade gliomas. The segmentation of various regions such as tumor core, enhancing tumor etc. plays an important role in determining severity and prognosis. Here, we have developed a multi-threshold model based on attention U-Net for identification of various regions of the tumor in magnetic resonance imaging (MRI). We propose a multi-path segmentation and built three separate models for the different regions of interest. The proposed model achieved mean Dice Coefficient of 0.59, 0.72, and 0.61 for enhancing tumor, whole tumor and tumor core respectively on the training dataset. The same model gave mean Dice Coefficient of 0.57, 0.73, and 0.61 on the validation dataset and 0.59, 0.72, and 0.57 on the test dataset.

cs.CV↗
arXiv2020

Electrical Flows over Spanning Trees

The network reconfiguration problem seeks to find a rooted tree $T$ such that the energy of the (unique) feasible electrical flow over $T$ is minimized. The tree requirement on the support of the flow is motivated by operational constraints in electricity distribution networks. The bulk of existing results on convex optimization over vertices of polytopes and on the structure of electrical flows do not easily give guarantees for this problem, while many heuristic methods have been developed in the power systems community as early as 1989. Our main contribution is to give the first provable approximation guarantees for the network reconfiguration problem. We provide novel lower bounds and corresponding approximation factors for various settings ranging from $\min\{O(m-n), O(n)\}$ for general graphs, to $O(\sqrt{n})$ over grids with uniform resistances on edges, and $O(1)$ for grids with uniform edge resistances and demands. To obtain the result for general graphs, we propose a new method for (approximate) spectral graph sparsification, which may be of independent interest. Using insights from our theoretical results, we propose a general heuristic for the network reconfiguration problem that is orders of magnitude faster than existing methods in the literature, while obtaining comparable performance.

cs.DS↗
arXiv2020

Group-Fair Online Allocation in Continuous Time

The theory of discrete-time online learning has been successfully applied in many problems that involve sequential decision-making under uncertainty. However, in many applications including contractual hiring in online freelancing platforms and server allocation in cloud computing systems, the outcome of each action is observed only after a random and action-dependent time. Furthermore, as a consequence of certain ethical and economic concerns, the controller may impose deadlines on the completion of each task, and require fairness across different groups in the allocation of total time budget $B$. In order to address these applications, we consider continuous-time online learning problem with fairness considerations, and present a novel framework based on continuous-time utility maximization. We show that this formulation recovers reward-maximizing, max-min fair and proportionally fair allocation rules across different groups as special cases. We characterize the optimal offline policy, which allocates the total time between different actions in an optimally fair way (as defined by the utility function), and impose deadlines to maximize time-efficiency. In the absence of any statistical knowledge, we propose a novel online learning algorithm based on dual ascent optimization for time averages, and prove that it achieves $\tilde{O}(B^{-1/2})$ regret bound.

cs.LG↗