arXiv subjects

Ayush Goel

Publications and source records attributed to Ayush Goel.

5 recordsLinked to original sources

arXiv2026

Curvature-Weighted Gradient Diversity: A Noise Measure for Geometry-Adaptive SGD Schedules

The standard convergence analysis of mini-batch stochastic gradient descent (SGD) models gradient noise using a single variance term that treats all parameter directions equally, ignoring the fact that noise in high-curvature directions has less impact because learning rates are already constrained there. We introduce Curvature-Weighted Gradient Diversity (CWGD), a geometry-aware measure that weights per-sample gradient diversity by the inverse square root of the Hessian, providing a tighter proxy for the effective optimization noise. For strongly convex quadratic objectives with diagonal Hessians and isotropic noise, we prove that a CWGD-modulated cosine learning-rate schedule can reduce the asymptotic optimization error floor by up to a factor of two compared with standard cosine annealing. We implement this idea as CWGD-Cosine using a Hutchinson-based diagonal Hessian estimator that is exact for quadratic objectives. Across a range of condition numbers, batch sizes, and noise structures, CWGD-Cosine consistently achieves approximately 20% lower final optimization error than standard cosine annealing while incurring negligible overhead in the quadratic setting. We also identify and correct a degenerate curvature estimator, analyze the robustness of the proposed estimator, and explicitly discuss the limitations of the method, including Hessian staleness in non-convex optimization. These results establish CWGD as a principled geometry-aware measure of optimization noise and motivate future extensions to more general learning problems.

cs.LG↗

arXiv2026

In-Context Learning in Linear vs. Quadratic Attention Models: An Empirical Study on Regression Tasks

Recent work has demonstrated that transformers and linear attention models can perform in-context learning (ICL) on simple function classes, such as linear regression. In this paper, we empirically study how these two attention mechanisms differ in their ICL behavior on the canonical linear-regression task of Garg et al. We evaluate learning quality (MSE), convergence, and generalization behavior of each architecture. We also analyze how increasing model depth affects ICL performance. Our results illustrate both the similarities and limitations of linear attention relative to quadratic attention in this setting.

cs.LG↗

arXiv2021

Near Real-time Learning and Extraction of Attack Models from Intrusion Alerts

Critical and sophisticated cyberattacks often take multitudes of reconnaissance, exploitations, and obfuscation techniques to penetrate through well protected enterprise networks. The discovery and detection of attacks, though needing continuous efforts, is no longer sufficient. Security Operation Center (SOC) analysts are overwhelmed by the significant volume of intrusion alerts without being able to extract actionable intelligence. Recognizing this challenge, this paper describes the advances and findings through deploying ASSERT to process intrusion alerts from OmniSOC in collaboration with the Center for Applied Cybersecurity Research (CACR) at Indiana University. ASSERT utilizes information theoretic unsupervised learning to extract and update `attack models' in near real-time without expert knowledge. It consumes streaming intrusion alerts and generates a small number of statistical models for SOC analysts to comprehend ongoing and emerging attacks in a timely manner. This paper presents the architecture and key processes of ASSERT and discusses a few real-world attack models to highlight the use-cases that benefit SOC operations. The research team is developing a light-weight containerized ASSERT that will be shared through a public repository to help the community combat the overwhelming intrusion alerts.

cs.CR↗

arXiv2020

An Empirical Review of Adversarial Defenses

From face recognition systems installed in phones to self-driving cars, the field of AI is witnessing rapid transformations and is being integrated into our everyday lives at an incredible pace. Any major failure in these system's predictions could be devastating, leaking sensitive information or even costing lives (as in the case of self-driving cars). However, deep neural networks, which form the basis of such systems, are highly susceptible to a specific type of attack, called adversarial attacks. A hacker can, even with bare minimum computation, generate adversarial examples (images or data points that belong to another class, but consistently fool the model to get misclassified as genuine) and crumble the basis of such algorithms. In this paper, we compile and test numerous approaches to defend against such adversarial attacks. Out of the ones explored, we found two effective techniques, namely Dropout and Denoising Autoencoders, and show their success in preventing such attacks from fooling the model. We demonstrate that these techniques are also resistant to both higher noise levels as well as different kinds of adversarial attacks (although not tested against all). We also develop a framework for deciding the suitable defense technique to use against attacks, based on the nature of the application and resource constraints of the Deep Neural Network.

cs.CR↗

arXiv2017

Incremental DFS algorithms: a theoretical and experimental study

Depth First Search (DFS) tree is a fundamental data structure for solving graph problems. The DFS tree of a graph $G$ with $n$ vertices and $m$ edges can be built in $O(m+n)$ time. Till date, only a few algorithms have been designed for maintaining incremental DFS. For undirected graphs, the two algorithms, namely, ADFS1 and ADFS2 [ICALP14] achieve total $O(n^{3/2}\sqrt{m})$ and $O(n^2)$ time respectively. For DAGs, the only non-trivial algorithm, namely, FDFS [IPL97] requires total $O(mn)$ time. In this paper, we carry out extensive experimental and theoretical evaluation of existing incremental DFS algorithms in random and real graphs, and derive the following results. 1- For insertion of uniformly random sequence of $n \choose 2$ edges, ADFS1, ADFS2 and FDFS perform equally well and are found to take $Θ(n^2)$ time experimentally. This is quite surprising because the worst case bounds of ADFS1 and FDFS are greater than $Θ(n^2)$ by a factor of $\sqrt{m/n}$ and $m/n$ respectively. We complement this result by probabilistic analysis of these algorithms proving $\tilde{O}(n^2)$ bound on the update time. Here, we derive results about the structure of a DFS tree in random graphs, which are of independent interest. 2- These insights led us to design an extremely simple incremental DFS algorithm for both undirected and directed graphs. This algorithm theoretically matches and experimentally outperforms the state-of-the-art in dense random graphs. It can also be used as a single-pass semi-streaming algorithm for incremental DFS and strong connectivity in random graphs. 3- Even for real graphs, both ADFS1 and FDFS perform much better than their theoretical bounds. Here again, we present two simple algorithms for incremental DFS for directed and undirected real graphs. In fact, our algorithm for directed graphs almost always matches the performance of FDFS.

cs.DS↗