arXiv subjects

Chetan Verma

Publications and source records attributed to Chetan Verma.

9 recordsLinked to original sources

arXiv2025

Matryoshka Model Learning for Improved Elastic Student Models

Industry-grade ML models are carefully designed to meet rapidly evolving serving constraints, which requires significant resources for model development. In this paper, we propose MatTA, a framework for training multiple accurate Student models using a novel Teacher-TA-Student recipe. TA models are larger versions of the Student models with higher capacity, and thus allow Student models to better relate to the Teacher model and also bring in more domain-specific expertise. Furthermore, multiple accurate Student models can be extracted from the TA model. Therefore, despite only one training run, our methodology provides multiple servable options to trade off accuracy for lower serving cost. We demonstrate the proposed method, MatTA, on proprietary datasets and models. Its practical efficacy is underscored by live A/B tests within a production ML system, demonstrating 20% improvement on a key metric. We also demonstrate our method on GPT-2 Medium, a public model, and achieve relative improvements of over 24% on SAT Math and over 10% on the LAMBADA benchmark.

cs.LG↗

arXiv2024

Unpacking the Resilience of SNLI Contradiction Examples to Attacks

Pre-trained models excel on NLI benchmarks like SNLI and MultiNLI, but their true language understanding remains uncertain. Models trained only on hypotheses and labels achieve high accuracy, indicating reliance on dataset biases and spurious correlations. To explore this issue, we applied the Universal Adversarial Attack to examine the model's vulnerabilities. Our analysis revealed substantial drops in accuracy for the entailment and neutral classes, whereas the contradiction class exhibited a smaller decline. Fine-tuning the model on an augmented dataset with adversarial examples restored its performance to near-baseline levels for both the standard and challenge sets. Our findings highlight the value of adversarial triggers in identifying spurious correlations and improving robustness while providing insights into the resilience of the contradiction class to adversarial attacks.

cs.CL↗

arXiv2023

URL-BERT: Training Webpage Representations via Social Media Engagements

Understanding and representing webpages is crucial to online social networks where users may share and engage with URLs. Common language model (LM) encoders such as BERT can be used to understand and represent the textual content of webpages. However, these representations may not model thematic information of web domains and URLs or accurately capture their appeal to social media users. In this work, we introduce a new pre-training objective that can be used to adapt LMs to understand URLs and webpages. Our proposed framework consists of two steps: (1) scalable graph embeddings to learn shallow representations of URLs based on user engagement on social media and (2) a contrastive objective that aligns LM representations with the aforementioned graph-based representation. We apply our framework to the multilingual version of BERT to obtain the model URL-BERT. We experimentally demonstrate that our continued pre-training approach improves webpage understanding on a variety of tasks and Twitter internal and external benchmarks.

cs.CL↗

arXiv2022

Detection of Gravitational Wave Signals from Precessing Binary Black Hole Systems using Convolutional Neural Network

Current searches for gravitational waves (GWs) from black hole binaries using the LIGO and Virgo observatories are limited to analytical models for systems with black hole spins aligned (or anti-aligned) with the orbital angular momentum of the binary. Detecting black hole binaries with precessing spinsis crucial for gaining unique astrophysical insights into the formation of these sources. Therefore, it is essential to develop a search strategy capable of identifying compact binaries with precessing spins. Aligned-spin waveform models are inadequate for detecting compact binaries with high precessing spins. While several efforts have been made to construct template banks for detecting precessing binaries using matched filtering, this approach requires many templates to cover the entire search parameter space, significantly increasing the computational cost. This work explores the detection of GW signals from binary black holes(BBH) with both aligned and precessing spins using a convolutional neural network (CNN). We frame the detection of GW signals from aligned or precessing BBH systems as a hierarchical binary classification problem. The first CNN model classifies strain data as either pure noise or noisy signals (GWs from BBH). A second CNN model then classifies the detected noisy signal data as originating from either precessing or non-precessing (aligned/anti-aligned) systems. Using simulated data, the trained classifier distinguishes between noise and noisy GW signals with more than 99% accuracy. The second classifier further differentiates between aligned and highly precessing signals with around 95% accuracy. We extended our analysis to a multi-detector framework by performing a coincident test. Additionally, we tested the performance of our trained architecture on data from the first three observation runs of LIGO to identify detected BBH events as either aligned or precessing.

gr-qc↗

arXiv2022

Electricity Consumption Forecasting for Out-of-distribution Time-of-Use Tariffs

In electricity markets, retailers or brokers want to maximize profits by allocating tariff profiles to end consumers. One of the objectives of such demand response management is to incentivize the consumers to adjust their consumption so that the overall electricity procurement in the wholesale markets is minimized, e.g. it is desirable that consumers consume less during peak hours when cost of procurement for brokers from wholesale markets are high. We consider a greedy solution to maximize the overall profit for brokers by optimal tariff profile allocation. This in-turn requires forecasting electricity consumption for each user for all tariff profiles. This forecasting problem is challenging compared to standard forecasting problems due to following reasons: i. the number of possible combinations of hourly tariffs is high and retailers may not have considered all combinations in the past resulting in a biased set of tariff profiles tried in the past, ii. the profiles allocated in the past to each user is typically based on certain policy. These reasons violate the standard i.i.d. assumptions, as there is a need to evaluate new tariff profiles on existing customers and historical data is biased by the policies used in the past for tariff allocation. In this work, we consider several scenarios for forecasting and optimization under these conditions. We leverage the underlying structure of how consumers respond to variable tariff rates by comparing tariffs across hours and shifting loads, and propose suitable inductive biases in the design of deep neural network based architectures for forecasting under such scenarios. More specifically, we leverage attention mechanisms and permutation equivariant networks that allow desirable processing of tariff profiles to learn tariff representations that are insensitive to the biases in the data and still representative of the task.

cs.LG↗

arXiv2022

TwHIN: Embedding the Twitter Heterogeneous Information Network for Personalized Recommendation

Social networks, such as Twitter, form a heterogeneous information network (HIN) where nodes represent domain entities (e.g., user, content, advertiser, etc.) and edges represent one of many entity interactions (e.g, a user re-sharing content or "following" another). Interactions from multiple relation types can encode valuable information about social network entities not fully captured by a single relation; for instance, a user's preference for accounts to follow may depend on both user-content engagement interactions and the other users they follow. In this work, we investigate knowledge-graph embeddings for entities in the Twitter HIN (TwHIN); we show that these pretrained representations yield significant offline and online improvement for a diverse range of downstream recommendation and classification tasks: personalized ads rankings, account follow-recommendation, offensive content detection, and search ranking. We discuss design choices and practical challenges of deploying industry-scale HIN embeddings, including compressing them to reduce end-to-end model latency and handling parameter drift across versions.

cs.SI↗

arXiv2022

Challenges and approaches to privacy preserving post-click conversion prediction

Online advertising has typically been more personalized than offline advertising, through the use of machine learning models and real-time auctions for ad targeting. One specific task, predicting the likelihood of conversion (i.e.\ the probability a user will purchase the advertised product), is crucial to the advertising ecosystem for both targeting and pricing ads. Currently, these models are often trained by observing individual user behavior, but, increasingly, regulatory and technical constraints are requiring privacy-preserving approaches. For example, major platforms are moving to restrict tracking individual user events across multiple applications, and governments around the world have shown steadily more interest in regulating the use of personal data. Instead of receiving data about individual user behavior, advertisers may receive privacy-preserving feedback, such as the number of installs of an advertised app that resulted from a group of users. In this paper we outline the recent privacy-related changes in the online advertising ecosystem from a machine learning perspective. We provide an overview of the challenges and constraints when learning conversion models in this setting. We introduce a novel approach for training these models that makes use of post-ranking signals. We show using offline experiments on real world data that it outperforms a model relying on opt-in data alone, and significantly reduces model degradation when no individual labels are available. Finally, we discuss future directions for research in this evolving area.

cs.LG↗

arXiv2021

Employing Deep Learning for Detection of Gravitational Waves from Compact Binary Coalescences

The matched filtering paradigm is the mainstay of gravitational wave (GW) searches from astrophysical coalescing compact binaries. The compact binary coalescence (CBC) search pipelines perform the matched filter between the GW detector's data and a large set of analytical waveforms. However, the computational cost of performing matched filter is very high as the required number of the analytical waveforms is also high. Recently, various deep learning-based methods have been deployed to identify a GW signal in the detector output as an alternative to computationally expensive matched filtering techniques. In past work, the researchers have considered the detection of GW signal mainly as a classification problem, in which they train the deep learning-based architecture by considering the noise and the GW signal as two different classes. However, in this work, for the first time, we have combined the Convolutional Neural Network (CNN) and matched filter methods to reduce the computational cost of the search by reducing the number of matched filtering operations. We have implemented the CNN based architecture not only for classification of the signal but also to identify the location of the signal in the intrinsic parameter space. Identifying the location in which the detected signal lies enables us to perform the matched filter operations between the data and the analytical waveforms generated for the smaller region of the parameter space only - thereby reducing the computational cost of the search. We demonstrate our method for two-dimensional parameter space for stellar to high mass binary black hole systems. In particular, we are able to classify between pure noise and noisy BBH signals with 99% accuracy. Further, the detected signals have been sub-classified into patches in mass components with an average accuracy > 97%

gr-qc↗

arXiv2021

Security Assessment Rating Framework for Enterprises using MITRE ATT&CK Matrix

Threats targeting cyberspace are becoming more prominent and intelligent day by day. This inherently leads to a dire demand for continuous security validation and testing. Using this paper, we aim to provide a holistic and precise security analysis rating framework for organizations that increases the overall coherency of the outcomes of such testing. This scorecard is based on the security assessment performed following the globally accessible knowledge base of adversary tactics and techniques called the MITRE ATTACK matrix. The scorecard for an evaluation is generated by ingesting the security testing results into our framework, which provides an organizations overall risk assessment rating and the risk related to each of the different tactics from the ATTACK matrix.

cs.CR↗