arXiv subjects

Mohit Sharma

Publications and source records attributed to Mohit Sharma.

At least 37 records · Page 2Linked to original sources

arXiv2024

Jacobi Set Simplification for Tracking Topological Features in Time-Varying Scalar Fields

The Jacobi set of a bivariate scalar field is the set of points where the gradients of the two constituent scalar fields align with each other. It captures the regions of topological changes in the bivariate field. The Jacobi set is a bivariate analog of critical points, and may correspond to features of interest. In the specific case of time-varying fields and when one of the scalar fields is time, the Jacobi set corresponds to temporal tracks of critical points, and serves as a feature-tracking graph. The Jacobi set of a bivariate field or a time-varying scalar field is complex, resulting in cluttered visualizations that are difficult to analyze. This paper addresses the problem of Jacobi set simplification. Specifically, we use the time-varying scalar field scenario to introduce a method that computes a reduced Jacobi set. The method is based on a stability measure called robustness that was originally developed for vector fields and helps capture the structural stability of critical points. We also present a mathematical analysis for the method, and describe an implementation for 2D time-varying scalar fields. Applications to both synthetic and real-world datasets demonstrate the effectiveness of the method for tracking features.

math.NA↗

arXiv2024

How Far Can Fairness Constraints Help Recover From Biased Data?

A general belief in fair classification is that fairness constraints incur a trade-off with accuracy, which biased data may worsen. Contrary to this belief, Blum & Stangl (2019) show that fair classification with equal opportunity constraints even on extremely biased data can recover optimally accurate and fair classifiers on the original data distribution. Their result is interesting because it demonstrates that fairness constraints can implicitly rectify data bias and simultaneously overcome a perceived fairness-accuracy trade-off. Their data bias model simulates under-representation and label bias in underprivileged population, and they show the above result on a stylized data distribution with i.i.d. label noise, under simple conditions on the data distribution and bias parameters. We propose a general approach to extend the result of Blum & Stangl (2019) to different fairness constraints, data bias models, data distributions, and hypothesis classes. We strengthen their result, and extend it to the case when their stylized distribution has labels with Massart noise instead of i.i.d. noise. We prove a similar recovery result for arbitrary data distributions using fair reject option classifiers. We further generalize it to arbitrary data distributions and arbitrary hypothesis classes, i.e., we prove that for any data distribution, if the optimally accurate classifier in a given hypothesis class is fair and robust, then it can be recovered through fair classification with equal opportunity constraints on the biased distribution whenever the bias parameters satisfy certain simple conditions. Finally, we show applications of our technique to time-varying data bias in classification and fair machine learning pipelines.

cs.LG↗

arXiv2024

MResT: Multi-Resolution Sensing for Real-Time Control with Vision-Language Models

Leveraging sensing modalities across diverse spatial and temporal resolutions can improve performance of robotic manipulation tasks. Multi-spatial resolution sensing provides hierarchical information captured at different spatial scales and enables both coarse and precise motions. Simultaneously multi-temporal resolution sensing enables the agent to exhibit high reactivity and real-time control. In this work, we propose a framework, MResT (Multi-Resolution Transformer), for learning generalizable language-conditioned multi-task policies that utilize sensing at different spatial and temporal resolutions using networks of varying capacities to effectively perform real time control of precise and reactive tasks. We leverage off-the-shelf pretrained vision-language models to operate on low-frequency global features along with small non-pretrained models to adapt to high frequency local feedback. Through extensive experiments in 3 domains (coarse, precise and dynamic manipulation tasks), we show that our approach significantly improves (2X on average) over recent multi-task baselines. Further, our approach generalizes well to visual and geometric variations in target objects and to varying interaction forces.

cs.RO↗

arXiv2023

On Comparing Fair Classifiers under Data Bias

In this paper, we consider a theoretical model for injecting data bias, namely, under-representation and label bias (Blum & Stangl, 2019). We empirically study the effect of varying data biases on the accuracy and fairness of fair classifiers. Through extensive experiments on both synthetic and real-world datasets (e.g., Adult, German Credit, Bank Marketing, COMPAS), we empirically audit pre-, in-, and post-processing fair classifiers from standard fairness toolkits for their fairness and accuracy by injecting varying amounts of under-representation and label bias in their training data (but not the test data). Our main observations are: 1. The fairness and accuracy of many standard fair classifiers degrade severely as the bias injected in their training data increases, 2. A simple logistic regression model trained on the right data can often outperform, in both accuracy and fairness, most fair classifiers trained on biased training data, and 3. A few, simple fairness techniques (e.g., reweighing, exponentiated gradients) seem to offer stable accuracy and fairness guarantees even when their training data is injected with under-representation and label bias. Our experiments also show how to integrate a measure of data bias risk in the existing fairness dashboards for real-world deployments.

cs.LG↗

arXiv2023

RoboAgent: Generalization and Efficiency in Robot Manipulation via Semantic Augmentations and Action Chunking

The grand aim of having a single robot that can manipulate arbitrary objects in diverse settings is at odds with the paucity of robotics datasets. Acquiring and growing such datasets is strenuous due to manual efforts, operational costs, and safety challenges. A path toward such an universal agent would require a structured framework capable of wide generalization but trained within a reasonable data budget. In this paper, we develop an efficient system (RoboAgent) for training universal agents capable of multi-task manipulation skills using (a) semantic augmentations that can rapidly multiply existing datasets and (b) action representations that can extract performant policies with small yet diverse multi-modal datasets without overfitting. In addition, reliable task conditioning and an expressive policy architecture enable our agent to exhibit a diverse repertoire of skills in novel situations specified using language commands. Using merely 7500 demonstrations, we are able to train a single agent capable of 12 unique skills, and demonstrate its generalization over 38 tasks spread across common daily activities in diverse kitchen scenes. On average, RoboAgent outperforms prior methods by over 40% in unseen situations while being more sample efficient and being amenable to capability improvements and extensions through fine-tuning. Videos at https://robopen.github.io/

cs.RO↗

arXiv2023

A comparative study of the correlation between the structure and the dynamics for systems interacting via attractive and repulsive potentials

We present the study of the structure-dynamics correlation for systems interacting via attractive Lennard- Jones and its repulsive counterpart, the WCA potentials. The structural order parameter (SOP) is related to the microscopic mean-field caging potential. At a particle level, the SOP shows a distribution. Although the two systems have similar pair structures, their average SOP differs. However, this difference alone is insufficient to explain the well known slowing down of the dynamics in LJ system at low temperatures. The slowing down can be explained in terms of a stronger coupling between the SOP and the dynamics. To understand the origin of this system specific coupling, we study the difference in the microscopic structure between the hard and soft particles. We find that for the LJ system, the structural differences of the hard and soft particles are more significant and have a much stronger temperature dependence compared to the WCA system. Thus the study suggests that attractive interaction creates more structurally different communities. This broader difference in the structural communities is probably responsible for stronger coupling between the structure and dynamics. Thus the system specific structure-dynamics correlation, which also leads to a faster slowing down in the dynamics, appears to have a structural origin. A comparison of the predictive power of our SOP with the local energy and two body excess entropy shows that in the LJ system, the dynamics is driven by enthalpy, whereas in the WCA system, it is driven by entropy, and our SOP can capture both these contributions.

physics.chem-ph↗

arXiv2023

Dynamic heterogeneity in polydisperse systems: A comparative study of the role of local structural order parameter and particle size

In polydisperse systems, describing the structure and any structural order parameter (SOP) is not trivial as it varies with the number of species we use to describe the system, M . Depending on the degree of polydispersity, there is an optimum value of M = M0 where we show that the mutual information of the system increases. However, surprisingly the correlation between a recently proposed SOP and the dynamics is highest for M = 1. This effect increases with polydispersity. We find that the SOP at M = 1 is coupled with the particle size, σ, and this coupling increases with polydispersity and decreases with an increase in M . Careful analysis shows that at lower polydispersity the SOP is a good predictor of the dynamics. However, at higher polydispersity, the dynamics is strongly dependent on σ. Since the coupling between the SOP and σ is higher for M = 1 thus, it appears to be a better predictor of the dynamics. We also study the Vibrality an order parameter independent of structural information. Compared to SOP, at high polydispersity we find Vibrality to be a marginally better predictor of the dynamics. However, this high predictive power of Vibrality, which is not there at lower polydispersity, appears to be due to its stronger coupling with σ. Thus our study suggests that for systems with high polydispersity, the correlation of any order parameter and σ will affect the correlation between the order parameter and dynamics and need not project a generic predictive power of the order parameter.

cond-mat.soft↗

arXiv2023

Lossless Adaptation of Pretrained Vision Models For Robotic Manipulation

Recent works have shown that large models pretrained on common visual learning tasks can provide useful representations for a wide range of specialized perception problems, as well as a variety of robotic manipulation tasks. While prior work on robotic manipulation has predominantly used frozen pretrained features, we demonstrate that in robotics this approach can fail to reach optimal performance, and that fine-tuning of the full model can lead to significantly better results. Unfortunately, fine-tuning disrupts the pretrained visual representation, and causes representational drift towards the fine-tuned task thus leading to a loss of the versatility of the original model. We introduce "lossless adaptation" to address this shortcoming of classical fine-tuning. We demonstrate that appropriate placement of our parameter efficient adapters can significantly reduce the performance gap between frozen pretrained representations and full end-to-end fine-tuning without changes to the original representation and thus preserving original capabilities of the pretrained model. We perform a comprehensive investigation across three major model architectures (ViTs, NFNets, and ResNets), supervised (ImageNet-1K classification) and self-supervised pretrained weights (CLIP, BYOL, Visual MAE) in 3 task domains and 35 individual tasks, and demonstrate that our claims are strongly validated in various settings.

cs.LG↗

arXiv2023

Continuous Scatterplot Operators for Bivariate Analysis and Study of Electronic Transitions

Electronic transitions in molecules due to the absorption or emission of light is a complex quantum mechanical process. Their study plays an important role in the design of novel materials. A common yet challenging task in the study is to determine the nature of electronic transitions, namely which subgroups of the molecule are involved in the transition by donating or accepting electrons, followed by an investigation of the variation in the donor-acceptor behavior for different transitions or conformations of the molecules. In this paper, we present a novel approach for the analysis of a bivariate field and show its applicability to the study of electronic transitions. This approach is based on two novel operators, the continuous scatterplot (CSP) lens operator and the CSP peel operator, that enable effective visual analysis of bivariate fields. Both operators can be applied independently or together to facilitate analysis. The operators motivate the design of control polygon inputs to extract fiber surfaces of interest in the spatial domain. The CSPs are annotated with a quantitative measure to further support the visual analysis. We study different molecular systems and demonstrate how the CSP peel and CSP lens operators help identify and study donor and acceptor characteristics in molecular systems.

cs.HC↗

arXiv2022

Jacobi Set Driven Search for Flexible Fiber Surface Extraction

Isosurfaces are an important tool for analysis and visualization of univariate scalar fields. Earlier works have demonstrated the presence of interesting isosurfaces at isovalues close to critical values. This motivated the development of efficient methods for computing individual components of isosurfaces restricted to a region of interest. Generalization of isosurfaces to fiber surfaces and critical points to Jacobi sets has resulted in new approaches for analyzing bivariate scalar fields. Unlike isosurfaces, there exists no output sensitive method for computing fiber surfaces. Existing methods traverse through all the tetrahedra in the domain. In this paper, we propose the use of the Jacobi set to identify fiber surface components of interest and present an output sensitive approach for its computation. The Jacobi edges are used to initiate the search towards seed tetrahedra that contain the fiber surface, thereby reducing the search space. This approach also leads to effective analysis of the bivariate field by supporting the identification of relevant fiber surfaces near Jacobi edges.

cs.GR↗

arXiv2022

Search-Based Task Planning with Learned Skill Effect Models for Lifelong Robotic Manipulation

Robots deployed in many real-world settings need to be able to acquire new skills and solve new tasks over time. Prior works on planning with skills often make assumptions on the structure of skills and tasks, such as subgoal skills, shared skill implementations, or task-specific plan skeletons, which limit adaptation to new skills and tasks. By contrast, we propose doing task planning by jointly searching in the space of parameterized skills using high-level skill effect models learned in simulation. We use an iterative training procedure to efficiently generate relevant data to train such models. Our approach allows flexible skill parameterizations and task specifications to facilitate lifelong learning in general-purpose domains. Experiments demonstrate the ability of our planner to integrate new skills in a lifelong manner, finding new task strategies with lower costs in both train and test tasks. We additionally show that our method can transfer to the real world without further fine-tuning.

cs.RO↗

arXiv2022

Identifying structural signature of dynamical heterogeneity via the local softness parameter

In a recent study by some of us, we have proposed a new measure of the structure of a liquid, the softness of the mean-field caging potential, and shown that it can describe the temperature dependence of the dynamics. In this work, we put this parameter through a stringent test and study the causal relationship between this softness parameter at the local level and the local dynamics. We first extend the earlier study to describe the local softness. We find a distribution of the softness value where the distribution has reasonably strong temperature dependence. This shows that the parameter can identify the local structural heterogeneity. We then study the lifetime of the softness parameter and show that its lifetime is connected to the well-known cage structure in the supercooled liquids. Finally, our theory predicts that the local softness and the local dynamics is causal below the onset temperature where there is a decoupling between the short and long time dynamics, thus allowing a static description of the cage. We find that at lower temperatures, the structural heterogeneity increases, and also, the structure becomes a better predictor of the dynamics, thus giving rise to dynamic heterogeneity. We also find that the softness of a hard, immobile region evolves with time and becomes soft and eventually mobile due to the rearrangements in the neighbourhood, confirming the well-known facilitation effect.

cond-mat.soft↗

arXiv2022

Improving third-generation leptoquark searches with combined signals and boosted top

Search strategies for the third-generation leptoquarks (LQs) are distinct from other LQ searches, especially when they decay to a top quark and a $τ$ lepton. We investigate the cases of all TeV-scale scalar and vector LQs that decay to either a top-tau pair (charge-$1/3$ and $5/3$ LQs) or a top-neutrino pair (charge-$2/3$ LQs). One can then use the boosted top (which can be tagged efficiently using jet-substructure techniques) and high-$p_{\rm T}$ $τ$ leptons to search for these LQs. We consider two search channels with either one or two taus along with at least one hadronically decaying boosted top quark. We estimate the high luminosity LHC (HL-LHC) search prospects of these LQs by considering both symmetric and asymmetric pair and single production processes. Our selection criteria are optimised to retain events from both pair and single production processes. The combined signal has better prospects than the traditional searches. We include new three-body single production processes to enhance the single production contributions to the combined signal. We identify the interference effect that appears in the dominant single production channel of charge-$1/3$ scalar LQ ($S^{1/3}$). This interference is constructive if $S^{1/3}$ is weak-triplet and destructive, for a singlet one. As a result, their LHC prospects differ appreciably.

hep-ph↗

arXiv2021

NoisyActions2M: A Multimedia Dataset for Video Understanding from Noisy Labels

Deep learning has shown remarkable progress in a wide range of problems. However, efficient training of such models requires large-scale datasets, and getting annotations for such datasets can be challenging and costly. In this work, we explore the use of user-generated freely available labels from web videos for video understanding. We create a benchmark dataset consisting of around 2 million videos with associated user-generated annotations and other meta information. We utilize the collected dataset for action classification and demonstrate its usefulness with existing small-scale annotated datasets, UCF101 and HMDB51. We study different loss functions and two pretraining strategies, simple and self-supervised learning. We also show how a network pretrained on the proposed dataset can help against video corruption and label noise in downstream datasets. We present this as a benchmark dataset in noisy learning for video understanding. The dataset, code, and trained models will be publicly available for future research.

cs.MM↗

arXiv2021

Segmentation Driven Peeling for Visual Analysis of Electronic Transitions

Electronic transitions in molecules due to absorption or emission of light is a complex quantum mechanical process. Their study plays an important role in the design of novel materials. A common yet challenging task in the study is to determine the nature of those electronic transitions, i.e. which subgroups of the molecule are involved in the transition by donating or accepting electrons, followed by an investigation of the variation in the donor-acceptor behavior for different transitions or conformations of the molecules. In this paper, we present a novel approach towards the study of electronic transitions based on the visual analysis of a bivariate field, namely the electron density in the hole and particle Natural Transition Orbital (NTO). The visual analysis focuses on the continuous scatter plots (CSPs) of the bivariate field linked to their spatial domain. The method supports selections in the CSP visualized as fiber surfaces in the spatial domain, the grouping of atoms, and segmentation of the density fields to peel the CSP. This peeling operator is central to the visual analysis process and helps identify donors and acceptors. We study different molecular systems, identifying local excitation and charge transfer excitations to demonstrate the utility of the method.

physics.chem-ph↗

arXiv2021

Generalizing Object-Centric Task-Axes Controllers using Keypoints

To perform manipulation tasks in the real world, robots need to operate on objects with various shapes, sizes and without access to geometric models. It is often unfeasible to train monolithic neural network policies across such large variance in object properties. Towards this generalization challenge, we propose to learn modular task policies which compose object-centric task-axes controllers. These task-axes controllers are parameterized by properties associated with underlying objects in the scene. We infer these controller parameters directly from visual input using multi-view dense correspondence learning. Our overall approach provides a simple, modular and yet powerful framework for learning manipulation tasks. We empirically evaluate our approach on multiple different manipulation tasks and show its ability to generalize to large variance in object size, shape and geometry.

cs.RO↗

arXiv2021

Inverse Reinforcement Learning with Explicit Policy Estimates

Various methods for solving the inverse reinforcement learning (IRL) problem have been developed independently in machine learning and economics. In particular, the method of Maximum Causal Entropy IRL is based on the perspective of entropy maximization, while related advances in the field of economics instead assume the existence of unobserved action shocks to explain expert behavior (Nested Fixed Point Algorithm, Conditional Choice Probability method, Nested Pseudo-Likelihood Algorithm). In this work, we make previously unknown connections between these related methods from both fields. We achieve this by showing that they all belong to a class of optimization problems, characterized by a common form of the objective, the associated policy and the objective gradient. We demonstrate key computational and algorithmic differences which arise between the methods due to an approximation of the optimal soft value function, and describe how this leads to more efficient algorithms. Using insights which emerge from our study of this class of optimization problems, we identify various problem scenarios and investigate each method's suitability for these problems.

cs.LG↗

arXiv2021

IR-VIC: Unsupervised Discovery of Sub-goals for Transfer in RL

We propose a novel framework to identify sub-goals useful for exploration in sequential decision making tasks under partial observability. We utilize the variational intrinsic control framework (Gregor et.al., 2016) which maximizes empowerment -- the ability to reliably reach a diverse set of states and show how to identify sub-goals as states with high necessary option information through an information theoretic regularizer. Despite being discovered without explicit goal supervision, our sub-goals provide better exploration and sample complexity on challenging grid-world navigation tasks compared to supervised counterparts in prior work.

cs.LG↗