arXiv subjects

Adway Mitra

Publications and source records attributed to Adway Mitra.

At least 19 recordsLinked to original sources

arXiv2026

MAUNet-Light: A Concise MAUNet Architecture for Bias Correction and Downscaling of Precipitation Estimates

Satellite-derived data products and climate model simulations of geophysical variables like precipitation, often exhibit systematic biases compared to in-situ measurements. Bias correction and spatial downscaling are fundamental components to develop operational weather forecast systems, as they seek to improve the consistency between coarse-resolution climate model simulations or satellite-based estimates and ground-based observations. In recent years, deep learning-based models have been increasingly replaced traditional statistical methods to generate high-resolution, bias free projections of climate variables. For example, Max-Average U-Net (MAUNet) architecture has been demonstrated for its ability to downscale precipitation estimates. The versatility and adaptability of these neural models make them highly effective across a range of applications, though this often come at the cost of high computational and memory requirements. The aim of this research is to develop light-weight neural network architectures for both bias correction and downscaling of precipitation, for which the teacher-student based learning paradigm is explored. This research demonstrates the adaptability of MAUNet to the task of bias correction, and further introduces a compact, lightweight neural network architecture termed MAUNet-Light.The proposed MAUNet-Light model is developed by transferring knowledge from the trained MAUNet, and it is designed to perform both downscaling and bias correction with reduced computational requirements without any significant loss in accuracy compared to state-of-the-art.

cs.LG↗

arXiv2026

AI-Driven Predictive Modelling for Groundwater Salinization in Israel

Increasing salinity and contamination of groundwater is a serious issue in many parts of the world, causing degradation of water resources. The aim of this work is to form a comprehensive understanding of groundwater salinization underlying causal factors and identify important meteorological, geological and anthropogenic drivers of salinity. We have integrated different datasets of potential covariates, to create a robust framework for machine learning based predictive models including Random Forest (RF), XGBoost, Neural network, Long Short-Term Memory (LSTM), convolution neural network (CNN) and linear regression (LR), of groundwater salinity. Additionally, Recursive Feature Elimination (RFE) followed by Global sensitivity analysis (GSA) and Explainable AI (XAI) based SHapley Additive exPlanations (SHAP) were used to estimate the importance scores and find insights into the drivers of salinization. We also did causality analysis via Double machine learning using various predictive models. From these analyses, key meteorological (Precipitation, Temperature), geological (Distance from river, Distance to saline body, TWI, Shoreline distance), and anthropogenic (Area of agriculture field, Treated Wastewater) covariates are identified to be influential drivers of groundwater salinity across Israel. XAI analysis also identified Treated Wastewater (TWW) as an essential anthropogenic driver of salinity, its significance being context-dependent but critical in vulnerable hydro-climatic environment. Our approach provides deeper insight into global salinization mechanisms at country scale, reducing AI model uncertainty and highlighting the need for tailored strategies to address salinity.

cs.LG↗

arXiv2024

Deep Learning for Slum Mapping in Remote Sensing Images: A Meta-analysis and Review

The major Sustainable Development Goals (SDG) 2030, set by the United Nations Development Program (UNDP), include sustainable cities and communities, no poverty, and reduced inequalities. However, millions of people live in slums or informal settlements with poor living conditions in many major cities around the world, especially in less developed countries. To emancipate these settlements and their inhabitants through government intervention, accurate data about slum location and extent is required. While ground survey data is the most reliable, such surveys are costly and time-consuming. An alternative is remotely sensed data obtained from very high-resolution (VHR) imagery. With the advancement of new technology, remote sensing based mapping of slums has emerged as a prominent research area. The parallel rise of Artificial Intelligence, especially Deep Learning has added a new dimension to this field as it allows automated analysis of satellite imagery to identify complex spatial patterns associated with slums. This article offers a detailed review and meta-analysis of research on slum mapping using remote sensing imagery from 2014 to 2024, with a special focus on deep learning approaches. Our analysis reveals a trend towards increasingly complex neural network architectures, with advancements in data preprocessing and model training techniques significantly enhancing slum identification accuracy. We have attempted to identify key methodologies that are effective across diverse geographic contexts. While acknowledging the transformative impact Convolutional Neural Networks (CNNs) in slum detection, our review underscores the absence of a universally optimal model, suggesting the need for context-specific adaptations. We also identify prevailing challenges in this field, such as data limitations and a lack of model explainability and suggest potential strategies for overcoming these.

cs.CV↗

arXiv2023

Evaluating District-based Election Surveys with Synthetic Dirichlet Likelihood

In district-based multi-party elections, electors cast votes in their respective districts. In each district, the party with maximum votes wins the corresponding seat in the governing body. Election Surveys try to predict the election outcome (vote shares and seat shares of parties) by querying a random sample of electors. However, the survey results are often inconsistent with the actual results, which could be due to multiple reasons. The aim of this work is to estimate a posterior distribution over the possible outcomes of the election, given one or more survey results. This is achieved using a prior distribution over vote shares, election models to simulate the complete election from the vote share, and survey models to simulate survey results from a complete election. The desired posterior distribution over the space of possible outcomes is constructed using Synthetic Dirichlet Likelihoods, whose parameters are estimated from Monte Carlo sampling of elections using the election models. We further show the same approach can also use be used to evaluate the surveys - whether they were biased or not, based on the true outcome once it is known. Our work offers the first-ever probabilistic model to analyze district-based election surveys. We illustrate our approach with extensive experiments on real and simulated data of district-based political elections in India.

stat.ME↗

arXiv2023

SepHRNet: Generating High-Resolution Crop Maps from Remote Sensing imagery using HRNet with Separable Convolution

The accurate mapping of crop production is crucial for ensuring food security, effective resource management, and sustainable agricultural practices. One way to achieve this is by analyzing high-resolution satellite imagery. Deep Learning has been successful in analyzing images, including remote sensing imagery. However, capturing intricate crop patterns is challenging due to their complexity and variability. In this paper, we propose a novel Deep learning approach that integrates HRNet with Separable Convolutional layers to capture spatial patterns and Self-attention to capture temporal patterns of the data. The HRNet model acts as a backbone and extracts high-resolution features from crop images. Spatially separable convolution in the shallow layers of the HRNet model captures intricate crop patterns more effectively while reducing the computational cost. The multi-head attention mechanism captures long-term temporal dependencies from the encoded vector representation of the images. Finally, a CNN decoder generates a crop map from the aggregated representation. Adaboost is used on top of this to further improve accuracy. The proposed algorithm achieves a high classification accuracy of 97.5\% and IoU of 55.2\% in generating crop maps. We evaluate the performance of our pipeline on the Zuericrop dataset and demonstrate that our results outperform state-of-the-art models such as U-Net++, ResNet50, VGG19, InceptionV3, DenseNet, and EfficientNet. This research showcases the potential of Deep Learning for Earth Observation Systems.

cs.CV↗

arXiv2023

Deep Learning-based Spatially Explicit Emulation of an Agent-Based Simulator for Pandemic in a City

Agent-Based Models are very useful for simulation of physical or social processes, such as the spreading of a pandemic in a city. Such models proceed by specifying the behavior of individuals (agents) and their interactions, and parameterizing the process of infection based on such interactions based on the geography and demography of the city. However, such models are computationally very expensive, and the complexity is often linear in the total number of agents. This seriously limits the usage of such models for simulations, which often have to be run hundreds of times for policy planning and even model parameter estimation. An alternative is to develop an emulator, a surrogate model that can predict the Agent-Based Simulator's output based on its initial conditions and parameters. In this paper, we discuss a Deep Learning model based on Dilated Convolutional Neural Network that can emulate such an agent based model with high accuracy. We show that use of this model instead of the original Agent-Based Model provides us major gains in the speed of simulations, allowing much quicker calibration to observations, and more extensive scenario analysis. The models we consider are spatially explicit, as the locations of the infected individuals are simulated instead of the gross counts. Another aspect of our emulation framework is its divide-and-conquer approach that divides the city into several small overlapping blocks and carries out the emulation in them parallelly, after which these results are merged together. This ensures that the same emulator can work for a city of any size, and also provides significant improvement of time complexity of the emulator, compared to the original simulator.

cs.MA↗

arXiv2023

Agent-based Simulation of District-based Elections

In district-based elections, electors cast votes in their respective districts. In each district, the party with maximum votes wins the corresponding seat in the governing body. The election result is based on the number of seats won by different parties. In this system, locations of electors across the districts may severely affect the election result even if the total number of votes obtained by different parties remains unchanged. A less popular party may end up winning more seats if their supporters are suitably distributed spatially. This happens due to various regional and social influences on individual voters which modulate their voting choice. In this paper, we explore agent-based models for district-based elections, where we consider each elector as an agent, and try to represent their social and geographical attributes and political inclinations using probability distributions. This model can be used to simulate election results by Monte Carlo sampling. The models allow us to explore the full space of possible outcomes of an electoral setting, though they can also be calibrated to actual election results for suitable values of parameters. We use Approximate Bayesian Computation (ABC) framework to estimate model parameters. We show that our model can reproduce the results of elections held in India and USA, and can also produce counterfactual scenarios.

cs.MA↗

arXiv2022

Exploring Fairness in District-based Multi-party Elections under different Voting Rules using Stochastic Simulations

Many democratic societies use district-based elections, where the region under consideration is geographically divided into districts and a representative is chosen for each district based on the preferences of the electors who reside there. These representatives belong to political parties, and the executive powers are acquired by that party which has a majority of the elected district representatives. In most systems, each elector can express preference for one candidate, though they may have a complete or partial ranking of the candidates/parties. We show that this can lead to situations where many electors are dissatisfied with the election results, which is not desirable in a democracy. The results may be biased towards the supporters of a particular party, and against others. Inspired by current literature on fairness of Machine Learning algorithms, we define measures of fairness to quantify the satisfaction of electors, irrespective of their political choices. We also consider alternative election policies using concepts of voting rules and rank aggregation, to enable voters to express their detailed preferences without making the electoral process cumbersome or opaque. We then evaluate these policies using the aforementioned fairness measures with the help of Monte Carlo simulations. Such simulations are obtained using a proposed stochastic model for election simulation, that takes into account community identities of electors and its role in influencing their residence and political preferences. We show that this model can simulate actual multi-party elections in India. Through extensive simulations, we find that allowing voters to provide 2 preferences reduces the disparity between supporters of different parties in terms of the election result.

cs.CY↗

arXiv2021

City-scale Simulation of Covid-19 Pandemic and Intervention Policies using Agent-based Modelling

During the Covid-19 pandemic, most governments across the world imposed policies like lock-down of public spaces and restrictions on people's movements to minimize the spread of the virus through physical contact. However, such policies have grave social and economic costs, and so it is important to pre-assess their impacts. In this work we aim to visualize the dynamics of the pandemic in a city under different intervention policies, by simulating the behavior of the residents. We develop a very detailed agent-based model for a city, including its residents, physical and social spaces like homes, marketplaces, workplaces, schools/colleges etc. We parameterize our model for Kolkata city in India using ward-level demographic and civic data. We demonstrate that under appropriate choice of parameters, our model is able to reproduce the observed dynamics of the Covid-19 pandemic in Kolkata, and also indicate the counter-factual outcomes of alternative intervention policies.

cs.MA↗

arXiv2020

Spatio-temporal relationships between rainfall and convective clouds during Indian Monsoon through a discrete lens

The Indian monsoon, a multi-variable process causing heavy rains during June-September every year, is very heterogeneous in space and time. We study the relationship between rainfall and Outgoing Longwave Radiation (OLR, convective cloud cover) for monsoon between 2004-2010. To identify, classify and visualize spatial patterns of rainfall and OLR we use a discrete and spatio-temporally coherent representation of the data, created using a statistical model based on Markov Random Field. Our approach clusters the days with similar spatial distributions of rainfall and OLR into a small number of spatial patterns. We find that eight daily spatial patterns each in rainfall and OLR, and seven joint patterns of rainfall and OLR, describe over 90\% of all days. Through these patterns, we find that OLR generally has a strong negative correlation with precipitation, but with significant spatial variations. In particular, peninsular India (except west coast) is under significant convective cloud cover over a majority of days but remains rainless. We also find that much of the monsoon rainfall co-occurs with low OLR, but some amount of rainfall in Eastern and North-western India in June occurs on OLR days, presumably from shallow clouds. To study day-to-day variations of both quantities, we identify spatial patterns in the temporal gradients computed from the observations. We find that changes in convective cloud activity across India most commonly occur due to the establishment of a north-south OLR gradient which persists for 1-2 days and shifts the convective cloud cover from light to deep or vice versa. Such changes are also accompanied by changes in the spatial distribution of precipitation. The present work thus provides a highly reduced description of the complex spatial patterns and their day-to-day variations, and could form a useful tool for future simplified descriptions of this process.

physics.ao-ph↗

arXiv2020

Electoral David vs Goliath: How does the Spatial Concentration of Electors affect District-based Elections?

Many democratic countries use district-based elections where there is a "seat" for each district in the governing body. In each district, the party whose candidate gets the maximum number of votes wins the corresponding seat. The result of the election is decided based on the number of seats won by the different parties. The electors (voters) can cast their votes only in the district of their residence. Thus, locations of the electors and boundaries of the districts may severely affect the election result even if the proportion of popular support (number of electors) of different parties remains unchanged. This has led to significant amount of research on whether the districts may be redrawn or electors may be moved to maximize seats for a particular party. In this paper, we frame the spatial distribution of electors in a probabilistic setting, and explore different models to capture the intra-district polarization of electors in favour of a party, or the spatial concentration of supporters of different parties. Our models are inspired by elections in India, where supporters of different parties tend to be concentrated in certain districts. We show with extensive simulations that our model can capture different statistical properties of real elections held in India. We frame parameter estimation problems to fit our models to the observed election results. Since analytical calculation of the likelihood functions are infeasible for our complex models, we use Likelihood-free Inference methods under the Approximate Bayesian Computation framework. Since this approach is highly time-consuming, we explore how supervised regression using Logistic Regression or Deep Neural Networks can be used to speed it up. We also explore how the election results can change by varying the spatial distributions of the voters, even when the proportions of popular support of the parties remain constant.

stat.AP↗

arXiv2018

Video Trajectory Classification and Anomaly Detection Using Hybrid CNN-VAE

Classifying time series data using neural networks is a challenging problem when the length of the data varies. Video object trajectories, which are key to many of the visual surveillance applications, are often found to be of varying length. If such trajectories are used to understand the behavior (normal or anomalous) of moving objects, they need to be represented correctly. In this paper, we propose video object trajectory classification and anomaly detection using a hybrid Convolutional Neural Network (CNN) and Variational Autoencoder (VAE) architecture. First, we introduce a high level representation of object trajectories using color gradient form. In the next stage, a semi-supervised way to annotate moving object trajectories extracted using Temporal Unknown Incremental Clustering (TUIC), has been applied for trajectory class labeling. Anomalous trajectories are separated using t-Distributed Stochastic Neighbor Embedding (t-SNE). Finally, a hybrid CNN-VAE architecture has been used for trajectory classification and anomaly detection. The results obtained using publicly available surveillance video datasets reveal that the proposed method can successfully identify some of the important traffic anomalies such as vehicles not following lane driving, sudden speed variations, abrupt termination of vehicle movement, and vehicles moving in wrong directions. The proposed method is able to detect above anomalies at higher accuracy as compared to existing anomaly detection methods.

cs.CV↗

arXiv2018

A Discrete View of the Indian Monsoon to Identify Spatial Patterns of Rainfall

We propose a representation of the Indian summer monsoon rainfall in terms of a probabilistic model based on a Markov Random Field, consisting of discrete state variables representing low and high rainfall at grid-scale and daily rainfall patterns across space and in time. These discrete states are conditioned on observed daily gridded rainfall data from the period 2000-2007. The model gives us a set of 10 spatial patterns of daily monsoon rainfall over India, which are robust over a range of user-chosen parameters as well as coherent in space and time. Each day in the monsoon season is assigned precisely one of the spatial patterns, that approximates the spatial distribution of rainfall on that day. Such approximations are quite accurate for nearly 95% of the days. Remarkably, these patterns are representative (with similar accuracy) of the monsoon seasons from 1901 to 2000 as well. Finally, we compare the proposed model with alternative approaches to extract spatial patterns of rainfall, using empirical orthogonal functions as well as clustering algorithms such as K-means and spectral clustering.

stat.AP↗

arXiv2018

Spatio-temporal Patterns of Indian Monsoon Rainfall

The primary objective of this paper is to analyze a set of canonical spatial patterns that approximate the daily rainfall across the Indian region, as identified in the companion paper where we developed a discrete representation of the Indian summer monsoon rainfall using state variables with spatio-temporal coherence maintained using a Markov Random Field prior. In particular, we use these spatio-temporal patterns to study the variation of rainfall during the monsoon season. Firstly, the ten patterns are divided into three families of patterns distinguished by their total rainfall amount and geographic spread. These families are then used to establish `active' and `break' spells of the Indian monsoon at the all-India level. Subsequently, we characterize the behavior of these patterns in time by estimating probabilities of transition from one pattern to another across days in a season. Patterns tend to be `sticky': the self-transition is the most common. We also identify most commonly occurring sequences of patterns. This leads to a simple seasonal evolution model for the summer monsoon rainfall. The discrete representation introduced in the companion paper also identifies typical temporal rainfall patterns for individual locations. This enables us to determine wet and dry spells at local and regional scales. Lastly, we specify sets of locations that tend to have such spells simultaneously, and thus come up with a new regionalization of the landmass.

stat.AP↗

arXiv2017

Detection of Spatiotemporally Coherent Rainfall Anomalies Using Markov Random Fields

Precipitation is a large-scale, spatio-temporally heterogeneous phenomenon, with frequent anomalies exhibiting unusually high or low values. We use Markov Random Fields (MRFs) to detect spatio-temporally coherent anomalies in gridded annual rainfall data across India from 1901-2005. MRFs are undirected graphical models where each node is associated with a \{location,year\} pair, with edges connecting nodes representing adjacent locations or years. Some nodes represent observations of precipitation, while the rest represent unobserved (\emph{latent}) states that can take one of three values: high/low/normal. The MRF represents a probability distribution over the variables, using \emph{node potential} and \emph{edge potential} functions defined on nodes and edges of the graph. Optimal values of latent state variables are estimated by maximizing the posterior probability of the observations, using Gibbs sampling. Edge potentials enforce spatial and temporal coherence, and node potentials influence threshold for anomalies by affecting the prior probabilities of the states. The model can be tuned to recover anomalies detected by threshold-based methods. The competing influences of spatial and temporal coherence can be adjusted through edge potentials. We study spatio-temporal properties of rainfall anomalies discovered by this method, using suitable measures. We identify nonstationarities in occurrence of positive and negative anomalies between the first and second halves of the 20th century. We find that between these periods, there has been decrease in rainfall during June-September (JJAS) and an increase during other months. These effects are highlighted prominently in the statistics of anomalies. Properties of anomalies learnt from this approach could present tests of regional-scale rainfall simulations by climate models and statistical simulators.

stat.AP↗

arXiv2017

Identifying Coherent Anomalies in Multi-Scale Spatio-Temporal Data using Markov Random Fields

Many physical processes involve spatio-temporal observations, which can be studied at different spatial and temporal scales. For example, rainfall data measured daily by rain gauges can be considered at daily, monthly or annual temporal scales, and local, grid-wise, region-wise or country-wise spatial scales. In this work, we focus on detection of anomalies in such multi-scale spatio-temporal data. We consider an anomaly as an event where the measured values over a spatio-temporally extended region are significantly different from their long-term means. However we aim to avoid setting any thresholds on the measured values and spatio-temporal sizes, because not only are thresholds subjective but also the long-term mean values often vary spatially and temporally. For this purpose we use spatio-Temporal Markov Random Field, where latent states indicate anomaly type (positive anomaly, negative anomaly, no anomaly/normal). Spatio-temporal coherence is maintained through suitable edge potentials. The model is extended to multiple spatio-temporal scales to achieve our second goal: anomalies at any scale should be defined both on the data at that scale, and also on anomalies at other scales. This allows us to trace an anomaly at a coarse scale to finer scales. For example, whether rainfall in a particular year is anomalous over a region should depend not only on the total volume of rainfall over the entire region, but also on whether there were such anomalies at the grid-scale, and the monthly scale. We use this approach to study rainfall anomalies over India -extremely diverse with respect to rainfall- for the period 1901-2011, and show its benefits over existing approaches.

stat.AP↗

arXiv2017

Bayesian approach to Spatio-temporally Consistent Simulation of Daily Monsoon Rainfall over India

Simulation of rainfall over a region for long time-sequences can be very useful for planning and policy-making, especially in India where the economy is heavily reliant on monsoon rainfall. However, such simulations should be able to preserve the known spatial and temporal characteristics of rainfall over India. General Circulation Models (GCMs) are unable to do so, and various rainfall generators designed by hydrologists using stochastic processes like Gaussian Processes are also difficult to apply over the vast and highly diverse landscape of India. In this paper, we explore a series of Bayesian models based on conditional distributions of latent variables that describe weather conditions at specific locations and over the whole country. During parameter estimation from observed data, we use spatio-temporal smoothing using Markov Random Field so that the parameters learnt are spatially and temporally coherent. Also, we use a nonparametric spatial clustering based on Chinese Restaurant Process to identify homogeneous regions, which are utilized by some of the proposed models to improve spatial correlations of the simulated rainfall. The models are able to simulate daily rainfall across India for years, and can also utilize contextual information for conditional simulation. We use two datasets of different spatial resolutions over India, and focus on the period 2000-2015. We propose a large number of metrics to study the spatio-temporal properties of the simulations by the models, and compare them with the observed data to evaluate the strengths and weaknesses of the models.

stat.AP↗

arXiv2016

Exploring Spatial Coherence in Inter-annual Changes and Annual Extremes of Rainfall over India

Forecasts of monsoon rainfall for India are made at national scale. But there is spatial coherence and heterogeneity that is relevant to forecasting. This paper considers year-to-year rainfall change and annual extremes at sub-national scales. We use Data Mining techniques to gridded rain-gauge data for 1901-2011 to characterize coherence and heterogeneity and identify spatially homogeneous clusters. We study the direction of change in rainfall between years (Phase), and extreme annual rainfall at both grid level and national level. Grid-level Phase is found to be spatially coherent, and significantly correlated with all-India mean rainfall (AIMR) phase. Grid-level extreme-rainfall years are not strongly associated with corresponding extremes in AIMR, although in extreme AIMR years local extremes of the same type occur with higher spatial coherence. Years of extremes in AIMR entail widespread phase of the corresponding sign. Furthermore, local extremes and phase are found to frequently co-occur in spatially contiguous clusters.

stat.AP↗