arXiv subjects

Manmeet Singh

Publications and source records attributed to Manmeet Singh.

At least 37 records · Page 2Linked to original sources

arXiv2024

CloudSense: A Model for Cloud Type Identification using Machine Learning from Radar data

The knowledge of type of precipitating cloud is crucial for radar based quantitative estimates of precipitation. We propose a novel model called CloudSense which uses machine learning to accurately identify the type of precipitating clouds over the complex terrain locations in the Western Ghats (WGs) of India. CloudSense uses vertical reflectivity profiles collected during July-August 2018 from an X-band radar to classify clouds into four categories namely stratiform,mixed stratiform-convective,convective and shallow clouds. The machine learning(ML) model used in CloudSense was trained using a dataset balanced by Synthetic Minority Oversampling Technique (SMOTE), with features selected based on physical characteristics relevant to different cloud types. Among various ML models evaluated Light Gradient Boosting Machine (LightGBM) demonstrate superior performance in classifying cloud types with a BAC of 0.8 and F1-Score of 0.82. CloudSense generated results are also compared against conventional radar algorithms and we find that CloudSense performs better than radar algorithms. For 200 samples tested, the radar algorithm achieved a BAC of 0.69 and F1-Score of 0.68, whereas CloudSense achieved a BAC and F1-Score of 0.77. Our results show that ML based approach can provide more accurate cloud detection and classification which would be useful to improve precipitation estimates over the complex terrain of the WG.

physics.ao-ph↗

arXiv2023

Transformative Effects of ChatGPT on Modern Education: Emerging Era of AI Chatbots

ChatGPT, an AI-based chatbot, was released to provide coherent and useful replies based on analysis of large volumes of data. In this article, leading scientists, researchers and engineers discuss the transformative effects of ChatGPT on modern education. This research seeks to improve our knowledge of ChatGPT capabilities and its use in the education sector, identifying potential concerns and challenges. Our preliminary evaluation concludes that ChatGPT performed differently in each subject area including finance, coding and maths. While ChatGPT has the ability to help educators by creating instructional content, offering suggestions and acting as an online educator to learners by answering questions and promoting group work, there are clear drawbacks in its use, such as the possibility of producing inaccurate or false data and circumventing duplicate content (plagiarism) detectors where originality is essential. The often reported hallucinations within Generative AI in general, and also relevant for ChatGPT, can render its use of limited benefit where accuracy is essential. What ChatGPT lacks is a stochastic measure to help provide sincere and sensitive communication with its users. Academic regulations and evaluation practices used in educational institutions need to be updated, should ChatGPT be used as a tool in education. To address the transformative effects of ChatGPT on the learning environment, educating teachers and students alike about its capabilities and limitations will be crucial.

cs.CY↗

arXiv2023

FlashBench: A lightning nowcasting framework based on the hybrid deep learning and physics-based dynamical models

Lightning strikes are a well-known danger, and are a leading cause of accidental fatality worldwide. Unfortunately, lightning hazards seldom make headlines in international media coverage because of their infrequency and the low number of casualties each incidence. According to readings from the TRMM LIS lightning sensor, thunderstorms are more common in the tropics while being extremely rare in the polar regions. To improve the precision of lightning forecasts, we develop a technique similar to LightNet's, with one key modification. We didn't just base our model off the results of preliminary numerical simulations; we also factored in the observed fields' time-dependent development. The effectiveness of the lightning forecast rose dramatically once this adjustment was made. The model was tested in a case study during a thunderstorm. Using lightning parameterization in the WRF model simulation, we compared the simulated fields. As the first of its type, this research has the potential to set the bar for how regional lightning predictions are conducted in the future because of its data-driven approach. In addition, we have built a cloud-based lightning forecast system based on Google Earth Engine. With this setup, lightning forecasts over West India may be made in real time, giving critically important information for the area.

physics.ao-ph↗

arXiv2023

Mind meets machine: Unravelling GPT-4's cognitive psychology

Cognitive psychology delves on understanding perception, attention, memory, language, problem-solving, decision-making, and reasoning. Large language models (LLMs) are emerging as potent tools increasingly capable of performing human-level tasks. The recent development in the form of GPT-4 and its demonstrated success in tasks complex to humans exam and complex problems has led to an increased confidence in the LLMs to become perfect instruments of intelligence. Although GPT-4 report has shown performance on some cognitive psychology tasks, a comprehensive assessment of GPT-4, via the existing well-established datasets is required. In this study, we focus on the evaluation of GPT-4's performance on a set of cognitive psychology datasets such as CommonsenseQA, SuperGLUE, MATH and HANS. In doing so, we understand how GPT-4 processes and integrates cognitive psychology with contextual information, providing insight into the underlying cognitive processes that enable its ability to generate the responses. We show that GPT-4 exhibits a high level of accuracy in cognitive psychology tasks relative to the prior state-of-the-art models. Our results strengthen the already available assessments and confidence on GPT-4's cognitive psychology abilities. It has significant potential to revolutionize the field of AI, by enabling machines to bridge the gap between human and machine reasoning.

cs.CL↗

arXiv2022

Short-range forecasts of global precipitation using deep learning-augmented numerical weather prediction

Precipitation governs Earth's hydroclimate, and its daily spatiotemporal fluctuations have major socioeconomic effects. Advances in Numerical weather prediction (NWP) have been measured by the improvement of forecasts for various physical fields such as temperature and pressure; however, large biases exist in precipitation prediction. We augment the output of the well-known NWP model CFSv2 with deep learning to create a hybrid model that improves short-range global precipitation at 1-, 2-, and 3-day lead times. To hybridise, we address the sphericity of the global data by using modified DLWP-CS architecture which transforms all the fields to cubed-sphere projection. Dynamical model precipitation and surface temperature outputs are fed into a modified DLWP-CS (UNET) to forecast ground truth precipitation. While CFSv2's average bias is +5 to +7 mm/day over land, the multivariate deep learning model decreases it to within -1 to +1 mm/day. Hurricane Katrina in 2005, Hurricane Ivan in 2004, China floods in 2010, India floods in 2005, and Myanmar storm Nargis in 2008 are used to confirm the substantial enhancement in the skill for the hybrid dynamical-deep learning model. CFSv2 typically shows a moderate to large bias in the spatial pattern and overestimates the precipitation at short-range time scales. The proposed deep learning augmented NWP model can address these biases and vastly improve the spatial pattern and magnitude of predicted precipitation. Deep learning enhanced CFSv2 reduces mean bias by 8x over important land regions for 1 day lead compared to CFSv2. The spatio-temporal deep learning system opens pathways to further the precision and accuracy in global short-range precipitation forecasts.

physics.ao-ph↗

arXiv2022

A curious case of the Indian Summer Monsoon 2020: The influence of Barotropic Rossby Waves and the monsoon depressions

Summer monsoon rainfall over India depends on synoptic-scale phenomena like lows and depressions. India's summer monsoon season 2020 saw above-average rainfall. North Indian Ocean had 12 LPAs in summer 2020 in JJAS season. This monsoon season has the highest all-India rainfall since 1976 in August 2020. 50% of seasonal LPAs emerged in August 2020, but none intensified into a monsoon depression (MDs). This study explores the potential factors behind LPAs not intensifying/concentrating as MDs. Warming over the Northern Parts of the Arabian Sea (NPAS) boosted convection in August 2020, resulting in substantial low-level convergence. Strong northwesterly winds from central Asia also combined with the cross-equatorial monsoon flow. This powerful flow across the Arabian Sea split into two branches: one extending to NW India following the monsoon trough, and another diverging into an anticyclone over the south Bay of Bengal (SBOB), reducing horizontal shear there (Barotropic Instability). Due to weak barotropic instability over the head BOB, LPAs cannot become MDs. In August 2020, the 200 hPa barotropic Rossby wave remains stationary over South Central Asia and retrogresses northeast to southwest. It affected all-India rainfall by increasing rainfall in the NW and Western Ghats (WG). Despite unfavourable dynamical conditions, the barotropic Rossby wave and low-level anticyclone over the WNP caused above-normal rainfall over India in August 2020. We also confirmed these mechanisms in Community Earth System Model Large Ensemble (CESM-LE) model simulations. Analysis showed that model has limited skill to mimic monsoon rainfall and associated circulation and failed to reflect mid-latitude circulation impact on ISM rainfall.

physics.ao-ph↗

arXiv2022

DeepClouds.ai: Deep learning enabled computationally cheap direct numerical simulations

Simulation of turbulent flows, especially at the edges of clouds in the atmosphere, is an inherently challenging task. Hitherto, the best possible computational method to perform such experiments is the Direct Numerical Simulation (DNS). DNS involves solving non-linear partial differential equations for fluid flows, also known as Navier-Stokes equations, on discretized grid boxes in a three-dimensional space. It is a valuable paradigm that has guided the numerical weather prediction models to compute rainfall formation. However, DNS cannot be performed for large domains of practical utility to the weather forecast community. Here, we introduce DeepClouds.ai, a 3D-UNET that simulates the outputs of a rising cloud DNS experiment. The problem of increasing the domain size in DNS is addressed by mapping an inner 3D cube to the complete 3D cube from the output of the DNS discretized grid simulation. Our approach effectively captures turbulent flow dynamics without having to solve the complex dynamical core. The baseline shows that the deep learning-based simulation is comparable to the partial-differential equation-based model as measured by various score metrics. This framework can be used to further the science of turbulence and cloud flows by enabling simulations over large physical domains in the atmosphere. It would lead to cascading societal benefits by improved weather predictions via advanced parameterization schemes.

physics.flu-dyn↗

arXiv2022

Trustworthy modelling of atmospheric formaldehyde powered by deep learning

Formaldehyde (HCHO) is one one of the most important trace gas in the atmosphere, as it is a pollutant causing respiratory and other diseases. It is also a precursor of tropospheric ozone which damages crops and deteriorates human health. Study of HCHO chemistry and long-term monitoring using satellite data is important from the perspective of human health, food security and air pollution. Dynamic atmospheric chemistry models struggle to simulate atmospheric formaldehyde and often overestimate by up to two times relative to satellite observations and reanalysis. Spatial distribution of modelled HCHO also fail to match satellite observations. Here, we present deep learning approach using a simple super-resolution based convolutional neural network towards simulating fast and reliable atmospheric HCHO. Our approach is an indirect method of HCHO estimation without the need to chemical equations. We find that deep learning outperforms dynamical model simulations which involves complicated atmospheric chemistry representation. Causality establishing the nonlinear relationships of different variables to target formaldehyde is established in our approach by using a variety of precursors from meteorology and chemical reanalysis to target OMI AURA satellite based HCHO predictions. We choose South Asia for testing our implementation as it doesnt have in situ measurements of formaldehyde and there is a need for improved quality data over the region. Moreover, there are spatial and temporal data gaps in the satellite product which can be removed by trustworthy modelling of atmospheric formaldehyde. This study is a novel attempt using computer vision for trustworthy modelling of formaldehyde from remote sensing can lead to cascading societal benefits.

physics.ao-ph↗

arXiv2022

Urban precipitation downscaling using deep learning: a smart city application over Austin, Texas, USA

Urban downscaling is a link to transfer the knowledge from coarser climate information to city scale assessments. These high-resolution assessments need multiyear climatology of past data and future projections, which are complex and computationally expensive to generate using traditional numerical weather prediction models. The city of Austin, Texas, USA has seen tremendous growth in the past decade. Systematic planning for the future requires the availability of fine resolution city-scale datasets. In this study, we demonstrate a novel approach generating a general purpose operator using deep learning to perform urban downscaling. The algorithm employs an iterative super-resolution convolutional neural network (Iterative SRCNN) over the city of Austin, Texas, USA. We show the development of a high-resolution gridded precipitation product (300 m) from a coarse (10 km) satellite-based product (JAXA GsMAP). High resolution gridded datasets of precipitation offer insights into the spatial distribution of heavy to low precipitation events in the past. The algorithm shows improvement in the mean peak-signal-to-noise-ratio and mutual information to generate high resolution gridded product of size 300 m X 300 m relative to the cubic interpolation baseline. Our results have implications for developing high-resolution gridded-precipitation urban datasets and the future planning of smart cities for other cities and other climatic variables.

physics.ao-ph↗

arXiv2022

On the modern deep learning approaches for precipitation downscaling

Deep Learning (DL) based downscaling has become a popular tool in earth sciences recently. Increasingly, different DL approaches are being adopted to downscale coarser precipitation data and generate more accurate and reliable estimates at local (~few km or even smaller) scales. Despite several studies adopting dynamical or statistical downscaling of precipitation, the accuracy is limited by the availability of ground truth. A key challenge to gauge the accuracy of such methods is to compare the downscaled data to point-scale observations which are often unavailable at such small scales. In this work, we carry out the DL-based downscaling to estimate the local precipitation data from the India Meteorological Department (IMD), which was created by approximating the value from station location to a grid point. To test the efficacy of different DL approaches, we apply four different methods of downscaling and evaluate their performance. The considered approaches are (i) Deep Statistical Downscaling (DeepSD), augmented Convolutional Long Short Term Memory (ConvLSTM), fully convolutional network (U-NET), and Super-Resolution Generative Adversarial Network (SR-GAN). A custom VGG network, used in the SR-GAN, is developed in this work using precipitation data. The results indicate that SR-GAN is the best method for precipitation data downscaling. The downscaled data is validated with precipitation values at IMD station. This DL method offers a promising alternative to statistical downscaling.

physics.ao-ph↗

arXiv2022

AI for Next Generation Computing: Emerging Trends and Future Directions

Autonomic computing investigates how systems can achieve (user) specified control outcomes on their own, without the intervention of a human operator. Autonomic computing fundamentals have been substantially influenced by those of control theory for closed and open-loop systems. In practice, complex systems may exhibit a number of concurrent and inter-dependent control loops. Despite research into autonomic models for managing computer resources, ranging from individual resources (e.g., web servers) to a resource ensemble (e.g., multiple resources within a data center), research into integrating Artificial Intelligence (AI) and Machine Learning (ML) to improve resource autonomy and performance at scale continues to be a fundamental challenge. The integration of AI/ML to achieve such autonomic and self-management of systems can be achieved at different levels of granularity, from full to human-in-the-loop automation. In this article, leading academics, researchers, practitioners, engineers, and scientists in the fields of cloud computing, AI/ML, and quantum computing join to discuss current research and potential future directions for these fields. Further, we discuss challenges and opportunities for leveraging AI and ML in next generation computing for emerging computing paradigms, including cloud, fog, edge, serverless and quantum computing environments.

cs.DC↗

arXiv2021

Machine learning for Earth System Science (ESS): A survey, status and future directions for South Asia

This survey focuses on the current problems in Earth systems science where machine learning algorithms can be applied. It provides an overview of previous work, ongoing work at the Ministry of Earth Sciences, Gov. of India, and future applications of ML algorithms to some significant earth science problems. We provide a comparison of previous work with this survey, a mind map of multidimensional areas related to machine learning and a Gartner's hype cycle for machine learning in Earth system science (ESS). We mainly focus on the critical components in Earth Sciences, including atmospheric, Ocean, Seismology, and biosphere, and cover AI/ML applications to statistical downscaling and forecasting problems.

cs.LG↗

arXiv2021

Quantum Artificial Intelligence for the Science of Climate Change

Climate change has become one of the biggest global problems increasingly compromising the Earth's habitability. Recent developments such as the extraordinary heat waves in California & Canada, and the devastating floods in Germany point to the role of climate change in the ever-increasing frequency of extreme weather. Numerical modelling of the weather and climate have seen tremendous improvements in the last five decades, yet stringent limitations remain to be overcome. Spatially and temporally localized forecasting is the need of the hour for effective adaptation measures towards minimizing the loss of life and property. Artificial Intelligence-based methods are demonstrating promising results in improving predictions, but are still limited by the availability of requisite hardware and software required to process the vast deluge of data at a scale of the planet Earth. Quantum computing is an emerging paradigm that has found potential applicability in several fields. In this opinion piece, we argue that new developments in Artificial Intelligence algorithms designed for quantum computers - also known as Quantum Artificial Intelligence (QAI) - may provide the key breakthroughs necessary to furthering the science of climate change. The resultant improvements in weather and climate forecasts are expected to cascade to numerous societal benefits.

cs.AI↗

arXiv2021

Quantum Computing: A Taxonomy, Systematic Review and Future Directions

Quantum computing is an emerging paradigm with the potential to offer significant computational advantage over conventional classical computing by exploiting quantum-mechanical principles such as entanglement and superposition. It is anticipated that this computational advantage of quantum computing will help to solve many complex and computationally intractable problems in several areas such as drug design, data science, clean energy, finance, industrial chemical development, secure communications, and quantum chemistry. In recent years, tremendous progress in both quantum hardware development and quantum software/algorithm have brought quantum computing much closer to reality. Indeed, the demonstration of quantum supremacy marks a significant milestone in the Noisy Intermediate Scale Quantum (NISQ) era - the next logical step being the quantum advantage whereby quantum computers solve a real-world problem much more efficiently than classical computing. As the quantum devices are expected to steadily scale up in the next few years, quantum decoherence and qubit interconnectivity are two of the major challenges to achieve quantum advantage in the NISQ era. Quantum computing is a highly topical and fast-moving field of research with significant ongoing progress in all facets. This article presents a comprehensive review of quantum computing literature, and taxonomy of quantum computing. Further, the proposed taxonomy is used to map various related studies to identify the research gaps. A detailed overview of quantum software tools and technologies, post-quantum cryptography and quantum computer hardware development to document the current state-of-the-art in the respective areas. We finish the article by highlighting various open challenges and promising future directions for research.

cs.ET↗

arXiv2021

Deep learning for improved global precipitation in numerical weather prediction systems

The formation of precipitation in state-of-the-art weather and climate models is an important process. The understanding of its relationship with other variables can lead to endless benefits, particularly for the world's monsoon regions dependent on rainfall as a support for livelihood. Various factors play a crucial role in the formation of rainfall, and those physical processes are leading to significant biases in the operational weather forecasts. We use the UNET architecture of a deep convolutional neural network with residual learning as a proof of concept to learn global data-driven models of precipitation. The models are trained on reanalysis datasets projected on the cubed-sphere projection to minimize errors due to spherical distortion. The results are compared with the operational dynamical model used by the India Meteorological Department. The theoretical deep learning-based model shows doubling of the grid point, as well as area averaged skill measured in Pearson correlation coefficients relative to operational system. This study is a proof-of-concept showing that residual learning-based UNET can unravel physical relationships to target precipitation, and those physical constraints can be used in the dynamical operational models towards improved precipitation forecasts. Our results pave the way for the development of online, hybrid models in the future.

physics.ao-ph↗

arXiv2021

Deep Learning Based Forecasting of Indian Summer Monsoon Rainfall

Accurate short range weather forecasting has significant implications for various sectors. Machine learning based approaches, e.g., deep learning, have gained popularity in this domain where the existing numerical weather prediction (NWP) models still have modest skill after a few days. Here we use a ConvLSTM network to develop a deep learning model for precipitation forecasting. The crux of the idea is to develop a forecasting model which involves convolution based feature selection and uses long term memory in the meteorological fields in conjunction with gradient based learning algorithm. Prior to using the input data, we explore various techniques to overcome dataset difficulties. We follow a strategic approach to deal with missing values and discuss the models fidelity to capture realistic precipitation. The model resolution used is (25 km). A comparison between 5 years of predicted data and corresponding observational records for 2 days lead time forecast show correlation coefficients of 0.67 and 0.42 for lead day 1 and 2 respectively. The patterns indicate higher correlation over the Western Ghats and Monsoon trough region (0.8 and 0.6 for lead day 1 and 2 respectively). Further, the model performance is evaluated based on skill scores, Mean Square Error, correlation coefficient and ROC curves. This study demonstrates that the adopted deep learning approach based only on a single precipitation variable, has a reasonable skill in the short range. Incorporating multivariable based deep learning has the potential to match or even better the short range precipitation forecasts based on the state of the art NWP models.

physics.ao-ph↗

arXiv2021

Quantifying COVID-19 enforced global changes in atmospheric pollutants using cloud computing based remote sensing

Global lockdowns in response to the COVID-19 pandemic have led to changes in the anthropogenic activities resulting in perceivable air quality improvements. Although several recent studies have analyzed these changes over different regions of the globe, these analyses have been constrained due to the usage of station-based data which is mostly limited upto the metropolitan cities. Also, the quantifiable changes have been reported only for the developed and developing regions leaving the poor economies (e.g. Africa) due to the shortage of in-situ data. Using a comprehensive set of high spatiotemporal resolution satellites and merged products of air pollutants, we analyze the air quality across the globe and quantify the improvement resulting from the suppressed anthropogenic activity during the lockdowns. In particular, we focus on megacities, capitals and cities with high standards of living to make the quantitative assessment. Our results offer valuable insights into the spatial distribution of changes in the air pollutants due to COVID-19 enforced lockdowns. Statistically significant reductions are observed over megacities with mean reduction by 19.74%, 7.38% and 49.9% in nitrogen dioxide (NO2), aerosol optical depth (AOD) and PM 2.5 concentrations. Google Earth Engine empowered cloud computing based remote sensing is used and the results provide a testbed for climate sensitivity experiments and validation of chemistry-climate models. Additionally, Google Earth Engine based apps have been developed to visualize the changes in a real-time fashion.

physics.ao-ph↗

arXiv2021

The IITM Earth System Model (IITM ESM)

Earth System Models (ESM) are important tools that allow us to understand and quantify the physical, chemical & biological mechanisms governing the rates of change of elements of the Earth System, comprising of the atmosphere, ocean, land, cryosphere and biosphere (terrestrial and marine) and related components. ESMs are essentially coupled numerical models which incorporate processes within and across the different Earth system components and are expressed as set of mathematical equations. ESMs are useful for enhancing our fundamental understanding of the climate system, its multi-scale variability, global and regional climatic phenomena and making projections of future climate change. In this chapter, we briefly describe the salient aspects of the Indian Institute of Tropical Meteorology ESM (IITM ESM), that has been developed recently at the IITM, Pune, India, for investigating long-term climate variability and change with focus on the South Asian monsoon.

physics.ao-ph↗