arXiv subjects

Kourosh Parand

Publications and source records attributed to Kourosh Parand.

At least 19 recordsLinked to original sources

arXiv2025

LyAm: Robust Non-Convex Optimization for Stable Learning in Noisy Environments

Training deep neural networks, particularly in computer vision tasks, often suffers from noisy gradients and unstable convergence, which hinder performance and generalization. In this paper, we propose LyAm, a novel optimizer that integrates Adam's adaptive moment estimation with Lyapunov-based stability mechanisms. LyAm dynamically adjusts the learning rate using Lyapunov stability theory to enhance convergence robustness and mitigate training noise. We provide a rigorous theoretical framework proving the convergence guarantees of LyAm in complex, non-convex settings. Extensive experiments on like as CIFAR-10 and CIFAR-100 show that LyAm consistently outperforms state-of-the-art optimizers in terms of accuracy, convergence speed, and stability, establishing it as a strong candidate for robust deep learning optimization.

cs.LG↗

arXiv2025

BridgeNet: A Hybrid, Physics-Informed Machine Learning Framework for Solving High-Dimensional Fokker-Planck Equations

BridgeNet is a novel hybrid framework that integrates convolutional neural networks with physics-informed neural networks to efficiently solve non-linear, high-dimensional Fokker-Planck equations (FPEs). Traditional PINNs, which typically rely on fully connected architectures, often struggle to capture complex spatial hierarchies and enforce intricate boundary conditions. In contrast, BridgeNet leverages adaptive CNN layers for effective local feature extraction and incorporates a dynamically weighted loss function that rigorously enforces physical constraints. Extensive numerical experiments across various test cases demonstrate that BridgeNet not only achieves significantly lower error metrics and faster convergence compared to conventional PINN approaches but also maintains robust stability in high-dimensional settings. This work represents a substantial advancement in computational physics, offering a scalable and accurate solution methodology with promising applications in fields ranging from financial mathematics to complex system dynamics.

physics.comp-ph↗

arXiv2025

Advanced Physics-Informed Neural Network with Residuals for Solving Complex Integral Equations

In this paper, we present the Residual Integral Solver Network (RISN), a novel neural network architecture designed to solve a wide range of integral and integro-differential equations, including one-dimensional, multi-dimensional, ordinary and partial integro-differential, systems, fractional types, and Helmholtz-type integral equations involving oscillatory kernels. RISN integrates residual connections with high-accuracy numerical methods such as Gaussian quadrature and fractional derivative operational matrices, enabling it to achieve higher accuracy and stability than traditional Physics-Informed Neural Networks (PINN). The residual connections help mitigate vanishing gradient issues, allowing RISN to handle deeper networks and more complex kernels, particularly in multi-dimensional problems. Through extensive experiments, we demonstrate that RISN consistently outperforms not only classical PINNs but also advanced variants such as Auxiliary PINN (A-PINN) and Self-Adaptive PINN (SA-PINN), achieving significantly lower Mean Absolute Errors (MAE) across various types of equations. These results highlight RISN's robustness and efficiency in solving challenging integral and integro-differential problems, making it a valuable tool for real-world applications where traditional methods often struggle.

cs.LG↗

arXiv2024

PINNIES: An Efficient Physics-Informed Neural Network Framework to Integral Operator Problems

This paper introduces an efficient tensor-vector product technique for the rapid and accurate approximation of integral operators within physics-informed deep learning frameworks. Our approach leverages neural network architectures to evaluate problem dynamics at specific points, while employing Gaussian quadrature formulas to approximate the integral components, even in the presence of infinite domains or singularities. We demonstrate the applicability of this method to both Fredholm and Volterra integral operators, as well as to optimal control problems involving continuous time. Additionally, we outline how this approach can be extended to approximate fractional derivatives and integrals and propose a fast matrix-vector product algorithm for efficiently computing the fractional Caputo derivative. In the numerical section, we conduct comprehensive experiments on forward and inverse problems. For forward problems, we evaluate the performance of our method on over 50 diverse mathematical problems, including multi-dimensional integral equations, systems of integral equations, partial and fractional integro-differential equations, and various optimal control problems in delay, fractional, multi-dimensional, and nonlinear configurations. For inverse problems, we test our approach on several integral equations and fractional integro-differential problems. Finally, we introduce the pinnies Python package to facilitate the implementation and usability of the proposed method.

cs.LG↗

arXiv2024

Solving the decision-making differential equations from eye fixation data in Unity software by using Hermite Long-Short-Term Memory neural network

Cognitive decision-making processes are crucial aspects of human behavior, influencing various personal and professional domains. This research delves into the application of differential equations in analyzing decision-making accuracy by leveraging eye-tracking data within a virtual industrial town setting. The study unveils a systematic approach to transforming raw data into a differential equation, essential for deciphering the relationship between eye movements during decision-making processes. Mathematical relationship extraction and variable-parameter definition pave the way for deriving a differential equation that encapsulates the growth of fixations on characters. The key factors in this equation encompass the fixation rate $(\lambda)$ and separation rate $(\mu)$, reflecting user interaction dynamics and their impact on decision-making complexities tied to user engagement with virtual characters. For a comprehensive grasp of decision dynamics, solving this differential equation requires initial fixation counts, fixation rate, and separation rate. The formulation of differential equations incorporates various considerations such as engagement duration, character-player distance, relative speed, and character attributes, enabling the representation of fixation changes, speed dynamics, distance variations, and the effects of character attributes. This comprehensive analysis not only enhances our comprehension of decision-making processes but also provides a foundational framework for predictive modeling and data-driven insights for future research and applications in cognitive science and virtual reality environments.

cs.HC↗

arXiv2024

Hermite Neural Network Simulation for Solving the 2D Schrodinger Equation

The Schrodinger equation is a mathematical equation describing the wave function's behavior in a quantum-mechanical system. It is a partial differential equation that provides valuable insights into the fundamental principles of quantum mechanics. In this paper, the aim was to solve the Schrodinger equation with sufficient accuracy by using a mixture of neural networks with the collocation method base Hermite functions. Initially, the Hermite functions roots were employed as collocation points, enhancing the efficiency of the solution. The Schrodinger equation is defined in an infinite domain, the use of Hermite functions as activation functions resulted in excellent precision. Finally, the proposed method was simulated using MATLAB's Simulink tool. The results were then compared with those obtained using Physics-informed neural networks and the presented method.

math.NA↗

arXiv2024

Accelerating Fractional PINNs using Operational Matrices of Derivative

This paper presents a novel operational matrix method to accelerate the training of fractional Physics-Informed Neural Networks (fPINNs). Our approach involves a non-uniform discretization of the fractional Caputo operator, facilitating swift computation of fractional derivatives within Caputo-type fractional differential problems with $0<\alpha<1$. In this methodology, the operational matrix is precomputed, and during the training phase, automatic differentiation is replaced with a matrix-vector product. While our methodology is compatible with any network, we particularly highlight its successful implementation in PINNs, emphasizing the enhanced accuracy achieved when utilizing the Legendre Neural Block (LNB) architecture. LNB incorporates Legendre polynomials into the PINN structure, providing a significant boost in accuracy. The effectiveness of our proposed method is validated across diverse differential equations, including Delay Differential Equations (DDEs) and Systems of Differential Algebraic Equations (DAEs). To demonstrate its versatility, we extend the application of the method to systems of differential equations, specifically addressing nonlinear Pantograph fractional-order DDEs/DAEs. The results are supported by a comprehensive analysis of numerical outcomes.

cs.LG↗

arXiv2024

An Orthogonal Polynomial Kernel-Based Machine Learning Model for Differential-Algebraic Equations

The recent introduction of the Least-Squares Support Vector Regression (LS-SVR) algorithm for solving differential and integral equations has sparked interest. In this study, we expand the application of this algorithm to address systems of differential-algebraic equations (DAEs). Our work presents a novel approach to solving general DAEs in an operator format by establishing connections between the LS-SVR machine learning model, weighted residual methods, and Legendre orthogonal polynomials. To assess the effectiveness of our proposed method, we conduct simulations involving various DAE scenarios, such as nonlinear systems, fractional-order derivatives, integro-differential, and partial DAEs. Finally, we carry out comparisons between our proposed method and currently established state-of-the-art approaches, demonstrating its reliability and effectiveness.

math.NA↗

arXiv2024

Using Singular Value Decomposition in a Convolutional Neural Network to Improve Brain Tumor Segmentation Accuracy

A brain tumor consists of cells showing abnormal brain growth. The area of the brain tumor significantly affects choosing the type of treatment and following the course of the disease during the treatment. At the same time, pictures of Brain MRIs are accompanied by noise. Eliminating existing noises can significantly impact the better segmentation and diagnosis of brain tumors. In this work, we have tried using the analysis of eigenvalues. We have used the MSVD algorithm, reducing the image noise and then using the deep neural network to segment the tumor in the images. The proposed method's accuracy was increased by 2.4% compared to using the original images. With Using the MSVD method, convergence speed has also increased, showing the proposed method's effectiveness

eess.IV↗

arXiv2023

deepFDEnet: A Novel Neural Network Architecture for Solving Fractional Differential Equations

The primary goal of this research is to propose a novel architecture for a deep neural network that can solve fractional differential equations accurately. A Gaussian integration rule and a $L_1$ discretization technique are used in the proposed design. In each equation, a deep neural network is used to approximate the unknown function. Three forms of fractional differential equations have been examined to highlight the method's versatility: a fractional ordinary differential equation, a fractional order integrodifferential equation, and a fractional order partial differential equation. The results show that the proposed architecture solves different forms of fractional differential equations with excellent precision.

cs.LG↗

arXiv2023

Solving Falkner-Skan type equations via Legendre and Chebyshev Neural Blocks

In this paper, a new deep-learning architecture for solving the non-linear Falkner-Skan equation is proposed. Using Legendre and Chebyshev neural blocks, this approach shows how orthogonal polynomials can be used in neural networks to increase the approximation capability of artificial neural networks. In addition, utilizing the mathematical properties of these functions, we overcome the computational complexity of the backpropagation algorithm by using the operational matrices of the derivative. The efficiency of the proposed method is carried out by simulating various configurations of the Falkner-Skan equation.

cs.LG↗

arXiv2023

Hyperparameter optimization of orthogonal functions in the numerical solution of differential equations

This paper considers the hyperparameter optimization problem of mathematical techniques that arise in the numerical solution of differential and integral equations. The well-known approaches grid and random search, in a parallel algorithm manner, are developed to find the optimal set of hyperparameters. Employing rational Jacobi functions, we ran these algorithms on two nonlinear benchmark differential equations on the semi-infinite domain. The configurations contain different rational mappings along with their length scale parameter and the Jacobi functions parameters. These trials are configured on the collocation Least-Squares Support Vector Regression (CLS-SVR), a novel numerical simulation approach based on spectral methods. In addition, we have addressed the sensitivity of these hyperparameters on the numerical stability and convergence of the CLS-SVR model. The experiments show that this technique can effectively improve state-of-the-art results.

math.NA↗

arXiv2022

JDNN: Jacobi Deep Neural Network for Solving Telegraph Equation

In this article, a new deep learning architecture, named JDNN, has been proposed to approximate a numerical solution to Partial Differential Equations (PDEs). The JDNN is capable of solving high-dimensional equations. Here, Jacobi Deep Neural Network (JDNN) has demonstrated various types of telegraph equations. This model utilizes the orthogonal Jacobi polynomials as the activation function to increase the accuracy and stability of the method for solving partial differential equations. The finite difference time discretization technique is used to overcome the computational complexity of the given equation. The proposed scheme utilizes a Graphics Processing Unit (GPU) to accelerate the learning process by taking advantage of the neural network platforms. Comparing the existing methods, the numerical experiments show that the proposed approach can efficiently learn the dynamics of the physical problem.

math.NA↗

arXiv2022

A least squares support vector regression for anisotropic diffusion filtering

Anisotropic diffusion filtering for signal smoothing as a low-pass filter has the advantage of the edge-preserving, i.e., it does not affect the edges that contain more critical data than the other parts of the signal. In this paper, we present a numerical algorithm based on least squares support vector regression by using Legendre orthogonal kernel with the discretization of the nonlinear diffusion problem in time by the Crank-Nicolson method. This method transforms the signal smoothing process into solving an optimization problem that can be solved by efficient numerical algorithms. In the final analysis, we have reported some numerical experiments to show the effectiveness of the proposed machine learning based approach for signal smoothing.

eess.SP↗

arXiv2021

Legendre Deep Neural Network (LDNN) and its application for approximation of nonlinear Volterra Fredholm Hammerstein integral equations

Various phenomena in biology, physics, and engineering are modeled by differential equations. These differential equations including partial differential equations and ordinary differential equations can be converted and represented as integral equations. In particular, Volterra Fredholm Hammerstein integral equations are the main type of these integral equations and researchers are interested in investigating and solving these equations. In this paper, we propose Legendre Deep Neural Network (LDNN) for solving nonlinear Volterra Fredholm Hammerstein integral equations (VFHIEs). LDNN utilizes Legendre orthogonal polynomials as activation functions of the Deep structure. We present how LDNN can be used to solve nonlinear VFHIEs. We show using the Gaussian quadrature collocation method in combination with LDNN results in a novel numerical solution for nonlinear VFHIEs. Several examples are given to verify the performance and accuracy of LDNN.

math.NA↗

arXiv2020

Symbolically Solving Partial Differential Equations using Deep Learning

We describe a neural-based method for generating exact or approximate solutions to differential equations in the form of mathematical expressions. Unlike other neural methods, our system returns symbolic expressions that can be interpreted directly. Our method uses a neural architecture for learning mathematical expressions to optimize a customizable objective, and is scalable, compact, and easily adaptable for a variety of tasks and configurations. The system has been shown to effectively find exact or approximate symbolic solutions to various differential equations with applications in natural sciences. In this work, we highlight how our method applies to partial differential equations over multiple variables and more complex boundary and initial value conditions.

cs.LG↗

arXiv2019

A comparison between pre-Newton and post-Newton approaches for solving a physical singular second-order boundary problem in the semi-infinite interval

In this paper, two numerical approaches based on the Newton iteration method with spectral algorithms are introduced to solve the Thomas-Fermi equation. That Thomas-Fermi equation is a nonlinear singular ordinary differential equation (ODE) with boundary condition in infinite. In these schemes, the Newton method is combined with a spectral method where in one of those, by Newton method we convert nonlinear ODE to a sequence of linear ODE then, solve them using the spectral method. In another one, by the spectral method the nonlinear ODE be converted to system of nonlinear algebraic equations, then, this system is solved by Newton method. In both approaches, the spectral method is based on fractional order of rational Gegenbauer functions. Finally, the obtained results of two introduced schemes are compared to each other in accuracy, runtime and iteration number. Numerical experiments are presented showing that our methods are as accurate as the best results which obtained until now.

math.NA↗

arXiv2019

Imposing various boundary conditions on positive definite kernels

This work is motivated by the frequent occurrence of boundary value problems with various boundary conditions in the modeling of some problems in engineering and physical science. Here we propose a new technique to force the positive definite kernels such as some radial basis functions to satisfy the boundary conditions exactly. It can improve the applications of existing methods based on positive definite kernels and radial basis functions especially the pseudospectral radial basis function method for handling the differential equations with more complicated boundary conditions. The proposed method is applied to a singularly perturbed steady-state convection-diffusion problem, two and three dimensional Poisson's equations with various boundary conditions.

math.NA↗