arXiv subjects

Shiv Dutt Joshi

Publications and source records attributed to Shiv Dutt Joshi.

9 recordsLinked to original sources

arXiv2022

Altering Backward Pass Gradients improves Convergence

In standard neural network training, the gradients in the backward pass are determined by the forward pass. As a result, the two stages are coupled. This is how most neural networks are trained currently. However, gradient modification in the backward pass has seldom been studied in the literature. In this paper we explore decoupled training, where we alter the gradients in the backward pass. We propose a simple yet powerful method called PowerGrad Transform, that alters the gradients before the weight update in the backward pass and significantly enhances the predictive performance of the neural network. PowerGrad Transform trains the network to arrive at a better optima at convergence. It is computationally extremely efficient, virtually adding no additional cost to either memory or compute, but results in improved final accuracies on both the training and test sets. PowerGrad Transform is easy to integrate into existing training routines, requiring just a few lines of code. PowerGrad Transform accelerates training and makes it possible for the network to better fit the training data. With decoupled training, PowerGrad Transform improves baseline accuracies for ResNet-50 by 0.73%, for SE-ResNet-50 by 0.66% and by more than 1.0% for the non-normalized ResNet-18 network on the ImageNet classification task.

cs.LG↗

arXiv2021

The Generalized Fourier Transform: A Unified Framework for the Fourier, Laplace, Mellin and $Z$ Transforms

This paper introduces Generalized Fourier transform (GFT) that is an extension or the generalization of the Fourier transform (FT). The Unilateral Laplace transform (LT) is observed to be the special case of GFT. GFT, as proposed in this work, contributes significantly to the scholarly literature. There are many salient contribution of this work. Firstly, GFT is applicable to a much larger class of signals, some of which cannot be analyzed with FT and LT. For example, we have shown the applicability of GFT on the polynomially decaying functions and super exponentials. Secondly, we demonstrate the efficacy of GFT in solving the initial value problems (IVPs). Thirdly, the generalization presented for FT is extended for other integral transforms with examples shown for wavelet transform and cosine transform. Likewise, generalized Gamma function is also presented. One interesting application of GFT is the computation of generalized moments, for the otherwise non-finite moments, of any random variable such as the Cauchy random variable. Fourthly, we introduce Fourier scale transform (FST) that utilizes GFT with the topological isomorphism of an exponential map. Lastly, we propose Generalized Discrete-Time Fourier transform (GDTFT). The DTFT and unilateral $z$-transform are shown to be the special cases of the proposed GDTFT. The properties of GFT and GDTFT have also been discussed.

eess.SP↗

arXiv2017

Unified Functorial Signal Representation III: Foundations, Redundancy, $L^0$ and $L^2$ functors

In this paper we propose and lay the foundations of a functorial framework for representing signals. By incorporating additional category-theoretic relative and generative perspective alongside the classic set-theoretic measure theory the fundamental concepts of redundancy, compression are formulated in a novel authentic arrow-theoretic way. The existing classic framework representing a signal as a vector of appropriate linear space is shown as a special case of the proposed framework. Next in the context of signal-spaces as a categories we study the various covariant and contravariant forms of $L^0$ and $L^2$ functors using categories of measurable or measure spaces and their opposites involving Boolean and measure algebras along with partial extension. Finally we contribute a novel definition of intra-signal redundancy using general concept of isomorphism arrow in a category covering the translation case and others as special cases. Through category-theory we provide a simple yet precise explanation for the well-known heuristic of lossless differential encoding standards yielding better compressions in image types such as line drawings, iconic image, text etc; as compared to classic representation techniques such as JPEG which choose bases or frames in a global Hilbert space.

eess.SP↗

arXiv2017

Unified Functorial Signal Representation I: From Grothendieck fibration to Base structured categories

In this paper we study categories $(F,\mathbf{C},\mathbf{D})$ and $(\mathbb{F},\mathbf{C},\mathbf{Set})$ and prove them to be fibred on $\mathbf{C}$. Then we examine Grothendieck construction in the context of an ordinary functor $F: \mathbf{C} \rightarrow \mathbf{D}$ through the concept of trivial categorification, using an appropriate functor $\mathbf{F}: \mathbf{C} \xrightarrow{F} \mathbf{D} \xrightarrow{I} \mathbf{Cat}$ to construct $\int_{\mathbf{C}^{op}} \bar{\mathbf{F}}$. This category characterizes a functor as an abstract right category action while its dual $\mathcal{X} \rtimes_{\mathbf{F}} \mathbf{C}$ or $(\int_{\mathbf{C}^{op}} \bar{\mathbf{F}})^{op}$ characterizes a functor as an abstract left category action. Similarly using $\mathbb{F}: \mathbf{C} \xrightarrow{F} \mathbf{D} \xrightarrow{U} \mathbf{Set}$ we define $\mathcal{X} \rtimes_{\mathbb{F}} \mathbf{C}$ and ${\int_{\mathbf{C}^{op}} \bar{\mathbb{F}}}$ as categories denoting concrete left and right actions of $\mathbf{C}$ respectively. Collectively referred to as `base structured categories', these are proven abstractly isomorphic to the base category $\mathbf{C}$ but concretely isomorphic to each other or $(\mathbb{F},\mathbf{C},\mathbf{Set}) \cong \int_{\mathbf{C}} \bar{\mathbb{F}} \cong \mathcal{X} \rtimes_{\mathbb{F}} \mathbf{C}$. These are special instances of fibred categories where the base category is $\mathbf{C}$ and fibres are $\mathbf{D}$ objects being treated as trivial categories. The perspective of making only base structure explicit through category theory concealing the vertical structure using identity morphisms enables one to combine intuitions of Grothendieck's relative and Leyton's generative theory. As explored further, it facilitates the application of functors in certain fundamental applications which hitherto have been treating objects of category $\mathbf{D}$ purely in a set theoretic way.

math.CT↗

arXiv2017

Unified Functorial Signal Representation II: Category action, Base Hierarchy, Geometries as Base structured categories

In this paper we propose and study few applications of the base structured categories $\mathcal{X} \rtimes_{\mathbf{F}} \mathbf{C}$, $\int_{\mathbf{C}} \bar{\mathbf{F}}$, $\mathcal{X} \rtimes_{\mathbb{F}} \mathbf{C}$ and ${\int_{\mathbf{C}} \bar{\mathbb{F}}}$. First we show classic transformation groupoid $X /\!\!/ G$ simply being a base-structured category ${\int_{\mathbf{G}} \bar{F}}$. Then using permutation action on a finite set, we introduce the notion of a hierarchy of base structured categories $[(\mathcal{X}_{2a} \rtimes_{\mathbf{F_{2a}}} \mathbf{B}_{2a}) \amalg (\mathcal{X}_{2b} \rtimes_{\mathbf{F_{2b}}} \mathbf{B}_{2b}) \amalg ...] \rtimes_{\mathbf{F_{1}}} \mathbf{B}_1$ that models local and global structures as a special case of composite Grothendieck fibration. Further utilizing the existing notion of transformation double category $(\mathcal{X}_{1} \rtimes_{\mathbf{F_{1}}} \mathbf{B}_{1}) /\!\!/ \mathbf{2G}$, we demonstrate that a hierarchy of bases naturally leads one from 2-groups to n-category theory. Finally we prove that every classic Klein geometry is the Grothendieck completion ($\mathbf{G} = \mathcal{X} \rtimes_{\mathbb{F}} \mathbf{H}$) of ${\mathbb{F}}: \mathbf{H} \xrightarrow{F} \mathbf{Man}^{\infty} \xrightarrow{U} \mathbf{Set}$. This is generalized to propose a set-theoretic definition of a groupoid geometry $(\mathcal{G},\mathcal{B})$ (originally conceived by Ehresmann through transport and later by Leyton using transfer) with a principal groupoid $\mathcal{G} = \mathcal{X} \rtimes \mathcal{B}$ and geometry space $\mathcal{X} = \mathcal{G}/\mathcal{B}$; which is essentially same as $\mathbf{G} = \mathcal{X} \rtimes_{\mathbb{F}} \mathbf{B}$ or precisely the completion of ${\mathbb{F}}: \mathbf{B} \xrightarrow{F} \mathbf{Man}^{\infty} \xrightarrow{U} \mathbf{Set}$.

math.CT↗

arXiv2015

The Fourier Decomposition Method for nonlinear and nonstationary time series analysis

Since many decades, there is a general perception in literature that the Fourier methods are not suitable for the analysis of nonlinear and nonstationary data. In this paper, we propose a Fourier Decomposition Method (FDM) and demonstrate its efficacy for the analysis of nonlinear (i.e. data generated by nonlinear systems) and nonstationary time series. The proposed FDM decomposes any data into a small number of `Fourier intrinsic band functions' (FIBFs). The FDM presents a generalized Fourier expansion with variable amplitudes and frequencies of a time series by the Fourier method itself. We propose an idea of zero-phase filter bank based multivariate FDM (MFDM) algorithm, for the analysis of multivariate nonlinear and nonstationary time series, from the FDM. We also present an algorithm to obtain cutoff frequencies for MFDM. The MFDM algorithm is generating finite number of band limited multivariate FIBFs (MFIBFs). The MFDM preserves some intrinsic physical properties of the multivariate data, such as scale alignment, trend and instantaneous frequency. The proposed methods produce the results in a time-frequency-energy distribution that reveal the intrinsic structures of a data. Simulations have been carried out and comparison is made with the Empirical Mode Decomposition (EMD) methods in the analysis of various simulated as well as real life time series, and results show that the proposed methods are powerful tools for analyzing and obtaining the time-frequency-energy representation of any data.

stat.ME↗

arXiv2015

Some studies on multidimensional Fourier theory for Hilbert transform, analytic signal and space-time series analysis

In this paper, we propose the Fourier frequency vector (FFV), inherently, associated with multidimensional Fourier transform. With the help of FFV, we are able to provide physical meaning of so called negative frequencies in multidimensional Fourier transform (MDFT), which in turn provide multidimensional spatial and space-time series analysis. The complex exponential representation of sinusoidal function always yields two frequencies, negative frequency corresponding to positive frequency and vice versa, in the multidimensional Fourier spectrum. Thus, using the MDFT, we propose multidimensional Hilbert transform (MDHT) and associated multidimensional analytic signal (MDAS) with following properties: (a) the extra and redundant positive, negative, or both frequencies, introduced due to complex exponential representation of multidimensional Fourier spectrum, are suppressed, (b) real part of MDAS is original signal, (c) real and imaginary part of MDAS are orthogonal, and (d) the magnitude envelope of a original signal is obtained as the magnitude of its associated MDAS, which is the instantaneous amplitude of the MDAS. The proposed MDHT and associated DMAS are generalization of the 1D HT and AS, respectively. We also provide the decomposition of an image into the AM-FM image model by the Fourier method and obtain explicit expression for the analytic image computation by 2DDFT.

cs.IT↗

arXiv2015

Computationally efficient MIMO system identification using Signal Matched Synthesis Filter Bank

We propose a multi input multi output(MIMO) system identification framework by interpreting the MIMO system in terms of a multirate synthesis filter bank. The proposed methodology is discussed in two steps: in the first step the MIMO system is interpreted as a synthesis filter bank and the second step is to convert the MIMO system into a SISO system "without any loss of information", which re-structures the system identification problem into a SISO form. The system identification problem, in its new form, is identical to the problem of obtaining the signal matched synthesis filter bank (SMSFB) as proposed in Part II. Since we have developed fast algorithms to obtain the filter bank coefficients in Part II, for "the given data case" as well as "the given statistics case", we can use these algorithm for the MIMO system identification as well. This framework can have an adaptive as well as block processing implementation. The algorithms, used here, involve only scalar computations, unlike the conventional MIMO system identification algorithms where one requires matrix computations. These order recursive algorithm can also be used to obtain approximate smaller order model for large order systems without using any model order reduction algorithm. The proposed identification framework can also be used for SISO LPTV system identification and also for a SIMO or MISO system. The efficacy of the proposed scheme is validated and its performance in the presence of measurement noise is illustrated using simulation results.

cs.IT↗

arXiv2015

The Hilbert spectrum and the Energy Preserving Empirical Mode Decomposition

In this paper, we propose algorithms which preserve energy in empirical mode decomposition (EMD), generating finite $n$ number of band limited Intrinsic Mode Functions (IMFs). In the first energy preserving EMD (EPEMD) algorithm, a signal is decomposed into linearly independent (LI), non orthogonal yet energy preserving (LINOEP) IMFs and residue (EPIMFs). It is shown that a vector in an inner product space can be represented as a sum of LI and non orthogonal vectors in such a way that Parseval's type property is satisfied. From the set of $n$ IMFs, through Gram-Schmidt orthogonalization method (GSOM), $n!$ set of orthogonal functions can be obtained. In the second algorithm, we show that if the orthogonalization process proceeds from lowest frequency IMF to highest frequency IMF, then the GSOM yields functions which preserve the properties of IMFs and the energy of a signal. With the Hilbert transform, these IMFs yield instantaneous frequencies and amplitudes as functions of time that reveal the imbedded structures of a signal. The instantaneous frequencies and square of amplitudes as functions of time produce a time-frequency-energy distribution, referred as the Hilbert spectrum, of a signal. Simulations have been carried out for the analysis of various time series and real life signals to show comparison among IMFs produced by EMD, EPEMD, ensemble EMD and multivariate EMD algorithms. Simulation results demonstrate the power of this proposed method.

cs.IT↗