arXiv subjects

Reshad Hosseini

Publications and source records attributed to Reshad Hosseini.

At least 37 records · Page 2Linked to original sources

arXiv2022

Introducing One Sided Margin Loss for Solving Classification Problems in Deep Networks

This paper introduces a new loss function, OSM (One-Sided Margin), to solve maximum-margin classification problems effectively. Unlike the hinge loss, in OSM the margin is explicitly determined with corresponding hyperparameters and then the classification problem is solved. In experiments, we observe that using OSM loss leads to faster training speeds and better accuracies than binary and categorical cross-entropy in several commonly used deep models for classification and optical character recognition problems. OSM has consistently shown better classification accuracies over cross-entropy and hinge losses for small to large neural networks. it has also led to a more efficient training procedure. We achieved state-of-the-art accuracies for small networks on several benchmark datasets of CIFAR10(98.82\%), CIFAR100(91.56\%), Flowers(98.04\%), Stanford Cars(93.91\%) with considerable improvements over other loss functions. Moreover, the accuracies are rather better than cross-entropy and hinge loss for large networks. Therefore, we strongly believe that OSM is a powerful alternative to hinge and cross-entropy losses to train deep neural networks on classification tasks.

cs.LG↗

arXiv2021

ParsiNorm: A Persian Toolkit for Speech Processing Normalization

In general, speech processing models consist of a language model along with an acoustic model. Regardless of the language model's complexity and variants, three critical pre-processing steps are needed in language models: cleaning, normalization, and tokenization. Among mentioned steps, the normalization step is so essential to format unification in pure textual applications. However, for embedded language models in speech processing modules, normalization is not limited to format unification. Moreover, it has to convert each readable symbol, number, etc., to how they are pronounced. To the best of our knowledge, there is no Persian normalization toolkits for embedded language models in speech processing modules, So in this paper, we propose an open-source normalization toolkit for text processing in speech applications. Briefly, we consider different readable Persian text like symbols (common currencies, #, @, URL, etc.), numbers (date, time, phone number, national code, etc.), and so on. Comparison with other available Persian textual normalization tools indicates the superiority of the proposed method in speech processing. Also, comparing the model's performance for one of the proposed functions (sentence separation) with other common natural language libraries such as HAZM and Parsivar indicates the proper performance of the proposed method. Besides, its evaluation of some Persian Wikipedia data confirms the proper performance of the proposed method.

eess.AS↗

arXiv2021

Vector Transport Free Riemannian LBFGS for Optimization on Symmetric Positive Definite Matrix Manifolds

This work concentrates on optimization on Riemannian manifolds. The Limited-memory Broyden-Fletcher-Goldfarb-Shanno (LBFGS) algorithm is a commonly used quasi-Newton method for numerical optimization in Euclidean spaces. Riemannian LBFGS (RLBFGS) is an extension of this method to Riemannian manifolds. RLBFGS involves computationally expensive vector transports as well as unfolding recursions using adjoint vector transports. In this article, we propose two mappings in the tangent space using the inverse second root and Cholesky decomposition. These mappings make both vector transport and adjoint vector transport identity and therefore isometric. Identity vector transport makes RLBFGS less computationally expensive and its isometry is also very useful in convergence analysis of RLBFGS. Moreover, under the proposed mappings, the Riemannian metric reduces to Euclidean inner product, which is much less computationally expensive. We focus on the Symmetric Positive Definite (SPD) manifolds which are beneficial in various fields such as data science and statistics. This work opens a research opportunity for extension of the proposed mappings to other well-known manifolds.

math.OC↗

arXiv2021

Accurate and fast matrix factorization for low-rank learning

In this paper, we tackle two important problems in low-rank learning, which are partial singular value decomposition and numerical rank estimation of huge matrices. By using the concepts of Krylov subspaces such as Golub-Kahan bidiagonalization (GK-bidiagonalization) as well as Ritz vectors, we propose two methods for solving these problems in a fast and accurate way. Our experiments show the advantages of the proposed methods compared to the traditional and randomized singular value decomposition methods. The proposed methods are appropriate for applications involving huge matrices where the accuracy of the desired singular values and also all of their corresponding singular vectors are essential. As a real application, we evaluate the performance of our methods on the problem of Riemannian similarity learning between two various image datasets of MNIST and USPS.

stat.ML↗

arXiv2021

Solving Viewing Graph Optimization for Simultaneous Position and Rotation Registration

A viewing graph is a set of unknown camera poses, as the vertices, and the observed relative motions, as the edges. Solving the viewing graph is an essential step in a Structure-from-Motion procedure, where a set of relative motions is obtained from a collection of 2D images. Almost all methods in the literature solve for the rotations separately, through rotation averaging process, and use them for solving the positions. Obtaining positions is the challenging part because the translation observations only tell the direction of the motions. It becomes more challenging when the set of edges comprises pairwise translation observations between either near and far cameras. In this paper an iterative method is proposed that overcomes these issues. Also a method is proposed which obtains the rotations and positions simultaneously. Experimental results show the-state-of-the-art performance of the proposed methods.

cs.CV↗

arXiv2021

Optimal Triangulation Method is Not Really Optimal

Triangulation refers to the problem of finding a 3D point from its 2D projections on multiple camera images. For solving this problem, it is the common practice to use so-called optimal triangulation method, which we call the L2 method in this paper. But, the method can be optimal only if we assume no uncertainty in the camera parameters. Through extensive comparison on synthetic and real data, we observed that the L2 method is actually not the best choice when there is uncertainty in the camera parameters. Interestingly, it can be observed that the simple mid-point method outperforms other methods. Apart from its high performance, the mid-point method has a simple closed formed solution for multiple camera images while the L2 method is hard to be used for more than two camera images. Therefore, in contrast to the common practice, we argue that the simple mid-point method should be used in structure-from-motion applications where there is uncertainty in camera parameters.

cs.CV↗

arXiv2021

Contour Integration using Graph-Cut and Non-Classical Receptive Field

Many edge and contour detection algorithms give a soft-value as an output and the final binary map is commonly obtained by applying an optimal threshold. In this paper, we propose a novel method to detect image contours from the extracted edge segments of other algorithms. Our method is based on an undirected graphical model with the edge segments set as the vertices. The proposed energy functions are inspired by the surround modulation in the primary visual cortex that help suppressing texture noise. Our algorithm can improve extracting the binary map, because it considers other important factors such as connectivity, smoothness, and length of the contour beside the soft-values. Our quantitative and qualitative experimental results show the efficacy of the proposed method.

cs.CV↗

arXiv2021

Learning with partially separable data

There are partially separable data types that make classification tasks very hard. In other words, only parts of the data are informative meaning that looking at the rest of the data would not give any distinguishable hint for classification. In this situation, the typical assumption of having the whole labeled data as an informative unit set for classification does not work. Consequently, typical classification methods with the mentioned assumption fail in such a situation. In this study, we propose a framework for the classification of partially separable data types that are not classifiable using typical methods. An algorithm based on the framework is proposed that tries to detect separable subgroups of the data using an iterative clustering approach. Then the detected subgroups are used in the classification process. The proposed approach was tested on a real dataset for autism screening and showed its capability by distinguishing children with autism from normal ones, while the other methods failed to do so.

cs.LG↗

arXiv2021

Simple online and real-time tracking with occlusion handling

Multiple object tracking is a challenging problem in computer vision due to difficulty in dealing with motion prediction, occlusion handling, and object re-identification. Many recent algorithms use motion and appearance cues to overcome these challenges. But using appearance cues increases the computation cost notably and therefore the speed of the algorithm decreases significantly which makes them inappropriate for online applications. In contrast, there are algorithms that only use motion cues to increase speed, especially for online applications. But these algorithms cannot handle occlusions and re-identify lost objects. In this paper, a novel online multiple object tracking algorithm is presented that only uses geometric cues of objects to tackle the occlusion and reidentification challenges simultaneously. As a result, it decreases the identity switch and fragmentation metrics. Experimental results show that the proposed algorithm could decrease identity switch by 40% and fragmentation by 28% compared to the state of the art online tracking algorithms. The code is also publicly available.

cs.CV↗

arXiv2020

Reinforcement Learning with Subspaces using Free Energy Paradigm

In large-scale problems, standard reinforcement learning algorithms suffer from slow learning speed. In this paper, we follow the framework of using subspaces to tackle this problem. We propose a free-energy minimization framework for selecting the subspaces and integrate the policy of the state-space into the subspaces. Our proposed free-energy minimization framework rests upon Thompson sampling policy and behavioral policy of subspaces and the state-space. It is therefore applicable to a variety of tasks, discrete or continuous state space, model-free and model-based tasks. Through a set of experiments, we show that this general framework highly improves the learning speed. We also provide a convergence proof.

cs.LG↗

arXiv2019

Active Transfer Learning for Persian Offline Signature Verification

Offline Signature Verification (OSV) remains a challenging pattern recognition task, especially in the presence of skilled forgeries that are not available during the training. This challenge is aggravated when there are small labeled training data available but with large intra-personal variations. In this study, we address this issue by employing an active learning approach, which selects the most informative instances to label and therefore reduces the human labeling effort significantly. Our proposed OSV includes three steps: feature learning, active learning, and final verification. We benefit from transfer learning using a pre-trained CNN for feature learning. We also propose SVM-based active learning for each user to separate his genuine signatures from the random forgeries. We finally used the SVMs to verify the authenticity of the questioned signature. We examined our proposed active transfer learning method on UTSig: A Persian offline signature dataset. We achieved near 13% improvement compared to the random selection of instances. Our results also showed 1% improvement over the state-of-the-art method in which a fully supervised setting with five more labeled instances per user was used.

cs.CV↗

arXiv2019

A Rate of Convergence for Two-Block Coordinate Descent

This short report gives a non-asymptotic rate of convergence proof for solving a two-block coordinate descent problem. This non-asymptotic proof is a simple result that can be derived easily from available results in the literature. We give the results in this report because in this general form we have not seen being stated in the literature.

math.OC↗

arXiv2018

Deep-RBF Networks Revisited: Robust Classification with Rejection

One of the main drawbacks of deep neural networks, like many other classifiers, is their vulnerability to adversarial attacks. An important reason for their vulnerability is assigning high confidence to regions with few or even no feature points. By feature points, we mean a nonlinear transformation of the input space extracting a meaningful representation of the input data. On the other hand, deep-RBF networks assign high confidence only to the regions containing enough feature points, but they have been discounted due to the widely-held belief that they have the vanishing gradient problem. In this paper, we revisit the deep-RBF networks by first giving a general formulation for them, and then proposing a family of cost functions thereof inspired by metric learning. In the proposed deep-RBF learning algorithm, the vanishing gradient problem does not occur. We make these networks robust to adversarial attack by adding the reject option to their output layer. Through several experiments on the MNIST dataset, we demonstrate that our proposed method not only achieves significant classification accuracy but is also very resistant to various adversarial attacks.

cs.LG↗

arXiv2018

A Recursive Least Square Method for 3D Pose Graph Optimization Problem

Pose Graph Optimization (PGO) is an important non-convex optimization problem and is the state-of-the-art formulation for SLAM in robotics. It also has applications like camera motion estimation, structure from motion and 3D reconstruction in machine vision. Recent researches have shown the importance of good initialization to bootstrap well-known iterative PGO solvers to converge to good solutions. The state-of-the-art initialization methods, however, works in low noise or eventually moderate noise problems, and they fail in challenging problems with high measurement noise. Consequently, iterative methods may get entangled in local minima in high noise scenarios. In this paper we present an initialization method which uses orientation measurements and then present a convergence analysis of our iterative algorithm. We show how the algorithm converges to global optima in noise-free cases and also obtain a bound for the difference between our result and the optimum solution in scenarios with noisy measurements. We then present our second algorithm that uses both relative orientation and position measurements to obtain a more accurate least squares approximation of the problem that is again solved iteratively. In the convergence proof, a structural coefficient arises that has important influence on the basin of convergence. Interestingly, simulation results show that this coefficient also affects the performance of other solvers and so it can indicate the complexity of the problem. Experimental results show the excellent performance of the proposed initialization algorithm, specially in high noise scenarios.

cs.RO↗

arXiv2017

Exploiting generalization in the subspaces for faster model-based learning

Due to the lack of enough generalization in the state-space, common methods in Reinforcement Learning (RL) suffer from slow learning speed especially in the early learning trials. This paper introduces a model-based method in discrete state-spaces for increasing learning speed in terms of required experience (but not required computational time) by exploiting generalization in the experiences of the subspaces. A subspace is formed by choosing a subset of features in the original state representation (full-space). Generalization and faster learning in a subspace are due to many-to-one mapping of experiences from the full-space to each state in the subspace. Nevertheless, due to inherent perceptual aliasing in the subspaces, the policy suggested by each subspace does not generally converge to the optimal policy. Our approach, called Model Based Learning with Subspaces (MoBLeS), calculates confidence intervals of the estimated Q-values in the full-space and in the subspaces. These confidence intervals are used in the decision making, such that the agent benefits the most from the possible generalization while avoiding from detriment of the perceptual aliasing in the subspaces. Convergence of MoBLeS to the optimal policy is theoretically investigated. Additionally, we show through several experiments that MoBLeS improves the learning speed in the early trials.

stat.ML↗

arXiv2017

An Alternative to EM for Gaussian Mixture Models: Batch and Stochastic Riemannian Optimization

We consider maximum likelihood estimation for Gaussian Mixture Models (Gmms). This task is almost invariably solved (in theory and practice) via the Expectation Maximization (EM) algorithm. EM owes its success to various factors, of which is its ability to fulfill positive definiteness constraints in closed form is of key importance. We propose an alternative to EM by appealing to the rich Riemannian geometry of positive definite matrices, using which we cast Gmm parameter estimation as a Riemannian optimization problem. Surprisingly, such an out-of-the-box Riemannian formulation completely fails and proves much inferior to EM. This motivates us to take a closer look at the problem geometry, and derive a better formulation that is much more amenable to Riemannian optimization. We then develop (Riemannian) batch and stochastic gradient algorithms that outperform EM, often substantially. We provide a non-asymptotic convergence analysis for our stochastic method, which is also the first (to our knowledge) such global analysis for Riemannian stochastic gradient. Numerous empirical results are included to demonstrate the effectiveness of our methods.

stat.ML↗

arXiv2016

Geometric Mean Metric Learning

We revisit the task of learning a Euclidean metric from data. We approach this problem from first principles and formulate it as a surprisingly simple optimization problem. Indeed, our formulation even admits a closed form solution. This solution possesses several very attractive properties: (i) an innate geometric appeal through the Riemannian geometry of positive definite matrices; (ii) ease of interpretability; and (iii) computational speed several orders of magnitude faster than the widely used LMNN and ITML methods. Furthermore, on standard benchmark datasets, our closed-form solution consistently attains higher classification accuracy.

stat.ML↗

arXiv2015

Inference and Mixture Modeling with the Elliptical Gamma Distribution

We study modeling and inference with the Elliptical Gamma Distribution (EGD). We consider maximum likelihood (ML) estimation for EGD scatter matrices, a task for which we develop new fixed-point algorithms. Our algorithms are efficient and converge to global optima despite nonconvexity. Moreover, they turn out to be much faster than both a well-known iterative algorithm of Kent & Tyler (1991) and sophisticated manifold optimization algorithms. Subsequently, we invoke our ML algorithms as subroutines for estimating parameters of a mixture of EGDs. We illustrate our methods by applying them to model natural image statistics---the proposed EGD mixture model yields the most parsimonious model among several competing approaches.

stat.CO↗