arXiv subjects

Kamal Youcef-Toumi

Publications and source records attributed to Kamal Youcef-Toumi.

27 records · Page 2Linked to original sources

arXiv2023

GSR: A Generalized Symbolic Regression Approach

Identifying the mathematical relationships that best describe a dataset remains a very challenging problem in machine learning, and is known as Symbolic Regression (SR). In contrast to neural networks which are often treated as black boxes, SR attempts to gain insight into the underlying relationships between the independent variables and the target variable of a given dataset by assembling analytical functions. In this paper, we present GSR, a Generalized Symbolic Regression approach, by modifying the conventional SR optimization problem formulation, while keeping the main SR objective intact. In GSR, we infer mathematical relationships between the independent variables and some transformation of the target variable. We constrain our search space to a weighted sum of basis functions, and propose a genetic programming approach with a matrix-based encoding scheme. We show that our GSR method is competitive with strong SR benchmark methods, achieving promising experimental performance on the well-known SR benchmark problem sets. Finally, we highlight the strengths of GSR by introducing SymSet, a new SR benchmark set which is more challenging relative to the existing benchmarks.

cs.LG↗

arXiv2022

D-DARTS: Distributed Differentiable Architecture Search

Differentiable ARchiTecture Search (DARTS) is one of the most trending Neural Architecture Search (NAS) methods. It drastically reduces search cost by resorting to weight-sharing. However, it also dramatically reduces the search space, thus excluding potential promising architectures. In this article, we propose D-DARTS, a solution that addresses this problem by nesting neural networks at the cell level instead of using weight-sharing to produce more diversified and specialized architectures. Moreover, we introduce a novel algorithm that can derive deeper architectures from a few trained cells, increasing performance and saving computation time. In addition, we also present an alternative search space (DARTOpti) in which we optimize existing handcrafted architectures (e.g., ResNet) rather than starting from scratch. This approach is accompanied by a novel metric that measures the distance between architectures inside our custom search space. Our solution reaches competitive performance on multiple computer vision tasks. Code and pretrained models can be accessed at https://github.com/aheuillet/D-DARTS.

cs.LG↗

arXiv2022

Reliable Neural Networks for Regression Uncertainty Estimation

While deep neural networks are highly performant and successful in a wide range of real-world problems, estimating their predictive uncertainty remains a challenging task. To address this challenge, we propose and implement a loss function for regression uncertainty estimation based on the Bayesian Validation Metric (BVM) framework while using ensemble learning. The proposed loss reproduces maximum likelihood estimation in the limiting case. A series of experiments on in-distribution data show that the proposed method is competitive with existing state-of-the-art methods. Experiments on out-of-distribution data show that the proposed method is robust to statistical change and exhibits superior predictive capability.

cs.LG↗

arXiv2021

Instance Semantic Segmentation Benefits from Generative Adversarial Networks

In design of instance segmentation networks that reconstruct masks, segmentation is often taken as its literal definition -- assigning each pixel a label. This has led to thinking the problem as a template matching one with the goal of minimizing the loss between the reconstructed and the ground truth pixels. Rethinking reconstruction networks as a generator, we define the problem of predicting masks as a GANs game framework: A segmentation network generates the masks, and a discriminator network decides on the quality of the masks. To demonstrate this game, we show effective modifications on the general segmentation framework in Mask R-CNN. We find that playing the game in feature space is more effective than the pixel space leading to stable training between the discriminator and the generator, predicting object coordinates should be replaced by predicting contextual regions for objects, and overall the adversarial loss helps the performance and removes the need for any custom settings per different data domain. We test our framework in various domains and report on cellphone recycling, autonomous driving, large-scale object detection, and medical glands. We observe in general GANs yield masks that account for crispier boundaries, clutter, small objects, and details, being in domain of regular shapes or heterogeneous and coalescing shapes. Our code for reproducing the results is available publicly.

cs.CV↗

arXiv2020

A Generalized Bayesian Approach to Model Calibration

In model development, model calibration and validation play complementary roles toward learning reliable models. In this article, we expand the Bayesian Validation Metric framework to a general calibration and validation framework by inverting the validation mathematics into a generalized Bayesian method for model calibration and regression. We perform Bayesian regression based on a user's definition of model-data agreement. This allows for model selection on any type of data distribution, unlike Bayesian and standard regression techniques, that "fail" in some cases. We show that our tool is capable of representing and combining least squares, likelihood-based, and Bayesian calibration techniques in a single framework while being able to generalize aspects of these methods. This tool also offers new insights into the interpretation of the predictive envelopes (also known as confidence bands) while giving the analyst more control over these envelopes. We demonstrate the validity of our method by providing three numerical examples to calibrate different models, including a model for energy dissipation in lap joints under impact loading. By calibrating models with respect to the validation metrics one desires a model to ultimately pass, reliability and safety metrics may be integrated into and automatically adopted by the model in the calibration phase.

stat.ME↗

arXiv2020

Why Simple Quadrature is just as good as Monte Carlo

We motive and calculate Newton--Cotes quadrature integration variance and compare it directly with Monte Carlo (MC) integration variance. We find an equivalence between deterministic quadrature sampling and random MC sampling by noting that MC random sampling is statistically indistinguishable from a method that uses deterministic sampling on a randomly shuffled (permuted) function. We use this statistical equivalence to regularize the form of permissible Bayesian quadrature integration priors such that they are guaranteed to be objectively comparable with MC. This leads to the proof that simple quadrature methods have expected variances that are less than or equal to their corresponding theoretical MC integration variances. Separately, using Bayesian probability theory, we find that the theoretical standard deviations of the unbiased errors of simple Newton--Cotes composite quadrature integrations improve over their worst case errors by an extra dimension independent factor $\propto N^{-1/2}$. This dimension independent factor is validated in our simulations.

math.ST↗

arXiv2019

A General Model Validation and Testing Tool

We construct and propose the "Bayesian Validation Metric" (BVM) as a general model validation and testing tool. We find the BVM to be capable of representing all of the standard validation metrics (square error, reliability, probability of agreement, frequentist, area, probability density comparison, statistical hypothesis testing, and Bayesian model testing) as special cases and find that it can be used to improve, generalize, or further quantify their uncertainties. Thus, the BVM allows us to assess the similarities and differences between existing validation metrics in a new light. The BVM has the capacity to allow users to invent and select models according to novel validation requirements. We formulate and test a few novel compound validation metrics that improve upon other validation metrics in the literature. Further, we construct the BVM Ratio for the purpose of quantifying model selection under user defined definitions of agreement in the presence or absence of uncertainty. This construction generalizes the Bayesian model testing framework.

stat.ME↗

arXiv2019

Vectorized Uncertainty Propagation and Input Probability Sensitivity Analysis

In this article we construct a theoretical and computational process for assessing Input Probability Sensitivity Analysis (IPSA) using a Graphics Processing Unit (GPU) enabled technique called Vectorized Uncertainty Propagation (VUP). VUP propagates probability distributions through a parametric computational model in a way that's computational time complexity grows sublinearly in the number of distinct propagated input probability distributions. VUP can therefore be used to efficiently implement IPSA, which estimates a model's probabilistic sensitivity to measurement and parametric uncertainty over each relevant measurement location. Theory and simulation illustrate the effectiveness of these methods.

stat.CO↗

arXiv2018

Finger Grip Force Estimation from Video using Two Stream Approach

Estimation of a hand grip force is essential for the understanding of force pattern during the execution of assembly or disassembly operations. Human demonstration of a correct way of doing an operation is a powerful source of information which can be used for guided robot teaching. Typically to assess this problem instrumented approach is used, which requires hand or object mounted devices and poses an inconvenience for an operator or limits the scope of addressable objects. The work demonstrates that contact force may be estimated using a noninvasive contactless method with the help of vision system alone. We propose a two-stream approach for video processing, which utilizes both spatial information of each frame and dynamic information of frame change. In this work, image processing and machine learning techniques are used along with dense optical flow for frame change tracking and Kalman filter is used for stream fusion. Our studies show that the proposed method can successfully estimate contact grip force with RMSE < 10% of sensor range (RMSE $\approx 0.2$ N), the performances of each stream and overall method performance are reported. The proposed method has a wide range of applications, including robot teaching through demonstration, haptic force feedback, and validation of human- performed operations.

cs.RO↗