arXiv subjects

Tirza Routtenberg

Publications and source records attributed to Tirza Routtenberg.

At least 37 records · Page 2Linked to original sources

arXiv2023

GSP-Based MAP Estimation of Graph Signals

In this paper, we consider the problem of recovering random graph signals from nonlinear measurements. We formulate the maximum a-posteriori probability (MAP) estimator, which results in a nonconvex optimization problem. Conventional iterative methods for minimizing nonconvex problems are sensitive to the initialization, have high computational complexity, and do not utilize the underlying graph structure behind the data. In this paper we propose two new estimators that are both based on the Gauss-Newton method: 1) the elementwise graph-frequency-domain MAP (eGFD-MAP) estimator; and 2) the graph signal processing MAP (GSP-MAP) estimator. At each iteration, these estimators are updated by the outputs of two graph filters, with the previous state estimator and the residual as the input graph signals. The eGFD-MAP estimator is an ad-hoc method that minimizes the MAP objective function in the graph frequency domain and neglects mixed-derivatives of different graph frequencies in the Jacobian matrix as well as off-diagonal elements in the covariance matrices. Consequently, it updates the elements of the graph signal independently, which reduces the computational complexity compared to the conventional MAP estimator. The GSP-MAP estimator is based on optimizing the graph filters at each iteration of the Gauss-Newton algorithm. We state conditions under which the eGFD-MAP and GSP- MAP estimators coincide with the MAP estimator, in the case of an observation model with orthogonal graph frequencies. We evaluate the performance of the estimators for nonlinear graph signal recovery tasks with synthetic data and with the real-world problem of state estimation in power systems. These simulations show the advantages of the proposed estimators in terms of computational complexity, mean-squared-error, and robustness to the initialization of the iterative algorithms.

eess.SP↗

arXiv2023

GSP-KalmanNet: Tracking Graph Signals via Neural-Aided Kalman Filtering

Dynamic systems of graph signals are encountered in various applications, including social networks, power grids, and transportation. While such systems can often be described as state space (SS) models, tracking graph signals via conventional tools based on the Kalman filter (KF) and its variants is typically challenging. This is due to the nonlinearity, high dimensionality, irregularity of the domain, and complex modeling associated with real-world dynamic systems of graph signals. In this work, we study the tracking of graph signals using a hybrid model-based/data-driven approach. We develop the GSP-KalmanNet, which tracks the hidden graphical states from the graphical measurements by jointly leveraging graph signal processing (GSP) tools and deep learning (DL) techniques. The derivations of the GSP-KalmanNet are based on extending the KF to exploit the inherent graph structure via graph frequency domain filtering, which considerably simplifies the computational complexity entailed in processing high-dimensional signals and increases the robustness to small topology changes. Then, we use data to learn the Kalman gain following the recently proposed KalmanNet framework, which copes with partial and approximated modeling, without forcing a specific model over the noise statistics. Our empirical results demonstrate that the proposed GSP-KalmanNet achieves enhanced accuracy and run time performance as well as improved robustness to model misspecifications compared with both model-based and data-driven benchmarks.

eess.SP↗

arXiv2023

Barankin-Type Bound for Constrained Parameter Estimation

In constrained parameter estimation, the classical constrained Cramer-Rao bound (CCRB) and the recent Lehmann-unbiased CCRB (LU-CCRB) are lower bounds on the performance of mean-unbiased and Lehmann-unbiased estimators, respectively. Both the CCRB and the LU-CCRB require differentiability of the likelihood function, which can be a restrictive assumption. Additionally, these bounds are local bounds that are inappropriate for predicting the threshold phenomena of the constrained maximum likelihood (CML) estimator. The constrained Barankin-type bound (CBTB) is a nonlocal mean-squared-error (MSE) lower bound for constrained parameter estimation that does not require differentiability of the likelihood function. However, this bound requires a restrictive mean-unbiasedness condition in the constrained set. In this work, we propose the Lehmann-unbiased CBTB (LU-CBTB) on the weighted MSE (WMSE). This bound does not require differentiability of the likelihood function and assumes uniform Lehmann-unbiasedness, which is less restrictive than the CBTB uniform mean-unbiasedness. We show that the LU-CBTB is tighter than or equal to the LU-CCRB and coincides with the CBTB for linear constraints. For nonlinear constraints the LU-CBTB and the CBTB are different and the LU-CBTB can be a lower bound on the WMSE of constrained estimators in cases, where the CBTB is not. In the simulations, we consider direction-of-arrival estimation of an unknown constant modulus discrete signal. In this case, the likelihood function is not differentiable and constrained Cramer-Rao-type bounds do not exist, while CBTBs exist. It is shown that the LU-CBTB better predicts the CML estimator performance than the CBTB, since the CML estimator is Lehmann-unbiased but not mean-unbiased.

eess.SP↗

arXiv2023

Non-Bayesian Post-Model-Selection Estimation as Estimation Under Model Misspecification

In many parameter estimation problems, the exact model is unknown and is assumed to belong to a set of candidate models. In such cases, a predetermined data-based selection rule selects a parametric model from a set of candidates before the parameter estimation. The existing framework for estimation under model misspecification does not account for the selection process that led to the misspecified model. Moreover, in post-model-selection estimation, there are multiple candidate models chosen based on the observations, making the interpretation of the assumed model in the misspecified setting non-trivial. In this work, we present three interpretations to address the problem of non-Bayesian post-model-selection estimation as an estimation under model misspecification problem: the naive interpretation, the normalized interpretation, and the selective inference interpretation, and discuss their properties. For each of these interpretations, we developed the corresponding misspecified maximum likelihood estimator and the misspecified Cram$\acute{\text{e}}$r-Rao-type lower bound. The relations between the estimators and the performance bounds, as well as their properties, are discussed. Finally, we demonstrate the performance of the proposed estimators and bounds via simulations of estimation after channel selection. We show that the proposed performance bounds are more informative than the oracle Cram$\acute{\text{e}}$r-Rao Bound (CRB), where the third interpretation (selective inference) results in the lowest mean-squared-error (MSE) among the estimators.

eess.SP↗

arXiv2023

Discriminative and Generative Learning for Linear Estimation of Random Signals [Lecture Notes]

Inference tasks in signal processing are often characterized by the availability of reliable statistical modeling with some missing instance-specific parameters. One conventional approach uses data to estimate these missing parameters and then infers based on the estimated model. Alternatively, data can also be leveraged to directly learn the inference mapping end-to-end. These approaches for combining partially-known statistical models and data in inference are related to the notions of generative and discriminative models used in the machine learning literature, typically considered in the context of classifiers. The goal of this lecture note is to introduce the concepts of generative and discriminative learning for inference with a partially-known statistical model. While machine learning systems often lack the interpretability of traditional signal processing methods, we focus on a simple setting where one can interpret and compare the approaches in a tractable manner that is accessible and relevant to signal processing readers. In particular, we exemplify the approaches for the task of Bayesian signal estimation in a jointly Gaussian setting with the mean-squared error (MSE) objective, i.e., a linear estimation setting.

eess.SP↗

arXiv2023

Latent-KalmanNet: Learned Kalman Filtering for Tracking from High-Dimensional Signals

The Kalman filter (KF) is a widely-used algorithm for tracking dynamic systems that are captured by state space (SS) models. The need to fully describe a SS model limits its applicability under complex settings, e.g., when tracking based on visual data, and the processing of high-dimensional signals often induces notable latency. These challenges can be treated by mapping the measurements into latent features obeying some postulated closed-form SS model, and applying the KF in the latent space. However, the validity of this approximated SS model may constitute a limiting factor. In this work, we study tracking from high-dimensional measurements under complex settings using a hybrid model-based/data-driven approach. By gradually tackling the challenges in handling the observations model and the task, we develop Latent-KalmanNet, which implements tracking from high-dimensional measurements by leveraging data to jointly learn the KF along with the latent space mapping. Latent-KalmanNet combines a learned encoder with data-driven tracking in the latent space using the recently proposed-KalmanNet, while identifying the ability of each of these trainable modules to assist its counterpart via providing a suitable prior (by KalmanNet) and by learning a latent representation that facilitates data-aided tracking (by the encoder). Our empirical results demonstrate that the proposed Latent-KalmanNet achieves improved accuracy and run-time performance over both model-based and data-driven techniques by learning a surrogate latent representation that most facilitates tracking, while operating with limited complexity and latency.

eess.SP↗

arXiv2023

Widely-Linear MMSE Estimation of Complex-Valued Graph Signals

In this paper, we consider the problem of recovering random graph signals with complex values. For general Bayesian estimation of complex-valued vectors, it is known that the widely-linear minimum mean-squared-error (WLMMSE) estimator can achieve a lower mean-squared-error (MSE) than that of the linear minimum MSE (LMMSE) estimator. Inspired by the WLMMSE estimator, in this paper we develop the graph signal processing (GSP)-WLMMSE estimator, which minimizes the MSE among estimators that are represented as a two-channel output of a graph filter, i.e. widely-linear GSP estimators. We discuss the properties of the proposed GSP-WLMMSE estimator. In particular, we show that the MSE of the GSP-WLMMSE estimator is always equal to or lower than the MSE of the GSP-LMMSE estimator. The GSP-WLMMSE estimator is based on diagonal covariance matrices in the graph frequency domain, and thus has reduced complexity compared with the WLMMSE estimator. This property is especially important when using the sample-mean versions of these estimators that are based on a training dataset. We then state conditions under which the low-complexity GSP-WLMMSE estimator coincides with the WLMMSE estimator. In the simulations, we investigate two synthetic estimation problems (with linear and nonlinear models) and the problem of state estimation in power systems. For these problems, it is shown that the GSP-WLMMSE estimator outperforms the GSP-LMMSE estimator and achieves similar performance to that of the WLMMSE estimator.

eess.SP↗

arXiv2023

Quickest Inference of Susceptible-Infected Cascades in Sparse Networks

We consider the task of estimating a network cascade as fast as possible. The cascade is assumed to spread according to a general Susceptible-Infected process with heterogeneous transmission rates from an unknown source in the network. While the propagation is not directly observable, noisy information about its spread can be gathered through multiple rounds of error-prone diagnostic testing. We propose a novel adaptive procedure which quickly outputs an estimate for the cascade source and the full spread under this observation model. Remarkably, under mild conditions on the network topology, our procedure is able to estimate the full spread of the cascade in an $n$-vertex network, before $\mathrm{poly log}(n)$ vertices are affected by the cascade. We complement our theoretical analysis with simulation results illustrating the effectiveness of our methods.

math.ST↗

arXiv2023

DA-MUSIC: Data-Driven DoA Estimation via Deep Augmented MUSIC Algorithm

Direction of arrival (DoA) estimation of multiple signals is pivotal in sensor array signal processing. A popular multi-signal DoA estimation method is the multiple signal classification (MUSIC) algorithm, which enables high-performance super-resolution DoA recovery while being highly applicable in practice. MUSIC is a model-based algorithm, relying on an accurate mathematical description of the relationship between the signals and the measurements and assumptions on the signals themselves (non-coherent, narrowband sources). As such, it is sensitive to model imperfections. In this work we propose to overcome these limitations of MUSIC by augmenting the algorithm with specifically designed neural architectures. Our proposed deep augmented MUSIC (DA-MUSIC) algorithm is thus a hybrid model-based/data-driven DoA estimator, which leverages data to improve performance and robustness while preserving the interpretable flow of the classic method. DA-MUSIC is shown to learn to overcome limitations of the purely model-based method, such as its inability to successfully localize coherent sources as well as estimate the number of coherent signal sources present. We further demonstrate the superior resolution of the DA-MUSIC algorithm in synthetic narrowband and broadband scenarios as well as with real-world data of DoA estimation from seismic signals.

eess.SP↗

arXiv2022

Structural-constrained Methods for the Identification of Unobservable False Data Injection Attacks in Power Systems

Power system functionality is determined on the basis of the power system state estimation (PSSE). Thus, corruption of the PSSE may lead to severe consequences, such as financial losses, maintenance damage, and disruptions in electricity distribution. Classical bad data detection (BDD) methods, developed to ensure PSSE reliability, are unable to detect well-designed attacks, named unobservable false data injection (FDI) attacks. In this paper, we develop novel structural-constrained methods for the detection of unobservable FDI attacks, the identification of the attacked buses' locations, and PSSE under the presence of such attacks. The proposed methods are based on formulating structural, sparse constraints on both the attack and the system loads. First, we exploit these constraints in order to compose an appropriate model selection problem. Then, we develop the associated generalized information criterion (GIC) for this problem. However, for large networks, the GIC method's computational complexity grows exponentially with the network size. Thus, based on the proposed structural and sparse constraints, we develop two novel low-complexity methods for unobservable FDI attack identification: 1) a modification of the state-of-the-art orthogonal matching pursuit (OMP); and 2) a method that utilizes the graph Markovian property in power systems, i.e. the second-neighbor relationship between the power data at the system's buses. The methods' performance is evaluated on a IEEE-30 bus test case system.

eess.SP↗

arXiv2022

Bayesian Estimation of Graph Signals

We consider the problem of recovering random graph signals from nonlinear measurements. For this case, closed-form Bayesian estimators are usually intractable and even numerical evaluation of these estimators may be hard to compute for large networks. In this paper, we propose a graph signal processing (GSP) framework for random graph signal recovery that utilizes the information of the structure behind the data. First, we develop the GSP-linear minimum mean-squared-error (GSP-LMMSE) estimator, which minimizes the mean-squared error (MSE) among estimators that are represented as an output of a graph filter. The GSP-LMMSE estimator is based on diagonal covariance matrices in the graph frequency domain, and thus, has reduced complexity compared with the LMMSE estimator. This property is especially important when using the sample-mean versions of these estimators that are based on a training dataset. We then state conditions under which the low-complexity GSP-LMMSE estimator coincides with the optimal LMMSE estimator. Next, we develop the approximated parametrization of the GSP-LMMSE estimator by shift-invariant graph filters by solving a weighted least squared (WLS) problem. We present three implementations of the parametric GSP-LMMSE estimator for typical graph filters. Parametric graph filters are more robust to outliers and to network topology changes. In our simulations, we evaluate the performance of the proposed GSP-LMMSE estimators for the problem of state estimation in power systems, which can be interpreted as a graph signal recovery task. We show that the proposed sample-GSP estimators outperform the sample-LMMSE estimator for a limited training dataset and that the parametric GSP-LMMSE estimators are more robust to topology changes in the form of adding/removing vertices/edges.

eess.SP↗

arXiv2022

Non-Bayesian Parametric Missing-Mass Estimation

We consider the classical problem of missing-mass estimation, which deals with estimating the total probability of unseen elements in a sample. The missing-mass estimation problem has various applications in machine learning, statistics, language processing, ecology, sensor networks, and others. The naive, constrained maximum likelihood (CML) estimator is inappropriate for this problem since it tends to overestimate the probability of the observed elements. Similarly, the conventional constrained Cramer-Rao bound (CCRB), which is a lower bound on the mean-squared-error (MSE) of unbiased estimators, does not provide a relevant bound on the performance for this problem. In this paper, we introduce a frequentist, non-Bayesian parametric model of the problem of missing-mass estimation. We introduce the concept of missing-mass unbiasedness by using the Lehmann unbiasedness definition. We derive a non-Bayesian CCRB-type lower bound on the missing-mass MSE (mmMSE), named the missing-mass CCRB (mmCCRB), based on the missing-mass unbiasedness. The missing-mass unbiasedness and the proposed mmCCRB can be used to evaluate the performance of existing estimators. Based on the new mmCCRB, we propose a new method to improve existing estimators by an iterative missing-mass Fisher scoring method. Finally, we demonstrate via numerical simulations that the proposed mmCCRB is a valid and informative lower bound on the mmMSE of state-of-the-art estimators for this problem: the CML, the Good-Turing, and Laplace estimators. We also show that the performance of the Laplace estimator is improved by using the new Fisher-scoring method.

eess.SP↗

arXiv2022

State Estimation in Unobservable Power Systems via Graph Signal Processing Tools

We consider the problem of estimating the states in an unobservable power system. To this end, we propose novel graph signal processing (GSP) methods. For simplicity, we start with analyzing the DC power flow (DC-PF) model and then extend our algorithms to the AC power flow (AC-PF) model. The main assumption behind the proposed GSP approach is that the grid states, which include the vector of phases and the vector of the magnitudes of the voltages in the system, is a smooth graph signal with respect to the system admittance matrix that represents the underlying graph. Thus, the first step in this paper is to validate the graph-smoothness assumption of the states, both empirically and theoretically. Then, we develop the regularized GSP weighted least squares (GSP-WLS) state estimator, which does not require observability of the network. We propose a sensor placement strategy that aims to optimize the estimation performance of the GSP-WLS estimator. Finally, we extend the GSP-WLS estimator method to the AC-PF model by integrating a smoothness regularization term into the Gauss-Newton algorithm. Numerical results on the IEEE 118-bus system demonstrate that the new GSP methods outperform commonly-used estimation approaches and are robust to missing data.

eess.SP↗

arXiv2021

Identification of Edge Disconnections in Networks Based on Graph Filter Outputs

Graphs are fundamental mathematical structures used in various fields to model statistical and physical relationships between data, signals, and processes. In some applications, such as data processing in graphs that represent physical networks, the initial network topology is known. However, disconnections of edges in the network change the topology and may affect the signals and processes over the network. In this paper, we consider the problem of edge disconnection identification in networks by using concepts from graph signal processing (GSP). We assume that the graph signals measured over the vertices of the network can be represented as white noise that has been filtered on the graph topology by a smooth graph filter. We develop the likelihood ratio test (LRT) to detect a specific set of edge disconnections. Then, we provide the maximum likelihood (ML) decision rule for identifying general scenarios of edge disconnections in the network. It is shown that the sufficient statistics of the LRT and ML decision rule are the graph frequency energy levels in the graph spectral domain. However, the ML decision rule leads to a high-complexity exhaustive search over the edges in the network and is practically infeasible. Thus, we propose a low-complexity greedy method that identifies a single disconnected edge at each iteration. Moreover, by using the smoothness of the considered graph filter, we suggest a local implementation of the decision rule, which is based solely on the measurements at neighboring vertices. Simulation results demonstrate that the proposed methods outperform existing detection and identification methods on a synthetic dataset and for line outage identification in power systems from the IEEE 118-bus test case.

eess.SP↗

arXiv2021

Cramer-Rao Bound for Estimation After Model Selection and its Application to Sparse Vector Estimation

In many practical parameter estimation problems, such as coefficient estimation of polynomial regression, the true model is unknown and thus, a model selection step is performed prior to estimation. The data-based model selection step affects the subsequent estimation. In particular, the oracle Cramér-Rao bound (CRB), which is based on knowledge of the true model, is inappropriate for post-model-selection performance analysis and system design outside the asymptotic region. In this paper, we investigate post-model-selection parameter estimation of a vector with an unknown support set, where this support set represents the model. We analyze the estimation performance of coherent estimators that force unselected parameters to zero. We use the mean-squared-selected-error (MSSE) criterion and introduce the concept of selective unbiasedness in the sense of Lehmann unbiasedness. We derive a non-Bayesian Cramér-Raotype bound on the MSSE and on the mean-squared-error (MSE) of any coherent estimator with a specific selective-bias function in the Lehmann sense. We implement the selective CRB for the special case of sparse vector estimation with an unknown support set. Finally, we demonstrate in simulations that the proposed selective CRB is an informative lower bound on the performance of the maximum selected likelihood estimator for a general linear model with the generalized information criterion and for sparse vector estimation with one step thresholding. It is shown that for these cases the selective CRB outperforms the oracle CRB and Sando-Mitra-Stoica CRB (SMS-CRB) [1].

eess.SP↗

arXiv2021

Non-Bayesian Estimation Framework for Signal Recovery on Graphs

Graph signals arise from physical networks, such as power and communication systems, or as a result of a convenient representation of data with complex structure, such as social networks. We consider the problem of general graph signal recovery from noisy, corrupted, or incomplete measurements and under structural parametric constraints, such as smoothness in the graph frequency domain. In this paper, we formulate the graph signal recovery as a non-Bayesian estimation problem under a weighted mean-squared-error (WMSE) criterion, which is based on a quadratic form of the Laplacian matrix of the graph and its trace WMSE is the Dirichlet energy of the estimation error w.r.t. the graph. The Laplacian-based WMSE penalizes estimation errors according to their graph spectral content and is a difference-based cost function which accounts for the fact that in many cases signal recovery on graphs can only be achieved up to a constant addend. We develop a new Cramér-Rao bound (CRB) on the Laplacian-based WMSE and present the associated Lehmann unbiasedness condition w.r.t. the graph. We discuss the graph CRB and estimation methods for the fundamental problems of 1) A linear Gaussian model with relative measurements; and 2) Bandlimited graph signal recovery. We develop sampling allocation policies that optimize sensor locations in a network for these problems based on the proposed graph CRB. Numerical simulations on random graphs and on electrical network data are used to validate the performance of the graph CRB and sampling policies.

eess.SP↗

arXiv2020

Resource Allocation and Dithering of Bayesian Parameter Estimation Using Mixed-Resolution Data

Quantization of signals is an integral part of modern signal processing applications, such as sensing, communication, and inference. While signal quantization provides many physical advantages, it usually degrades the subsequent estimation performance that is based on quantized data. In order to maintain physical constraints and simultaneously bring substantial performance gain, in this work we consider systems with mixed-resolution, 1-bit quantized and continuous-valued, data. First, we describe the linear minimum mean-squared error (LMMSE) estimator and its associated mean-squared error (MSE) for the general mixed-resolution model. However, the MSE of the LMMSE requires matrix inversion in which the number of measurements defines the matrix dimensions and thus, is not a tractable tool for optimization and system design. Therefore, we present the linear Gaussian orthonormal (LGO) measurement model and derive a closed-form analytic expression for the MSE of the LMMSE estimator under this model. In addition, we present two common special cases of the LGO model: 1) scalar parameter estimation and 2) channel estimation in mixed-ADC multiple-input multiple-output (MIMO) communication systems. We then solve the resource allocation optimization problem of the LGO model with the proposed tractable form of the MSE as an objective function and under a power constraint using a one-dimensional search. Moreover, we present the concept of dithering for mixed-resolution models and optimize the dithering noise as part of the resource allocation optimization problem for two dithering schemes: 1) adding noise only to the quantized measurements and 2) adding noise to both measurement types. Finally, we present simulations that demonstrate the advantages of using mixed-resolution measurements and the possible improvement introduced with dithering and resource allocation.

eess.SP↗

arXiv2020

Low-Complexity Detection of Small Frequency Changes by the Generalized LMPU Test

In this paper, we consider the detection of a small change in the frequency of sinusoidal signals, which arises in various signal processing applications. The generalized likelihood ratio test (GLRT) for this problem uses the maximum likelihood (ML) estimator of the frequency, and therefore suffers from high computational complexity. In addition, the GLRT is not necessarily optimal and its performance may degrade for non-asymptotic scenarios that are characterized by close hypotheses and small sample sizes. In this paper we propose a new detection method, named the generalized locally most powerful unbiased (GLMPU) test, which is a general method for local detection in the presence of nuisance parameters. A closed-form expression of the GLMPU test is developed for the detection of frequency deviation in the case where the complex amplitudes of the measured signals are unknown. Numerical simulations show improved performance over the GLRT in terms of probability of detection performance and computational complexity.

eess.SP↗