SearcharxivSearch

arXiv subjects

Frank de Hoog

Publications and source records attributed to Frank de Hoog.

18 recordsLinked to original sources

Determinant-Based Error Bounds for CUR Matrix Approximation: Oversampling and Volume Sampling

We derive error bounds for CUR matrix approximation using determinant-based methods that relate local projection errors to global approximation quality. For general matrices, we establish determinant identities for bordered Gramian matrices that decompose CUR approximation errors into interpretable local components. These identities connect projection errors onto submatrix column spaces directly to determinants, providing geometric insight into approximation degradation. We develop a probabilistic framework based on volume sampling that yields interpolation-type error bounds quantifying the benefits of oversampling: when $r > k$ rows are selected for $k$ columns, the expected error factor transitions linearly from $(k+1)^2$ (no oversampling) to $(k+1)$ (full oversampling). Our analysis establishes that the expected squared error is bounded by this interpolation factor times the squared error of the best rank-$k$ approximation, directly connecting CUR approximation quality to the optimal low-rank approximation. The framework applies to both CUR decomposition for general matrices and the Nyström method for symmetric positive semi-definite matrices, providing a unified theoretical foundation for determinant-based low-rank approximation analysis.

math.NA

Lighter-X: An Efficient and Plug-and-play Strategy for Graph-based Recommendation through Decoupled Propagation

Graph Neural Networks (GNNs) have demonstrated remarkable effectiveness in recommendation systems. However, conventional graph-based recommenders, such as LightGCN, require maintaining embeddings of size $d$ for each node, resulting in a parameter complexity of $\mathcal{O}(n \times d)$, where $n$ represents the total number of users and items. This scaling pattern poses significant challenges for deployment on large-scale graphs encountered in real-world applications. To address this scalability limitation, we propose \textbf{Lighter-X}, an efficient and modular framework that can be seamlessly integrated with existing GNN-based recommender architectures. Our approach substantially reduces both parameter size and computational complexity while preserving the theoretical guarantees and empirical performance of the base models, thereby enabling practical deployment at scale. Specifically, we analyze the original structure and inherent redundancy in their parameters, identifying opportunities for optimization. Based on this insight, we propose an efficient compression scheme for the sparse adjacency structure and high-dimensional embedding matrices, achieving a parameter complexity of $\mathcal{O}(h \times d)$, where $h \ll n$. Furthermore, the model is optimized through a decoupled framework, reducing computational complexity during the training process and enhancing scalability. Extensive experiments demonstrate that Lighter-X achieves comparable performance to baseline models with significantly fewer parameters. In particular, on large-scale interaction graphs with millions of edges, we are able to attain even better results with only 1\% of the parameter over LightGCN.

cs.LG

Understanding the Effects of Projectors in Knowledge Distillation

Conventionally, during the knowledge distillation process (e.g. feature distillation), an additional projector is often required to perform feature transformation due to the dimension mismatch between the teacher and the student networks. Interestingly, we discovered that even if the student and the teacher have the same feature dimensions, adding a projector still helps to improve the distillation performance. In addition, projectors even improve logit distillation if we add them to the architecture too. Inspired by these surprising findings and the general lack of understanding of the projectors in the knowledge distillation process from existing literature, this paper investigates the implicit role that projectors play but so far have been overlooked. Our empirical study shows that the student with a projector (1) obtains a better trade-off between the training accuracy and the testing accuracy compared to the student without a projector when it has the same feature dimensions as the teacher, (2) better preserves its similarity to the teacher beyond shallow and numeric resemblance, from the view of Centered Kernel Alignment (CKA), and (3) avoids being over-confident as the teacher does at the testing phase. Motivated by the positive effects of projectors, we propose a projector ensemble-based feature distillation method to further improve distillation performance. Despite the simplicity of the proposed strategy, empirical results from the evaluation of classification tasks on benchmark datasets demonstrate the superior classification performance of our method on a broad range of teacher-student pairs and verify from the aspects of CKA and model calibration that the student's features are of improved quality with the projector ensemble design.

cs.CV

Improved Feature Distillation via Projector Ensemble

In knowledge distillation, previous feature distillation methods mainly focus on the design of loss functions and the selection of the distilled layers, while the effect of the feature projector between the student and the teacher remains under-explored. In this paper, we first discuss a plausible mechanism of the projector with empirical evidence and then propose a new feature distillation method based on a projector ensemble for further performance improvement. We observe that the student network benefits from a projector even if the feature dimensions of the student and the teacher are the same. Training a student backbone without a projector can be considered as a multi-task learning process, namely achieving discriminative feature extraction for classification and feature matching between the student and the teacher for distillation at the same time. We hypothesize and empirically verify that without a projector, the student network tends to overfit the teacher's feature distributions despite having different architecture and weights initialization. This leads to degradation on the quality of the student's deep features that are eventually used in classification. Adding a projector, on the other hand, disentangles the two learning tasks and helps the student network to focus better on the main feature extraction task while still being able to utilize teacher features as a guidance through the projector. Motivated by the positive effect of the projector in feature distillation, we propose an ensemble of projectors to further improve the quality of student features. Experimental results on different datasets with a series of teacher-student pairs illustrate the effectiveness of the proposed method.

cs.CV

A Note on Error Bounds for Pseudo Skeleton Approximations of Matrices

Due to their importance in both data analysis and numerical algorithms, low rank approximations have recently been widely studied. They enable the handling of very large matrices. Tight error bounds for the computationally efficient Gaussian elimination based methods (skeleton approximations) are available. In practice, these bounds are useful for matrices with singular values which decrease quickly. Using the Chebyshev norm, this paper provides improved bounds for the errors of the matrix elements. These bounds are substantially better in the practically relevant cases where the eigenvalues decrease polynomially. Results are proven for general real rectangular matrices. Even stronger bounds are obtained for symmetric positive definite matrices. A simple example is given, comparing these new bounds to earlier ones.

math.NA

Vaccination strategies on dynamic networks with indirect transmission links and limited contact information

Infectious diseases are still a major global burden for modern society causing 13 million deaths annually. One way to reduce the morbidity and mortality rates from infectious diseases is through preventative or targeted vaccinations. Current vaccination strategies, however, rely on the highly specific individual contact information that is difficult and costly to obtain, in order to identify influential spreading individuals. Current approaches also focus only on direct contacts between individuals for spreading, and disregard indirect transmission where a pathogen can spread between one infected individual and one susceptible individual that visit the same location within a short time-frame without meeting. This paper presents a novel vaccination strategy that relies on coarse-grained contact information, both direct and indirect, that can be easily and efficiently collected. Rather than tracking exact contact degrees of individuals, our strategy uses the types of places people visit to estimate a range of contact degrees for individuals, considering both direct and indirect contacts. We conduct extensive simulations to evaluate the performance of our strategy in comparison to the state of the art's vaccination strategies. Results show that our strategy achieves comparable performance to the oracle approach and outperforms all existing strategies when considering indirect links.

cs.SI

Indirect interactions influence contact network structure and diffusion dynamics

Interaction patterns at the individual level influence the behaviour of diffusion over contact networks. Most of the current diffusion models only consider direct interactions among individuals to build underlying infectious items transmission networks. However, delayed indirect interactions, where a susceptible individual interacts with infectious items after the infected individual has left the interaction space, can also cause transmission events. We define a diffusion model called the same place different time transmission (SPDT) based diffusion that considers transmission links for these indirect interactions. Our SPDT model changes the network dynamics where the connectivity among individuals varies with the decay rates of link infectivity. We investigate SPDT diffusion behaviours by simulating airborne disease spreading on data-driven contact networks. The SPDT model significantly increases diffusion dynamics (particularly for networks with low link densities where indirect interactions create new infection pathways) and is capable of producing realistic disease reproduction number. Our results show that the SPDT model is significantly more likely to lead to outbreaks compared to current diffusion models with direct interactions. We find that the diffusion dynamics with including indirect links are not reproducible by the current models, highlighting the importance of the indirect links for predicting outbreaks.

cs.SI

A Graph Model with Indirect Co-location Links

Graph models are widely used to analyse diffusion processes embedded in social contacts and to develop applications. A range of graph models are available to replicate the underlying social structures and dynamics realistically. However, most of the current graph models can only consider concurrent interactions among individuals in the co-located interaction networks. However, they do not account for indirect interactions that can transmit spreading items to individuals who visit the same locations at different times but within a certain time limit. The diffusion phenomena occurring through direct and indirect interactions is called same place different time (SPDT) diffusion. This paper introduces a model to synthesize co-located interaction graphs capturing both direct interactions, where individuals meet at a location, and indirect interactions, where individuals visit the same location at different times within a set timeframe. We analyze 60 million location updates made by 2 million users from a social networking application to characterize the graph properties, including the space-time correlations and its time evolving characteristics, such as bursty or ongoing behaviors. The generated synthetic graph reproduces diffusion dynamics of a realistic contact graph, and reduces the prediction error by up to 82% when compare to other contact graph models demonstrating its potential for forecasting epidemic spread.

cs.SI

Hyperspectral Image Recovery via Hybrid Regularization

Natural images tend to mostly consist of smooth regions with individual pixels having highly correlated spectra. This information can be exploited to recover hyperspectral images of natural scenes from their incomplete and noisy measurements. To perform the recovery while taking full advantage of the prior knowledge, we formulate a composite cost function containing a square-error data-fitting term and two distinct regularization terms pertaining to spatial and spectral domains. The regularization for the spatial domain is the sum of total-variation of the image frames corresponding to all spectral bands. The regularization for the spectral domain is the l1-norm of the coefficient matrix obtained by applying a suitable sparsifying transform to the spectra of the pixels. We use an accelerated proximal-subgradient method to minimize the formulated cost function. We analyze the performance of the proposed algorithm and prove its convergence. Numerical simulations using real hyperspectral images exhibit that the proposed algorithm offers an excellent recovery performance with a number of measurements that is only a small fraction of the hyperspectral image data size. Simulation results also show that the proposed algorithm significantly outperforms an accelerated proximal-gradient algorithm that solves the classical basis-pursuit denoising problem to recover the hyperspectral image.

cs.CV

On spatial resolution, signal-to-noise and information capacity of linear imaging systems

A simple model for image formation in linear shift-invariant systems is considered, in which both the detected signal and the noise variance are varying slowly compared to the point-spread function of the system. It is shown that within the constraints of this model, the square of the signal-to-noise ratio is always proportional to the "volume" of the spatial resolution unit. In the case of Poisson statistics, the ratio of these two quantities divided by the incident density of the imaging particles (e.g. photons) represents a dimensionless invariant of the imaging system, which was previously termed the intrinsic imaging quality. The relationship of this invariant to the notion of information capacity of communication and imaging systems, which was previously considered by Shannon, Gabor and others, is investigated. The results are then applied to a simple generic model of quantitative imaging of weakly scattering objects, leading to an estimate of the upper limit for the amount of information about the sample that can be obtained in such experiments. It is shown that this limit depends only on the total number of imaging particles incident on the sample, the average scattering coefficient, the size of the sample and the number of spatial resolution units.

physics.med-ph

Compressive hyperspectral imaging via adaptive sampling and dictionary learning

In this paper, we propose a new sampling strategy for hyperspectral signals that is based on dictionary learning and singular value decomposition (SVD). Specifically, we first learn a sparsifying dictionary from training spectral data using dictionary learning. We then perform an SVD on the dictionary and use the first few left singular vectors as the rows of the measurement matrix to obtain the compressive measurements for reconstruction. The proposed method provides significant improvement over the conventional compressive sensing approaches. The reconstruction performance is further improved by reconditioning the sensing matrix using matrix balancing. We also demonstrate that the combination of dictionary learning and SVD is robust by applying them to different datasets.

cs.CV

Orthogonal Matching Pursuit with Thresholding and its Application in Compressive Sensing

Greed is good. However, the tighter you squeeze, the less you have. In this paper, a less greedy algorithm for sparse signal reconstruction in compressive sensing, named orthogonal matching pursuit with thresholding is studied. Using the global 2-coherence , which provides a "bridge" between the well known mutual coherence and the restricted isometry constant, the performance of orthogonal matching pursuit with thresholding is analyzed and more general results for sparse signal reconstruction are obtained. It is also shown that given the same assumption on the coherence index and the restricted isometry constant as required for orthogonal matching pursuit, the thresholding variation gives exactly the same reconstruction performance with significantly less complexity.

cs.IT

Evaluating the Performance of BSBL Methodology for EEG Source Localization On a Realistic Head Model

Source localization in EEG represents a high dimensional inverse problem, which is severely ill-posed by nature. Fortunately, sparsity constraints have come into rescue as it helps solving the ill-posed problems when the signal is sparse. When the signal has a structure such as block structure, consideration of block sparsity produces better results. Knowing sparse Bayesian learning is an important member in the family of sparse recovery, and a superior choice when the projection matrix is highly coherent (which is typical the case for EEG), in this work we evaluate the performance of block sparse Bayesian learning (BSBL) method for EEG source localization. It is already accepted by the EEG community that a group of dipoles rather than a single dipole are activated during brain activities; thus, block structure is a reasonable choice for EEG. In this work we use two definitions of blocks: Brodmann areas and automated anatomical labelling (AAL), and analyze the reconstruction performance of BSBL methodology for them. A realistic head model is used for the experiment, which was obtained from segmentation of MRI images. When the number of simultaneously active blocks is 2, the BSBL produces overall localization accuracy of less than 5 mm without the presence of noise. The presence of more than 3 simultaneously active blocks and noise significantly affect the localization performance. Consideration of AAL based blocks results more accurate source localization in comparison to Brodmann area based blocks.

physics.med-ph

Sparse Bayesian Learning for EEG Source Localization

Purpose: Localizing the sources of electrical activity from electroencephalographic (EEG) data has gained considerable attention over the last few years. In this paper, we propose an innovative source localization method for EEG, based on Sparse Bayesian Learning (SBL). Methods: To better specify the sparsity profile and to ensure efficient source localization, the proposed approach considers grouping of the electrical current dipoles inside human brain. SBL is used to solve the localization problem in addition with imposed constraint that the electric current dipoles associated with the brain activity are isotropic. Results: Numerical experiments are conducted on a realistic head model that is obtained by segmentation of MRI images of the head and includes four major components, namely the scalp, the skull, the cerebrospinal fluid (CSF) and the brain, with appropriate relative conductivity values. The results demonstrate that the isotropy constraint significantly improves the performance of SBL. In a noiseless environment, the proposed method was 1 found to accurately (with accuracy of >75%) locate up to 6 simultaneously active sources, whereas for SBL without the isotropy constraint, the accuracy of finding just 3 simultaneously active sources was <75%. Conclusions: Compared to the state-of-the-art algorithms, the proposed method is potentially more consistent in specifying the sparsity profile of human brain activity and is able to produce better source localization for EEG.

q-bio.QM

EEG source localization using a sparsity prior based on Brodmann areas

Localizing the sources of electrical activity in the brain from Electroencephalographic (EEG) data is an important tool for non-invasive study of brain dynamics. Generally, the source localization process involves a high-dimensional inverse problem that has an infinite number of solutions and thus requires additional constraints to be considered to have a unique solution. In the context of EEG source localization, we propose a novel approach that is based on dividing the cerebral cortex of the brain into a finite number of Functional Zones which correspond to unitary functional areas in the brain. In this paper we investigate the use of Brodmanns areas as the Functional Zones. This approach allows us to apply a sparsity constraint to find a unique solution for the inverse EEG problem. Compared to previously published algorithms which use different sparsity constraints to solve this problem, the proposed method is potentially more consistent with the known sparsity profile of the human brain activity and thus may be able to ensure better localization. Numerical experiments are conducted on a realistic head model obtained from segmentation of MRI images of the head and includes four major compartments namely scalp, skull, cerebrospinal fluid (CSF) and brain with relative conductivity values. Three different electrode setups are tested in the numerical experiments.

q-bio.QM

New Coherence and RIP Analysis for Weak Orthogonal Matching Pursuit

In this paper we define a new coherence index, named the global 2-coherence, of a given dictionary and study its relationship with the traditional mutual coherence and the restricted isometry constant. By exploring this relationship, we obtain more general results on sparse signal reconstruction using greedy algorithms in the compressive sensing (CS) framework. In particular, we obtain an improved bound over the best known results on the restricted isometry constant for successful recovery of sparse signals using orthogonal matching pursuit (OMP).

cs.IT

The application of compressive sampling to radio astronomy I: Deconvolution

Compressive sampling is a new paradigm for sampling, based on sparseness of signals or signal representations. It is much less restrictive than Nyquist-Shannon sampling theory and thus explains and systematises the widespread experience that methods such as the Högbom CLEAN can violate the Nyquist-Shannon sampling requirements. In this paper, a CS-based deconvolution method for extended sources is introduced. This method can reconstruct both point sources and extended sources (using the isotropic undecimated wavelet transform as a basis function for the reconstruction step). We compare this CS-based deconvolution method with two CLEAN-based deconvolution methods: the Högbom CLEAN and the multiscale CLEAN. This new method shows the best performance in deconvolving extended sources for both uniform and natural weighting of the sampled visibilities. Both visual and numerical results of the comparison are provided.

astro-ph.IM

The application of compressive sampling to radio astronomy II: Faraday rotation measure synthesis

Faraday rotation measure (RM) synthesis is an important tool to study and analyze galactic and extra-galactic magnetic fields. Since there is a Fourier relation between the Faraday dispersion function and the polarized radio emission, full reconstruction of the dispersion function requires knowledge of the polarized radio emission at both positive and negative square wavelengths $λ^2$. However, one can only make observations for $λ^2 > 0$. Furthermore observations are possible only for a limited range of wavelengths. Thus reconstructing the Faraday dispersion function from these limited measurements is ill-conditioned. In this paper, we propose three new reconstruction algorithms for RM synthesis based upon compressive sensing/sampling (CS). These algorithms are designed to be appropriate for Faraday thin sources only, thick sources only, and mixed sources respectively. Both visual and numerical results show that the new RM synthesis methods provide superior reconstructions of both magnitude and phase information than RM-CLEAN

astro-ph.IM