Searcharxiv⌕ Search

arXiv subjects

Michał P. Karpowicz

Publications and source records attributed to Michał P. Karpowicz.

5 recordsLinked to original sources

On the Fundamental Impossibility of Hallucination Control in Large Language Models

Large language models hallucinate. This paper shows when that is unavoidable and what we can do about it. We model inference as an auction of ideas, in which a model's components, each holding partial knowledge, compete to shape the answer. We then prove Impossibility Theorems showing that whenever a query makes LLM components contest a fact they hold in common, no aggregation of their reports can at once report that knowledge truthfully, avoid manufacturing confidence beyond what it supports, keep the relevant components engaged, and give the best answer. Something must give, and each failure is familiar: a fabricated detail, unearned confidence, ignored knowledge, or a needlessly weak reply. This is no artifact of one design. It reappears when components report probabilities, and inside the transformer itself, where the combined answer is credited more confidence than the internal contributions supplied. The unbalanced semantic budget cannot be settled from within. Factual truth lies outside the model, and in the worst case no internal signal can certify it. What can be certified is support. Given externally authorized evidence, checking that an answer stays within what the evidence entails needs only the answer and the evidence, and we prove when that check is computable. However, a correct answer can lack support, and a supported answer can be false. What counts as evidence, how far beyond it we allow answers to reach, and which failures we can live with are choices no model can make for us.

stat.ML↗

Generalized Inverses of Matrix Products: From Fundamental Subspaces to Randomized Decompositions

We investigate the Moore-Penrose pseudoinverse and generalized inverse of a matrix product $A=CR$ to establish a unifying framework for generalized and randomized matrix inverses. This analysis is rooted in first principles, focusing on the geometry of the four fundamental subspaces. We examine: (1) the reverse order law, $A^+ = R^+C^+$, which holds when $C$ has independent columns and $R$ has independent rows, (2) the universally correct formula, $A^+ = (C^+CR)^+(CRR^+)^+$, providing a geometric interpretation of the mappings between the involved subspaces, (3) a new generalized randomized formula, $A^+_p = (P^TA)^+P^TAQ(AQ)^+$, which gives $A^+_p = A^+$ if and only if the sketching matrices $P$ and $Q$ preserve the rank of $A$, i.e., $\mathrm{rank}(P^TA) = \mathrm{rank}(AQ) = \mathrm{rank}(A)$. The framework is extended to generalized $\{1,2\}$-inverses and specialized forms, revealing the underlying structure of established randomized linear algebra algorithms, including randomized SVD, the Nyström approximation, and CUR decomposition. We demonstrate applications in sparse sensor placement and effective resistance estimation. For the latter, we provide a rigorous quantitative analysis of an approximation scheme, establishing that it always underestimates the true resistance and deriving a worst-case spectral bound on the error of resistance differences.

math.NA↗

The Pseudoinverse of $A=CR$ is $A^+=R^+C^+$ (?)

This paper gives three formulas for the pseudoinverse of a matrix product $A = CR$. The first is sometimes correct, the second is always correct, and the third is almost never correct. But that third randomized pseudoinverse $A^+_r$ may be very useful when $A$ is a very large matrix. 1. $A^+ = R^+C^+$ when $A = CR$ and $C$ has independent columns and $R$ has independent rows. 2. $A^+ = (C^+CR)^+(CRR^+)^+$ is always correct. 3. $A^+_r = (P^TCR)^+P^TCRQ(CRQ)^+ = A^+$ only when $\mathrm{rank}(P^TA) = \mathrm{rank}(AQ) = \mathrm{rank}(A)$ with $A = CR$.

math.NA↗

The secret life of matrix factorizations: how matrix decompositions reveal and keep secrets of linear equations and what we can do about it

This paper explores the relationship between matrix factorizations and linear matrix equations. It shows that every matrix factorization defines two hidden projectors, one for the column space and one for the row space of a matrix, and how to calculate them. The projectors can be applied to solve linear matrix equations, generate low-rank approximations, or design randomized matrix algorithms. But also, as demonstrated, they can be applied in cryptography to encrypt and decrypt messages. The paper discusses some of the security implications of this application and leaves some questions open for further investigation. The basic concepts are illustrated with source code listings. Finally, this work shares some personal reflections on the meaning and importance of understanding in the time of the artificial intelligence revolution.

math.NA↗

A theory of meta-factorization

We introduce meta-factorization, a theory that describes matrix decompositions as solutions of linear matrix equations: the projector and the reconstruction equation. Meta-factorization reconstructs known factorizations, reveals their internal structures, and allows for introducing modifications, as illustrated with SVD, QR, and UTV factorizations. The prospect of meta-factorization also provides insights into computational aspects of generalized matrix inverses and randomized linear algebra algorithms. The relations between the Moore-Penrose pseudoinverse, generalized Nyström method, and the CUR decomposition are revealed here as an illustration. Finally, meta-factorization offers hints on the structure of new factorizations and provides the potential of creating them.

math.NA↗