arXiv subjects

Albert Cohen

Publications and source records attributed to Albert Cohen.

At least 19 recordsLinked to original sources

arXiv2026

Relaxed activation analysis of dataflow networks - A clock calculus for machine learning and real-time scheduling

Previous work has shown that the simple dataflow primitives of the Lustre language allow the natural, semantically unambiguous, and compact representation of machine learning (ML) applications, including models featuring complex conditional execution and recurrent state. The Lustre clock calculus is responsible for the static determination of important properties such as liveness (absence of deadlocks) and static memory bounds. Yet existing clock calculi are tailored for embedded control applications. We show they do not cater for the representation of control patterns commonly found in training algorithms, resulting in cumbersome expressions and inefficient compilation. We propose a conservative extension of Lustre's clock calculus addressing this limitation, thereby facilitating the embedding of ML models in reactive applications.

cs.PL↗

arXiv2026

Sampling and reconstruction of convex functions

We discuss optimal recovery for classes of multivariate convex functions from given point samples, as well as the sampling numbers of these classes, corresponding to optimal sample choices. Upper and lower bounds for either variant are established when the reconstruction error is measured in $L_p$ for $1\leq p\leq \infty$. These bounds match, sometimes up to logarithmic factors, and therefore characterize the respective optimal rate of decay. For classical smoothness classes such as Sobolev, H\"older or Besov spaces, it is well known that the optimal decay rate of sampling numbers can be achieved by sampling on uniform tensor product grids and using linear methods of reconstruction, such as piecewise polynomial interpolation. One of the main findings in this paper is that for classes of convex functions, these procedures generally produce suboptimal rates, except when $p=1$ and $p=\infty$, and are outperformed by nonlinear reconstruction methods that do not employ tensor product grids.

math.NA↗

arXiv2024

The MLIR Transform Dialect. Your compiler is more powerful than you think

To take full advantage of a specific hardware target, performance engineers need to gain control on compilers in order to leverage their domain knowledge about the program and hardware. Yet, modern compilers are poorly controlled, usually by configuring a sequence of coarse-grained monolithic black-box passes, or by means of predefined compiler annotations/pragmas. These can be effective, but often do not let users precisely optimize their varying compute loads. As a consequence, performance engineers have to resort to implementing custom passes for a specific optimization heuristic, requiring compiler engineering expert knowledge. In this paper, we present a technique that provides fine-grained control of general-purpose compilers by introducing the Transform dialect, a controllable IR-based transformation system implemented in MLIR. The Transform dialect empowers performance engineers to optimize their various compute loads by composing and reusing existing - but currently hidden - compiler features without the need to implement new passes or even rebuilding the compiler. We demonstrate in five case studies that the Transform dialect enables precise, safe composition of compiler transformations and allows for straightforward integration with state-of-the-art search methods.

cs.PL↗

arXiv2024

Constructions of bounded solutions of $div\, {\mathbf u}=f$ in critical spaces

We construct uniformly bounded solutions of the equation $div\, {\mathbf u}=f$ for arbitrary data $f$ in the critical spaces $L^d(\Omega)$, where $\Omega$ is a domain of ${\mathbb R}^d$. This question was addressed by Bourgain & Brezis, [On the equation ${\rm div}\, Y=f$ and application to control of phases, JAMS 16(2) (2003) 393-426], who proved that although the problem has a uniformly bounded solution, it is critical in the sense that there exists no linear solution operator for general $L^d$-data. We first discuss the validity of this existence result under weaker conditions than $f\in L^d(\Omega)$, and then focus our work on constructive processes for such uniformly bounded solutions. In the $d=2$ case, we present a direct one-step explicit construction, which generalizes for $d>2$ to a $(d-1)$-step construction based on induction. An explicit construction is proposed for compactly supported data in $L^{2,\infty}(\Omega)$ in the $d=2$ case. We also present constructive approaches based on optimization of a certain loss functional adapted to the problem. This approach provides a two-step construction in the $d=2$ case. This optimization is used as the building block of a hierarchical multistep process introduced in [E. Tadmor, Hierarchical construction of bounded solutions in critical regularity spaces, CPAM 69(6) (2016) 1087-1109] that converges to a solution in more general situations.

math.AP↗

arXiv2024

Strided Difference Bound Matrices

A wide range of symbolic analysis and optimization problems can be formalized using polyhedra. Sub-classes of polyhedra, also known as sub-polyhedral domains, are sought for their lower space and time complexity. We introduce the Strided Difference Bound Matrix (SDBM) domain, which represents a sweet spot in the context of optimizing compilers. Its expressiveness and efficient algorithms are particularly well suited to the construction of machine learning compilers. We present decision algorithms, abstract domain operators and computational complexity proofs for SDBM. We also conduct an empirical study with the MLIR compiler framework to validate the domain's practical applicability. We characterize a sub-class of SDBMs that frequently occurs in practice, and demonstrate even faster algorithms on this sub-class.

cs.SC↗

arXiv2024

High order recovery of geometric interfaces from cell-average data

We consider the problem of recovering characteristic functions $u:=\chi_\Omega$ from cell-average data on a coarse grid, and where $\Omega$ is a compact set of $\mathbb{R}^d$. This task arises in very different contexts such as image processing, inverse problems, and the accurate treatment of interfaces in finite volume schemes. While linear recovery methods are known to perform poorly, nonlinear strategies based on local reconstructions of the jump interface $\Gamma:=\partial\Omega$ by geometrically simpler interfaces may offer significant improvements. We study two main families of local reconstruction schemes, the first one based on nonlinear least-squares fitting, the second one based on the explicit computation of a polynomial-shaped curve fitting the data, which yields simpler numerical computations and high order geometric fitting. For each of them, we derive a general theoretical framework which allows us to control the recovery error by the error of best approximation up to a fixed multiplicative constant. Numerical tests in 2d illustrate the expected approximation order of these strategies. Several extensions are discussed, in particular the treatment of piecewise smooth interfaces with corners.

math.NA↗

arXiv2023

European Football Player Valuation: Integrating Financial Models and Network Theory

This paper presents a new framework for player valuation in European football, by fusing principles from financial mathematics and network theory. The valuation model leverages a "passing matrix" to encapsulate player interactions on the field, utilizing centrality measures to quantify individual influence. Unlike traditional approaches, such as regressing on past performance-salary data, this model focuses on in-game performance as a player's contributions evolve over time. Consequently, our model provides a dynamic and individualized framework for ascertaining a player's fair market value. The methodology is empirically validated through a case study in European football, employing real-world match and financial data. This cross-disciplinary mechanism for player valuation adapts the effect of connecting pay with performance, first seen in Scully (1974), to include in-game contributions as well as expected present valuation of stochastic variables.

physics.soc-ph↗

arXiv2023

Bidirectional Reactive Programming for Machine Learning

Reactive languages are dedicated to the programming of systems which interact continuously and concurrently with their environment. Values take the form of unbounded streams modeling the (discrete) passing of time or the sequence of concurrent interactions. While conventional reactivity models recurrences forward in time, we introduce a symmetric reactive construct enabling backward recurrences. Constraints on the latter allow to make the implementation practical. Machine Learning (ML) systems provide numerous motivations for all of this: we demonstrate that reverse-mode automatic differentiation, backpropagation, batch normalization, bidirectional recurrent neural networks, training and reinforcement learning algorithms, are all naturally captured as bidirectional reactive programs.

cs.PL↗

arXiv2023

The Next 700 ML-Enabled Compiler Optimizations

There is a growing interest in enhancing compiler optimizations with ML models, yet interactions between compilers and ML frameworks remain challenging. Some optimizations require tightly coupled models and compiler internals,raising issues with modularity, performance and framework independence. Practical deployment and transparency for the end-user are also important concerns. We propose ML-Compiler-Bridge to enable ML model development within a traditional Python framework while making end-to-end integration with an optimizing compiler possible and efficient. We evaluate it on both research and production use cases, for training and inference, over several optimization problems, multiple compilers and its versions, and gym infrastructures.

cs.PL↗

arXiv2023

Reduced order modeling for elliptic problems with high contrast diffusion coefficients

We consider the parametric elliptic PDE $-{\rm div} (a(y)\nabla u)=f$ on a spatial domain $\Omega$, with $a(y)$ a scalar piecewise constant diffusion coefficient taking any positive values $y=(y_1, \dots, y_d)\in ]0,\infty[^d$ on fixed subdomains $\Omega_1,\dots,\Omega_d$. This problem is not uniformly elliptic as the contrast $\kappa(y)=\frac{\max y_j}{\min y_j}$ can be arbitrarily high, contrarily to the Uniform Ellipticity Assumption (UEA) that is commonly made on parametric elliptic PDEs. Based on local polynomial approximations in the $y$ variable, we construct local and global reduced model spaces $V_n$ of moderate dimension $n$ that approximate uniformly well all solutions $u(y)$. Since the solution $u(y)$ blows as $y\to 0$, the solution manifold is not a compact set and does not have finite $n$-width. Therefore, our results for approximation by such spaces are formulated in terms of relative $H^1_0$-projection error, that is, after normalization by $\|u(y)\|_{H^1_0}$. We prove that this relative error decays exponentially with $n$, yet exhibiting the curse of dimensionality as the number $d$ of subdomains grows. We also show similar rates for the Galerkin projection despite the fact that high contrast is well-known to deteriorate the multiplicative constant when applying Cea's lemma. We finally establish uniform estimates in relative error for the state estimation and parameter estimation inverse problems, when $y$ is unknown and a limited number of linear measurements $\ell_i(u)$ are observed. A key ingredient in our construction and analysis is the study of the convergence of $u(y)$ to limit solutions when some of the parameters $y_j$ tend to infinity.

math.AP↗

arXiv2023

Solving PDEs with Incomplete Information

We consider the problem of numerically approximating the solutions to a partial differential equation (PDE) when there is insufficient information to determine a unique solution. Our main example is the Poisson boundary value problem, when the boundary data is unknown and instead one observes finitely many linear measurements of the solution. We view this setting as an optimal recovery problem and develop theory and numerical algorithms for its solution. The main vehicle employed is the derivation and approximation of the Riesz representers of these functionals with respect to relevant Hilbert spaces of harmonic functions.

math.NA↗

arXiv2022

Nonlinear approximation spaces for inverse problems

This paper is concerned with the ubiquitous inverse problem of recovering an unknown function u from finitely many measurements possibly affected by noise. In recent years, inversion methods based on linear approximation spaces were introduced in [MPPY15, BCDDPW17] with certified recovery bounds. It is however known that linear spaces become ineffective for approximating simple and relevant families of functions, such as piecewise smooth functions that typically occur in hyperbolic PDEs (shocks) or images (edges). For such families, nonlinear spaces [Devore98] are known to significantly improve the approximation performance. The first contribution of this paper is to provide with certified recovery bounds for inversion procedures based on nonlinear approximation spaces. The second contribution is the application of this framework to the recovery of general bidimensional shapes from cell-average data. We also discuss how the application of our results to n-term approximation relates to classical results in compressed sensing.

math.NA↗

arXiv2022

RL4ReAl: Reinforcement Learning for Register Allocation

We aim to automate decades of research and experience in register allocation, leveraging machine learning. We tackle this problem by embedding a multi-agent reinforcement learning algorithm within LLVM, training it with the state of the art techniques. We formalize the constraints that precisely define the problem for a given instruction-set architecture, while ensuring that the generated code preserves semantic correctness. We also develop a gRPC based framework providing a modular and efficient compiler interface for training and inference. Our approach is architecture independent: we show experimental results targeting Intel x86 and ARM AArch64. Our results match or out-perform the heavily tuned, production-grade register allocators of LLVM.

cs.LG↗

arXiv2022

Composable and Modular Code Generation in MLIR: A Structured and Retargetable Approach to Tensor Compiler Construction

Despite significant investment in software infrastructure, machine learning systems, runtimes and compilers do not compose properly. We propose a new design aiming at providing unprecedented degrees of modularity, composability and genericity. This paper discusses a structured approach to the construction of domain-specific code generators for tensor compilers, with the stated goal of improving the productivity of both compiler engineers and end-users. The approach leverages the natural structure of tensor algebra. It has been the main driver for the design of progressive lowering paths in \MLIR. The proposed abstractions and transformations span data structures and control flow with both functional (SSA form) and imperative (side-effecting) semantics. We discuss the implications of this infrastructure on compiler construction and present preliminary experimental results.

cs.PL↗

arXiv2021

Optimal pointwise sampling for $L^2$ approximation

Given a function $u\in L^2=L^2(D,μ)$, where $D\subset \mathbb R^d$ and $μ$ is a measure on $D$, and a linear subspace $V_n\subset L^2$ of dimension $n$, we show that near-best approximation of $u$ in $V_n$ can be computed from a near-optimal budget of $Cn$ pointwise evaluations of $u$, with $C>1$ a universal constant. The sampling points are drawn according to some random distribution, the approximation is computed by a weighted least-squares method, and the error is assessed in expected $L^2$ norm. This result improves on the results in [6,8] which require a sampling budget that is sub-optimal by a logarithmic factor, thanks to a sparsification strategy introduced in [17,18]. As a consequence, we obtain for any compact class $\mathcal K\subset L^2$ that the sampling number $ρ_{Cn}^{\rm rand}(\mathcal K)_{L^2}$ in the randomized setting is dominated by the Kolmogorov $n$-width $d_n(\mathcal K)_{L^2}$. While our result shows the existence of a randomized sampling with such near-optimal properties, we discuss remaining issues concerning its generation by a computationally efficient algorithm.

math.NA↗

arXiv2021

Recursive Estimation of a Failure Probability for a Lipschitz Function

Let g : $Ω$ = [0, 1] d $\rightarrow$ R denote a Lipschitz function that can be evaluated at each point, but at the price of a heavy computational time. Let X stand for a random variable with values in $Ω$ such that one is able to simulate, at least approximately, according to the restriction of the law of X to any subset of $Ω$. For example, thanks to Markov chain Monte Carlo techniques, this is always possible when X admits a density that is known up to a normalizing constant. In this context, given a deterministic threshold T such that the failure probability p := P(g(X) > T) may be very low, our goal is to estimate the latter with a minimal number of calls to g. In this aim, building on Cohen et al. [9], we propose a recursive and optimal algorithm that selects on the fly areas of interest and estimate their respective probabilities.

math.PR↗

arXiv2021

Near-optimal approximation methods for elliptic PDEs with lognormal coefficients

This paper studies numerical methods for the approximation of elliptic PDEs with lognormal coefficients of the form $-{\rm div}(a\nabla u)=f$ where $a=\exp(b)$ and $b$ is a Gaussian random field. The approximant of the solution $u$ is an $n$-term polynomial expansion in the scalar Gaussian random variables that parametrize $b$. We present a general convergence analysis of weighted least-squares approximants for smooth and arbitrarily rough random field, using a suitable random design, for which we prove optimality in the following sense: their convergence rate matches exactly or closely the rate that has been established in \cite{BCDM} for best $n$-term approximation by Hermite polynomials, under the same minimial assumptions on the Gaussian random field. This is in contrast with the current state of the art results for the stochastic Galerkin method that suffers the lack of coercivity due to the lognormal nature of the diffusion field. Numerical tests with $b$ as the Brownian bridge confirm our theoretical findings.

math.NA↗

arXiv2021

Secure Optimization Through Opaque Observations

Secure applications implement software protections against side-channel and physical attacks. Such protections are meaningful at machine code or micro-architectural level, but they typically do not carry observable semantics at source level. To prevent optimizing compilers from altering the protection, security engineers embed input/output side-effects into the protection. These side-effects are error-prone and compiler-dependent, and the current practice involves analyzing the generated machine code to make sure security or privacy properties are still enforced. Vu et al. recently demonstrated how to automate the insertion of volatile side-effects in a compiler [52], but these may be too expensive in fined-grained protections such as control-flow integrity. We introduce observations of the program state that are intrinsic to the correct execution of security protections, along with means to specify and preserve observations across the compilation flow. Such observations complement the traditional input/output-preservation contract of compilers. We show how to guarantee their preservation without modifying compilation passes and with as little performance impact as possible. We validate our approach on a range of benchmarks, expressing the secure compilation of these applications in terms of observations to be made at specific program points.

cs.CR↗