arXiv subjects

Feilong Cao

Publications and source records attributed to Feilong Cao.

9 recordsLinked to original sources

arXiv2026

HeterSEED: Semantics-Structure Decoupling for Heterogeneous Graph Learning under Heterophily

Many real-world heterogeneous graphs exhibit pronounced heterophily, where connected nodes often have dissimilar labels or play different semantic roles. In such settings, standard heterogeneous graph neural networks that aggregate messages along metapaths or meta-relations primarily based on feature similarity can propagate misleading information, since feature similarity may be misaligned with underlying relational semantics. In this paper, we propose HeterSEED, a semantics-structure decoupling framework for heterogeneous graph learning under heterophily. HeterSEED decouples representation learning into a heterogeneous semantic channel that captures type- and relation-aware local semantics and a structure-aware heterophily channel that separates homophilic and heterophilic neighborhoods via pseudo-label-guided partitioning and aggregates them using metapath-based structural weights. A node-level adaptive fusion mechanism then combines the two channels to produce context-dependent node representations. Theoretically, we establish that, on heterogeneous graphs under heterophily, HeterSEED is strictly more expressive than standard heterogeneous graph neural networks that rely primarily on feature similarity and provably reduces the prediction bias introduced by heterophilic neighbors. Experiments on five real-world heterogeneous graphs, including two large-scale networks at the million-node and hundred-million-edge scale, demonstrate that HeterSEED consistently outperforms representative heterogeneous graph neural networks and recent heterophily-aware baselines, especially in strongly heterophilic regimes.

cs.LG↗

arXiv2025

Feature Qualification by Deep Nets: A Constructive Approach

The great success of deep learning has stimulated avid research activities in verifying the power of depth in theory, a common consensus of which is that deep net are versatile in approximating and learning numerous functions. Such a versatility certainly enhances the understanding of the power of depth, but makes it difficult to judge which data features are crucial in a specific learning task. This paper proposes a constructive approach to equip deep nets for the feature qualification purpose. Using the product-gate nature and localized approximation property of deep nets with sigmoid activation (deep sigmoid nets), we succeed in constructing a linear deep net operator that possesses optimal approximation performance in approximating smooth and radial functions. Furthermore, we provide theoretical evidences that the constructed deep net operator is capable of qualifying multiple features such as the smoothness and radialness of the target functions.

cs.LG↗

arXiv2024

Component-based Sketching for Deep ReLU Nets

Deep learning has made profound impacts in the domains of data mining and AI, distinguished by the groundbreaking achievements in numerous real-world applications and the innovative algorithm design philosophy. However, it suffers from the inconsistency issue between optimization and generalization, as achieving good generalization, guided by the bias-variance trade-off principle, favors under-parameterized networks, whereas ensuring effective convergence of gradient-based algorithms demands over-parameterized networks. To address this issue, we develop a novel sketching scheme based on deep net components for various tasks. Specifically, we use deep net components with specific efficacy to build a sketching basis that embodies the advantages of deep networks. Subsequently, we transform deep net training into a linear empirical risk minimization problem based on the constructed basis, successfully avoiding the complicated convergence analysis of iterative algorithms. The efficacy of the proposed component-based sketching is validated through both theoretical analysis and numerical experiments. Theoretically, we show that the proposed component-based sketching provides almost optimal rates in approximating saturated functions for shallow nets and also achieves almost optimal generalization error bounds. Numerically, we demonstrate that, compared with the existing gradient-based training methods, component-based sketching possesses superior generalization performance with reduced training costs.

cs.LG↗

arXiv2023

How Powerful are Shallow Neural Networks with Bandlimited Random Weights?

We investigate the expressive power of depth-2 bandlimited random neural networks. A random net is a neural network where the hidden layer parameters are frozen with random assignment, and only the output layer parameters are trained by loss minimization. Using random weights for a hidden layer is an effective method to avoid non-convex optimization in standard gradient descent learning. It has also been adopted in recent deep learning theories. Despite the well-known fact that a neural network is a universal approximator, in this study, we mathematically show that when hidden parameters are distributed in a bounded domain, the network may not achieve zero approximation error. In particular, we derive a new nontrivial approximation error lower bound. The proof utilizes the technique of ridgelet analysis, a harmonic analysis method designed for neural networks. This method is inspired by fundamental principles in classical signal processing, specifically the idea that signals with limited bandwidth may not always be able to perfectly recreate the original signal. We corroborate our theoretical results with various simulation studies, and generally, two main take-home messages are offered: (i) Not any distribution for selecting random weights is feasible to build a universal approximator; (ii) A suitable assignment of random weights exists but to some degree is associated with the complexity of the target function.

cs.LG↗

arXiv2014

Approximation by boolean sums of Jackson operators on the sphere

This paper concerns the approximation by the Boolean sums of Jackson operators $\oplus^rJ_{k,s}(f)$ on the unit sphere $\mathbb S^{n-1}$ of $\mathbb{R}^{n}$. We prove the following the direct and inverse theorem for $\oplus^rJ_{k,s}(f)$: there are constants $C_1$ and $C_2$ such that \begin{equation*} C_1\|\oplus^rJ_{k,s}f-f\|_p \leq ω^{2r}(f,k^{-1})_p \leq C_2 \max_{v\geq k}\|\oplus^rJ_{k,s}f-f\|_p \end{equation*} for any positive integer $k$ and any $p$th Lebesgue integrable functions $f$ defined on $\mathbb S^{n-1}$, where $ω^{2r}(f,t)_p$ is the modulus of smoothness of degree $2r$ of $f$. We also prove that the saturation order for $\oplus^rJ_{k,s}$ is $k^{-2r}$.

math.CA↗

arXiv2014

A study on effectiveness of extreme learning machine

Extreme learning machine (ELM), proposed by Huang et al., has been shown a promising learning algorithm for single-hidden layer feedforward neural networks (SLFNs). Nevertheless, because of the random choice of input weights and biases, the ELM algorithm sometimes makes the hidden layer output matrix H of SLFN not full column rank, which lowers the effectiveness of ELM. This paper discusses the effectiveness of ELM and proposes an improved algorithm called EELM that makes a proper selection of the input weights and bias before calculating the output weights, which ensures the full column rank of H in theory. This improves to some extend the learning rate (testing accuracy, prediction accuracy, learning time) and the robustness property of the networks. The experimental results based on both the benchmark function approximation and real-world problems including classification and regression applications show the good performances of EELM.

cs.NE↗

arXiv2014

The Direct and Converse Inequalities for Jackson-Type Operators on Spherical Cap

Approximation on the spherical cap is different from that on the sphere which requires us to construct new operators. This paper discusses the approximation on the spherical cap. That is, so called Jackson-type operator $\{J_{k,s}^m\}_{k=1}^{\infty}$ is constructed to approximate the function defined on the spherical cap $D(x_0,γ)$. We thus establish the direct and inverse inequalities and obtain saturation theorems for $\{J_{k,s}^m\}_{k=1}^{\infty}$ on the cap $D(x_0,γ)$. Using methods of $K$-functional and multiplier, we obtain the inequality \begin{eqnarray*} C_1\:\| J_{k,s}^m(f)-f\|_{D,p}\leq ω^2\left(f,\:k^{-1}\right)_{D,p} \leq C_2 \max_{v\geq k}\| J_{v,s}^m(f) - f\|_{D,p} \end{eqnarray*} and that the saturation order of these operators is $O(k^{-2})$, where $ω^2\left(f,\:t\right)_{D,p}$ is the modulus of smoothness of degree 2, the constants $C_1$ and $C_2$ are independent of $k$ and $f$.

math.CA↗

arXiv2011

The strong converse inequality for de la Vallée Poussin means on the sphere

This paper discusses the approximation by de la Vallée Poussin means $V_nf$ on the unit sphere. Especially, the lower bound of approximation is studied. As a main result, the strong converse inequality for the means is established. Namely, it is proved that there are constants $C_1$ and $C_2$ such that \begin{eqnarray*} C_1ω(f,\frac{1}{\sqrt n})_p \leq \|V_{n}f-f\|_p \leq C_2ω(f,\frac{1}{\sqrt n})_p \end{eqnarray*} for any $p$-th Lebesgue integrable or continuous function $f$ defined on the sphere, where $ω(f,t)_p$ is the modulus of smoothness of $f$.

math.CA↗

arXiv2011

Approximation by Semigroups of Spherical Operators

This paper discusses the approximation by %semigroups of operators of class ($\mathscr{C}_0$) on the sphere and focuses on a class of so called exponential-type multiplier operators. It is proved that such operators form a strongly continuous semigroup of contraction operators of class ($\mathscr{C}_0$), from which the equivalence between approximation for these operators and $K$-functionals introduced by the operators is given. As examples, the constructed $r$-th Boolean of generalized spherical Abel-Poisson operator and $r$-th Boolean of generalized spherical Weierstrass operator denoted by $\oplus^r V_t^γ$ and $\oplus^r W_t^κ$ separately ($r$ is any positive integer, $0<γ,κ\leq1$ and $t>0$) satisfy that $\|\oplus^r V_t^γf - f\|_{\mathcal{X}}\approx ω^{rγ}(f,t^{1/γ})_{\mathcal{X}}$ and $\|\oplus^r W_t^κf - f\|_{\mathcal{X}}\approx ω^{2rγ}(f,t^{1/(2κ)})_{\mathcal{X}}$, for all $f\in \mathcal{X}$, where $\mathcal{X}$ is a Banach space of continuous functions or $\mathcal{L}^p$-integrable functions ($1\leq p<\infty$) and $\|\cdot\|_{\mathcal{X}}$ is the norm on $\mathcal{X}$ and $ω^s(f,t)_{\mathcal{X}}$ is the moduli of smoothness of degree $s>0$ for $f\in \mathcal{X}$. The saturation order and saturation class of the regular exponential-type multiplier operators with positive kernels are also obtained. Moreover, it is proved that $\oplus^r V_t^γ$ and $\oplus^r W_t^κ$ have the same saturation class if $γ=2κ$.

math.CA↗