arXiv subjects

John Chiang

Publications and source records attributed to John Chiang.

At least 19 recordsLinked to original sources

arXiv2026

Generalized Quadratic Gradient: A New Direction in Optimization via the Fusion of Positive-Definite Curvature Matrices and Gradients into A Unified Framework

Quadratic Gradient (QG) is a Newton-type optimization framework that bridges first-order gradient descent and second-order optimization by incorporating curvature information into gradient updates. Simplified Quadratic Gradient (SQG) reduces the complexity of QG construction while preserving its optimization capability, whereas Quasi-Quadratic Gradient (QQG) extends the quadratic gradient principle to quasi-Newton methods such as BFGS. In this paper, we propose **Generalized Quadratic Gradient (GQG)**, a unified framework that extends the quadratic gradient principle to a broader class of Newton-type optimization algorithms. By abstracting the common structure of existing quadratic gradient methods, we show that the fundamental requirement of quadratic gradient construction is not limited to specific Hessian approximations, such as constant Hessian matrices, diagonal Hessian approximations, or BFGS-based Hessian surrogates. Instead, it can be generalized to any positive-definite curvature matrix satisfying the stationary condition of a local quadratic model. Based on this perspective, we investigate the construction of generalized quadratic gradients using various positive-definite Hessian surrogates beyond BFGS, providing a broader foundation for developing curvature-aware optimization algorithms.

math.OC↗

arXiv2026

Trimming: Decoupling Multiplicative Depth from Modulus Chains in RNS-CKKS via Rational Levels

Recent work on Grafting decouples scale factors from ciphertext moduli, enabling more flexible precision management in RNS-CKKS. However, the multiplicative depth remains fundamentally constrained by the modulus chain structure. In this paper, we propose \emph{Trimming}, a novel fine-grained level management mechanism that decouples multiplicative depth from modulus chains in RNS-CKKS via rational levels. The key idea is to introduce an auxiliary trimming modulus chain composed of smaller NTT-friendly modulus factors, which enables partial modulus transitions instead of directly discarding an entire modulus factor. By replacing conventional discrete level reductions with fine-grained modulus factor refinement, Trimming provides a rational-level abstraction beyond the traditional integer-based modulus chain representation. Our approach preserves the compatibility with existing RNS-CKKS arithmetic while enabling more flexible depth management and adaptive modulus transitions. Similar to Grafting, which addresses the precision bottleneck, Trimming targets the depth bottleneck in RNS-CKKS and contributes toward a fully decoupled RNS-CKKS architecture. The proposed framework will be further validated through concrete implementation and experimental evaluation to investigate its practical performance and computational overhead in real-world homomorphic encryption applications.

cs.CR↗

arXiv2026

LibFHE: A Numba-Based CUDA-Python Library for Non-RNS CKKS-BGV Fully Homomorphic Encryption on GPUs

It has been a decade since the fourth-generation FHE framework, CKKS, was proposed; yet, there is still no indicator pointing toward a fifth-generation successor; and in recent years, numerous studies have explored GPU acceleration to improve the efficiency of homomorphic computations. In this paper, we propose LibFHE, a high-performance GPU-accelerated framework that features CUDA-Python bindings to achieve both high-level programmability and bare-metal GPU performance for homomorphic workloads. A large majority of state-of-the-art implementations adopt the RNS-CKKS variant. In contrast, this work deliberately revisits the original (non-RNS) CKKS-BGV framework, and develops a GPU-based implementation along with corresponding optimizations. Experimental results demonstrate that optimized CUDA-Python implementations can achieve performance comparable to highly optimized CPU-based C++ FHE libraries, while significantly reducing implementation complexity and improving programmability.

cs.CR↗

arXiv2026

Quasi-Quadratic Gradient: A New Direction for Accelerating the BFGS Method in Quasi-Newton Optimization

In this paper, we introduce the Quasi-Quadratic Gradient (QQG), a novel search direction designed to accelerate the BFGS method within the quasi-Newton framework. By defining the QQG as the product of the inverse Hessian approximation and the current gradient, we explicitly leverage local second-order curvature to rectify the search path. Theoretical analysis and empirical results demonstrate that our approach significantly outperforms vanilla BFGS in convergence speed while maintaining computational efficiency.

math.OC↗

arXiv2025

Volley Revolver: A Novel Matrix-Encoding Method for Privacy-Preserving Deep Learning (Inference++)

Privacy-preserving inference of convolutional neural networks (CNNs) using homomorphic encryption has emerged as a promising approach for enabling secure machine learning in untrusted environments. In our previous work, we introduced a matrix-encoding strategy that allows convolution and matrix multiplication to be efficiently evaluated over encrypted data, enabling practical CNN inference without revealing either the input data or the model parameters. The core idea behind this strategy is to construct a three-dimensional representation within ciphertexts that preserves the intrinsic spatial structure of both input image data and model weights, rather than flattening them into conventional two-dimensional encodings. However, this approach can operate efficiently $only$ when the number of available plaintext slots within a ciphertext is sufficient to accommodate an entire input image, which becomes a critical bottleneck when processing high-resolution images. In this paper, we address this fundamental limitation by proposing an improved encoding and computation framework that removes the requirement that a single encrypted ciphertext must fully contain one input image. Our method reformulates the data layout and homomorphic operations to partition high-resolution inputs across multiple ciphertexts while preserving the algebraic structure required for efficient convolution and matrix multiplication. As a result, our approach enables privacy-preserving CNN inference to scale naturally beyond the slot-capacity constraints of prior methods, making homomorphic evaluation of CNNs practical for higher-resolution and more complex datasets.

cs.CR↗

arXiv2025

CryptoUNets: Applying Convolutional Networks to Encrypted Data for Biomedical Image Segmentation

In this manuscript, we demonstrate the feasibility of a privacy-preserving U-Net deep learning inference framework, namely, homomorphic encryption-based U-Net inference. That is, U-Net inference can be performed solely using homomorphic encryption techniques. To our knowledge, this is the first work to achieve support perform implement enable U-Net inference entirely based on homomorphic encryption ?. The primary technical challenge lies in data encoding. To address this, we employ a flexible encoding scheme, termed Double Volley Revolver, which enables effective support for skip connections and upsampling operations within the U-Net architecture. We adopt a tailored HE-friendly U-Net design incorporating square activation functions, mean pooling layers, and transposed convolution layers (implemented as ConvTranspose2d in PyTorch) with a kernel size of 2 and stride of 2. After training the model in plaintext, we deploy the resulting parameters using the HEAAN homomorphic encryption library to perform encrypted U-Net inference.

cs.CR↗

arXiv2025

Privacy-Preserving CNN Training with Transfer Learning: Two Hidden Layers

In this paper, we present the demonstration of training a four-layer neural network entirely using fully homomorphic encryption (FHE), supporting both single-output and multi-output classification tasks in a non-interactive setting. A key contribution of our work is identifying that replacing \textit{Softmax} with \textit{Sigmoid}, in conjunction with the Binary Cross-Entropy (BCE) loss function, provides an effective and scalable solution for homomorphic classification. Moreover, we show that the BCE loss function, originally designed for multi-output tasks, naturally extends to the multi-class setting, thereby enabling broader applicability. We also highlight the limitations of prior loss functions such as the SLE loss and the one proposed in the 2019 CVPR Workshop, both of which suffer from vanishing gradients as network depth increases. To address the challenges posed by large-scale encrypted data, we further introduce an improved version of the previously proposed data encoding scheme, \textit{Double Volley Revolver}, which achieves a better trade-off between computational and memory efficiency, making FHE-based neural network training more practical. The complete, runnable C++ code to implement our work can be found at: \href{https://github.com/petitioner/ML.NNtraining}{$\texttt{https://github.com/petitioner/ML.NNtraining}$}.

cs.CR↗

arXiv2024

LFFR: Logistic Function For (multi-output) Regression

In this manuscript, we extend our previous work on privacy-preserving regression to address multi-output regression problems using data encrypted under a fully homomorphic encryption scheme. We build upon the simplified fixed Hessian approach for linear and ridge regression and adapt our novel LFFR algorithm, initially designed for single-output logistic regression, to handle multiple outputs. We further refine the constant simplified Hessian method for the multi-output context, ensuring computational efficiency and robustness. Evaluations on multiple real-world datasets demonstrate the effectiveness of our multi-output LFFR algorithm, highlighting its capability to maintain privacy while achieving high predictive accuracy. Normalizing both data and target predictions remains essential for optimizing homomorphic encryption parameters, confirming the practicality of our approach for secure and efficient multi-output regression tasks.

cs.CR↗

arXiv2024

LFFR: Logistic Function For (single-output) Regression

Privacy-preserving regression in machine learning is a crucial area of research, aimed at enabling the use of powerful machine learning techniques while protecting individuals' privacy. In this paper, we implement privacy-preserving regression training using data encrypted under a fully homomorphic encryption scheme. We first examine the common linear regression algorithm and propose a (simplified) fixed Hessian for linear regression training, which can be applied for any datasets even not normalized into the range $[0, 1]$. We also generalize this constant Hessian matrix to the ridge regression version, namely linear regression which includes a regularization term to penalize large coefficients. However, our main contribution is to develop a novel and efficient algorithm called LFFR for homomorphic regression using the logistic function, which could model more complex relations between input values and output prediction in comparison with linear regression. We also find a constant simplified Hessian to train our LFFR algorithm using the Newton-like method and compare it against to with our new fixed Hessian linear regression training over two real-world datasets. We suggest normalizing not only the data but also the target predictions even for the original linear regression used in a privacy-preserving manner, which is helpful to remain weights in a small range, say $[-5, +5]$ good for refreshing ciphertext setting parameters, and avoid tuning the regularization parameter $\lambda$ via cross validation. The linear regression with normalized predictions could be a viable alternative to ridge regression.

cs.LG↗

arXiv2024

Privacy-Preserving Logistic Regression Training on Large Datasets

Privacy-preserving machine learning is one class of cryptographic methods that aim to analyze private and sensitive data while keeping privacy, such as homomorphic logistic regression training over large encrypted data. In this paper, we propose an efficient algorithm for logistic regression training on large encrypted data using Homomorphic Encryption (HE), which is the mini-batch version of recent methods using a faster gradient variant called $\texttt{quadratic gradient}$. It is claimed that $\texttt{quadratic gradient}$ can integrate curve information (Hessian matrix) into the gradient and therefore can effectively accelerate the first-order gradient (descent) algorithms. We also implement the full-batch version of their method when the encrypted dataset is so large that it has to be encrypted in the mini-batch manner. We compare our mini-batch algorithm with our full-batch implementation method on real financial data consisting of 422,108 samples with 200 freatures. %Our experiments show that Nesterov's accelerated gradient (NAG) Given the inefficiency of HEs, our results are inspiring and demonstrate that the logistic regression training on large encrypted dataset is of practical feasibility, marking a significant milestone in our understanding.

cs.CR↗

arXiv2024

A Simple Solution for Homomorphic Evaluation on Large Intervals

Homomorphic encryption (HE) is a promising technique used for privacy-preserving computation. Since HE schemes only support primitive polynomial operations, homomorphic evaluation of polynomial approximations for non-polynomial functions plays an important role in privacy-preserving machine learning. In this paper, we introduce a simple solution to approximating any functions, which might be overmissed by researchers: just using the neural networks for regressions. By searching decent superparameters, neural networks can achieve near-optimal computation depth for a given function with fixed precision, thereby reducing the modulus consumed. There are three main reasons why we choose neural networks for homomorphic evaluation of polynomial approximations. Firstly, neural networks with polynomial activation functions can be used to approximate whatever functions are needed in an encrypted state. This means that we can compute by one unified process for any polynomial approximation, such as that of Sigmoid or of ReLU. Secondly, by carefully finding an appropriate architecture, neural networks can efficiently evaluate a polynomial using near-optimal multiplicative depth, which would consume less modulus and therefore employ less ciphertext refreshing. Finally, as popular tools, model neural networks have many well-studied techniques that can conveniently serve our solution. Experiments showed that our method can be used for approximation of various functions. We exploit our method to the evaluation of the Sigmoid function on large intervals $[-30, +30]$, $[-50, +50]$, and $[-70, +70]$, respectively.

cs.CR↗

arXiv2023

Privacy-Preserving 3-Layer Neural Network Training

In this manuscript, we consider the problem of privacy-preserving training of neural networks in the mere homomorphic encryption setting. We combine several exsiting techniques available, extend some of them, and finally enable the training of 3-layer neural networks for both the regression and classification problems using mere homomorphic encryption technique.

cs.CR↗

arXiv2023

Activation Functions Not To Active: A Plausible Theory on Interpreting Neural Networks

Researchers commonly believe that neural networks model a high-dimensional space but cannot give a clear definition of this space. What is this space? What is its dimension? And does it has finite dimensions? In this paper, we develop a plausible theory on interpreting neural networks in terms of the role of activation functions in neural networks and define a high-dimensional (more precisely, an infinite-dimensional) space that neural networks including deep-learning networks could create. We show that the activation function acts as a magnifying function that maps the low-dimensional linear space into an infinite-dimensional space, which can distinctly identify the polynomial approximation of any multivariate continuous function of the variable values being the same features of the given dataset. Given a dataset with each example of $d$ features $f_1$, $f_2$, $\cdots$, $f_d$, we believe that neural networks model a special space with infinite dimensions, each of which is a monomial $$\prod_{i_1, i_2, \cdots, i_d} f_1^{i_1} f_2^{i_2} \cdots f_d^{i_d}$$ for some non-negative integers ${i_1, i_2, \cdots, i_d} \in \mathbb{Z}_{0}^{+}=\{0,1,2,3,\ldots\} $. We term such an infinite-dimensional space a $\textit{ Super Space (SS)}$. We see such a dimension as the minimum information unit. Every neuron node previously through an activation layer in neural networks is a $\textit{ Super Plane (SP) }$, which is actually a polynomial of infinite degree. This $\textit{ Super Space }$ is something like a coordinate system, in which every multivalue function can be represented by a $\textit{ Super Plane }$. We also show that training NNs could at least be reduced to solving a system of nonlinear equations. %solve sets of nonlinear equations

cs.LG↗

arXiv2023

Privacy-Preserving CNN Training with Transfer Learning: Multiclass Logistic Regression

In this paper, we present a practical solution to implement privacy-preserving CNN training based on mere Homomorphic Encryption (HE) technique. To our best knowledge, this is the first attempt successfully to crack this nut and no work ever before has achieved this goal. Several techniques combine to accomplish the task:: (1) with transfer learning, privacy-preserving CNN training can be reduced to homomorphic neural network training, or even multiclass logistic regression (MLR) training; (2) via a faster gradient variant called $\texttt{Quadratic Gradient}$, an enhanced gradient method for MLR with a state-of-the-art performance in convergence speed is applied in this work to achieve high performance; (3) we employ the thought of transformation in mathematics to transform approximating Softmax function in the encryption domain to the approximation of the Sigmoid function. A new type of loss function termed $\texttt{Squared Likelihood Error}$ has been developed alongside to align with this change.; and (4) we use a simple but flexible matrix-encoding method named $\texttt{Volley Revolver}$ to manage the data flow in the ciphertexts, which is the key factor to complete the whole homomorphic CNN training. The complete, runnable C++ code to implement our work can be found at: \href{https://github.com/petitioner/HE.CNNtraining}{$\texttt{https://github.com/petitioner/HE.CNNtraining}$}. We select $\texttt{REGNET\_X\_400MF}$ as our pre-trained model for transfer learning. We use the first 128 MNIST training images as training data and the whole MNIST testing dataset as the testing data. The client only needs to upload 6 ciphertexts to the cloud and it takes $\sim 21$ mins to perform 2 iterations on a cloud with 64 vCPUs, resulting in a precision of $21.49\%$.

cs.CR↗

arXiv2023

Multinomial Logistic Regression Algorithms via Quadratic Gradient

Multinomial logistic regression, also known by other names such as multiclass logistic regression and softmax regression, is a fundamental classification method that generalizes binary logistic regression to multiclass problems. A recently work proposed a faster gradient called $\texttt{quadratic gradient}$ that can accelerate the binary logistic regression training, and presented an enhanced Nesterov's accelerated gradient (NAG) method for binary logistic regression. In this paper, we extend this work to multiclass logistic regression and propose an enhanced Adaptive Gradient Algorithm (Adagrad) that can accelerate the original Adagrad method. We test the enhanced NAG method and the enhanced Adagrad method on some multiclass-problem datasets. Experimental results show that both enhanced methods converge faster than their original ones respectively.

cs.LG↗

arXiv2022

Simplified Quadratic Gradient: A Unified Framework Bridging Gradient Descent and Newton-Type Methods by Synthesizing Hessians and Gradients

Accelerating the convergence of second-order optimization, particularly Newton-type methods, remains a pivotal challenge in algorithmic research. In this paper, we extend previous work on the \textbf{Quadratic Gradient (QG)} and rigorously validate its applicability to general convex numerical optimization problems. We introduce a novel variant of the Quadratic Gradient that departs from the conventional fixed Hessian Newton framework. We present a new way to build a new version of the quadratic gradient. This new quadratic gradient doesn't satisfy the convergence conditions of the fixed Hessian Newton's method. However, experimental results show that it sometimes has a better performance than the original one in convergence rate. While this variant relaxes certain classical convergence constraints, it maintains a positive-definite Hessian proxy and demonstrates comparable, or in some cases superior, empirical performance in convergence rates. Furthermore, we demonstrate that both the original and the proposed QG variants can be effectively applied to non-convex optimization landscapes. A key motivation of our work is the limitation of traditional scalar learning rates. We argue that a diagonal matrix can more effectively accelerate gradient elements at heterogeneous rates. Our findings establish the Quadratic Gradient as a versatile and potent framework for modern optimization. Furthermore, we integrate Hutchinson's Estimator to estimate the Hessian diagonal efficiently via Hessian-vector products. Notably, we demonstrate that the proposed Quadratic Gradient variant is highly effective for Deep Learning architectures, providing a robust second-order alternative to standard adaptive optimizers.

math.OC↗

arXiv2022

Volley Revolver: A Novel Matrix-Encoding Method for Privacy-Preserving Neural Networks (Inference)

In this work, we present a novel matrix-encoding method that is particularly convenient for neural networks to make predictions in a privacy-preserving manner using homomorphic encryption. Based on this encoding method, we implement a convolutional neural network for handwritten image classification over encryption. For two matrices $A$ and $B$ to perform homomorphic multiplication, the main idea behind it, in a simple version, is to encrypt matrix $A$ and the transpose of matrix $B$ into two ciphertexts respectively. With additional operations, the homomorphic matrix multiplication can be calculated over encrypted matrices efficiently. For the convolution operation, we in advance span each convolution kernel to a matrix space of the same size as the input image so as to generate several ciphertexts, each of which is later used together with the ciphertext encrypting input images for calculating some of the final convolution results. We accumulate all these intermediate results and thus complete the convolution operation. In a public cloud with 40 vCPUs, our convolutional neural network implementation on the MNIST testing dataset takes $\sim$ 287 seconds to compute ten likelihoods of 32 encrypted images of size $28 \times 28$ simultaneously. The data owner only needs to upload one ciphertext ($\sim 19.8$ MB) encrypting these 32 images to the public cloud.

cs.CR↗

arXiv2022

On Polynomial Approximation of Activation Function

In this work, we propose an interesting method that aims to approximate an activation function over some domain by polynomials of the presupposing low degree. The main idea behind this method can be seen as an extension of the ordinary least square method and includes the gradient of activation function into the cost function to minimize.

cs.LG↗