arXiv subjects

Zhi Zhou

Publications and source records attributed to Zhi Zhou.

At least 91 records · Page 5Linked to original sources

arXiv2024

TexLiverNet: Leveraging Medical Knowledge and Spatial-Frequency Perception for Enhanced Liver Tumor Segmentation

Integrating textual data with imaging in liver tumor segmentation is essential for enhancing diagnostic accuracy. However, current multi-modal medical datasets offer only general text annotations, lacking lesion-specific details critical for extracting nuanced features, especially for fine-grained segmentation of tumor boundaries and small lesions. To address these limitations, we developed datasets with lesion-specific text annotations for liver tumors and introduced the TexLiverNet model. TexLiverNet employs an agent-based cross-attention module that integrates text features efficiently with visual features, significantly reducing computational costs. Additionally, enhanced spatial and adaptive frequency domain perception is proposed to precisely delineate lesion boundaries, reduce background interference, and recover fine details in small lesions. Comprehensive evaluations on public and private datasets demonstrate that TexLiverNet achieves superior performance compared to current state-of-the-art methods.

eess.IV↗

arXiv2024

Drag and torque coefficients of a translating particle with slip at a gas-liquid interface

The dynamics of colloid-size particles trapped at a liquid interface is an extensively studied problem owing to its relevance to a wide range of engineering applications. Here we investigate the impact of interfacial deformations on the hydrodynamic force and torque exerted on a spherical particle with surface slip moving along a gas-liquid interface. Following a two-parameter asymptotic modeling approach, we perturb the interface from its planar state and apply the Lorentz reciprocal theorem to the zeroth and first-order approximations to analytically calculate the drag and torque on the particle. This allows us to explicitly account for the effect of physical parameters like the three-phase contact angle, the Bond number, and the slip coefficient on the particle motion. In addition, we study the interactions between two translating and rotating particles at a large separation. The interaction forces and torques exerted by the flow-induced deformations are calculated via the linear superposition approximation, where the interaction forces are identified as dipolar in terms of the azimuthal angle.

cond-mat.soft↗

arXiv2024

On an Inverse Problem of the Generalized Bathtub Model of Network Trip Flows

In this work, we investigate the generalized bathtub model, a nonlocal transport equation for describing network trip flows served by privately operated vehicles inside a road network. First, we establish the well-posedness of the mathematical model for both classical and weak solutions. Then we consider an inverse source problem of the model with model parameters embodying particular traffic situations. We establish a conditional Lipschitz stability of the inverse problem under suitable a priori regularity assumption on the problem data, using a Volterra integral formulation of the problem. Inspired by the analysis, we develop an easy-to-implement numerical method for reconstructing the flow rates, and provide the error analysis of the method. Further we present several numerical experiments to complement the theoretical analysis.

math.AP↗

arXiv2024

Point Source Identification Using Singularity Enriched Neural Networks

The inverse problem of recovering point sources represents an important class of applied inverse problems. However, there is still a lack of neural network-based methods for point source identification, mainly due to the inherent solution singularity. In this work, we develop a novel algorithm to identify point sources, utilizing a neural network combined with a singularity enrichment technique. We employ the fundamental solution and neural networks to represent the singular and regular parts, respectively, and then minimize an empirical loss involving the intensities and locations of the unknown point sources, as well as the parameters of the neural network. Moreover, by combining the conditional stability argument of the inverse problem with the generalization error of the empirical loss, we conduct a rigorous error analysis of the algorithm. We demonstrate the effectiveness of the method with several challenging experiments.

math.NA↗

arXiv2024

Regularity Analysis and High-Order Time Stepping Scheme for Quasilinear Subdiffusion

In this work, we investigate a quasilinear subdiffusion model which involves a fractional derivative of order $α\in (0,1)$ in time and a nonlinear diffusion coefficient. First, using smoothing properties of solution operators for linear subdiffusion and a perturbation argument, we prove several pointwise-in-time regularity estimates that are useful for numerical analysis. Then we develop a high-order time stepping scheme for solving quasilinear subdiffusion, based on convolution quadrature generated by second-order backward differentiation formula with correction at the first step. Further, we establish that the convergence order of the scheme is $O(τ^{1+α-ε})$ without imposing any additional assumption on the regularity of the solution. The analysis relies on refined Sobolev regularity of the nonlinear perturbation remainder and smoothing properties of discrete solution operators. Several numerical experiments in two space dimensions show the sharpness of the error estimate.

math.NA↗

arXiv2024

Determining a Time-Varying Potential in Time-Fractional Diffusion from Observation at a Single Point

We discuss the identification of a time-dependent potential in a time-fractional diffusion model from a boundary measurement taken at a single point. Theoretically, we establish a conditional Lipschitz stability for this inverse problem. Numerically, we develop an easily implementable iterative algorithm to recover the unknown coefficient, and also derive rigorous error bounds for the discrete reconstruction. These results are attained by using the (discrete) solution theory of direct problems, and applying error estimates that are optimal with respect to problem data regularity. Numerical simulations are provided to demonstrate the theoretical results.

math.NA↗

arXiv2024

Online Optimization of DNN Inference Network Utility in Collaborative Edge Computing

Collaborative Edge Computing (CEC) is an emerging paradigm that collaborates heterogeneous edge devices as a resource pool to compute DNN inference tasks in proximity such as edge video analytics. Nevertheless, as the key knob to improve network utility in CEC, existing works mainly focus on the workload routing strategies among edge devices with the aim of minimizing the routing cost, remaining an open question for joint workload allocation and routing optimization problem from a system perspective. To this end, this paper presents a holistic, learned optimization for CEC towards maximizing the total network utility in an online manner, even though the utility functions of task input rates are unknown a priori. In particular, we characterize the CEC system in a flow model and formulate an online learning problem in a form of cross-layer optimization. We propose a nested-loop algorithm to solve workload allocation and distributed routing iteratively, using the tools of gradient sampling and online mirror descent. To improve the convergence rate over the nested-loop version, we further devise a single-loop algorithm. Rigorous analysis is provided to show its inherent convexity, efficient convergence, as well as algorithmic optimality. Finally, extensive numerical simulations demonstrate the superior performance of our solutions.

cs.DC↗

arXiv2024

LawGPT: A Chinese Legal Knowledge-Enhanced Large Language Model

Large language models (LLMs), including both proprietary and open-source models, have showcased remarkable capabilities in addressing a wide range of downstream tasks. Nonetheless, when it comes to practical Chinese legal tasks, these models fail to meet the actual requirements. Proprietary models do not ensure data privacy for sensitive legal cases, while open-source models demonstrate unsatisfactory performance due to their lack of legal knowledge. To address this problem, we introduce LawGPT, the first open-source model specifically designed for Chinese legal applications. LawGPT comprises two key components: legal-oriented pre-training and legal supervised fine-tuning. Specifically, we employ large-scale Chinese legal documents for legal-oriented pre-training to incorporate legal domain knowledge. To further improve the model's performance on downstream legal tasks, we create a knowledge-driven instruction dataset for legal supervised fine-tuning. Our experimental results demonstrate that LawGPT outperforms the open-source LLaMA 7B model. Our code and resources are publicly available at https://github.com/pengxiao-song/LaWGPT and have received 5.7K stars on GitHub.

cs.CL↗

arXiv2024

Long-Tail Learning with Foundation Model: Heavy Fine-Tuning Hurts

The fine-tuning paradigm in addressing long-tail learning tasks has sparked significant interest since the emergence of foundation models. Nonetheless, how fine-tuning impacts performance in long-tail learning was not explicitly quantified. In this paper, we disclose that heavy fine-tuning may even lead to non-negligible performance deterioration on tail classes, and lightweight fine-tuning is more effective. The reason is attributed to inconsistent class conditions caused by heavy fine-tuning. With the observation above, we develop a low-complexity and accurate long-tail learning algorithms LIFT with the goal of facilitating fast prediction and compact models by adaptive lightweight fine-tuning. Experiments clearly verify that both the training time and the learned parameters are significantly reduced with more accurate predictive performance compared with state-of-the-art approaches. The implementation code is available at https://github.com/shijxcs/LIFT.

cs.CV↗

arXiv2024

DeCoOp: Robust Prompt Tuning with Out-of-Distribution Detection

Vision-language models (VLMs), such as CLIP, have demonstrated impressive zero-shot capabilities for various downstream tasks. Their performance can be further enhanced through few-shot prompt tuning methods. However, current studies evaluate the performance of learned prompts separately on base and new classes. This evaluation lacks practicality for real-world applications since downstream tasks cannot determine whether the data belongs to base or new classes in advance. In this paper, we explore a problem setting called Open-world Prompt Tuning (OPT), which involves tuning prompts on base classes and evaluating on a combination of base and new classes. By introducing Decomposed Prompt Tuning framework (DePT), we theoretically demonstrate that OPT can be solved by incorporating out-of-distribution detection into prompt tuning, thereby enhancing the base-to-new discriminability. Based on DePT, we present a novel prompt tuning approach, namely, Decomposed Context Optimization (DeCoOp), which introduces new-class detectors and sub-classifiers to further enhance the base-class and new-class discriminability. Experimental results on 11 benchmark datasets validate the effectiveness of DePT and demonstrate that DeCoOp outperforms current state-of-the-art methods, providing a significant 2% average accuracy improvement.

cs.CV↗

arXiv2024

Opara: Exploiting Operator Parallelism for Expediting DNN Inference on GPUs

GPUs have become the \emph{defacto} hardware devices for accelerating Deep Neural Network (DNN) inference workloads. However, the conventional \emph{sequential execution mode of DNN operators} in mainstream deep learning frameworks cannot fully utilize GPU resources, even with the operator fusion enabled, due to the increasing complexity of model structures and a greater diversity of operators. Moreover, the \emph{inadequate operator launch order} in parallelized execution scenarios can lead to GPU resource wastage and unexpected performance interference among operators. In this paper, we propose \emph{Opara}, a resource- and interference-aware DNN \underline{Op}erator \underline{para}llel scheduling framework to accelerate DNN inference on GPUs. Specifically, \emph{Opara} first employs \texttt{CUDA Streams} and \texttt{CUDA Graph} to \emph{parallelize} the execution of multiple operators automatically. To further expedite DNN inference, \emph{Opara} leverages the resource demands of operators to judiciously adjust the operator launch order on GPUs, overlapping the execution of compute-intensive and memory-intensive operators. We implement and open source a prototype of \emph{Opara} based on PyTorch in a \emph{non-intrusive} manner. Extensive prototype experiments with representative DNN and Transformer-based models demonstrate that \emph{Opara} outperforms the default sequential \texttt{CUDA Graph} in PyTorch and the state-of-the-art operator parallelism systems by up to $1.68\times$ and $1.29\times$, respectively, yet with acceptable runtime overhead.

cs.DC↗

arXiv2024

Online Resource Allocation for Edge Intelligence with Colocated Model Retraining and Inference

With edge intelligence, AI models are increasingly pushed to the edge to serve ubiquitous users. However, due to the drift of model, data, and task, AI model deployed at the edge suffers from degraded accuracy in the inference serving phase. Model retraining handles such drifts by periodically retraining the model with newly arrived data. When colocating model retraining and model inference serving for the same model on resource-limited edge servers, a fundamental challenge arises in balancing the resource allocation for model retraining and inference, aiming to maximize long-term inference accuracy. This problem is particularly difficult due to the underlying mathematical formulation being time-coupled, non-convex, and NP-hard. To address these challenges, we introduce a lightweight and explainable online approximation algorithm, named ORRIC, designed to optimize resource allocation for adaptively balancing the accuracy of model training and inference. The competitive ratio of ORRIC outperforms that of the traditional Inference-Only paradigm, especially when data drift persists for a sufficiently lengthy time. This highlights the advantages and applicable scenarios of colocating model retraining and inference. Notably, ORRIC can be translated into several heuristic algorithms for different resource environments. Experiments conducted in real scenarios validate the effectiveness of ORRIC.

cs.LG↗

arXiv2024

Numerical Recovery of the Diffusion Coefficient in Diffusion Equations from Terminal Measurement

In this work, we investigate a numerical procedure for recovering a space-dependent diffusion coefficient in a (sub)diffusion model from the given terminal data, and provide a rigorous numerical analysis of the procedure. By exploiting decay behavior of the observation in time, we establish a novel H{ö}lder type stability estimate for a large terminal time $T$. This is achieved by novel decay estimates of the (fractional) time derivative of the solution. To numerically recover the diffusion coefficient, we employ the standard output least-squares formulation with an $H^1(Ω)$-seminorm penalty, and discretize the regularized problem by the Galerkin finite element method with continuous piecewise linear finite elements in space and backward Euler convolution quadrature in time. Further, we provide an error analysis of discrete approximations, and prove a convergence rate that matches the stability estimate. The derived $L^2(Ω)$ error bound depends explicitly on the noise level, regularization parameter and discretization parameter(s), which gives a useful guideline of the \textsl{a priori} choice of discretization parameters with respect to the noise level in practical implementation. The error analysis is achieved using the conditional stability argument and discrete maximum-norm resolvent estimates. Several numerical experiments are also given to illustrate and complement the theoretical analysis.

math.NA↗

arXiv2024

HarmonyBatch: Batching multi-SLO DNN Inference with Heterogeneous Serverless Functions

Deep Neural Network (DNN) inference on serverless functions is gaining prominence due to its potential for substantial budget savings. Existing works on serverless DNN inference solely optimize batching requests from one application with a single Service Level Objective (SLO) on CPU functions. However, production serverless DNN inference traces indicate that the request arrival rate of applications is surprisingly low, which inevitably causes a long batching time and SLO violations. Hence, there is an urgent need for batching multiple DNN inference requests with diverse SLOs (i.e., multi-SLO DNN inference) in serverless platforms. Moreover, the potential performance and cost benefits of deploying heterogeneous (i.e., CPU and GPU) functions for DNN inference have received scant attention. In this paper, we present HarmonyBatch, a cost-efficient resource provisioning framework designed to achieve predictable performance for multi-SLO DNN inference with heterogeneous serverless functions. Specifically, we construct an analytical performance and cost model of DNN inference on both CPU and GPU functions, by explicitly considering the GPU time-slicing scheduling mechanism and request arrival rate distribution. Based on such a model, we devise a two-stage merging strategy in HarmonyBatch to judiciously batch the multi-SLO DNN inference requests into application groups. It aims to minimize the budget of function provisioning for each application group while guaranteeing diverse performance SLOs of inference applications. We have implemented a prototype of HarmonyBatch on Alibaba Cloud Function Compute. Extensive prototype experiments with representative DNN inference workloads demonstrate that HarmonyBatch can provide predictable performance to serverless DNN inference workloads while reducing the monetary cost by up to 82.9% compared to the state-of-the-art methods.

cs.DC↗

arXiv2024

Solving Elliptic Optimal Control Problems via Neural Networks and Optimality System

In this work, we investigate a neural network based solver for optimal control problems (without / with box constraint) for linear and semilinear second-order elliptic problems. It utilizes a coupled system derived from the first-order optimality system of the optimal control problem, and employs deep neural networks to represent the solutions to the reduced system. We present an error analysis of the scheme, and provide $L^2(Ω)$ error bounds on the state, control and adjoint in terms of neural network parameters (e.g., depth, width, and parameter bounds) and the numbers of sampling points. The main tools in the analysis include offset Rademacher complexity and boundedness and Lipschitz continuity of neural network functions. We present several numerical examples to illustrate the method and compare it with two existing ones.

math.OC↗

arXiv2024

Maximum bound principle and original energy dissipation of arbitrarily high-order rescaled exponential time differencing Runge-Kutta schemes for Allen--Cahn equations

The energy dissipation law and the maximum bound principle are two critical physical properties of the Allen--Cahn equations. While many existing time-stepping methods are known to preserve the energy dissipation law, most apply to a modified form of energy. In this work, we demonstrate that, when the nonlinear term of the Allen--Cahn equation is Lipschitz continuous, a class of arbitrarily high-order exponential time differencing Runge--Kutta (ETDRK) schemes preserve the original energy dissipation property, under a mild step-size constraint. Additionally, we guarantee the Lipschitz condition on the nonlinear term by applying a rescaling post-processing technique, which ensures that the numerical solution unconditionally satisfies the maximum bound principle. Consequently, our proposed schemes maintain both the original energy dissipation law and the maximum bound principle and can achieve arbitrarily high-order accuracy. We also establish an optimal error estimate for the proposed schemes. Some numerical experiments are carried out to verify our theoretical results.

math.NA↗

arXiv2024

Solving Poisson Problems in Polygonal Domains with Singularity Enriched Physics Informed Neural Networks

Physics-Informed Neural Networks (PINNs) are a powerful class of numerical solvers for partial differential equations, employing deep neural networks with successful applications across a diverse set of problems. However, their effectiveness is somewhat diminished when addressing issues involving singularities, such as point sources or geometric irregularities, where the approximations they provide often suffer from reduced accuracy due to the limited regularity of the exact solution. In this work, we investigate PINNs for solving Poisson equations in polygonal domains with geometric singularities and mixed boundary conditions. We propose a novel singularity enriched PINN (SEPINN), by explicitly incorporating the singularity behavior of the analytic solution, e.g., corner singularity, mixed boundary condition and edge singularities, into the ansatz space, and present a convergence analysis of the scheme. We present extensive numerical simulations in two and three-dimensions to illustrate the efficiency of the method, and also a comparative study with several existing neural network based approaches.

math.NA↗

arXiv2024

Inverse Coefficient Problem for One-Dimensional Subdiffusion with Data on Disjoint Sets in Time

In this work we investigate an inverse coefficient problem for the one-dimensional subdiffusion model, which involves a Caputo fractional derivative in time. The inverse problem is to determine two coefficients and multiple parameters (the order, and length of the interval) from one pair of lateral Cauchy data. The lateral Cauchy data are given on disjoint sets in time with a single excitation and the measurement is made on a time sequence located outside the support of the excitation. We prove two uniqueness results for different lateral Cauchy data. The analysis is based on the solution representation, analyticity of the observation and a refined version of inverse Sturm-Liouville theory due to Sini [35]. Our results heavily exploit the memory effect of fractional diffusion for the unique recovery of the coefficients in the model. Several numerical experiments are also presented to complement the analysis.

math.AP↗