arXiv subjects

Yang Zheng

Publications and source records attributed to Yang Zheng.

At least 73 records · Page 4Linked to original sources

arXiv2025

CustomContrast: A Multilevel Contrastive Perspective For Subject-Driven Text-to-Image Customization

Subject-driven text-to-image (T2I) customization has drawn significant interest in academia and industry. This task enables pre-trained models to generate novel images based on unique subjects. Existing studies adopt a self-reconstructive perspective, focusing on capturing all details of a single image, which will misconstrue the specific image's irrelevant attributes (e.g., view, pose, and background) as the subject intrinsic attributes. This misconstruction leads to both overfitting or underfitting of irrelevant and intrinsic attributes of the subject, i.e., these attributes are over-represented or under-represented simultaneously, causing a trade-off between similarity and controllability. In this study, we argue an ideal subject representation can be achieved by a cross-differential perspective, i.e., decoupling subject intrinsic attributes from irrelevant attributes via contrastive learning, which allows the model to focus more on intrinsic attributes through intra-consistency (features of the same subject are spatially closer) and inter-distinctiveness (features of different subjects have distinguished differences). Specifically, we propose CustomContrast, a novel framework, which includes a Multilevel Contrastive Learning (MCL) paradigm and a Multimodal Feature Injection (MFI) Encoder. The MCL paradigm is used to extract intrinsic features of subjects from high-level semantics to low-level appearance through crossmodal semantic contrastive learning and multiscale appearance contrastive learning. To facilitate contrastive learning, we introduce the MFI encoder to capture cross-modal representations. Extensive experiments show the effectiveness of CustomContrast in subject similarity and text controllability.

cs.CV↗

arXiv2025

Road Traffic Sign Recognition method using Siamese network Combining Efficient-CNN based Encoder

Traffic signs recognition (TSR) plays an essential role in assistant driving and intelligent transportation system. However, the noise of complex environment may lead to motion-blur or occlusion problems, which raise the tough challenge to real-time recognition with high accuracy and robust. In this article, we propose IECES-network which with improved encoders and Siamese net. The three-stage approach of our method includes Efficient-CNN based encoders, Siamese backbone and the fully-connected layers. We firstly use convolutional encoders to extract and encode the traffic sign features of augmented training samples and standard images. Then, we design the Siamese neural network with Efficient-CNN based encoder and contrastive loss function, which can be trained to improve the robustness of TSR problem when facing the samples of motion-blur and occlusion by computing the distance between inputs and templates. Additionally, the template branch of the proposed network can be stopped when executing the recognition tasks after training to raise the process speed of our real-time model, and alleviate the computational resource and parameter scale. Finally, we recombined the feature code and a fully-connected layer with SoftMax function to classify the codes of samples and recognize the category of traffic signs. The results of experiments on the Tsinghua-Tencent 100K dataset and the German Traffic Sign Recognition Benchmark dataset demonstrate the performance of the proposed IECESnetwork. Compared with other state-of-the-art methods, in the case of motion-blur and occluded environment, the proposed method achieves competitive performance precision-recall and accuracy metric average is 88.1%, 86.43% and 86.1% with a 2.9M lightweight scale, respectively. Moreover, processing time of our model is 0.1s per frame, of which the speed is increased by 1.5 times compared with existing methods.

cs.CV↗

arXiv2025

A Bundle-based Augmented Lagrangian Framework: Algorithm, Convergence, and Primal-dual Principles

We propose a new bundle-based augmented Lagrangian framework for solving constrained convex problems. Unlike the classical (inexact) augmented Lagrangian method (ALM) that has a nested double-loop structure, our framework features a $\textit{single-loop}$ process. Motivated by the proximal bundle method (PBM), we use a $\textit{bundle}$ of past iterates to approximate the subproblem in ALM to get a computationally efficient update at each iteration. We establish sub-linear convergences for primal feasibility, primal cost values, and dual iterates under mild assumptions. With further regularity conditions, such as quadratic growth, our algorithm enjoys $\textit{linear}$ convergences. Importantly, this linear convergence can happen for a class of conic optimization problems, including semidefinite programs. Our proof techniques leverage deep connections with inexact ALM and primal-dual principles with PBM.

math.OC↗

arXiv2024

Simultaneous imaging of bidirectional guided waves enables synchronous probing of mechanical anisotropy, local blood pressure, and stress in arteries

Arterial biomechanical indicators have long been recognized as fundamental contributors to the physiology and pathology of cardiovascular systems. Probing the multiple biomechanical parameters of arteries simultaneously at different time points within one cardiac cycle is of great importance but remains challenging. Here we report an ultrasound elastography method to quantify arterial anisotropic stiffness, mechanical stresses in arterial wall, and local blood pressure in a single measurement. With programmed acoustic radiation force, arterial axial and circumferential guided elastic waves were induced simultaneously and recorded at multiple time points within one cardiac cycle. Then a mechanical model incorporating acoustoelasticity and viscoelasticity of arteries was proposed to quantitatively predict the correlation of arterial guided elastic waves with arterial biomechanical parameters. Our experimental design and biomechanical model lead to an elastography method to interrogate the variation of blood pressure, arterial bidirectional stiffnesses and mechanical stresses in arterial walls with time. In vivo experiments were performed on healthy young, normotensive older and hypertensive older volunteers. The results demonstrate that the reported method can find applications in understanding aging of cardiovascular system and diagnosis of cardiovascular diseases.

physics.med-ph↗

arXiv2024

PLPP: Prompt Learning with Perplexity Is Self-Distillation for Vision-Language Models

Pre-trained Vision-Language (VL) models such as CLIP have demonstrated their excellent performance across numerous downstream tasks. A recent method, Context Optimization (CoOp), further improves the performance of VL models on downstream tasks by introducing prompt learning. CoOp optimizes a set of learnable vectors, aka prompt, and freezes the whole CLIP model. However, relying solely on CLIP loss to fine-tune prompts can lead to models that are prone to overfitting on downstream task. To address this issue, we propose a plug-in prompt-regularization method called PLPP (Prompt Learning with PerPlexity), which use perplexity loss to regularize prompt learning. PLPP designs a two-step operation to compute the perplexity for prompts: (a) calculating cosine similarity between the weight of the embedding layer and prompts to get labels, (b) introducing a language model (LM) head that requires no training behind text encoder to output word probability distribution. Meanwhile, we unveil that the essence of PLPP is inherently a form of self-distillation. To further prevent overfitting as well as to reduce the additional computation introduced by PLPP, we turn the hard label to soft label and choose top-$k$ values for calculating the perplexity loss. For accelerating model convergence, we introduce mutual self-distillation learning, that is perplexity and inverted perplexity loss. The experiments conducted on four classification tasks indicate that PLPP exhibits superior performance compared to existing methods.

cs.CL↗

arXiv2024

Willems' Fundamental Lemma for Nonlinear Systems with Koopman Linear Embedding

Koopman operator theory and Willems' fundamental lemma both can provide (approximated) data-driven linear representation for nonlinear systems. However, choosing lifting functions for the Koopman operator is challenging, and the quality of the data-driven model from Willems' fundamental lemma has no guarantee for general nonlinear systems. In this paper, we extend Willems' fundamental lemma for a class of nonlinear systems that admit a Koopman linear embedding. We first characterize the relationship between the trajectory space of a nonlinear system and that of its Koopman linear embedding. We then prove that the trajectory space of Koopman linear embedding can be formed by a linear combination of rich-enough trajectories from the nonlinear system. Combining these two results leads to a data-driven representation of the nonlinear system, which bypasses the need for the lifting functions and thus eliminates the associated bias errors. Our results illustrate that both the width (more trajectories) and depth (longer trajectories) of the trajectory library are important to ensure the accuracy of the data-driven model.

math.OC↗

arXiv2024

Inexact Augmented Lagrangian Methods for Conic Programs: Quadratic Growth and Linear Convergence

Augmented Lagrangian Methods (ALMs) are widely employed in solving constrained optimizations, and some efficient solvers are developed based on this framework. Under the quadratic growth assumption, it is known that the dual iterates and the Karush-Kuhn-Tucker (KKT) residuals of ALMs applied to semidefinite programs (SDPs) converge linearly. In contrast, the convergence rate of the primal iterates has remained elusive. In this paper, we resolve this challenge by establishing new $\textit{quadratic growth}$ and $\textit{error bound}$ properties for primal and dual SDPs under the strict complementarity condition. Our main results reveal that both primal and dual iterates of the ALMs converge linearly contingent solely upon the assumption of strict complementarity and a bounded solution set. This finding provides a positive answer to an open question regarding the asymptotically linear convergence of the primal iterates of ALMs applied to semidefinite optimization.

math.OC↗

arXiv2024

On equivalence relations induced by locally compact TSI Polish groups admitting open identity component

For a Polish group $G$, let $E(G)$ be the right coset equivalence relation $G^ω/c(G)$, where $c(G)$ is the group of all convergent sequences in $G$. We prove a Rigid theorem on locally compact TSI Polish groups admitting open identity component, as follows: Let $G$ be a locally compact TSI Polish group such that $G_0$ is open in $G$, and let $H$ be a nontrivial pro-Lie TSI Polish group. Then $E(G)\leq_BE(H)$ iff there exists a continuous homomorphism $ϕ:G_0\to H_0$ satisfying the following conditions: (i) $\ker(ϕ)$ is non-archimedean; (ii) $ϕ{\rm Inn}_G(G_0)\subseteq\overline{{\rm Inn}_H(H_0)ϕ}$ under pointwise convergence topology. An application of the Rigid theorem yields a negative answer to Question 7.5 of [2].

math.LO↗

arXiv2024

Training Overhead Ratio: A Practical Reliability Metric for Large Language Model Training Systems

Large Language Models (LLMs) are revolutionizing the AI industry with their superior capabilities. Training these models requires large-scale GPU clusters and significant computing time, leading to frequent failures that significantly increase training costs. Despite its significance, this field lacks a metric for evaluating reliability. In this work, we introduce a novel reliability metric called \emph{Training Overhead Ratio} (TOR) to evaluate the reliability of fault-tolerant LLM training systems. TOR is defined as the ratio of optimal training time to the observed training time of a system, serving as a practical tool for users to estimate the actual time required to train an LLM on a given system. Furthermore, our investigation identifies the key factor for enhancing reliability and present TOR equations for various types of failures encountered in practice.

cs.DC↗

arXiv2024

Decentralized Robust Data-driven Predictive Control for Smoothing Mixed Traffic Flow

In a mixed traffic with connected automated vehicles (CAVs) and human-driven vehicles (HDVs) coexisting, data-driven predictive control of CAVs promises system-wide traffic performance improvements. Yet, most existing approaches focus on a centralized setup, which is not computationally scalable while failing to protect data privacy. The robustness against unknown disturbances has not been well addressed either, causing safety concerns. In this paper, we propose a decentralized robust DeeP-LCC (Data-EnablEd Predictive Leading Cruise Control) approach for CAVs to smooth mixed traffic flow. In particular, each CAV computes its control input based on locally available data from its involved subsystem. Meanwhile, the interaction between neighboring subsystems is modeled as a bounded disturbance, for which appropriate estimation methods are proposed. Then, we formulate a robust optimization problem and present its tractable computational solutions. Compared with the centralized formulation, our method greatly reduces computation burden with better safety performance, while naturally preserving data privacy. Extensive traffic simulations validate its wave-dampening ability, safety performance, and computational benefits.

math.OC↗

arXiv2024

On equivalence relations induced by Polish groups

The motivation of this article is to introduce a kind of orbit equivalence relations which can well describe structures and properties of Polish groups from the perspective of Borel reducibility. Given a Polish group $G$, let $E(G)$ be the right coset equivalence relation $G^ω/c(G)$, where $c(G)$ is the group of all convergent sequences in $G$. Let $G$ be a Polish group. (1) $G$ is a discrete countable group containing at least two elements iff $E(G)\sim_BE_0$; (2) if $G$ is TSI uncountable non-archimedean, then $E(G)\sim_BE_0^ω$; (3) $G$ is non-archimedean iff $E(G)\le_B=^+$; (4) if $H$ is a CLI Polish group but $G$ is not, then $E(G)\not\le_BE(H)$; (5) if $H$ is a non-archimedean Polish group but $G$ is not, then $E(G)\not\le_BE(H)$. The notion of $α$-l.m.-unbalanced Polish group for $α<ω_1$ is introduced. Let $G,H$ be Polish groups, $0<α<ω_1$. If $G$ is $α$-l.m.-unbalanced but $H$ is not, then $E(G)\not\le_B E(H)$. For TSI Polish groups, the existence of Borel reduction is transformed into the existence of a well-behaved continuous mapping between topological groups. As its applications, for any Polish group $G$, let $G_0$ be the connected component of the identity element $1_G$. Let $G$ and $H$ be two separable TSI Lie groups. If $E(G)\le_BE(H)$, then there exists a continuous locally injective map $S:G_0\to H_0$. Moreover, if $G_0,H_0$ are abelian, $S$ is a group homomorphism. In particular, for $c_0,e_0,c_1,e_1\in{\mathbb N}$, $E({\mathbb R}^{c_0}\times{\mathbb T}^{e_0})\le_BE({\mathbb R}^{c_1}\times{\mathbb T}^{e_1})$ iff $e_0\le e_1$ and $c_0+e_0\le c_1+e_1$.

math.LO↗

arXiv2024

Verification of Geometric Robustness of Neural Networks via Piecewise Linear Approximation and Lipschitz Optimisation

We address the problem of verifying neural networks against geometric transformations of the input image, including rotation, scaling, shearing, and translation. The proposed method computes provably sound piecewise linear constraints for the pixel values by using sampling and linear approximations in combination with branch-and-bound Lipschitz optimisation. The method obtains provably tighter over-approximations of the perturbation region than the present state-of-the-art. We report results from experiments on a comprehensive set of verification benchmarks on MNIST and CIFAR10. We show that our proposed implementation resolves up to 32% more verification cases than present approaches.

cs.LG↗

arXiv2024

Fault-Tolerant Hybrid-Parallel Training at Scale with Reliable and Efficient In-memory Checkpointing

To efficiently scale large model (LM) training, researchers transition from data parallelism (DP) to hybrid parallelism (HP) on GPU clusters, which frequently experience hardware and software failures. Existing works introduce in-memory checkpointing optimizations that snapshot parameters to device memory for rapid failure recovery. However, these methods introduce severe resource competition between checkpointing and training, which can work under DP but can hardly scale under resource-intensive HP. To ensure low checkpointing overhead for hybrid-parallel training, this paper introduces a distributed in-memory checkpointing system with near-zero in-memory saving overhead. It strives from two aspects to mitigate the on-host resource competition caused by in-memory checkpointing: (1) It introduces Hierarchical Asynchronous Snapshotting Coordination in the checkpoint saving stage. This approach uses three-level asynchronous on-device scheduling to enhance parallelism between snapshotting and training, thereby minimizing snapshotting overhead. (2) It proposes Hybrid In-memory Checkpoint Protection to enhance checkpoint completeness during hardware failures. Unlike methods that require inter-node communications, which may block training under HP, it creates intra-node redundancy with efficient resource utilization, protecting training against hardware failures with minimal overhead. With these methods, this work enables fast restart for failed HP training with Distributed In-memory Checkpoint Loading, bypassing inefficiencies in NFS reads. In our evaluation, we achieve zero in-memory checkpoint saving overhead on Frontier while training Llama-2-34B on 256 MI250X devices (512 GPUs).

cs.DC↗

arXiv2024

Boosting Diffusion Model for Spectrogram Up-sampling in Text-to-speech: An Empirical Study

Scaling text-to-speech (TTS) with autoregressive language model (LM) to large-scale datasets by quantizing waveform into discrete speech tokens is making great progress to capture the diversity and expressiveness in human speech, but the speech reconstruction quality from discrete speech token is far from satisfaction depending on the compressed speech token compression ratio. Generative diffusion models trained with score-matching loss and continuous normalized flow trained with flow-matching loss have become prominent in generation of images as well as speech. LM based TTS systems usually quantize speech into discrete tokens and generate these tokens autoregressively, and finally use a diffusion model to up sample coarse-grained speech tokens into fine-grained codec features or mel-spectrograms before reconstructing into waveforms with vocoder, which has a high latency and is not realistic for real time speech applications. In this paper, we systematically investigate varied diffusion models for up sampling stage, which is the main bottleneck for streaming synthesis of LM and diffusion-based architecture, we present the model architecture, objective and subjective metrics to show quality and efficiency improvement.

eess.AS↗

arXiv2024

Benign Nonconvex Landscapes in Optimal and Robust Control, Part II: Extended Convex Lifting

Many optimal and robust control problems are nonconvex and potentially nonsmooth in their policy optimization forms. In Part II of this paper, we introduce a new and unified Extended Convex Lifting (ECL) framework to reveal hidden convexity in classical optimal and robust control problems from a modern optimization perspective. Our ECL offers a bridge between nonconvex policy optimization and convex reformulations, enabling convex analysis for nonconvex problems. Despite non-convexity and non-smoothness, the existence of an ECL not only reveals that minimizing the original function is equivalent to a convex problem but also certifies a class of first-order non-degenerate stationary points to be globally optimal. Therefore, no spurious stationarity exists in the set of non-degenerate policies. This ECL framework can cover many benchmark control problems, including state feedback linear quadratic regulator (LQR), dynamic output feedback linear quadratic Gaussian (LQG) control, and $\mathcal{H}_\infty$ robust control. ECL can also handle a class of distributed control problems when the notion of quadratic invariance (QI) holds. We further show that all static stabilizing policies are non-degenerate for state feedback LQR and $\mathcal{H}_\infty$ control under standard assumptions. We believe that the new ECL framework may be of independent interest for analyzing nonconvex problems beyond control.

math.OC↗

arXiv2024

Policy Optimization in Control: Geometry and Algorithmic Implications

This survey explores the geometric perspective on policy optimization within the realm of feedback control systems, emphasizing the intrinsic relationship between control design and optimization. By adopting a geometric viewpoint, we aim to provide a nuanced understanding of how various ``complete parameterization'' -- referring to the policy parameters together with its Riemannian geometry -- of control design problems, influence stability and performance of local search algorithms. The paper is structured to address key themes such as policy parameterization, the topology and geometry of stabilizing policies, and their implications for various (non-convex) dynamic performance measures. We focus on a few iconic control design problems, including the Linear Quadratic Regulator (LQR), Linear Quadratic Gaussian (LQG) control, and $\mathcal{H}_\infty$ control. In particular, we first discuss the topology and Riemannian geometry of stabilizing policies, distinguishing between their static and dynamic realizations. Expanding on this geometric perspective, we then explore structural properties of the aforementioned performance measures and their interplay with the geometry of stabilizing policies in presence of policy constraints; along the way, we address issues such as spurious stationary points, symmetries of dynamic feedback policies, and (non-)smoothness of the corresponding performance measures. We conclude the survey with algorithmic implications of policy optimization in feedback design.

math.OC↗

arXiv2024

PhysAvatar: Learning the Physics of Dressed 3D Avatars from Visual Observations

Modeling and rendering photorealistic avatars is of crucial importance in many applications. Existing methods that build a 3D avatar from visual observations, however, struggle to reconstruct clothed humans. We introduce PhysAvatar, a novel framework that combines inverse rendering with inverse physics to automatically estimate the shape and appearance of a human from multi-view video data along with the physical parameters of the fabric of their clothes. For this purpose, we adopt a mesh-aligned 4D Gaussian technique for spatio-temporal mesh tracking as well as a physically based inverse renderer to estimate the intrinsic material properties. PhysAvatar integrates a physics simulator to estimate the physical parameters of the garments using gradient-based optimization in a principled manner. These novel capabilities enable PhysAvatar to create high-quality novel-view renderings of avatars dressed in loose-fitting clothes under motions and lighting conditions not seen in the training data. This marks a significant advancement towards modeling photorealistic digital humans using physically based inverse rendering with physics in the loop. Our project website is at: https://qingqing-zhao.github.io/PhysAvatar

cs.GR↗

arXiv2024

Multimodal Fusion Method with Spatiotemporal Sequences and Relationship Learning for Valence-Arousal Estimation

This paper presents our approach for the VA (Valence-Arousal) estimation task in the ABAW6 competition. We devised a comprehensive model by preprocessing video frames and audio segments to extract visual and audio features. Through the utilization of Temporal Convolutional Network (TCN) modules, we effectively captured the temporal and spatial correlations between these features. Subsequently, we employed a Transformer encoder structure to learn long-range dependencies, thereby enhancing the model's performance and generalization ability. Our method leverages a multimodal data fusion approach, integrating pre-trained audio and video backbones for feature extraction, followed by TCN-based spatiotemporal encoding and Transformer-based temporal information capture. Experimental results demonstrate the effectiveness of our approach, achieving competitive performance in VA estimation on the AffWild2 dataset.

cs.CV↗