arXiv subjects

Wenzhe Wang

Publications and source records attributed to Wenzhe Wang.

10 recordsLinked to original sources

arXiv2026

FlowTrain: Flow-Based Decoupled Training for Industrial-Grade Vision-Language Models

Industrial-grade distributed training of vision-language models (VLMs) remains far less efficient than that of unimodal LLMs. Existing solutions either follow a monolithic design that assigns uniform parallelism to heterogeneous modules or adopt a disaggregated deployment that separates modules while executing them as a batch-synchronized pipeline. In this paper, we highlight that the above solutions are still not sufficient, and VLM training can be further decoupled. To this end, we present FlowTrain, a flow-based decoupled training framework that reformulates VLM training as a producer-consumer dataflow coordinated through a unified memory pool. The encoder and backbone can progress independently over a global virtual address space. Since this execution decoupling fundamentally changes the optimization objective of allocation and scheduling, FlowTrain further introduces a heterogeneous parallel allocator that assigns module-specific parallelism strategies by solving a throughput matching problem. The dynamic packing scheduler is used to construct balanced microbatches at runtime according to the actual LLM-side computation cost. Extensive experiments on real-world workloads show that FlowTrain achieves over 50% MFU and up to 1.7x throughput improvement, narrowing the efficiency gap to LLM-only training.

cs.LG↗

arXiv2025

Effects of Prompt Length on Domain-specific Tasks for Large Language Models

In recent years, Large Language Models have garnered significant attention for their strong performance in various natural language tasks, such as machine translation and question answering. These models demonstrate an impressive ability to generalize across diverse tasks. However, their effectiveness in tackling domain-specific tasks, such as financial sentiment analysis and monetary policy understanding, remains a topic of debate, as these tasks often require specialized knowledge and precise reasoning. To address such challenges, researchers design various prompts to unlock the models' abilities. By carefully crafting input prompts, researchers can guide these models to produce more accurate responses. Consequently, prompt engineering has become a key focus of study. Despite the advancements in both models and prompt engineering, the relationship between the two-specifically, how prompt design impacts models' ability to perform domain-specific tasks-remains underexplored. This paper aims to bridge this research gap.

cs.CL↗

arXiv2024

Handling The Non-Smooth Challenge in Tensor SVD: A Multi-Objective Tensor Recovery Framework

Recently, numerous tensor singular value decomposition (t-SVD)-based tensor recovery methods have shown promise in processing visual data, such as color images and videos. However, these methods often suffer from severe performance degradation when confronted with tensor data exhibiting non-smooth changes. It has been commonly observed in real-world scenarios but ignored by the traditional t-SVD-based methods. In this work, we introduce a novel tensor recovery model with a learnable tensor nuclear norm to address such a challenge. We develop a new optimization algorithm named the Alternating Proximal Multiplier Method (APMM) to iteratively solve the proposed tensor completion model. Theoretical analysis demonstrates the convergence of the proposed APMM to the Karush-Kuhn-Tucker (KKT) point of the optimization problem. In addition, we propose a multi-objective tensor recovery framework based on APMM to efficiently explore the correlations of tensor data across its various dimensions, providing a new perspective on extending the t-SVD-based method to higher-order tensor cases. Numerical experiments demonstrated the effectiveness of the proposed method in tensor completion.

stat.ML↗

arXiv2023

A Novel Tensor Factorization-Based Method with Robustness to Inaccurate Rank Estimation

This study aims to solve the over-reliance on the rank estimation strategy in the standard tensor factorization-based tensor recovery and the problem of a large computational cost in the standard t-SVD-based tensor recovery. To this end, we proposes a new tensor norm with a dual low-rank constraint, which utilizes the low-rank prior and rank information at the same time. In the proposed tensor norm, a series of surrogate functions of the tensor tubal rank can be used to achieve better performance in harness low-rankness within tensor data. It is proven theoretically that the resulting tensor completion model can effectively avoid performance degradation caused by inaccurate rank estimation. Meanwhile, attributed to the proposed dual low-rank constraint, the t-SVD of a smaller tensor instead of the original big one is computed by using a sample trick. Based on this, the total cost at each iteration of the optimization algorithm is reduced to $\mathcal{O}(n^3\log n +kn^3)$ from $\mathcal{O}(n^4)$ achieved with standard methods, where $k$ is the estimation of the true tensor rank and far less than $n$. Our method was evaluated on synthetic and real-world data, and it demonstrated superior performance and efficiency over several existing state-of-the-art tensor completion methods.

cs.LG↗

arXiv2021

PR-Net: Preference Reasoning for Personalized Video Highlight Detection

Personalized video highlight detection aims to shorten a long video to interesting moments according to a user's preference, which has recently raised the community's attention. Current methods regard the user's history as holistic information to predict the user's preference but negating the inherent diversity of the user's interests, resulting in vague preference representation. In this paper, we propose a simple yet efficient preference reasoning framework (PR-Net) to explicitly take the diverse interests into account for frame-level highlight prediction. Specifically, distinct user-specific preferences for each input query frame are produced, presented as the similarity weighted sum of history highlights to the corresponding query frame. Next, distinct comprehensive preferences are formed by the user-specific preferences and a learnable generic preference for more overall highlight measurement. Lastly, the degree of highlight and non-highlight for each query frame is calculated as semantic similarity to its comprehensive and non-highlight preferences, respectively. Besides, to alleviate the ambiguity due to the incomplete annotation, a new bi-directional contrastive loss is proposed to ensure a compact and differentiable metric space. In this way, our method significantly outperforms state-of-the-art methods with a relative improvement of 12% in mean accuracy precision.

cs.CV↗

arXiv2020

Frame Aggregation and Multi-Modal Fusion Framework for Video-Based Person Recognition

Video-based person recognition is challenging due to persons being blocked and blurred, and the variation of shooting angle. Previous research always focused on person recognition on still images, ignoring similarity and continuity between video frames. To tackle the challenges above, we propose a novel Frame Aggregation and Multi-Modal Fusion (FAMF) framework for video-based person recognition, which aggregates face features and incorporates them with multi-modal information to identify persons in videos. For frame aggregation, we propose a novel trainable layer based on NetVLAD (named AttentionVLAD), which takes arbitrary number of features as input and computes a fixed-length aggregation feature based on feature quality. We show that introducing an attention mechanism to NetVLAD can effectively decrease the impact of low-quality frames. For the multi-model information of videos, we propose a Multi-Layer Multi-Modal Attention (MLMA) module to learn the correlation of multi-modality by adaptively updating Gram matrix. Experimental results on iQIYI-VID-2019 dataset show that our framework outperforms other state-of-the-art methods.

cs.CV↗

arXiv2014

Minimum (maximum) rank of tensors and the sign nonsingular tensors

In this paper, we define the minimum (maximum) rank, term rank and the sign nonsingular of tensors. The sufficiency and necessity for the minimum rank of a real tensor to be $1$ is given. And we show that the maximum rank of a tensor is not less than the term rank. We also prove that the minimum rank of a sign nonsingular tensor is not less than the dimension of it. And we get some characterizations of a tensor having sign left or sign right inverses.

math.CO↗

arXiv2014

Some spectral properties of uniform hypergraphs

For a $k$-uniform hypergraph $H$, we obtain some trace formulas for the Laplacian tensor of $H$, which imply that $\sum_{i=1}^nd_i^s$ ($s=1,\ldots,k$) is determined by the Laplacian spectrum of $H$, where $d_1,\ldots,d_n$ is the degree sequence of $H$. Using trace formulas for the Laplacian tensor, we obtain expressions for some coefficients of the Laplacian polynomial of a regular hypergraph. We give some spectral characterizations of odd-bipartite hypergraphs, and give a partial answer to a question posed by Shao et al \cite{ShaoShanWu}. We also give some spectral properties of power hypergraphs, and show that a conjecture posed by Hu et al \cite{HuQiShao} holds under certain conditons.

math.CO↗

arXiv2011

Continuous variable quantum key distribution based on optical entangled states without signal modulation

In this paper, we present the first experimental demonstration on continuous variable quantum key distribution using determinant Einstein-Podolsky-Rosen entangled states of optical field. By means of the instantaneous measurements of the quantum fluctuations of optical modes respectively distributed at sender and receiver, the random bits of secret key are obtained without the need for signal modulation. The post-selection boundaries for the presented entanglement-based scheme against both Gaussian collective and individual attacks are theoretically concluded. The final secret key rates of 84 kbits/s and 3 kbits/s are completed under the collective attack for the transmission efficiency of 80% and 40%, respectively.

quant-ph↗

arXiv2007

Experimental investigation to influence of pump phase fluctuation on phase-correlation of output optical fields from a non-degenerate parametric oscillator

The influence of the phase fluctuation of the pump laser on the phase-correlation between the signal and idler modes of the output fields from as non-degenerate optical parametric oscillator operating above oscillation threshold was experimentally investigated. The noise spectra of the intensity-difference and the phase-sum of the entangled optical beams were measured with a pair of unbalanced fiber Match-Zehnder interferometers specifically designed. The experimental results proved the previously theoretical prediction and are in reasonable agreement with the calculation based on semiclassical theory involving the phase fluctuation of pump laser.

quant-ph↗