arXiv subjects

Jianzhi Liu

Publications and source records attributed to Jianzhi Liu.

6 recordsLinked to original sources

arXiv2024

ALF: Adaptive Label Finetuning for Scene Graph Generation

Scene Graph Generation (SGG) endeavors to predict the relationships between subjects and objects in a given image. Nevertheless, the long-tail distribution of relations often leads to biased prediction on coarse labels, presenting a substantial hurdle in SGG. To address this issue, researchers focus on unbiased SGG and introduce data transfer methods to transfer coarse-grained predicates into fine-grained ones across the entire dataset. However, these methods encounter two primary challenges: 1) They overlook the inherent context constraints imposed by subject-object pairs, leading to erroneous relations transfer. 2) Additional retraining process are required after the data transfer, which incurs substantial computational costs. To overcome these limitations, we introduce the first plug-and-play one-stage data transfer pipeline in SGG, termed Adaptive Label Finetuning (ALF), which eliminates the need for extra retraining sessions and meanwhile significantly enhance models' relation recognition capability across various SGG benchmark approaches. Specifically, ALF consists of two components: Adaptive Label Construction (ALC) and Adaptive Iterative Learning (AIL). By imposing Predicate-Context Constraints within relation space, ALC adaptively re-ranks and selects candidate relations in reference to model's predictive logits utilizing the Restriction-Based Judgment techniques, achieving robust relation transfer. Supervised with labels transferred by ALC, AIL iteratively finetunes the SGG models in an auto-regressive manner, which mitigates the substantial computational costs arising from the retraining process. Extensive experiments demonstrate that ALF achieves a 16% improvement in mR@100 compared to the typical SGG method Motif, with only a 6% increase in calculation costs compared to the state-of-the-art method IETrans.

cs.CV↗

arXiv2024

AICL: Action In-Context Learning for Video Diffusion Model

The open-domain video generation models are constrained by the scale of the training video datasets, and some less common actions still cannot be generated. Some researchers explore video editing methods and achieve action generation by editing the spatial information of the same action video. However, this method mechanically generates identical actions without understanding, which does not align with the characteristics of open-domain scenarios. In this paper, we propose AICL, which empowers the generative model with the ability to understand action information in reference videos, similar to how humans do, through in-context learning. Extensive experiments demonstrate that AICL effectively captures the action and achieves state-of-the-art generation performance across three typical video diffusion models on five metrics when using randomly selected categories from non-training datasets.

cs.CV↗

arXiv2024

Dynamic Generation of Personalities with Large Language Models

In the realm of mimicking human deliberation, large language models (LLMs) show promising performance, thereby amplifying the importance of this research area. Deliberation is influenced by both logic and personality. However, previous studies predominantly focused on the logic of LLMs, neglecting the exploration of personality aspects. In this work, we introduce Dynamic Personality Generation (DPG), a dynamic personality generation method based on Hypernetworks. Initially, we embed the Big Five personality theory into GPT-4 to form a personality assessment machine, enabling it to evaluate characters' personality traits from dialogues automatically. We propose a new metric to assess personality generation capability based on this evaluation method. Then, we use this personality assessment machine to evaluate dialogues in script data, resulting in a personality-dialogue dataset. Finally, we fine-tune DPG on the personality-dialogue dataset. Experiments prove that DPG's personality generation capability is stronger after fine-tuning on this dataset than traditional fine-tuning methods, surpassing prompt-based GPT-4.

cs.CL↗

arXiv2024

SSP: A Simple and Safe automatic Prompt engineering method towards realistic image synthesis on LVM

Recently, text-to-image (T2I) synthesis has undergone significant advancements, particularly with the emergence of Large Language Models (LLM) and their enhancement in Large Vision Models (LVM), greatly enhancing the instruction-following capabilities of traditional T2I models. Nevertheless, previous methods focus on improving generation quality but introduce unsafe factors into prompts. We explore that appending specific camera descriptions to prompts can enhance safety performance. Consequently, we propose a simple and safe prompt engineering method (SSP) to improve image generation quality by providing optimal camera descriptions. Specifically, we create a dataset from multi-datasets as original prompts. To select the optimal camera, we design an optimal camera matching approach and implement a classifier for original prompts capable of automatically matching. Appending camera descriptions to original prompts generates optimized prompts for further LVM image generation. Experiments demonstrate that SSP improves semantic consistency by an average of 16% compared to others and safety metrics by 48.9%.

cs.CV↗

arXiv2023

F3-Pruning: A Training-Free and Generalized Pruning Strategy towards Faster and Finer Text-to-Video Synthesis

Recently Text-to-Video (T2V) synthesis has undergone a breakthrough by training transformers or diffusion models on large-scale datasets. Nevertheless, inferring such large models incurs huge costs.Previous inference acceleration works either require costly retraining or are model-specific.To address this issue, instead of retraining we explore the inference process of two mainstream T2V models using transformers and diffusion models.The exploration reveals the redundancy in temporal attention modules of both models, which are commonly utilized to establish temporal relations among frames.Consequently, we propose a training-free and generalized pruning strategy called F3-Pruning to prune redundant temporal attention weights.Specifically, when aggregate temporal attention values are ranked below a certain ratio, corresponding weights will be pruned.Extensive experiments on three datasets using a classic transformer-based model CogVideo and a typical diffusion-based model Tune-A-Video verify the effectiveness of F3-Pruning in inference acceleration, quality assurance and broad applicability.

cs.CV↗

arXiv2019

Intelligent Surveillance of World Health Organization (WHO) Integrated Disease Surveillance and Response (IDSR) Data in Cameroon Using Multivariate Cross-Correlation

As developing countries continue to face challenges associated with infectious diseases, the need to improve infrastructure to systematically collect data which can be used to understand their outbreak patterns becomes more critical. The World Health Organization (WHO) Integrated Disease Surveillance and Response (IDSR) strategy seeks to drive the systematic collection of surveillance data to strengthen district-level reporting and to translate them into public health actions. Since the analysis of this surveillance data at the central levels of government in many developing nations has traditionally not included advanced analytics, there are opportunities for the development and exploration of computational approaches that can provide proactive insights and improve general health outcomes of infectious disease outbreaks. We propose and demonstrate a multivariate time series cross-correlation analysis as a foundational step towards gaining insight on infectious disease patterns via the pairwise computation of weighted cross-correlation scores for a specified disease across different health districts using surveillance data from Cameroon. Following the computation of weighted cross-correlation scores, we apply an anomaly detection algorithm to assess how outbreak alarm patterns align in highly correlated health districts. We demonstrate how multivariate cross-correlation analysis of weekly surveillance data can provide insight into infectious disease incidence patterns in Cameroon by identifying highly correlated health districts for a given disease. We further demonstrate scenarios in which identification of highly correlated districts aligns with alarms flagged using a standard anomaly detection algorithm, hinting at the potential of end to end solutions combining anomaly detection algorithms for flagging alarms in combination with multivariate cross-correlation analysis.

stat.AP↗