arXiv subjects

Jie Wu

Publications and source records attributed to Jie Wu.

At least 145 records · Page 8Linked to original sources

arXiv2025

DreamFuse: Adaptive Image Fusion with Diffusion Transformer

Image fusion seeks to seamlessly integrate foreground objects with background scenes, producing realistic and harmonious fused images. Unlike existing methods that directly insert objects into the background, adaptive and interactive fusion remains a challenging yet appealing task. It requires the foreground to adjust or interact with the background context, enabling more coherent integration. To address this, we propose an iterative human-in-the-loop data generation pipeline, which leverages limited initial data with diverse textual prompts to generate fusion datasets across various scenarios and interactions, including placement, holding, wearing, and style transfer. Building on this, we introduce DreamFuse, a novel approach based on the Diffusion Transformer (DiT) model, to generate consistent and harmonious fused images with both foreground and background information. DreamFuse employs a Positional Affine mechanism to inject the size and position of the foreground into the background, enabling effective foreground-background interaction through shared attention. Furthermore, we apply Localized Direct Preference Optimization guided by human feedback to refine DreamFuse, enhancing background consistency and foreground harmony. DreamFuse achieves harmonious fusion while generalizing to text-driven attribute editing of the fused results. Experimental results demonstrate that our method outperforms state-of-the-art approaches across multiple metrics.

cs.CV↗

arXiv2025

A Convex and Global Solution for the P$n$P Problem in 2D Forward-Looking Sonar

The perspective-$n$-point (P$n$P) problem is important for robotic pose estimation. It is well studied for optical cameras, but research is lacking for 2D forward-looking sonar (FLS) in underwater scenarios due to the vastly different imaging principles. In this paper, we demonstrate that, despite the nonlinearity inherent in sonar image formation, the P$n$P problem for 2D FLS can still be effectively addressed within a point-to-line (PtL) 3D registration paradigm through orthographic approximation. The registration is then resolved by a duality-based optimal solver, ensuring the global optimality. For coplanar cases, a null space analysis is conducted to retrieve the solutions from the dual formulation, enabling the methods to be applied to more general cases. Extensive simulations have been conducted to systematically evaluate the performance under different settings. Compared to non-reprojection-optimized state-of-the-art (SOTA) methods, the proposed approach achieves significantly higher precision. When both methods are optimized, ours demonstrates comparable or slightly superior precision.

cs.RO↗

arXiv2025

Sufficient conditions for a graph with minimum degree to have a component factor

Let $\mathcal{T}_{\frac{k}{r}}$ denote the set of trees $T$ such that $i(T-S)\leq\frac{k}{r}|S|$ for any $S\subset V(T)$ and for any $e\in E(T)$ there exists a set $S^{*}\subset V(T)$ with $i((T-e)-S^{*})>\frac{k}{r}|S^{*}|$, where $r<k$ are two positive integers. A $\{C_{2i+1},T:1\leq i<\frac{r}{k-r},T\in\mathcal{T}_{\frac{k}{r}}\}$-factor of a graph $G$ is a spanning subgraph of $G$, in which every component is isomorphic to an element in $\{C_{2i+1},T:1\leq i<\frac{r}{k-r},T\in\mathcal{T}_{\frac{k}{r}}\}$. Let $A(G)$ and $Q(G)$ denote the adjacency matrix and the signless Laplacian matrix of $G$, respectively. The adjacency spectral radius and the signless Laplacian spectral radius of $G$, denoted by $ρ(G)$ and $q(G)$, are the largest eigenvalues of $A(G)$ and $Q(G)$, respectively. In this paper, we study the connections between the spectral radius and the existence of a $\{C_{2i+1},T:1\leq i<\frac{r}{k-r},T\in\mathcal{T}_{\frac{k}{r}}\}$-factor in a graph. We first establish a tight sufficient condition involving the adjacency spectral radius to guarantee the existence of a $\{C_{2i+1},T:1\leq i<\frac{r}{k-r},T\in\mathcal{T}_{\frac{k}{r}}\}$-factor in a graph. Then we propose a tight signless Laplacian spectral radius condition for the existence of a $\{C_{2i+1},T:1\leq i<\frac{r}{k-r},T\in\mathcal{T}_{\frac{k}{r}}\}$-factor in a graph.

math.CO↗

arXiv2025

Calculating Higher Digraph Homotopy Groups

We give the first tractable and systematic examples of nontrivial higher digraph homotopy groups. To do this we define relative digraph homotopy groups and show these satisfy a long exact sequence analogous to the relative homotopy groups of spaces. We then define digraph suspension and Hurewicz homomorphisms and show they commute with each other. The existence of nontrivial digraph homotopy groups then reduces to the existence of corresponding groups in the degree 1 path homology of digraphs.

math.AT↗

arXiv2025

Step-Video-TI2V Technical Report: A State-of-the-Art Text-Driven Image-to-Video Generation Model

We present Step-Video-TI2V, a state-of-the-art text-driven image-to-video generation model with 30B parameters, capable of generating videos up to 102 frames based on both text and image inputs. We build Step-Video-TI2V-Eval as a new benchmark for the text-driven image-to-video task and compare Step-Video-TI2V with open-source and commercial TI2V engines using this dataset. Experimental results demonstrate the state-of-the-art performance of Step-Video-TI2V in the image-to-video generation task. Both Step-Video-TI2V and Step-Video-TI2V-Eval are available at https://github.com/stepfun-ai/Step-Video-TI2V.

cs.CV↗

arXiv2025

Seedream 2.0: A Native Chinese-English Bilingual Image Generation Foundation Model

Rapid advancement of diffusion models has catalyzed remarkable progress in the field of image generation. However, prevalent models such as Flux, SD3.5 and Midjourney, still grapple with issues like model bias, limited text rendering capabilities, and insufficient understanding of Chinese cultural nuances. To address these limitations, we present Seedream 2.0, a native Chinese-English bilingual image generation foundation model that excels across diverse dimensions, which adeptly manages text prompt in both Chinese and English, supporting bilingual image generation and text rendering. We develop a powerful data system that facilitates knowledge integration, and a caption system that balances the accuracy and richness for image description. Particularly, Seedream is integrated with a self-developed bilingual large language model as a text encoder, allowing it to learn native knowledge directly from massive data. This enable it to generate high-fidelity images with accurate cultural nuances and aesthetic expressions described in either Chinese or English. Beside, Glyph-Aligned ByT5 is applied for flexible character-level text rendering, while a Scaled ROPE generalizes well to untrained resolutions. Multi-phase post-training optimizations, including SFT and RLHF iterations, further improve the overall capability. Through extensive experimentation, we demonstrate that Seedream 2.0 achieves state-of-the-art performance across multiple aspects, including prompt-following, aesthetics, text rendering, and structural correctness. Furthermore, Seedream 2.0 has been optimized through multiple RLHF iterations to closely align its output with human preferences, as revealed by its outstanding ELO score. In addition, it can be readily adapted to an instruction-based image editing model, such as SeedEdit, with strong editing capability that balances instruction-following and image consistency.

cs.CV↗

arXiv2025

ResAdapter: Domain Consistent Resolution Adapter for Diffusion Models

Recent advancement in text-to-image models (e.g., Stable Diffusion) and corresponding personalized technologies (e.g., DreamBooth and LoRA) enables individuals to generate high-quality and imaginative images. However, they often suffer from limitations when generating images with resolutions outside of their trained domain. To overcome this limitation, we present the Resolution Adapter (ResAdapter), a domain-consistent adapter designed for diffusion models to generate images with unrestricted resolutions and aspect ratios. Unlike other multi-resolution generation methods that process images of static resolution with complex post-process operations, ResAdapter directly generates images with the dynamical resolution. Especially, after learning a deep understanding of pure resolution priors, ResAdapter trained on the general dataset, generates resolution-free images with personalized diffusion models while preserving their original style domain. Comprehensive experiments demonstrate that ResAdapter with only 0.5M can process images with flexible resolutions for arbitrary diffusion models. More extended experiments demonstrate that ResAdapter is compatible with other modules (e.g., ControlNet, IP-Adapter and LCM-LoRA) for image generation across a broad range of resolutions, and can be integrated into other multi-resolution model (e.g., ElasticDiffusion) for efficiently generating higher-resolution images. Project link is https://res-adapter.github.io

cs.CV↗

arXiv2025

Identification of Stochastic Gravitational Wave Backgrounds from Cosmic String Using Machine Learning

Cosmic strings play a crucial role in enhancing our understanding of the fundamental structure and evolution of the universe, unifying our knowledge of cosmology, and potentially unveiling new physical laws and phenomena. The advent and operation of space-based detectors provide an important opportunity for detecting stochastic gravitational wave backgrounds (SGWB) generated by cosmic strings. However, the intricate nature of SGWB poses a formidable challenge in distinguishing its signal from the complex noise by some traditional methods. Therefore, we attempt to identify SGWB based on machine learning. Our findings show that the joint detection of LISA and Taiji significantly outperforms individual detectors, and even in the presence of numerous low signal-to-noise ratio(SNR) signals, the identification accuracy remains exceptionally high with 95%. Although our discussion is based solely on simulated data, the relevant methods can provide data-driven analytical capabilities for future observations of SGWB.

gr-qc↗

arXiv2025

Uncovering the Dark Side of Telegram: Fakes, Clones, Scams, and Conspiracy Movements

Telegram is one of the most used instant messaging apps worldwide. Some of its success lies in providing high privacy protection and social network features like the channels -- virtual rooms in which only the admins can post and broadcast messages to all its subscribers. However, these same features contributed to the emergence of borderline activities and, as is common with Online Social Networks, the heavy presence of fake accounts. Telegram started to address these issues by introducing the verified and scam marks for the channels. Unfortunately, the problem is far from being solved. In this work, we perform a large-scale analysis of Telegram by collecting 35,382 different channels and over 130,000,000 messages. We study the channels that Telegram marks as verified or scam, highlighting analogies and differences. Then, we move to the unmarked channels. Here, we find some of the infamous activities also present on privacy-preserving services of the Dark Web, such as carding, sharing of illegal adult and copyright protected content. In addition, we identify and analyze two other types of channels: the clones and the fakes. Clones are channels that publish the exact content of another channel to gain subscribers and promote services. Instead, fakes are channels that attempt to impersonate celebrities or well-known services. Fakes are hard to identify even by the most advanced users. To detect the fake channels automatically, we propose a machine learning model that is able to identify them with an accuracy of 86%. Lastly, we study Sabmyk, a conspiracy theory that exploited fakes and clones to spread quickly on the platform reaching over 1,000,000 users.

cs.CY↗

arXiv2025

Air-stable lithiation engineering of $\mathrm{MoS}_{2}$ for direct-bandgap multilayers

Due to its sizable direct bandgap and strong light-matter interactions, the preparation of monolayer $\mathrm{MoS}_{2}$ has attracted significant attention and intensive research efforts. However, multilayer $\mathrm{MoS}_{2}$ is largely overlooked because of its optically inactive indirect bandgap caused by interlayer coupling. It is highly desirable to modulate and decrease the interlayer coupling so that each layer in multilayer $\mathrm{MoS}_{2}$ can exhibit a monolayer-like direct-gap behavior. Here, we demonstrate the nanoprobe fabrication of $\mathrm{Li}_{x}\mathrm{MoS}_{2}$-based multilayers exhibiting a direct bandgap and strong photoluminescence emission from tightly bound excitons and trions. The fabrication is facilitated by our newly developed Li-ion platform, featuring tip-induced Li intercalation, air stability and rewritability. Raman characterizations reveal that controlled Li intercalation effectively transforms multilayer $\mathrm{MoS}_{2}$ into the stack of multiple monolayers, leading to a 26-fold enhancement of photoluminescence, compared to a monolayer. This intercalation result is different from existing observations of transforming $\mathrm{MoS}_{2}$ multilayers into metallic phases.

cond-mat.mes-hall↗

arXiv2025

Step-Video-T2V Technical Report: The Practice, Challenges, and Future of Video Foundation Model

We present Step-Video-T2V, a state-of-the-art text-to-video pre-trained model with 30B parameters and the ability to generate videos up to 204 frames in length. A deep compression Variational Autoencoder, Video-VAE, is designed for video generation tasks, achieving 16x16 spatial and 8x temporal compression ratios, while maintaining exceptional video reconstruction quality. User prompts are encoded using two bilingual text encoders to handle both English and Chinese. A DiT with 3D full attention is trained using Flow Matching and is employed to denoise input noise into latent frames. A video-based DPO approach, Video-DPO, is applied to reduce artifacts and improve the visual quality of the generated videos. We also detail our training strategies and share key observations and insights. Step-Video-T2V's performance is evaluated on a novel video generation benchmark, Step-Video-T2V-Eval, demonstrating its state-of-the-art text-to-video quality when compared with both open-source and commercial engines. Additionally, we discuss the limitations of current diffusion-based model paradigm and outline future directions for video foundation models. We make both Step-Video-T2V and Step-Video-T2V-Eval available at https://github.com/stepfun-ai/Step-Video-T2V. The online version can be accessed from https://yuewen.cn/videos as well. Our goal is to accelerate the innovation of video foundation models and empower video content creators.

cs.CV↗

arXiv2025

Step-Audio: Unified Understanding and Generation in Intelligent Speech Interaction

Real-time speech interaction, serving as a fundamental interface for human-machine collaboration, holds immense potential. However, current open-source models face limitations such as high costs in voice data collection, weakness in dynamic control, and limited intelligence. To address these challenges, this paper introduces Step-Audio, the first production-ready open-source solution. Key contributions include: 1) a 130B-parameter unified speech-text multi-modal model that achieves unified understanding and generation, with the Step-Audio-Chat version open-sourced; 2) a generative speech data engine that establishes an affordable voice cloning framework and produces the open-sourced lightweight Step-Audio-TTS-3B model through distillation; 3) an instruction-driven fine control system enabling dynamic adjustments across dialects, emotions, singing, and RAP; 4) an enhanced cognitive architecture augmented with tool calling and role-playing abilities to manage complex tasks effectively. Based on our new StepEval-Audio-360 evaluation benchmark, Step-Audio achieves state-of-the-art performance in human evaluations, especially in terms of instruction following. On open-source benchmarks like LLaMA Question, shows 9.3% average performance improvement, demonstrating our commitment to advancing the development of open-source multi-modal language technologies. Our code and models are available at https://github.com/stepfun-ai/Step-Audio.

cs.CL↗

arXiv2025

Fine-tuning Multimodal Large Language Models for Product Bundling

Recent advances in product bundling have leveraged multimodal information through sophisticated encoders, but remain constrained by limited semantic understanding and a narrow scope of knowledge. Therefore, some attempts employ In-context Learning (ICL) to explore the potential of large language models (LLMs) for their extensive knowledge and complex reasoning abilities. However, these efforts are inadequate in understanding mulitmodal data and exploiting LLMs' knowledge for product bundling. To bridge the gap, we introduce Bundle-MLLM, a novel framework that fine-tunes LLMs through a hybrid item tokenization approach within a well-designed optimization strategy. Specifically, we integrate textual, media, and relational data into a unified tokenization, introducing a soft separation token to distinguish between textual and non-textual tokens. Additionally, a streamlined yet powerful multimodal fusion module is employed to embed all non-textual features into a single, informative token, significantly boosting efficiency. To tailor product bundling tasks for LLMs, we reformulate the task as a multiple-choice question with candidate items as options. We further propose a progressive optimization strategy that fine-tunes LLMs for disentangled objectives: 1) learning bundle patterns and 2) enhancing multimodal semantic understanding specific to product bundling. Extensive experiments on four datasets across two domains demonstrate that our approach outperforms a range of state-of-the-art (SOTA) methods.

cs.IR↗

arXiv2025

Integrate Temporal Graph Learning into LLM-based Temporal Knowledge Graph Model

Temporal Knowledge Graph Forecasting (TKGF) aims to predict future events based on the observed events in history. Recently, Large Language Models (LLMs) have exhibited remarkable capabilities, generating significant research interest in their application for reasoning over temporal knowledge graphs (TKGs). Existing LLM-based methods have integrated retrieved historical facts or static graph representations into LLMs. Despite the notable performance of LLM-based methods, they are limited by the insufficient modeling of temporal patterns and ineffective cross-modal alignment between graph and language, hindering the ability of LLMs to fully grasp the temporal and structural information in TKGs. To tackle these issues, we propose a novel framework TGL-LLM to integrate temporal graph learning into LLM-based temporal knowledge graph model. Specifically, we introduce temporal graph learning to capture the temporal and relational patterns and obtain the historical graph embedding. Furthermore, we design a hybrid graph tokenization to sufficiently model the temporal patterns within LLMs. To achieve better alignment between graph and language, we employ a two-stage training paradigm to finetune LLMs on high-quality and diverse data, thereby resulting in better performance. Extensive experiments on three real-world datasets show that our approach outperforms a range of state-of-the-art (SOTA) methods.

cs.IR↗

arXiv2024

JailPO: A Novel Black-box Jailbreak Framework via Preference Optimization against Aligned LLMs

Large Language Models (LLMs) aligned with human feedback have recently garnered significant attention. However, it remains vulnerable to jailbreak attacks, where adversaries manipulate prompts to induce harmful outputs. Exploring jailbreak attacks enables us to investigate the vulnerabilities of LLMs and further guides us in enhancing their security. Unfortunately, existing techniques mainly rely on handcrafted templates or generated-based optimization, posing challenges in scalability, efficiency and universality. To address these issues, we present JailPO, a novel black-box jailbreak framework to examine LLM alignment. For scalability and universality, JailPO meticulously trains attack models to automatically generate covert jailbreak prompts. Furthermore, we introduce a preference optimization-based attack method to enhance the jailbreak effectiveness, thereby improving efficiency. To analyze model vulnerabilities, we provide three flexible jailbreak patterns. Extensive experiments demonstrate that JailPO not only automates the attack process while maintaining effectiveness but also exhibits superior performance in efficiency, universality, and robustness against defenses compared to baselines. Additionally, our analysis of the three JailPO patterns reveals that attacks based on complex templates exhibit higher attack strength, whereas covert question transformations elicit riskier responses and are more likely to bypass defense mechanisms.

cs.CR↗

arXiv2024

Prompt-A-Video: Prompt Your Video Diffusion Model via Preference-Aligned LLM

Text-to-video models have made remarkable advancements through optimization on high-quality text-video pairs, where the textual prompts play a pivotal role in determining quality of output videos. However, achieving the desired output often entails multiple revisions and iterative inference to refine user-provided prompts. Current automatic methods for refining prompts encounter challenges such as Modality-Inconsistency, Cost-Discrepancy, and Model-Unaware when applied to text-to-video diffusion models. To address these problem, we introduce an LLM-based prompt adaptation framework, termed as Prompt-A-Video, which excels in crafting Video-Centric, Labor-Free and Preference-Aligned prompts tailored to specific video diffusion model. Our approach involves a meticulously crafted two-stage optimization and alignment system. Initially, we conduct a reward-guided prompt evolution pipeline to automatically create optimal prompts pool and leverage them for supervised fine-tuning (SFT) of the LLM. Then multi-dimensional rewards are employed to generate pairwise data for the SFT model, followed by the direct preference optimization (DPO) algorithm to further facilitate preference alignment. Through extensive experimentation and comparative analyses, we validate the effectiveness of Prompt-A-Video across diverse generation models, highlighting its potential to push the boundaries of video generation.

cs.CV↗

arXiv2024

IntComplex for high-order interactions

Graphs serve as powerful tools for modeling pairwise interactions in diverse fields such as biology, material science, and social networks. However, they inherently overlook interactions involving more than two entities. Simplicial complexes and hypergraphs have emerged as prominent frameworks for modeling many-body interactions; nevertheless, they exhibit limitations in capturing specific high-order interactions, particularly those involving transitions from $n$-interactions to $m$-interactions. Addressing this gap, we propose IntComplex as an innovative framework to characterize such high-order interactions comprehensively. Our framework leverages homology theory to provide a quantitative representation of the topological structure inherent in such interactions. IntComplex is defined as a collection of interactions, each of which can be equivalently represented by a binary tree. Drawing inspiration from GLMY homology, we introduce homology for the detailed analysis of structural patterns formed by interactions across adjacent dimensions, $p$-layer homology to elucidate loop structures within $p$-interactions in specific dimensions, and multilayer homology to analyze loop structures of interactions across multiple dimensions. Furthermore, we introduce persistent homology through a filtration process and establish its stability to ensure robust quantitative analysis of these complex interactions. The proposed IntComplex framework establishes a foundational paradigm for the analysis of topological properties in high-order interactions, presenting significant potential to drive forward the advancements in the domain of complex network analysis.

math.AT↗

arXiv2024

Primitive path homology

In this paper we introduce a primitive path homology theory on the category of simple digraphs. On the subcategory of asymmetric digraphs, this theory coincides with the path homology theory which was introduced by Grigor'yan, Lin, Muranov, and Yau, but these theories are different in general case. We study properties of the primitive path homology and describe relations between the primitive path homology and the path homology. Let $a,b$ two different vertices of a digraph. Our approach gives a possibility to construct primitive homology theories of paths which have a given tail vertex $a$ or (and) a given head vertex $b$. We study these theories and describe also relationships between them and the path homology theory.

math.AT↗