arXiv subjects

Hanghang Ma

Publications and source records attributed to Hanghang Ma.

8 recordsLinked to original sources

arXiv2026

A Simple Baseline for Unifying Understanding, Generation, and Editing via Vanilla Next-token Prediction

In this work, we introduce Wallaroo, a simple autoregressive baseline that leverages next-token prediction to unify multi-modal understanding, image generation, and editing at the same time. Moreover, Wallaroo supports multi-resolution image input and output, as well as bilingual support for both Chinese and English. We decouple the visual encoding into separate pathways and apply a four-stage training strategy to reshape the model's capabilities. Experiments are conducted on various benchmarks where Wallaroo produces competitive performance or exceeds other unified models, suggesting the great potential of autoregressive models in unifying multi-modality understanding and generation. Our code is available at https://github.com/JiePKU/Wallaroo.

cs.CV↗

arXiv2026

Forge-and-Quench: Enhancing Image Generation for Higher Fidelity in Unified Multimodal Models

Integrating image generation and understanding into a single framework has become a pivotal goal in the multimodal domain. However, how understanding can effectively assist generation has not been fully explored. Unlike previous works that focus on leveraging reasoning abilities and world knowledge from understanding models, this paper introduces a novel perspective: leveraging understanding to enhance the fidelity and detail richness of generated images. To this end, we propose Forge-and-Quench, a new unified framework that puts this principle into practice. In the generation process of our framework, an MLLM first reasons over the entire conversational context, including text instructions, to produce an enhanced text instruction. This refined instruction is then mapped to a virtual visual representation, termed the Bridge Feature, via a novel Bridge Adapter. This feature acts as a crucial link, forging insights from the understanding model to quench and refine the generation process. It is subsequently injected into the T2I backbone as a visual guidance signal, alongside the enhanced text instruction that replaces the original input. To validate this paradigm, we conduct comprehensive studies on the design of the Bridge Feature and Bridge Adapter. Our framework demonstrates exceptional extensibility and flexibility, enabling efficient migration across different MLLM and T2I models with significant savings in training overhead, all without compromising the MLLM's inherent multimodal understanding capabilities. Experiments show that Forge-and-Quench significantly improves image fidelity and detail across multiple models, while also maintaining instruction-following accuracy and enhancing world knowledge application. Models and codes are available at https://github.com/YanbingZeng/Forge-and-Quench.

cs.CV↗

arXiv2025

LongCat-Image Technical Report

We introduce LongCat-Image, a pioneering open-source and bilingual (Chinese-English) foundation model for image generation, designed to address core challenges in multilingual text rendering, photorealism, deployment efficiency, and developer accessibility prevalent in current leading models. 1) We achieve this through rigorous data curation strategies across the pre-training, mid-training, and SFT stages, complemented by the coordinated use of curated reward models during the RL phase. This strategy establishes the model as a new state-of-the-art (SOTA), delivering superior text-rendering capabilities and remarkable photorealism, and significantly enhancing aesthetic quality. 2) Notably, it sets a new industry standard for Chinese character rendering. By supporting even complex and rare characters, it outperforms both major open-source and commercial solutions in coverage, while also achieving superior accuracy. 3) The model achieves remarkable efficiency through its compact design. With a core diffusion model of only 6B parameters, it is significantly smaller than the nearly 20B or larger Mixture-of-Experts (MoE) architectures common in the field. This ensures minimal VRAM usage and rapid inference, significantly reducing deployment costs. Beyond generation, LongCat-Image also excels in image editing, achieving SOTA results on standard benchmarks with superior editing consistency compared to other open-source works. 4) To fully empower the community, we have established the most comprehensive open-source ecosystem to date. We are releasing not only multiple model versions for text-to-image and image editing, including checkpoints after mid-training and post-training stages, but also the entire toolchain of training procedure. We believe that the openness of LongCat-Image will provide robust support for developers and researchers, pushing the frontiers of visual content creation.

cs.CV↗

arXiv2025

MagicMirror: A Large-Scale Dataset and Benchmark for Fine-Grained Artifacts Assessment in Text-to-Image Generation

Text-to-image (T2I) generation has achieved remarkable progress in instruction following and aesthetics. However, a persistent challenge is the prevalence of physical artifacts, such as anatomical and structural flaws, which severely degrade perceptual quality and limit application. Given the diversity and complexity of these artifacts, a systematic and fine-grained evaluation framework is required, which is lacking in current benchmarks. To fill this gap, we introduce MagicMirror, a comprehensive framework for artifacts assessment. We first establish a detailed taxonomy of generated image artifacts. Guided by this taxonomy, we manually annotate MagicData340K, the first human-annotated large-scale dataset of 340K generated images with fine-grained artifact labels. Building on this dataset, we train MagicAssessor, a Vision-Language Model (VLM) that provides detailed assessments and corresponding labels. To overcome challenges like class imbalance and reward hacking, we design a novel data sampling strategy and a multi-level reward system for Group Relative Policy Optimization (GRPO). Finally, we leverage MagicAssessor to construct MagicBench, an automated benchmark for evaluating the image artifacts of current T2I models. Our evaluation with MagicBench reveals that despite their widespread adoption, even top-tier models like GPT-image-1 are consistently plagued by significant artifacts, highlighting artifact reduction as a critical frontier for future T2I development. Project page: https://wj-inf.github.io/MagicMirror-page/.

cs.CV↗

arXiv2025

Image Editing with Diffusion Models: A Survey

With deeper exploration of diffusion model, developments in the field of image generation have triggered a boom in image creation. As the quality of base-model generated images continues to improve, so does the demand for further application like image editing. In recent years, many remarkable works are realizing a wide variety of editing effects. However, the wide variety of editing types and diverse editing approaches have made it difficult for researchers to establish a comprehensive view of the development of this field. In this survey, we summarize the image editing field from four aspects: tasks definition, methods classification, results evaluation and editing datasets. First, we provide a definition of image editing, which in turn leads to a variety of editing task forms from the perspective of operation parts and manipulation actions. Subsequently, we categorize and summary methods for implementing editing into three categories: inversion-based, fine-tuning-based and adapter-based. In addition, we organize the currently used metrics, available datasets and corresponding construction methods. At the end, we present some visions for the future development of the image editing field based on the previous summaries.

cs.GR↗

arXiv2024

Kangaroo: A Powerful Video-Language Model Supporting Long-context Video Input

Rapid advancements have been made in extending Large Language Models (LLMs) to Large Multi-modal Models (LMMs). However, extending input modality of LLMs to video data remains a challenging endeavor, especially for long videos. Due to insufficient access to large-scale high-quality video data and the excessive compression of visual features, current methods exhibit limitations in effectively processing long videos. In this paper, we introduce Kangaroo, a powerful Video LMM aimed at addressing these challenges. Confronted with issue of inadequate training data, we develop a data curation system to build a large-scale dataset with high-quality annotations for vision-language pre-training and instruction tuning. In addition, we design a curriculum training pipeline with gradually increasing resolution and number of input frames to accommodate long videos. Evaluation results demonstrate that, with 8B parameters, Kangaroo achieves state-of-the-art performance across a variety of video understanding benchmarks while exhibiting competitive results on others. Particularly, on benchmarks specialized for long videos, Kangaroo excels some larger models with over 10B parameters and proprietary models.

cs.CV↗

arXiv2024

PM2D: A parallel GPU-based code for the kinetic simulation of laser plasma instabilities in large scale plasmas

Laser plasma instabilities (LPIs) have significant influences on the laser energy deposition efficiency, hot electron generation, and uniformity of irradiation in inertial confined fusion (ICF). In contrast to theoretical analysis of linear development of LPIs, numerical simulations play a more and more important role in revealing the complex physics of LPIs. Since LPIs are typically a three-wave coupling process, the precise kinetic simulation of LPIs requires to resolve the laser period (around one femtosecond) and laser wavelength (less than one micron). In this paper, a full wave fluid model of LPIs is constructed and numerically solved by the particle-mesh method, where the plasma is described by macro particles that can move across the mesh grids freely. Based upon this model, a two-dimensional (2D) GPU code named PM2D is developed. It can simulate the kinetic effects of LPIs self-consistently as normal particle-in-cell (PIC) codes. Moreover, as the physical model adopted in the PM2D code is specifically constructed for LPIs, the required macro particles per grid in the simulations can be largely reduced and thus overall simulation cost is considerably reduced comparing with typical PIC codes. Moreover, the numerical noise in our PM2D code is much lower, which makes it more robust than PIC codes in the simulation of LPIs for the long-time scale above 10 picoseconds. After the distributed computing is realized, our PM2D code is able to run on GPU clusters with a total mesh grids up to several billions, which meets the typical requirements for the simulations of LPIs at ICF experimental scale with reasonable cost.

physics.plasm-ph↗

arXiv2019

Simultaneous polarization transformation and amplification of multi-petawatt laser pulses in magnetized plasmas

With increasing laser peak power, the generation and manipulation of high-power laser pulses becomes a growing challenge for conventional solid-state optics due to their limited damage threshold. As a result, plasma-based optical components which can sustain extremely high fields are attracting increasing interest. Here, we propose a type of plasma waveplate based on magneto-optical birefringence under a transverse magnetic field, which can work under extremely high laser power. Importantly, this waveplate can simultaneously alter the polarization state and boost the peak laser power. It is demonstrated numerically that an initially linearly polarized laser pulse with 5 petawatt peak power can be converted into a circularly polarized pulse with a peak power higher than 10 petawatts by such a waveplate with a centimeter-scale diameter. The energy conversion efficiency of the polarization transformation is about $98\%$. The necessary waveplate thickness is shown to scale inversely with plasma electron density $n_e$ and the square of magnetic field $B_0$, and it is about 1 cm for $n_e=3\times 10^{20}$ cm$^{-3}$ and $B_0=100$ T. The proposed plasma waveplate and other plasma-based optical components can play a critical role for the effective utilization of multi-petawatt laser systems.

physics.plasm-ph↗