arXiv subjects

Yunlong Tang

Publications and source records attributed to Yunlong Tang.

28 records · Page 2Linked to original sources

arXiv2024

Large Enhancement of Properties in Strained Lead-free Multiferroic Solid Solutions with Strong Deviation from Vegard's Law

Efforts to combine the advantages of multiple systems to enhance functionlities through solid solution design present a great challenge due to the constraint imposed by the classical Vegard law. Here, we successfully navigate this trade off by leveraging the synergistic effect of chemical doping and strain engineering in solid solution system of BiFeO3 BaTiO3. Unlike bulks, a significant deviation from the Vegard law accompanying with enhanced multiferroism is observed in the strained solid solution epitaxial films, where we achieve a pronounced tetragonality, enhanced saturated magnetization, substantial polarization, high ferroelectric Curie temperature, all while maintaining impressively low leakage current. These characteristics surpass the properties of their parent BiFeO3 and BaTiO3 films. Moreover, the superior ferroelectricity has never been reported in corresponding bulks. These findings underscore the potential of strained BiFeO3 BaTiO3 films as lead-free, room-temperature multiferroics.

cond-mat.mtrl-sci↗

arXiv2024

MMCOMPOSITION: Revisiting the Compositionality of Pre-trained Vision-Language Models

The advent of large Vision-Language Models (VLMs) has significantly advanced multimodal understanding, enabling more sophisticated and accurate integration of visual and textual information across various tasks, including image and video captioning, visual question answering, and cross-modal retrieval. Despite VLMs' superior capabilities, researchers lack a comprehensive understanding of their compositionality -- the ability to understand and produce novel combinations of known visual and textual components. Prior benchmarks provide only a relatively rough compositionality evaluation from the perspectives of objects, relations, and attributes while neglecting deeper reasoning about object interactions, counting, and complex compositions. However, compositionality is a critical ability that facilitates coherent reasoning and understanding across modalities for VLMs. To address this limitation, we propose MMCOMPOSITION, a novel human-annotated benchmark for comprehensively and accurately evaluating VLMs' compositionality. Our proposed benchmark serves as a complement to these earlier works. With MMCOMPOSITION, we can quantify and explore the compositionality of the mainstream VLMs. Surprisingly, we find GPT-4o's compositionality inferior to the best open-source model, and we analyze the underlying reasons. Our experimental analysis reveals the limitations of VLMs in fine-grained compositional perception and reasoning, and points to areas for improvement in VLM design and training. Resources available at: https://hanghuacs.github.io/MMComposition/

cs.CV↗

arXiv2024

EAGLE: Egocentric AGgregated Language-video Engine

The rapid evolution of egocentric video analysis brings new insights into understanding human activities and intentions from a first-person perspective. Despite this progress, the fragmentation in tasks like action recognition, procedure learning, and moment retrieval, \etc, coupled with inconsistent annotations and isolated model development, hinders a holistic interpretation of video content. In response, we introduce the EAGLE (Egocentric AGgregated Language-video Engine) model and the EAGLE-400K dataset to provide a unified framework that integrates various egocentric video understanding tasks. EAGLE-400K, the \textit{first} large-scale instruction-tuning dataset tailored for egocentric video, features 400K diverse samples to enhance a broad spectrum of tasks from activity recognition to procedure knowledge learning. Moreover, EAGLE, a strong video multimodal large language model (MLLM), is designed to effectively capture both spatial and temporal information. In addition, we propose a set of evaluation metrics designed to facilitate a thorough assessment of MLLM for egocentric video understanding. Our extensive experiments demonstrate EAGLE's superior performance over existing models, highlighting its ability to balance task-specific understanding with holistic video interpretation. With EAGLE, we aim to pave the way for research opportunities and practical applications in real-world scenarios.

cs.CV↗

arXiv2024

AIM 2024 Challenge on Video Saliency Prediction: Methods and Results

This paper reviews the Challenge on Video Saliency Prediction at AIM 2024. The goal of the participants was to develop a method for predicting accurate saliency maps for the provided set of video sequences. Saliency maps are widely exploited in various applications, including video compression, quality assessment, visual perception studies, the advertising industry, etc. For this competition, a previously unused large-scale audio-visual mouse saliency (AViMoS) dataset of 1500 videos with more than 70 observers per video was collected using crowdsourced mouse tracking. The dataset collection methodology has been validated using conventional eye-tracking data and has shown high consistency. Over 30 teams registered in the challenge, and there are 7 teams that submitted the results in the final phase. The final phase solutions were tested and ranked by commonly used quality metrics on a private test subset. The results of this evaluation and the descriptions of the solutions are presented in this report. All data, including the private test subset, is made publicly available on the challenge homepage - https://challenges.videoprocessing.ai/challenges/video-saliency-prediction.html.

cs.CV↗

arXiv2024

GaussianStyle: Gaussian Head Avatar via StyleGAN

Existing methods like Neural Radiation Fields (NeRF) and 3D Gaussian Splatting (3DGS) have made significant strides in facial attribute control such as facial animation and components editing, yet they struggle with fine-grained representation and scalability in dynamic head modeling. To address these limitations, we propose GaussianStyle, a novel framework that integrates the volumetric strengths of 3DGS with the powerful implicit representation of StyleGAN. The GaussianStyle preserves structural information, such as expressions and poses, using Gaussian points, while projecting the implicit volumetric representation into StyleGAN to capture high-frequency details and mitigate the over-smoothing commonly observed in neural texture rendering. Experimental outcomes indicate that our method achieves state-of-the-art performance in reenactment, novel view synthesis, and animation.

cs.CV↗

arXiv2024

DPStyler: Dynamic PromptStyler for Source-Free Domain Generalization

Source-Free Domain Generalization (SFDG) aims to develop a model that works for unseen target domains without relying on any source domain. Research in SFDG primarily bulids upon the existing knowledge of large-scale vision-language models and utilizes the pre-trained model's joint vision-language space to simulate style transfer across domains, thus eliminating the dependency on source domain images. However, how to efficiently simulate rich and diverse styles using text prompts, and how to extract domain-invariant information useful for classification from features that contain both semantic and style information after the encoder, are directions that merit improvement. In this paper, we introduce Dynamic PromptStyler (DPStyler), comprising Style Generation and Style Removal modules to address these issues. The Style Generation module refreshes all styles at every training epoch, while the Style Removal module eliminates variations in the encoder's output features caused by input styles. Moreover, since the Style Generation module, responsible for generating style word vectors using random sampling or style mixing, makes the model sensitive to input text prompts, we introduce a model ensemble method to mitigate this sensitivity. Extensive experiments demonstrate that our framework outperforms state-of-the-art methods on benchmark datasets.

cs.CV↗

arXiv2024

Do More Details Always Introduce More Hallucinations in LVLM-based Image Captioning?

Large Vision-Language Models (LVLMs) excel in integrating visual and linguistic contexts to produce detailed content, facilitating applications such as image captioning. However, using LVLMs to generate descriptions often faces the challenge of object hallucination (OH), where the output text misrepresents actual objects in the input image. While previous studies attribute the occurrence of OH to the inclusion of more details, our study finds technical flaws in existing metrics, leading to unreliable evaluations of models and conclusions about OH. This has sparked a debate on the question: Do more details always introduce more hallucinations in LVLM-based image captioning? In this paper, we address this debate by proposing a novel decoding strategy, Differentiated Beam Decoding (DBD), along with a reliable new set of evaluation metrics: CLIP-Precision, CLIP-Recall, and CLIP-F1. DBD decodes the wealth of information hidden in visual input into distinct language representations called unit facts in parallel. This decoding is achieved via a well-designed differential score that guides the parallel search and candidate screening. The selected unit facts are then aggregated to generate the final caption. Our proposed metrics evaluate the comprehensiveness and accuracy of image captions by comparing the embedding groups of ground-truth image regions and generated text partitions. Extensive experiments on the Visual Genome dataset validate the effectiveness of our approach, demonstrating that it produces detailed descriptions while maintaining low hallucination levels.

cs.CV↗

arXiv2023

Caption Anything: Interactive Image Description with Diverse Multimodal Controls

Controllable image captioning is an emerging multimodal topic that aims to describe the image with natural language following human purpose, $\textit{e.g.}$, looking at the specified regions or telling in a particular text style. State-of-the-art methods are trained on annotated pairs of input controls and output captions. However, the scarcity of such well-annotated multimodal data largely limits their usability and scalability for interactive AI systems. Leveraging unimodal instruction-following foundation models is a promising alternative that benefits from broader sources of data. In this paper, we present Caption AnyThing (CAT), a foundation model augmented image captioning framework supporting a wide range of multimodel controls: 1) visual controls, including points, boxes, and trajectories; 2) language controls, such as sentiment, length, language, and factuality. Powered by Segment Anything Model (SAM) and ChatGPT, we unify the visual and language prompts into a modularized framework, enabling the flexible combination between different controls. Extensive case studies demonstrate the user intention alignment capabilities of our framework, shedding light on effective user interaction modeling in vision-language applications. Our code is publicly available at https://github.com/ttengwang/Caption-Anything.

cs.CV↗

arXiv2021

Local Manipulation of Polar Skyrmions and Topological Phase Transitions

Topological phases such as polar skyrmions have been a fertile playground for ferroelectric oxide superlattices, with exotic physical phenomena such as negative capacitance. Herein, using phase-field simulations, we demonstrate the local control of the skyrmion phase with electric potential applied through a top electrode. Under a relatively small electric potential, the skyrmions underneath the electrode can be erased and recovered reversibly. A topologically protected transition from the symmetric to asymmetric skyrmion bubbles is observed at the edge of the electrode. While a topological transition to a labyrinthine domain requires a high applied potential, it can switch back to the skyrmion state with a relatively small electric potential. The topological transition from +1 to 0 occurs before the full destruction of the bubble state. It is shown that the shrinking and bursting of the skyrmions leads to a large reduction in the dielectric permittivity, the magnitude of which depends on the size of the electrode.

cond-mat.mtrl-sci↗

arXiv2021

Mechanical Properties of Gradient Copper Nano-Gyroid Cellular Structures: A Molecular Dynamics Study

Advanced manufacturing (AM) technologies, such as nanoscale additive manufacturing process, enable the fabrication of nanoscale architected materials which has received great attention due to their prominent properties. However, few studies delve into the functional gradient cellular architecture on nanoscale. This work studied the gradient nano-Gyroid architected material made of copper (Cu) by molecular dynamic (MD) simulations. The result reveals that, unlike homogeneous architecture, gradient Gyroid not only shows novel layer-by-layer deformation behavior, but also processes significantly better energy absorption ability. Moreover, this deformation behavior and energy absorption are predictable and designable, which demonstrates its highly programmable potential.

physics.app-ph↗