arXiv subjects

Yang Luo

Publications and source records attributed to Yang Luo.

At least 73 records · Page 4Linked to original sources

arXiv2024

How Does the Textual Information Affect the Retrieval of Multimodal In-Context Learning?

The increase in parameter size of multimodal large language models (MLLMs) introduces significant capabilities, particularly in-context learning, where MLLMs enhance task performance without updating pre-trained parameters. This effectiveness, however, hinges on the appropriate selection of in-context examples, a process that is currently biased towards visual data, overlooking textual information. Furthermore, the area of supervised retrievers for MLLMs, crucial for optimal in-context example selection, continues to be uninvestigated. Our study offers an in-depth evaluation of the impact of textual information on the unsupervised selection of in-context examples in multimodal contexts, uncovering a notable sensitivity of retriever performance to the employed modalities. Responding to this, we introduce a novel supervised MLLM-retriever MSIER that employs a neural network to select examples that enhance multimodal in-context learning efficiency. This approach is validated through extensive testing across three distinct tasks, demonstrating the method's effectiveness. Additionally, we investigate the influence of modalities on our supervised retrieval method's training and pinpoint factors contributing to our model's success. This exploration paves the way for future advancements, highlighting the potential for refined in-context learning in MLLMs through the strategic use of multimodal data.

cs.CL↗

arXiv2024

Nuclear burning in an accretion flow around a stellar-mass black hole embedded within an AGN disk

A stellar-mass black hole, embedded within the accretion disk of an active galactic nuclei (AGN), has the potential to accrete gas at a rate that can reach approximately $\sim 10^9$ times the Eddington limit. This study explores the potential for nuclear burning in the rapidly accreting flow towards this black hole and studies how nucleosynthesis affects metal production. Using numerical methods, we have obtained the disk structure while considering nuclear burning and assessed the stability of the disk. In contrast to gas accretion onto the surface of a neutron star or white dwarf, the disk remains stable against the thermal and secular instabilities because advection cooling offsets the nuclear heating effects. The absence of a solid surface for a black hole prevents excessive mass accumulation in the inner disk region. Notably, nuclear fusion predominantly takes place in the inner disk region, resulting in substantial burning of $\rm ^{12}C$ and $\rm ^{3}He$, particularly for black holes around $M = 10\, M_\odot$ with accretion rates exceeding approximately $\sim 10^7$ times the Eddington rate. The ejection of carbon-depleted gas through outflows can lead to an increase in the mass ratio of oxygen or nitrogen to carbon, which may be reflected in observed line ratios such as $\rm N\, V/C\, IV$ and $\rm O\, IV/C\, IV$. Consequently, these elevated spectral line ratios could be interpreted as indications of super-solar metallicity in the broad line region.

astro-ph.HE↗

arXiv2024

Visualizing hot carrier dynamics by nonlinear optical microscopy at the atomic length scale

Probing and manipulating the spatiotemporal dynamics of hot carriers in nanoscale metals is crucial to a plethora of applications ranging from nonlinear nanophotonics to single molecule photochemistry. The direct investigation of these highly non-equilibrium carriers requires the experimental capability of high energy resolution (~ meV) broadband femtosecond spectroscopy. When considering the ultimate limits of atomic scale structures, this capability has remained out of reach until date. Using a two color femtosecond pump-probe spectroscopy, we present here the real-time tracking of hot carrier dynamics in a well-defined plasmonic picocavity, formed in the tunnel junction of a scanning tunneling microscope (STM). The excitation of hot carriers in the picocavity enables ultrafast all optical control over the broadband (~ eV) anti Stokes electronic resonance Raman scattering (ERRS) and the four-wave mixing (FWM) signals generated at the atomic length scale. By mapping the ERRS and FWM signals from a single graphene nanoribbon (GNR), we demonstrate that both signals are more efficiently generated along the edges of the GNR: a manifestation of atomic-scale nonlinear optical microscopy. This demonstration paves the way to the development of novel ultrafast nonlinear picophotonic platforms, affording unique opportunities in a variety of contexts, from the direct investigation of non equilibrium light matter interactions in complex quantum materials, to the development of robust strategies for hot carriers harvesting in single molecules and the next generation of active metasurfaces with deep-sub-wavelength meta-atoms.

physics.optics↗

arXiv2024

Direct Collapse Accretion Disks Within Dark Matter Halos: Saturation of the Magnetorotational Instability and the Field Expulsion

We have used high-resolution zoom-in simulations of direct collapse to supermassive black hole (SMBH) seeds within dark mater (DM) halos in the presence of magnetic fields generated during the collapse, down to $10^{-5}$ pc or 2 AU. We confirm an efficient amplification of magnetic field during collapse, the formation of a geometrically thick self-gravitating accretion disk inside 0.1 pc, and damping of fragmentation in the disk by the field. This disk differs profoundly from SMBH accretion disks. We find the following: (1) The accretion disk is subject to the magnetorotational instability which further amplifies the field to near equipartition. No artificial seeding of the disk field has been used. (2) The equipartition toroidal field changes its polarity in the midplane. (3) The nonlinear Parker instability develops, accompanied by the vertical buckling of the field lines, which injects material above the disk, leading to an increase in the disk scale height; (4) With the Coriolis force producing a coherent helicity above the disk, vertical poloidal field has been generated and amplified. (5) We estimate that the associated outflow will be most probably squashed by accretion. The resulting configuration consists of a magnetized disk with $β> 0.1$ and its magnetosphere with $β<< 1$, where $β= P_{\rm th}/P_{\rm B}$ is the ratio of thermal to magnetic energy density. (6) The disk is highly variable, due to feeding by variable accretion flow, and strong vortical motions are present. (7) Finally, the negative gradient of the total vertical stress drives an equatorial outflow sandwiched by an inward accretion flow.

astro-ph.GA↗

arXiv2024

Exploring Adversarial Robustness of Deep State Space Models

Deep State Space Models (SSMs) have proven effective in numerous task scenarios but face significant security challenges due to Adversarial Perturbations (APs) in real-world deployments. Adversarial Training (AT) is a mainstream approach to enhancing Adversarial Robustness (AR) and has been validated on various traditional DNN architectures. However, its effectiveness in improving the AR of SSMs remains unclear. While many enhancements in SSM components, such as integrating Attention mechanisms and expanding to data-dependent SSM parameterizations, have brought significant gains in Standard Training (ST) settings, their potential benefits in AT remain unexplored. To investigate this, we evaluate existing structural variants of SSMs with AT to assess their AR performance. We observe that pure SSM structures struggle to benefit from AT, whereas incorporating Attention yields a markedly better trade-off between robustness and generalization for SSMs in AT compared to other components. Nonetheless, the integration of Attention also leads to Robust Overfitting (RO) issues. To understand these phenomena, we empirically and theoretically analyze the output error of SSMs under AP. We find that fixed-parameterized SSMs have output error bounds strictly related to their parameters, limiting their AT benefits, while input-dependent SSMs may face the problem of error explosion. Furthermore, we show that the Attention component effectively scales the output error of SSMs during training, enabling them to benefit more from AT, but at the cost of introducing RO due to its high model complexity. Inspired by this, we propose a simple and effective Adaptive Scaling (AdS) mechanism that brings AT performance close to Attention-integrated SSMs without introducing the issue of RO. Our code is available at https://github.com/Biqing-Qi/Exploring-Adversarial-Robustness-of-Deep-State-Space-Models.git.

cs.LG↗

arXiv2024

FreeEnhance: Tuning-Free Image Enhancement via Content-Consistent Noising-and-Denoising Process

The emergence of text-to-image generation models has led to the recognition that image enhancement, performed as post-processing, would significantly improve the visual quality of the generated images. Exploring diffusion models to enhance the generated images nevertheless is not trivial and necessitates to delicately enrich plentiful details while preserving the visual appearance of key content in the original image. In this paper, we propose a novel framework, namely FreeEnhance, for content-consistent image enhancement using the off-the-shelf image diffusion models. Technically, FreeEnhance is a two-stage process that firstly adds random noise to the input image and then capitalizes on a pre-trained image diffusion model (i.e., Latent Diffusion Models) to denoise and enhance the image details. In the noising stage, FreeEnhance is devised to add lighter noise to the region with higher frequency to preserve the high-frequent patterns (e.g., edge, corner) in the original image. In the denoising stage, we present three target properties as constraints to regularize the predicted noise, enhancing images with high acutance and high visual quality. Extensive experiments conducted on the HPDv2 dataset demonstrate that our FreeEnhance outperforms the state-of-the-art image enhancement models in terms of quantitative metrics and human preference. More remarkably, FreeEnhance also shows higher human preference compared to the commercial image enhancement solution of Magnific AI.

cs.CV↗

arXiv2024

Robust Domain Generalization for Multi-modal Object Recognition

In multi-label classification, machine learning encounters the challenge of domain generalization when handling tasks with distributions differing from the training data. Existing approaches primarily focus on vision object recognition and neglect the integration of natural language. Recent advancements in vision-language pre-training leverage supervision from extensive visual-language pairs, enabling learning across diverse domains and enhancing recognition in multi-modal scenarios. However, these approaches face limitations in loss function utilization, generality across backbones, and class-aware visual fusion. This paper proposes solutions to these limitations by inferring the actual loss, broadening evaluations to larger vision-language backbones, and introducing Mixup-CLIPood, which incorporates a novel mix-up loss for enhanced class-aware visual fusion. Our method demonstrates superior performance in domain generalization across multiple datasets.

cs.CV↗

arXiv2024

Adaptive Two-Stage Cloud Resource Scaling via Hierarchical Multi-Indicator Forecasting and Bayesian Decision-Making

The surging demand for cloud computing resources, driven by the rapid growth of sophisticated large-scale models and data centers, underscores the critical importance of efficient and adaptive resource allocation. As major tech enterprises deploy massive infrastructures with thousands of GPUs, existing cloud platforms still struggle with low resource utilization due to key challenges: capturing hierarchical indicator structures, modeling non-Gaussian distributions, and decision-making under uncertainty. To address these challenges, we propose HRAMONY, an adaptive Hierarchical Attention-based Resource Modeling and Decision-Making System. HARMONY combines hierarchical multi-indicator distribution forecasting and uncertainty-aware Bayesian decision-making. It introduces a novel hierarchical attention mechanism that comprehensively models complex inter-indicator dependencies, enabling accurate predictions that can adapt to evolving environment states. By transforming Gaussian projections into adaptive non-Gaussian distributions via Normalizing Flows. Crucially, HARMONY leverages the full predictive distributions in an adaptive Bayesian process, proactively incorporating uncertainties to optimize resource allocation while robustly meeting SLA constraints under varying conditions. Extensive evaluations across four large-scale cloud datasets demonstrate HARMONY's state-of-the-art performance, significantly outperforming nine established methods. A month-long real-world deployment validated HARMONY's substantial practical impact, realizing over 35,000 GPU hours in savings and translating to $100K+ in cost reduction, showcasing its remarkable economic value through adaptive, uncertainty-aware scaling. Our code is available at https://github.com/Floating-LY/HARMONY1.

cs.LG↗

arXiv2024

Towards Open-World Mobile Manipulation in Homes: Lessons from the Neurips 2023 HomeRobot Open Vocabulary Mobile Manipulation Challenge

In order to develop robots that can effectively serve as versatile and capable home assistants, it is crucial for them to reliably perceive and interact with a wide variety of objects across diverse environments. To this end, we proposed Open Vocabulary Mobile Manipulation as a key benchmark task for robotics: finding any object in a novel environment and placing it on any receptacle surface within that environment. We organized a NeurIPS 2023 competition featuring both simulation and real-world components to evaluate solutions to this task. Our baselines on the most challenging version of this task, using real perception in simulation, achieved only an 0.8% success rate; by the end of the competition, the best participants achieved an 10.8\% success rate, a 13x improvement. We observed that the most successful teams employed a variety of methods, yet two common threads emerged among the best solutions: enhancing error detection and recovery, and improving the integration of perception with decision-making processes. In this paper, we detail the results and methodologies used, both in simulation and real-world settings. We discuss the lessons learned and their implications for future research. Additionally, we compare performance in real and simulated environments, emphasizing the necessity for robust generalization to novel settings.

cs.RO↗

arXiv2024

Robust Multimodal Learning via Representation Decoupling

Multimodal learning robust to missing modality has attracted increasing attention due to its practicality. Existing methods tend to address it by learning a common subspace representation for different modality combinations. However, we reveal that they are sub-optimal due to their implicit constraint on intra-class representation. Specifically, the sample with different modalities within the same class will be forced to learn representations in the same direction. This hinders the model from capturing modality-specific information, resulting in insufficient learning. To this end, we propose a novel Decoupled Multimodal Representation Network (DMRNet) to assist robust multimodal learning. Specifically, DMRNet models the input from different modality combinations as a probabilistic distribution instead of a fixed point in the latent space, and samples embeddings from the distribution for the prediction module to calculate the task loss. As a result, the direction constraint from the loss minimization is blocked by the sampled representation. This relaxes the constraint on the inference representation and enables the model to capture the specific information for different modality combinations. Furthermore, we introduce a hard combination regularizer to prevent DMRNet from unbalanced training by guiding it to pay more attention to hard modality combinations. Finally, extensive experiments on multimodal classification and segmentation tasks demonstrate that the proposed DMRNet outperforms the state-of-the-art significantly.

cs.CV↗

arXiv2024

Flat bands and distinct density wave orders in correlated Kagome superconductor CsCr$_3$Sb$_5$

Kagome metal CsV$_3$Sb$_5$ has attracted much recent attention due to the coexistence of multiple exotic orders and the associated proposals to mimic unconventional high temperature superconductors. Nevertheless, magnetism and strong electronic correlations -- two essential ingredients for unconventional superconductivity, are absent in this V-based Kagome metal. CsCr$_3$Sb$_5$ is a newly discovered Cr-based parallel of CsV$_3$Sb$_5$, in which magnetism appears with charge density wave and superconductivity at different temperature and pressure regions. Enhanced electronic correlations are also suggested by theoretical proposals due to the calculated flat bands. Here, we report angle-resolved photoemission measurements and first-principles calculations on this new material system. Electron energy bands and the associated orbitals are resolved. Flat bands are observed near the Fermi level. Doping dependent measurements on Cs(Cr$_x$V$_{1-x}$)$_3$Sb$_5$ reveal a gradually enhanced band renormalization from CsV$_3$Sb$_5$ to CsCr$_3$Sb$_5$, accompanied by distinct spatial symmetry breaking states in the phase diagram.

cond-mat.supr-con↗

arXiv2024

Learning to Rank Patches for Unbiased Image Redundancy Reduction

Images suffer from heavy spatial redundancy because pixels in neighboring regions are spatially correlated. Existing approaches strive to overcome this limitation by reducing less meaningful image regions. However, current leading methods rely on supervisory signals. They may compel models to preserve content that aligns with labeled categories and discard content belonging to unlabeled categories. This categorical inductive bias makes these methods less effective in real-world scenarios. To address this issue, we propose a self-supervised framework for image redundancy reduction called Learning to Rank Patches (LTRP). We observe that image reconstruction of masked image modeling models is sensitive to the removal of visible patches when the masking ratio is high (e.g., 90\%). Building upon it, we implement LTRP via two steps: inferring the semantic density score of each patch by quantifying variation between reconstructions with and without this patch, and learning to rank the patches with the pseudo score. The entire process is self-supervised, thus getting out of the dilemma of categorical inductive bias. We design extensive experiments on different datasets and tasks. The results demonstrate that LTRP outperforms both supervised and other self-supervised methods due to the fair assessment of image content.

cs.CV↗

arXiv2024

From Two-Stream to One-Stream: Efficient RGB-T Tracking via Mutual Prompt Learning and Knowledge Distillation

Due to the complementary nature of visible light and thermal infrared modalities, object tracking based on the fusion of visible light images and thermal images (referred to as RGB-T tracking) has received increasing attention from researchers in recent years. How to achieve more comprehensive fusion of information from the two modalities at a lower cost has been an issue that researchers have been exploring. Inspired by visual prompt learning, we designed a novel two-stream RGB-T tracking architecture based on cross-modal mutual prompt learning, and used this model as a teacher to guide a one-stream student model for rapid learning through knowledge distillation techniques. Extensive experiments have shown that, compared to similar RGB-T trackers, our designed teacher model achieved the highest precision rate, while the student model, with comparable precision rate to the teacher model, realized an inference speed more than three times faster than the teacher model.(Codes will be available if accepted.)

cs.CV↗

arXiv2024

MCformer: Multivariate Time Series Forecasting with Mixed-Channels Transformer

The massive generation of time-series data by largescale Internet of Things (IoT) devices necessitates the exploration of more effective models for multivariate time-series forecasting. In previous models, there was a predominant use of the Channel Dependence (CD) strategy (where each channel represents a univariate sequence). Current state-of-the-art (SOTA) models primarily rely on the Channel Independence (CI) strategy. The CI strategy treats all channels as a single channel, expanding the dataset to improve generalization performance and avoiding inter-channel correlation that disrupts long-term features. However, the CI strategy faces the challenge of interchannel correlation forgetting. To address this issue, we propose an innovative Mixed Channels strategy, combining the data expansion advantages of the CI strategy with the ability to counteract inter-channel correlation forgetting. Based on this strategy, we introduce MCformer, a multivariate time-series forecasting model with mixed channel features. The model blends a specific number of channels, leveraging an attention mechanism to effectively capture inter-channel correlation information when modeling long-term features. Experimental results demonstrate that the Mixed Channels strategy outperforms pure CI strategy in multivariate time-series forecasting tasks.

cs.LG↗

arXiv2024

Selective Excitation of Vibrations in a Single Molecule

The capability to excite, probe, and manipulate vibrational modes is essential for understanding and controlling chemical reactions at the molecular level. Recent advancements in tip-enhanced Raman spectroscopies have enabled the probing of vibrational fingerprints in a single molecule with Angstrom-scale spatial resolution. However, achieving controllable excitation of specific vibrational modes in individual molecules remains challenging. Here, we demonstrate the selective excitation and probing of vibrational modes in single deprotonated phthalocyanine molecules utilizing resonance Raman spectroscopy in a scanning tunneling microscope. Selective excitation is achieved by finely tuning the excitation wavelength of the laser to be resonant with the vibronic transitions between the molecular ground electronic state and the vibrational levels in the excited electronic state, resulting in the state-selective enhancement of the resonance Raman signal. Our approach sets the stage for steering chemical transformations in molecules on surfaces by selective excitation of molecular vibrations.

physics.optics↗

arXiv2024

Intelligent Reflecting Surfaces vs. Full-Duplex Relays: A Comparison in the Air

This letter aims to provide a fundamental analytical comparison for the two major types of relaying methods: intelligent reflecting surfaces and full-duplex relays, particularly focusing on unmanned aerial vehicle communication scenarios. Both amplify-and-forward and decode-and-forward relaying schemes are included in the comparison. In addition, optimal 3D UAV deployment and minimum transmit power under the quality of service constraint are derived. Our numerical results show that IRSs of medium size exhibit comparable performance to AF relays, meanwhile outperforming DF relays under extremely large surface size and high data rates.

eess.SP↗

arXiv2024

Towards the THz Networks in the 6G Era

This commentary dedicates to envision what role THz is going to play in the coming human-centric 6G era. Three distinct THz network types including outdoor, indoor, and body area networks are discussed, with an emphasis on their capabilities in human body detection. Synthesizing these networks will unlock a bunch of fascinating applications across industrial, biomedical and entertainment fields, significantly enhancing the quality of human life.

eess.SY↗

arXiv2024

Two-dimensional phase diagram of the charge density wave in doped CsV$_3$Sb$_5$

Kagome superconductors AV$_3$Sb$_5$ (A = K, Rb and Cs) have attracted much recent attention due to the coexistence of multiple exotic orders. Among them, the charge density wave (CDW) order has been shown to host various unconventional behaviors. Here, we investigate the CDW order by a combination of both bulk and surface doping methods. While element substitutions in bulk doping change both carriers and the crystal lattice, the surface doping primarily tunes the carrier concentration. As such, our results reveal a two-dimensional phase diagram of the CDW in doped CsV$_3$Sb$_5$. In the lightly bulk doped regime, the existence of CDW order is reversible by tuning the carrier concentration. But excessive bulk doping permanently destroys the CDW, regardless of the carrier doping level. These results provide insights to the origin of the CDW from both electronic and structural degrees of freedom. They also open an avenue for manipulating the exotic CDW order in Kagome superconductors.

cond-mat.supr-con↗