Searcharxiv⌕ Search

arXiv subjects

Yi Yang

Publications and source records attributed to Yi Yang.

At least 541 records · Page 30Linked to original sources
arXiv2024

BootsTAP: Bootstrapped Training for Tracking-Any-Point

To endow models with greater understanding of physics and motion, it is useful to enable them to perceive how solid surfaces move and deform in real scenes. This can be formalized as Tracking-Any-Point (TAP), which requires the algorithm to track any point on solid surfaces in a video, potentially densely in space and time. Large-scale groundtruth training data for TAP is only available in simulation, which currently has a limited variety of objects and motion. In this work, we demonstrate how large-scale, unlabeled, uncurated real-world data can improve a TAP model with minimal architectural changes, using a selfsupervised student-teacher setup. We demonstrate state-of-the-art performance on the TAP-Vid benchmark surpassing previous results by a wide margin: for example, TAP-Vid-DAVIS performance improves from 61.3% to 67.4%, and TAP-Vid-Kinetics from 57.2% to 62.5%. For visualizations, see our project webpage at https://bootstap.github.io/

cs.CV↗
arXiv2024

Ground-based and JWST Observations of SN 2022pul: II. Evidence from Nebular Spectroscopy for a Violent Merger in a Peculiar Type-Ia Supernova

We present an analysis of ground-based and JWST observations of SN~2022pul, a peculiar "03fg-like" (or "super-Chandrasekhar") Type Ia supernova (SN Ia), in the nebular phase at 338d post explosion. Our combined spectrum continuously covers 0.4--14 $μ$m and includes the first mid-infrared spectrum of an 03fg-like SN Ia. Compared to normal SN Ia 2021aefx, SN 2022pul exhibits a lower mean ionization state, asymmetric emission-line profiles, stronger emission from the intermediate-mass elements (IMEs) argon and calcium, weaker emission from iron-group elements (IGEs), and the first unambiguous detection of neon in a SN Ia. Strong, broad, centrally peaked [Ne II] line at 12.81 $μ$m was previously predicted as a hallmark of "violent merger'' SN Ia models, where dynamical interaction between two sub-$M_{ch}$ white dwarfs (WDs) causes disruption of the lower mass WD and detonation of the other. The violent merger scenario was already a leading hypothesis for 03fg-like SNe Ia; in SN 2022pul it can explain the large-scale ejecta asymmetries seen between the IMEs and IGEs and the central location of narrow oxygen and broad neon. We modify extant models to add clumping of the ejecta to better reproduce the optical iron emission, and add mass in the innermost region ($< 2000$ km s$^{-1}$) to account for the observed narrow [O I]~$λ\lambda6300$, 6364 emission. A violent WD-WD merger explains many of the observations of SN 2022pul, and our results favor this model interpretation for the subclass of 03fg-like SN Ia.

astro-ph.HE↗
arXiv2024

Progressive Volume Distillation with Active Learning for Efficient NeRF Architecture Conversion

Neural Radiance Fields (NeRF) have been widely adopted as practical and versatile representations for 3D scenes, facilitating various downstream tasks. However, different architectures, including the plain Multi-Layer Perceptron (MLP), Tensors, low-rank Tensors, Hashtables, and their combinations, entail distinct trade-offs. For instance, representations based on Hashtables enable faster rendering but lack clear geometric meaning, thereby posing challenges for spatial-relation-aware editing. To address this limitation and maximize the potential of each architecture, we propose Progressive Volume Distillation with Active Learning (PVD-AL), a systematic distillation method that enables any-to-any conversion between diverse architectures. PVD-AL decomposes each structure into two parts and progressively performs distillation from shallower to deeper volume representation, leveraging effective information retrieved from the rendering process. Additionally, a three-level active learning technique provides continuous feedback from teacher to student during the distillation process, achieving high-performance outcomes. Experimental evidence showcases the effectiveness of our method across multiple benchmark datasets. For instance, PVD-AL can distill an MLP-based model from a Hashtables-based model at a 10~20X faster speed and 0.8dB~2dB higher PSNR than training the MLP-based model from scratch. Moreover, PVD-AL permits the fusion of diverse features among distinct structures, enabling models with multiple editing properties and providing a more efficient model to meet real-time requirements like mobile devices. Project website: https://sk-fun.fun/PVD-AL.

cs.CV↗
arXiv2024

CapHuman: Capture Your Moments in Parallel Universes

We concentrate on a novel human-centric image synthesis task, that is, given only one reference facial photograph, it is expected to generate specific individual images with diverse head positions, poses, facial expressions, and illuminations in different contexts. To accomplish this goal, we argue that our generative model should be capable of the following favorable characteristics: (1) a strong visual and semantic understanding of our world and human society for basic object and human image generation. (2) generalizable identity preservation ability. (3) flexible and fine-grained head control. Recently, large pre-trained text-to-image diffusion models have shown remarkable results, serving as a powerful generative foundation. As a basis, we aim to unleash the above two capabilities of the pre-trained model. In this work, we present a new framework named CapHuman. We embrace the "encode then learn to align" paradigm, which enables generalizable identity preservation for new individuals without cumbersome tuning at inference. CapHuman encodes identity features and then learns to align them into the latent space. Moreover, we introduce the 3D facial prior to equip our model with control over the human head in a flexible and 3D-consistent manner. Extensive qualitative and quantitative analyses demonstrate our CapHuman can produce well-identity-preserved, photo-realistic, and high-fidelity portraits with content-rich representations and various head renditions, superior to established baselines. Code and checkpoint will be released at https://github.com/VamosC/CapHuman.

cs.CV↗
arXiv2024

Light curves of the explosion of ONe WD+CO WD merger remnant and type Icn supernovae

Type Icn supernovae (SNe Icn) are a newly detected rare subtype of interacting stripped-envelope supernovae which show narrow P-Cygni lines of highly ionized carbon, oxygen, and neon in their early spectra due to the interactions of the SNe ejecta with dense hydrogen- and helium-deficient circumstellar material (CSM). It has been suggested that SNe Icn may have multiple progenitor channels, such as the explosion of carbon-rich Wolf-Rayet stars, or the explosion of stripped-envelope SNe which undergo binary interactions. Among the SNe Icn, SN 2019jc shows unique properties, and previous work inferred that it may stem from the ultra-stripped supernova, but other possibilities still exist. In this work, we aim to simulate the light curves from the explosions of oxygen-neon and carbon-oxygen double white dwarf (WD) merger remnants, and to further investigate whether the corresponding explosions can appear as some particular SNe Icn. We generate the light curves from the explosive remnants and analyse the influence of different parameters on the light curves, such as the ejecta mass, explosion energy, mass of Ni56 and CSM properties. Comparing our results with some SNe Icn, we found that the light curves from the explosions of double WD merger remnants can explain the observable properties of SN 2019jc, which inferred that this special SN Icn may have a different progenitor. Our results indicated that double WD merger may be an alternative model in producing at least one of the SNe Icn.

astro-ph.HE↗
arXiv2024

BrainODE: Dynamic Brain Signal Analysis via Graph-Aided Neural Ordinary Differential Equations

Brain network analysis is vital for understanding the neural interactions regarding brain structures and functions, and identifying potential biomarkers for clinical phenotypes. However, widely used brain signals such as Blood Oxygen Level Dependent (BOLD) time series generated from functional Magnetic Resonance Imaging (fMRI) often manifest three challenges: (1) missing values, (2) irregular samples, and (3) sampling misalignment, due to instrumental limitations, impacting downstream brain network analysis and clinical outcome predictions. In this work, we propose a novel model called BrainODE to achieve continuous modeling of dynamic brain signals using Ordinary Differential Equations (ODE). By learning latent initial values and neural ODE functions from irregular time series, BrainODE effectively reconstructs brain signals at any time point, mitigating the aforementioned three data challenges of brain signals altogether. Comprehensive experimental results on real-world neuroimaging datasets demonstrate the superior performance of BrainODE and its capability of addressing the three data challenges.

cs.LG↗
arXiv2024

Revealing the Two Sides of Data Augmentation: An Asymmetric Distillation-based Win-Win Solution for Open-Set Recognition

In this paper, we reveal the two sides of data augmentation: enhancements in closed-set recognition correlate with a significant decrease in open-set recognition. Through empirical investigation, we find that multi-sample-based augmentations would contribute to reducing feature discrimination, thereby diminishing the open-set criteria. Although knowledge distillation could impair the feature via imitation, the mixed feature with ambiguous semantics hinders the distillation. To this end, we propose an asymmetric distillation framework by feeding teacher model extra raw data to enlarge the benefit of teacher. Moreover, a joint mutual information loss and a selective relabel strategy are utilized to alleviate the influence of hard mixed samples. Our method successfully mitigates the decline in open-set and outperforms SOTAs by 2%~3% AUROC on the Tiny-ImageNet dataset and experiments on large-scale dataset ImageNet-21K demonstrate the generalization of our method.

cs.CV↗
arXiv2024

RulePrompt: Weakly Supervised Text Classification with Prompting PLMs and Self-Iterative Logical Rules

Weakly supervised text classification (WSTC), also called zero-shot or dataless text classification, has attracted increasing attention due to its applicability in classifying a mass of texts within the dynamic and open Web environment, since it requires only a limited set of seed words (label names) for each category instead of labeled data. With the help of recently popular prompting Pre-trained Language Models (PLMs), many studies leveraged manually crafted and/or automatically identified verbalizers to estimate the likelihood of categories, but they failed to differentiate the effects of these category-indicative words, let alone capture their correlations and realize adaptive adjustments according to the unlabeled corpus. In this paper, in order to let the PLM effectively understand each category, we at first propose a novel form of rule-based knowledge using logical expressions to characterize the meanings of categories. Then, we develop a prompting PLM-based approach named RulePrompt for the WSTC task, consisting of a rule mining module and a rule-enhanced pseudo label generation module, plus a self-supervised fine-tuning module to make the PLM align with this task. Within this framework, the inaccurate pseudo labels assigned to texts and the imprecise logical rules associated with categories mutually enhance each other in an alternative manner. That establishes a self-iterative closed loop of knowledge (rule) acquisition and utilization, with seed words serving as the starting point. Extensive experiments validate the effectiveness and robustness of our approach, which markedly outperforms state-of-the-art weakly supervised methods. What is more, our approach yields interpretable category rules, proving its advantage in disambiguating easily-confused categories.

cs.CL↗
arXiv2024

Neural Interaction Energy for Multi-Agent Trajectory Prediction

Maintaining temporal stability is crucial in multi-agent trajectory prediction. Insufficient regularization to uphold this stability often results in fluctuations in kinematic states, leading to inconsistent predictions and the amplification of errors. In this study, we introduce a framework called Multi-Agent Trajectory prediction via neural interaction Energy (MATE). This framework assesses the interactive motion of agents by employing neural interaction energy, which captures the dynamics of interactions and illustrates their influence on the future trajectories of agents. To bolster temporal stability, we introduce two constraints: inter-agent interaction constraint and intra-agent motion constraint. These constraints work together to ensure temporal stability at both the system and agent levels, effectively mitigating prediction fluctuations inherent in multi-agent systems. Comparative evaluations against previous methods on four diverse datasets highlight the superior prediction accuracy and generalization capabilities of our model.

cs.AI↗
arXiv2024

AudioScenic: Audio-Driven Video Scene Editing

Audio-driven visual scene editing endeavors to manipulate the visual background while leaving the foreground content unchanged, according to the given audio signals. Unlike current efforts focusing primarily on image editing, audio-driven video scene editing has not been extensively addressed. In this paper, we introduce AudioScenic, an audio-driven framework designed for video scene editing. AudioScenic integrates audio semantics into the visual scene through a temporal-aware audio semantic injection process. As our focus is on background editing, we further introduce a SceneMasker module, which maintains the integrity of the foreground content during the editing process. AudioScenic exploits the inherent properties of audio, namely, audio magnitude and frequency, to guide the editing process, aiming to control the temporal dynamics and enhance the temporal consistency. First, we present an audio Magnitude Modulator module that adjusts the temporal dynamics of the scene in response to changes in audio magnitude, enhancing the visual dynamics. Second, the audio Frequency Fuser module is designed to ensure temporal consistency by aligning the frequency of the audio with the dynamics of the video scenes, thus improving the overall temporal coherence of the edited videos. These integrated features enable AudioScenic to not only enhance visual diversity but also maintain temporal consistency throughout the video. We present a new metric named temporal score for more comprehensive validation of temporal consistency. We demonstrate substantial advancements of AudioScenic over competing methods on DAVIS and Audioset datasets.

cs.CV↗
arXiv2024

Understanding Privacy Risks of Embeddings Induced by Large Language Models

Large language models (LLMs) show early signs of artificial general intelligence but struggle with hallucinations. One promising solution to mitigate these hallucinations is to store external knowledge as embeddings, aiding LLMs in retrieval-augmented generation. However, such a solution risks compromising privacy, as recent studies experimentally showed that the original text can be partially reconstructed from text embeddings by pre-trained language models. The significant advantage of LLMs over traditional pre-trained models may exacerbate these concerns. To this end, we investigate the effectiveness of reconstructing original knowledge and predicting entity attributes from these embeddings when LLMs are employed. Empirical findings indicate that LLMs significantly improve the accuracy of two evaluated tasks over those from pre-trained models, regardless of whether the texts are in-distribution or out-of-distribution. This underscores a heightened potential for LLMs to jeopardize user privacy, highlighting the negative consequences of their widespread use. We further discuss preliminary strategies to mitigate this risk.

cs.CL↗
arXiv2024

Deep SE(3)-Equivariant Geometric Reasoning for Precise Placement Tasks

Many robot manipulation tasks can be framed as geometric reasoning tasks, where an agent must be able to precisely manipulate an object into a position that satisfies the task from a set of initial conditions. Often, task success is defined based on the relationship between two objects - for instance, hanging a mug on a rack. In such cases, the solution should be equivariant to the initial position of the objects as well as the agent, and invariant to the pose of the camera. This poses a challenge for learning systems which attempt to solve this task by learning directly from high-dimensional demonstrations: the agent must learn to be both equivariant as well as precise, which can be challenging without any inductive biases about the problem. In this work, we propose a method for precise relative pose prediction which is provably SE(3)-equivariant, can be learned from only a few demonstrations, and can generalize across variations in a class of objects. We accomplish this by factoring the problem into learning an SE(3) invariant task-specific representation of the scene and then interpreting this representation with novel geometric reasoning layers which are provably SE(3) equivariant. We demonstrate that our method can yield substantially more precise placement predictions in simulated placement tasks than previous methods trained with the same amount of data, and can accurately represent relative placement relationships data collected from real-world demonstrations. Supplementary information and videos can be found at https://sites.google.com/view/reldist-iclr-2023.

cs.RO↗
arXiv2024

Multiple charge-density-wave gaps in LaSbTe and CeSbTe as revealed by ultrafast spectroscopy

Utilizing ultrafast time-resolved pump-probe spectroscopy measurements, we investigate the photoinduced quasiparticle dynamics of the topological materials LaSbTe and CeSbTe. In LaSbTe, the relaxation of quasiparticles is dominated by two different mechanisms: electron-phonon coupling, and phonon-assisted electron-hole recombination. Significantly, the amplitude of photoinduced reflectivity related to the former one shows two pronounced peaks at 156 K and 263 K, indicating the occurrence of two charge density wave (CDW) phase transitions. The ultrafast responses of CeSbTe share a lot of similarities with LaSbTe, and an additional CDW phase transition at 154 K is revealed in CeSbTe. However, the slower relaxation of CeSbTe exhibits an exotic behavior that deviates from the typical phonon-assisted electron-hole recombination process, probably due to the imbalance between the electron- and hole-type carriers. Unlike LaSbTe, the relaxation times of CeSbTe vary slightly with the pump power, inferring the possible participation of 4$f$ electron in the decay process. In addition, two oscillation modes around 1 THz and 3 THz are identified in both LaSbTe and CeSbTe, which are mostly likely to be coherent phonon modes. These findings unravel the existence of multiple CDW orders in LaSbTe and CeSbTe, offering insights into the underlying physics of these systems.

cond-mat.str-el↗
arXiv2024

Double fast algorithm for solving time-space fractional diffusion problems with spectral fractional Laplacian

This paper presents an efficient and concise double fast algorithm to solve high dimensional time-space fractional diffusion problems with spectral fractional Laplacian. We first establish semi-discrete scheme of time-space fractional diffusion equation, which uses linear finite element or fourth-order compact difference method combining with matrix transfer technique to approximate spectral fractional Laplacian. Then we introduce a fast time-stepping L1 scheme for time discretization. The proposed scheme can exactly evaluate fractional power of matrix and perform matrix-vector multiplication at per time level by using discrete sine transform, which doesn't need to resort to any iteration method and can significantly reduce computation cost and memory requirement. Further, we address stability and convergence analyses of full discrete scheme based on fast time-stepping L1 scheme on graded time mesh. Our analysis shows that the choice of graded mesh factor $ω=(2-α)/α$ shall give an optimal temporal convergence $\mathcal{O}(N^{-(2-α)})$ with $N$ denoting the number of time mesh. Finally, ample numerical examples are delivered to illustrate our theoretical analysis and the efficiency of the suggested scheme.

math.NA↗
arXiv2024

The Early Ultraviolet Light-Curves of Type II Supernovae and the Radii of Their Progenitor Stars

We present a sample of 34 normal SNe II detected with the Zwicky Transient Facility, with multi-band UV light-curves starting at $t \leq 4$ days after explosion, as well as X-ray detections and upper limits. We characterize the early UV-optical colors and provide prescriptions for empirical host-extinction corrections. We show that the $t > 2\,$days UV-optical colors and the blackbody evolution of the sample are consistent with the predictions of spherical phase shock-cooling (SC), independently of the presence of `flash ionization" features. We present a framework for fitting SC models which can reproduce the parameters of a set of multi-group simulations without a significant bias up to 20% in radius and velocity. Observations of about half of the SNe II in the sample are well-fit by models with breakout radii $<10^{14}\,$cm. The other half are typically more luminous, with observations from day 1 onward that are better fit by a model with a large $>10^{14}\,$cm breakout radius. However, these fits predict an early rise during the first day that is too slow. We suggest these large-breakout events are explosions of stars with an inflated envelope or a confined CSM with a steep density profile, at which breakout occurs. Using the X-ray data, we derive constraints on the extended ($\sim10^{15}$ cm) CSM density independent of spectral modeling, and find most SNe II progenitors lose $<10^{-4} M_{\odot}\, \rm yr^{-1}$ a few years before explosion. This provides independent evidence the CSM around many SNe II progenitors is confined. We show that the overall observed breakout radius distribution is skewed to higher radii due to a luminosity bias. We argue that the $66^{+11}_{-22}\%$ of red supergiants (RSG) explode as SNe II with breakout radii consistent with the observed distribution of field RSG, with a tail extending to large radii, likely due to the presence of CSM.

astro-ph.HE↗
arXiv2024

NICEST: Noisy Label Correction and Training for Robust Scene Graph Generation

Nearly all existing scene graph generation (SGG) models have overlooked the ground-truth annotation qualities of mainstream SGG datasets, i.e., they assume: 1) all the manually annotated positive samples are equally correct; 2) all the un-annotated negative samples are absolutely background. In this paper, we argue that neither of the assumptions applies to SGG: there are numerous noisy ground-truth predicate labels that break these two assumptions and harm the training of unbiased SGG models. To this end, we propose a novel NoIsy label CorrEction and Sample Training strategy for SGG: NICEST. Specifically, it consists of two parts: NICE and NIST, which rule out these noisy label issues by generating high-quality samples and the effective training strategy, respectively. NICE first detects noisy samples and then reassigns them more high-quality soft predicate labels. NIST is a multi-teacher knowledge distillation based training strategy, which enables the model to learn unbiased fusion knowledge. And a dynamic trade-off weighting strategy in NIST is designed to penalize the bias of different teachers. Due to the model-agnostic nature of both NICE and NIST, our NICEST can be seamlessly incorporated into any SGG architecture to boost its performance on different predicate categories. In addition, to better evaluate the generalization of SGG models, we further propose a new benchmark VG-OOD, by re-organizing the prevalent VG dataset and deliberately making the predicate distributions of the training and test sets as different as possible for each subject-object category pair. This new benchmark helps disentangle the influence of subject-object category based frequency biases. Extensive ablations and results on different backbones and tasks have attested to the effectiveness and generalization ability of each component of NICEST.

cs.CV↗
arXiv2024

Joint Conditional Diffusion Model for Image Restoration with Mixed Degradations

Image restoration is rather challenging in adverse weather conditions, especially when multiple degradations occur simultaneously. Blind image decomposition was proposed to tackle this issue, however, its effectiveness heavily relies on the accurate estimation of each component. Although diffusion-based models exhibit strong generative abilities in image restoration tasks, they may generate irrelevant contents when the degraded images are severely corrupted. To address these issues, we leverage physical constraints to guide the whole restoration process, where a mixed degradation model based on atmosphere scattering model is constructed. Then we formulate our Joint Conditional Diffusion Model (JCDM) by incorporating the degraded image and degradation mask to provide precise guidance. To achieve better color and detail recovery results, we further integrate a refinement network to reconstruct the restored image, where Uncertainty Estimation Block (UEB) is employed to enhance the features. Extensive experiments performed on both multi-weather and weather-specific datasets demonstrate the superiority of our method over state-of-the-art competing methods.

cs.CV↗
arXiv2024

SIFU: Side-view Conditioned Implicit Function for Real-world Usable Clothed Human Reconstruction

Creating high-quality 3D models of clothed humans from single images for real-world applications is crucial. Despite recent advancements, accurately reconstructing humans in complex poses or with loose clothing from in-the-wild images, along with predicting textures for unseen areas, remains a significant challenge. A key limitation of previous methods is their insufficient prior guidance in transitioning from 2D to 3D and in texture prediction. In response, we introduce SIFU (Side-view Conditioned Implicit Function for Real-world Usable Clothed Human Reconstruction), a novel approach combining a Side-view Decoupling Transformer with a 3D Consistent Texture Refinement pipeline.SIFU employs a cross-attention mechanism within the transformer, using SMPL-X normals as queries to effectively decouple side-view features in the process of mapping 2D features to 3D. This method not only improves the precision of the 3D models but also their robustness, especially when SMPL-X estimates are not perfect. Our texture refinement process leverages text-to-image diffusion-based prior to generate realistic and consistent textures for invisible views. Through extensive experiments, SIFU surpasses SOTA methods in both geometry and texture reconstruction, showcasing enhanced robustness in complex scenarios and achieving an unprecedented Chamfer and P2S measurement. Our approach extends to practical applications such as 3D printing and scene building, demonstrating its broad utility in real-world scenarios. Project page https://river-zhang.github.io/SIFU-projectpage/ .

cs.CV↗