arXiv subjects

Yutong He

Publications and source records attributed to Yutong He.

At least 55 records · Page 3Linked to original sources

arXiv2024

Weather Prediction with Diffusion Guided by Realistic Forecast Processes

Weather forecasting remains a crucial yet challenging domain, where recently developed models based on deep learning (DL) have approached the performance of traditional numerical weather prediction (NWP) models. However, these DL models, often complex and resource-intensive, face limitations in flexibility post-training and in incorporating NWP predictions, leading to reliability concerns due to potential unphysical predictions. In response, we introduce a novel method that applies diffusion models (DM) for weather forecasting. In particular, our method can achieve both direct and iterative forecasting with the same modeling framework. Our model is not only capable of generating forecasts independently but also uniquely allows for the integration of NWP predictions, even with varying lead times, during its sampling process. The flexibility and controllability of our model empowers a more trustworthy DL system for the general weather community. Additionally, incorporating persistence and climatology data further enhances our model's long-term forecasting stability. Our empirical findings demonstrate the feasibility and generalizability of this approach, suggesting a promising direction for future, more sophisticated diffusion models without the need for retraining.

physics.ao-ph↗

arXiv2024

Unbiased Compression Saves Communication in Distributed Optimization: When and How Much?

Communication compression is a common technique in distributed optimization that can alleviate communication overhead by transmitting compressed gradients and model parameters. However, compression can introduce information distortion, which slows down convergence and incurs more communication rounds to achieve desired solutions. Given the trade-off between lower per-round communication costs and additional rounds of communication, it is unclear whether communication compression reduces the total communication cost. This paper explores the conditions under which unbiased compression, a widely used form of compression, can reduce the total communication cost, as well as the extent to which it can do so. To this end, we present the first theoretical formulation for characterizing the total communication cost in distributed optimization with communication compression. We demonstrate that unbiased compression alone does not necessarily save the total communication cost, but this outcome can be achieved if the compressors used by all workers are further assumed independent. We establish lower bounds on the communication rounds required by algorithms using independent unbiased compressors to minimize smooth convex functions and show that these lower bounds are tight by refining the analysis for ADIANA. Our results reveal that using independent unbiased compression can reduce the total communication cost by a factor of up to $Θ(\sqrt{\min\{n, κ\}})$ when all local smoothness constants are constrained by a common upper bound, where $n$ is the number of workers and $κ$ is the condition number of the functions being minimized. These theoretical findings are supported by experimental results.

cs.LG↗

arXiv2023

Manifold Preserving Guided Diffusion

Despite the recent advancements, conditional image generation still faces challenges of cost, generalizability, and the need for task-specific training. In this paper, we propose Manifold Preserving Guided Diffusion (MPGD), a training-free conditional generation framework that leverages pretrained diffusion models and off-the-shelf neural networks with minimal additional inference cost for a broad range of tasks. Specifically, we leverage the manifold hypothesis to refine the guided diffusion steps and introduce a shortcut algorithm in the process. We then propose two methods for on-manifold training-free guidance using pre-trained autoencoders and demonstrate that our shortcut inherently preserves the manifolds when applied to latent diffusion models. Our experiments show that MPGD is efficient and effective for solving a variety of conditional generation applications in low-compute settings, and can consistently offer up to 3.8x speed-ups with the same number of diffusion steps while maintaining high sample quality compared to the baselines.

cs.LG↗

arXiv2023

Towards reporting bias in visual-language datasets: bimodal augmentation by decoupling object-attribute association

Reporting bias arises when people assume that some knowledge is universally understood and hence, do not necessitate explicit elaboration. In this paper, we focus on the wide existence of reporting bias in visual-language datasets, embodied as the object-attribute association, which can subsequentially degrade models trained on them. To mitigate this bias, we propose a bimodal augmentation (BiAug) approach through object-attribute decoupling to flexibly synthesize visual-language examples with a rich array of object-attribute pairing and construct cross-modal hard negatives. We employ large language models (LLMs) in conjunction with a grounding object detector to extract target objects. Subsequently, the LLM generates a detailed attribute description for each object and produces a corresponding hard negative counterpart. An inpainting model is then used to create images based on these detailed object descriptions. By doing so, the synthesized examples explicitly complement omitted objects and attributes to learn, and the hard negative pairs steer the model to distinguish object attributes. Our experiments demonstrated that BiAug is superior in object-attribute understanding. In addition, BiAug also improves the performance on zero-shot retrieval tasks on general benchmarks like MSCOCO and Flickr30K. BiAug refines the way of collecting text-image datasets. Mitigating the reporting bias helps models achieve a deeper understanding of visual-language phenomena, expanding beyond mere frequent patterns to encompass the richness and diversity of real-world scenarios.

cs.CV↗

arXiv2023

Sphere2Vec: A General-Purpose Location Representation Learning over a Spherical Surface for Large-Scale Geospatial Predictions

Generating learning-friendly representations for points in space is a fundamental and long-standing problem in ML. Recently, multi-scale encoding schemes (such as Space2Vec and NeRF) were proposed to directly encode any point in 2D/3D Euclidean space as a high-dimensional vector, and has been successfully applied to various geospatial prediction and generative tasks. However, all current 2D and 3D location encoders are designed to model point distances in Euclidean space. So when applied to large-scale real-world GPS coordinate datasets, which require distance metric learning on the spherical surface, both types of models can fail due to the map projection distortion problem (2D) and the spherical-to-Euclidean distance approximation error (3D). To solve these problems, we propose a multi-scale location encoder called Sphere2Vec which can preserve spherical distances when encoding point coordinates on a spherical surface. We developed a unified view of distance-reserving encoding on spheres based on the DFS. We also provide theoretical proof that the Sphere2Vec preserves the spherical surface distance between any two points, while existing encoding schemes do not. Experiments on 20 synthetic datasets show that Sphere2Vec can outperform all baseline models on all these datasets with up to 30.8% error rate reduction. We then apply Sphere2Vec to three geo-aware image classification tasks - fine-grained species recognition, Flickr image recognition, and remote sensing image classification. Results on 7 real-world datasets show the superiority of Sphere2Vec over multiple location encoders on all three tasks. Further analysis shows that Sphere2Vec outperforms other location encoder models, especially in the polar regions and data-sparse areas because of its nature for spherical surface distance preservation. Code and data are available at https://gengchenmai.github.io/sphere2vec-website/.

cs.CV↗

arXiv2023

Localized Text-to-Image Generation for Free via Cross Attention Control

Despite the tremendous success in text-to-image generative models, localized text-to-image generation (that is, generating objects or features at specific locations in an image while maintaining a consistent overall generation) still requires either explicit training or substantial additional inference time. In this work, we show that localized generation can be achieved by simply controlling cross attention maps during inference. With no additional training, model architecture modification or inference time, our proposed cross attention control (CAC) provides new open-vocabulary localization abilities to standard text-to-image models. CAC also enhances models that are already trained for localized generation when deployed at inference time. Furthermore, to assess localized text-to-image generation performance automatically, we develop a standardized suite of evaluations using large pretrained recognition models. Our experiments show that CAC improves localized generation performance with various types of location information ranging from bounding boxes to semantic segmentation maps, and enhances the compositional capability of state-of-the-art text-to-image generative models.

cs.CV↗

arXiv2023

Modified propagation of gravitational waves from the early radiation era

We study the propagation of cosmological gravitational wave (GW) backgrounds from the early radiation era until the present day in modified theories of gravity. Comparing to general relativity (GR), we study the effects that modified gravity parameters, such as the GW friction $α_{\rm M}$ and the tensor speed excess $α_{\rm T}$, have on the present-day GW spectrum. We use both the WKB estimate, which provides an analytical description but fails at superhorizon scales, and numerical simulations that allow us to go beyond the WKB approximation. We show that a constant $α_{\rm T}$ makes relatively insignificant changes to the GR solution, especially taking into account the constraints on its value from GW observations by the LIGO--Virgo collaboration, while $α_{\rm M}$ can introduce modifications to the spectral slopes of the GW energy spectrum in the low-frequency regime depending on the considered time evolution of $α_{\rm M}$. The latter effect is additional to the damping or growth occurring equally at all scales that can be predicted by the WKB approximation. In light of the recent observations by pulsar timing array (PTA) collaborations, and the potential observations by future detectors such as SKA, LISA, DECIGO, BBO, or ET, we show that, in most of the cases, constraints cannot be placed on the effects of $α_{\rm M}$ and the initial GW energy density $\mathcal{E}_{\rm GW}^*$ separately, but only on the combined effects of the two, unless the signal is observed at different frequency ranges. In particular, we provide some constraints on the combined effects from the reported PTA observations.

gr-qc↗

arXiv2023

CSP: Self-Supervised Contrastive Spatial Pre-Training for Geospatial-Visual Representations

Geo-tagged images are publicly available in large quantities, whereas labels such as object classes are rather scarce and expensive to collect. Meanwhile, contrastive learning has achieved tremendous success in various natural image and language tasks with limited labeled data. However, existing methods fail to fully leverage geospatial information, which can be paramount to distinguishing objects that are visually similar. To directly leverage the abundant geospatial information associated with images in pre-training, fine-tuning, and inference stages, we present Contrastive Spatial Pre-Training (CSP), a self-supervised learning framework for geo-tagged images. We use a dual-encoder to separately encode the images and their corresponding geo-locations, and use contrastive objectives to learn effective location representations from images, which can be transferred to downstream supervised tasks such as image classification. Experiments show that CSP can improve model performance on both iNat2018 and fMoW datasets. Especially, on iNat2018, CSP significantly boosts the model performance with 10-34% relative improvement with various labeled training data sampling ratios.

cs.CV↗

arXiv2023

Neural Generation Meets Real People: Building a Social, Informative Open-Domain Dialogue Agent

We present Chirpy Cardinal, an open-domain social chatbot. Aiming to be both informative and conversational, our bot chats with users in an authentic, emotionally intelligent way. By integrating controlled neural generation with scaffolded, hand-written dialogue, we let both the user and bot take turns driving the conversation, producing an engaging and socially fluent experience. Deployed in the fourth iteration of the Alexa Prize Socialbot Grand Challenge, Chirpy Cardinal handled thousands of conversations per day, placing second out of nine bots with an average user rating of 3.58/5.

cs.CL↗

arXiv2023

SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagery

Unsupervised pre-training methods for large vision models have shown to enhance performance on downstream supervised tasks. Developing similar techniques for satellite imagery presents significant opportunities as unlabelled data is plentiful and the inherent temporal and multi-spectral structure provides avenues to further improve existing pre-training strategies. In this paper, we present SatMAE, a pre-training framework for temporal or multi-spectral satellite imagery based on Masked Autoencoder (MAE). To leverage temporal information, we include a temporal embedding along with independently masking image patches across time. In addition, we demonstrate that encoding multi-spectral data as groups of bands with distinct spectral positional encodings is beneficial. Our approach yields strong improvements over previous state-of-the-art techniques, both in terms of supervised learning performance on benchmark datasets (up to $\uparrow$ 7%), and transfer learning performance on downstream remote sensing tasks, including land cover classification (up to $\uparrow$ 14%) and semantic segmentation. Code and data are available on the project website: https://sustainlab-group.github.io/SatMAE/

cs.CV↗

arXiv2022

Modified gravity approaches to the cosmological constant problem

The cosmological constant and its phenomenology remain among the greatest puzzles in theoretical physics. We review how modifications of Einstein's general relativity could alleviate the different problems associated with it that result from the interplay of classical gravity and quantum field theory. We introduce a modern and concise language to describe the problems associated with its phenomenology, and inspect no-go theorems and their loopholes to motivate the approaches discussed here. Constrained gravity approaches exploit minimal departures from general relativity; massive gravity introduces mass to the graviton; Horndeski theories lead to the breaking of translational invariance of the vacuum; and models with extra dimensions change the symmetries of the vacuum. We also review screening mechanisms that have to be present in some of these theories if they aim to recover the success of general relativity on small scales as well. Finally, we summarise the statuses of these models in their attempt to solve the different cosmological constant problems while being able to account for current astrophysical and cosmological observations.

gr-qc↗

arXiv2022

Passive Non-line-of-sight Imaging for Moving Targets with an Event Camera

Non-line-of-sight (NLOS) imaging is an emerging technique for detecting objects behind obstacles or around corners. Recent studies on passive NLOS mainly focus on steady-state measurement and reconstruction methods, which show limitations in recognition of moving targets. To the best of our knowledge, we propose a novel event-based passive NLOS imaging method. We acquire asynchronous event-based data which contains detailed dynamic information of the NLOS target, and efficiently ease the degradation of speckle caused by movement. Besides, we create the first event-based NLOS imaging dataset, NLOS-ES, and the event-based feature is extracted by time-surface representation. We compare the reconstructions through event-based data with frame-based data. The event-based method performs well on PSNR and LPIPS, which is 20% and 10% better than frame-based method, while the data volume takes only 2% of traditional method.

cs.CV↗

arXiv2022

Spatial-Temporal Super-Resolution of Satellite Imagery via Conditional Pixel Synthesis

High-resolution satellite imagery has proven useful for a broad range of tasks, including measurement of global human population, local economic livelihoods, and biodiversity, among many others. Unfortunately, high-resolution imagery is both infrequently collected and expensive to purchase, making it hard to efficiently and effectively scale these downstream tasks over both time and space. We propose a new conditional pixel synthesis model that uses abundant, low-cost, low-resolution imagery to generate accurate high-resolution imagery at locations and times in which it is unavailable. We show that our model attains photo-realistic sample quality and outperforms competing baselines on a key downstream task -- object counting -- particularly in geographic locations where conditions on the ground are changing rapidly.

cs.CV↗

arXiv2022

Tracking Urbanization in Developing Regions with Remote Sensing Spatial-Temporal Super-Resolution

Automated tracking of urban development in areas where construction information is not available became possible with recent advancements in machine learning and remote sensing. Unfortunately, these solutions perform best on high-resolution imagery, which is expensive to acquire and infrequently available, making it difficult to scale over long time spans and across large geographies. In this work, we propose a pipeline that leverages a single high-resolution image and a time series of publicly available low-resolution images to generate accurate high-resolution time series for object tracking in urban construction. Our method achieves significant improvement in comparison to baselines using single image super-resolution, and can assist in extending the accessibility and scalability of building construction tracking across the developing world.

eess.IV↗

arXiv2022

SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations

Guided image synthesis enables everyday users to create and edit photo-realistic images with minimum effort. The key challenge is balancing faithfulness to the user input (e.g., hand-drawn colored strokes) and realism of the synthesized image. Existing GAN-based methods attempt to achieve such balance using either conditional GANs or GAN inversions, which are challenging and often require additional training data or loss functions for individual applications. To address these issues, we introduce a new image synthesis and editing method, Stochastic Differential Editing (SDEdit), based on a diffusion model generative prior, which synthesizes realistic images by iteratively denoising through a stochastic differential equation (SDE). Given an input image with user guide of any type, SDEdit first adds noise to the input, then subsequently denoises the resulting image through the SDE prior to increase its realism. SDEdit does not require task-specific training or inversions and can naturally achieve the balance between realism and faithfulness. SDEdit significantly outperforms state-of-the-art GAN-based methods by up to 98.09% on realism and 91.72% on overall satisfaction scores, according to a human perception study, on multiple tasks, including stroke-based image synthesis and editing as well as image compositing.

cs.CV↗

arXiv2021

Leading-order nonlinear gravitational waves from reheating magnetogeneses

We study the leading-order nonlinear gravitational waves (GWs) produced by an electromagnetic (EM) stress in reheating magnetogenesis scenarios. Both nonhelical and helical magnetic fields are considered. By numerically solving the linear and leading-order nonlinear GW equations, we find that the GW energy from the latter is usually larger. We compare their differences in terms of the GW spectrum and parameterize the GW energy difference due to the nonlinear term, $Δ\mathcal{E}_{\rm GW}$, in terms of EM energy $\mathcal{E}_{\rm EM}$ as $Δ\mathcal{E}_{\rm GW}=(\tilde p\mathcal{E}_{\rm EM}/k_*)^3$, where $k_*$ is the characteristic wave number, $\tilde p=0.84$ and $0.88$ are found in the nonhelical and helical cases, respectively, with reheating around the QCD energy scale, while $\tilde p=0.45$ is found at the electroweak energy scale. We also compare the polarization spectrum of the linear and nonlinear cases and find that adding the nonlinear term usually yields a decrease in the polarization that is proportional to the EM energy density. We parameterize the fractional polarization suppression as $|Δ\mathcal{P}_{\rm GW}/\mathcal{P}_{\rm GW}|=\tilde r \mathcal{E}_{\rm EM}/k_*$ and find $\tilde r = 1.2 \times 10^{-1}$, $7.2 \times 10^{-4}$, and $3.2 \times 10^{-2}$ for the helical cases with reheating temperatures $T_{\rm r} = 300 {\rm TeV}$, $8 {\rm GeV}$, and $120 {\rm MeV}$, respectively. Prospects of observation by pulsar timing arrays, space-based interferometers, and other novel detection proposals are also discussed.

astro-ph.CO↗

arXiv2021

Can we observe the QCD phase transition-generated gravitational waves through pulsar timing arrays?

We perform numerical simulations of gravitational waves (GWs) induced by hydrodynamic and hydromagnetic turbulent sources that might have been present at cosmological quantum chromodynamic (QCD) phase transitions. For turbulent energies of about 4% of the radiation energy density, the typical scale of such motions may have been a sizable fraction of the Hubble scale at that time. The resulting GWs are found to have an energy fraction of about $10^{-9}$ of the critical energy density in the nHz range today and may already have been observed by the NANOGrav collaboration. This is further made possible by our findings of shallower spectra proportional to the square root of the frequency for nonhelical hydromagnetic turbulence. This implies more power at low frequencies than for the steeper spectra previously anticipated. The behavior toward higher frequencies depends strongly on the nature of the turbulence. For vortical hydrodynamic and hydromagnetic turbulence, there is a sharp drop of spectral GW energy by up to five orders of magnitude in the presence of helicity, and somewhat less in the absence of helicity. For acoustic hydrodynamic turbulence, the sharp drop is replaced by a power law decay, albeit with a rather steep slope. Our study supports earlier findings of a quadratic scaling of the GW energy with the magnetic energy of the turbulence and inverse quadratic scaling with the peak frequency, which leads to larger GW energies under QCD conditions.

astro-ph.CO↗

arXiv2021

Simulations of helical inflationary magnetogenesis and gravitational waves

Using numerical simulations of helical inflationary magnetogenesis in a low reheating temperature scenario, we show that the magnetic energy spectrum is strongly peaked at a particular wavenumber that depends on the reheating temperature. Gravitational waves (GWs) are produced at frequencies between 3 nHz and 50 mHz for reheating temperatures between 150 MeV and 3x10^5 GeV, respectively. At and below the peak frequency, the stress spectrum is always found to be that of white noise. This implies a linear increase of GW energy per logarithmic wavenumber interval, instead of a cubic one, as previously thought. Both in the helical and nonhelical cases, the GW spectrum is followed by a sharp drop for frequencies above the respective peak frequency. In this magnetogenesis scenario, the presence of a helical term extends the peak of the GW spectrum and therefore also the position of the aforementioned drop toward larger frequencies compared to the case without helicity. This might make a difference in it being detectable with space interferometers. The efficiency of GW production is found to be almost the same as in the nonhelical case, and independent of the reheating temperature, provided the electromagnetic energy at the end of reheating is fixed to be a certain fraction of the radiation energy density. Also, contrary to the case without helicity, the electric energy is now less than the magnetic energy during reheating. The fractional circular polarization is found to be nearly hundred per cent in a certain range below the peak frequency range.

astro-ph.CO↗