arXiv subjects

Hao Chen

Publications and source records attributed to Hao Chen.

At least 775 records · Page 43Linked to original sources

arXiv2024

ExtremeCast: Boosting Extreme Value Prediction for Global Weather Forecast

Data-driven weather forecast based on machine learning (ML) has experienced rapid development and demonstrated superior performance in the global medium-range forecast compared to traditional physics-based dynamical models. However, most of these ML models struggle with accurately predicting extreme weather, which is related to training loss and the uncertainty of weather systems. Through mathematical analysis, we prove that the use of symmetric losses, such as the Mean Squared Error (MSE), leads to biased predictions and underestimation of extreme values. To address this issue, we introduce Exloss, a novel loss function that performs asymmetric optimization and highlights extreme values to obtain accurate extreme weather forecast. Beyond the evolution in training loss, we introduce a training-free extreme value enhancement module named ExBooster, which captures the uncertainty in prediction outcomes by employing multiple random samples, thereby increasing the hit rate of low-probability extreme events. Combined with an advanced global weather forecast model, extensive experiments show that our solution can achieve state-of-the-art performance in extreme weather prediction, while maintaining the overall forecast accuracy comparable to the top medium-range forecast models.

cs.LG↗

arXiv2024

Multi-Antenna Broadband Backscatter Communications

Backscatter communication offers a promising solution to connect massive Internet-of-Things (IoT) devices with low cost and high energy efficiency. Nevertheless, its inherently passive nature limits transmission reliability, thereby hindering improvements in communication range and data rate. To overcome these challenges, we introduce a bistatic broadband backscatter communication (BBBC) system, which equips the backscatter device (BD) with multiple antennas. In the proposed BBBC system, a radio frequency (RF) source directs a sinusoidal signal to the BD, facilitating single-carrier block transmission at the BD. Meanwhile, without requiring channel state information (CSI), cyclic delay diversity (CDD) is employed at the multi-antenna BD to enhance transmission reliability through additional cyclically delayed backscattered signals. We also propose a receiver design that includes preprocessing of the time-domain received signal, pilot-based parameter estimation, and frequency-domain equalization, enabling low-complexity detection of the backscattered signal. Leveraging the matched filter bound (MFB), we analyze the achievable diversity gains in terms of outage probability. Our analysis reveals that spatial diversity is achievable under general Rayleigh fading conditions, and both frequency and spatial diversity are attainable in scenarios where the forward link experiences a line-of-sight (LoS) channel. Simulation results validate the effectiveness of the proposed BBBC system. As the number of BD antennas increases, our results show that the proposed scheme not only enhances array gain but also improves diversity order, significantly reducing both outage probability and bit error rate (BER). Consequently, it outperforms conventional schemes that yield only minor gains.

cs.IT↗

arXiv2024

Thermodynamic topology of Phantom AdS Black Holes in Massive Gravity

In this work, we explore the thermodynamic topology of phantom AdS black holes in the context of massive gravity. To this end, we evaluate these black holes in two distinct ensembles: the canonical and grand canonical ensembles (GCE). We begin by examining the topological charge linked to the critical point and confirming the existence of a conventional critical point $(CP_{1})$ in the canonical ensemble (CE), this critical point has a topological charge of $-1$ and acts as a point of phase annihilation, this situation can only be considered within the context of the classical Einstein-Maxwell (CEM) theory $(η=1)$, while no critical point is identified in the GCE. Furthermore, we consider black holes as a topological defect within the thermodynamic space. To gain an understanding of the local and global topological configuration of this defect, we will analyze its winding numbers, and observe that the total topological charge in the CE consistently remains at $1$. When the system experiences a pressure below the critical threshold, it gives rise to the occurrence of annihilation and generation points. The value of electric potential determines whether the total topological charge in the GCE is zero or one. As a result, we detect a point of generation point or absence of generation/annihilation point. Based on our analysis, it can be inferred that ensembles significantly impact the topological class of phantom AdS black holes in massive gravity.

gr-qc↗

arXiv2024

Self-Supervised Video Desmoking for Laparoscopic Surgery

Due to the difficulty of collecting real paired data, most existing desmoking methods train the models by synthesizing smoke, generalizing poorly to real surgical scenarios. Although a few works have explored single-image real-world desmoking in unpaired learning manners, they still encounter challenges in handling dense smoke. In this work, we address these issues together by introducing the self-supervised surgery video desmoking (SelfSVD). On the one hand, we observe that the frame captured before the activation of high-energy devices is generally clear (named pre-smoke frame, PS frame), thus it can serve as supervision for other smoky frames, making real-world self-supervised video desmoking practically feasible. On the other hand, in order to enhance the desmoking performance, we further feed the valuable information from PS frame into models, where a masking strategy and a regularization term are presented to avoid trivial solutions. In addition, we construct a real surgery video dataset for desmoking, which covers a variety of smoky scenes. Extensive experiments on the dataset show that our SelfSVD can remove smoke more effectively and efficiently while recovering more photo-realistic details than the state-of-the-art methods. The dataset, codes, and pre-trained models are available at \url{https://github.com/ZcsrenlongZ/SelfSVD}.

cs.CV↗

arXiv2024

Kohayakawa-Nagle-R{ö}dl-Schacht conjecture for subdivisions

In this paper, we study the well-known Kohayakawa-Nagle-R{ö}dl-Schacht (KNRS) conjecture, with a specific focus on graph subdivisions. The KNRS conjecture asserts that for any graph $H$, locally dense graphs contain asymptotically at least the number of copies of $H$ found in a random graph with the same edge density. We prove the following results about $k$-subdivisions of graphs (obtained by replacing edges with paths of length $k+1$): (1). If $H$ satisfies the KNRS conjecture, then its $(2k-1)$-subdivision satisfies Sidorenko's conjecture, extending a prior result of Conlon, Kim, Lee and Lee; (2). If $H$ satisfies the KNRS conjecture, then its $2k$-subdivision satisfies a constant-fraction version of the KNRS conjecture; (3). If $H$ is regular and satisfies the KNRS conjecture, then its $2k$-subdivision also satisfies the KNRS conjecture. These findings imply that all balanced subdivisions of cliques satisfy the KNRS conjecture, improving upon a recent result of Bradač, Sudakov and Wigerson. Our work provides new insights into this pivotal conjecture in extremal graph theory.

math.CO↗

arXiv2024

A new graph-based two-sample test for multivariate and object data

Two-sample tests for multivariate data and especially for non-Euclidean data are not well explored. This paper presents a novel test statistic based on a similarity graph constructed on the pooled observations from the two samples. It can be applied to multivariate data and non-Euclidean data as long as a dissimilarity measure on the sample space can be defined, which can usually be provided by domain experts. Existing tests based on a similarity graph lack power either for location or for scale alternatives. The new test utilizes a common pattern that was overlooked previously, and works for both types of alternatives. The test exhibits substantial power gains in simulation studies. Its asymptotic permutation null distribution is derived and shown to work well under finite samples, facilitating its application to large data sets. The new test is illustrated on two applications: The assessment of covariate balance in a matched observational study, and the comparison of network data under different conditions.

stat.ME↗

arXiv2024

LoRAPrune: Structured Pruning Meets Low-Rank Parameter-Efficient Fine-Tuning

Large Language Models (LLMs), such as LLaMA and T5, have shown exceptional performance across various tasks through fine-tuning. Although low-rank adaption (LoRA) has emerged to cheaply fine-tune these LLMs on downstream tasks, their deployment is still hindered by the vast model scale and computational costs. Post-training model pruning offers a way to compress LLMs. However, the current pruning methods designed for LLMs are not compatible with LoRA. This is due to their utilization of unstructured pruning on LLMs, impeding the merging of LoRA weights, or their dependence on the gradients of pre-trained weights to guide pruning, which can impose significant memory overhead. To this end, we propose LoRAPrune, a new framework that delivers an accurate structured pruned model in a highly memory-efficient manner. Specifically, we first design a LoRA-guided pruning criterion, which uses the weights and gradients of LoRA, rather than the gradients of pre-trained weights for importance estimation. We subsequently integrate this criterion into an iterative pruning process, effectively removing redundant channels and heads. Extensive experimental results demonstrate the superior performance of our LoRAPrune over existing approaches on the LLaMA series models. At a 50\% compression rate, LoRAPrune demonstrates superior performance over LLM-Pruner, achieving a reduction in perplexity by 4.81 on WikiText2 and 3.46 on PTB, while also decreasing memory usage by 52.6%. Besides, LoRAPrune also matches semi-structural pruning across multiple LLMs, proving its wide applicability. The code is available at https://github.com/aim-uofa/LoRAPrune.

cs.LG↗

arXiv2024

Surgformer: Surgical Transformer with Hierarchical Temporal Attention for Surgical Phase Recognition

Existing state-of-the-art methods for surgical phase recognition either rely on the extraction of spatial-temporal features at a short-range temporal resolution or adopt the sequential extraction of the spatial and temporal features across the entire temporal resolution. However, these methods have limitations in modeling spatial-temporal dependency and addressing spatial-temporal redundancy: 1) These methods fail to effectively model spatial-temporal dependency, due to the lack of long-range information or joint spatial-temporal modeling. 2) These methods utilize dense spatial features across the entire temporal resolution, resulting in significant spatial-temporal redundancy. In this paper, we propose the Surgical Transformer (Surgformer) to address the issues of spatial-temporal modeling and redundancy in an end-to-end manner, which employs divided spatial-temporal attention and takes a limited set of sparse frames as input. Moreover, we propose a novel Hierarchical Temporal Attention (HTA) to capture both global and local information within varied temporal resolutions from a target frame-centric perspective. Distinct from conventional temporal attention that primarily emphasizes dense long-range similarity, HTA not only captures long-term information but also considers local latent consistency among informative frames. HTA then employs pyramid feature aggregation to effectively utilize temporal information across diverse temporal resolutions, thereby enhancing the overall temporal representation. Extensive experiments on two challenging benchmark datasets verify that our proposed Surgformer performs favorably against the state-of-the-art methods. The code is released at https://github.com/isyangshu/Surgformer.

cs.CV↗

arXiv2024

M1 Radiative and spin-nonflip $ππ$ transitions of $B_c$ states in the Cornell potential model

In this paper, we mainly predict the rates of $M$1 radiative and spin-nonflip $ππ$ transitions of the $B_{c}$-meson under the nonrelativistic Cornell potential model with a screening potential effect. We employ the numerical wave function to determine the $M$1 radiative transition widths of $B_c$ excited states and utilize the Kuang-Yan proposed method for the spin-nonflip $ππ$ transitions among $B_c$ states. Our theoretical results are valuable for studying the $M$1 radiative and spin-nonflip $ππ$ transition processes of $B_c$ states in experiments.

hep-ph↗

arXiv2024

Explain via Any Concept: Concept Bottleneck Model with Open Vocabulary Concepts

The concept bottleneck model (CBM) is an interpretable-by-design framework that makes decisions by first predicting a set of interpretable concepts, and then predicting the class label based on the given concepts. Existing CBMs are trained with a fixed set of concepts (concepts are either annotated by the dataset or queried from language models). However, this closed-world assumption is unrealistic in practice, as users may wonder about the role of any desired concept in decision-making after the model is deployed. Inspired by the large success of recent vision-language pre-trained models such as CLIP in zero-shot classification, we propose "OpenCBM" to equip the CBM with open vocabulary concepts via: (1) Aligning the feature space of a trainable image feature extractor with that of a CLIP's image encoder via a prototype based feature alignment; (2) Simultaneously training an image classifier on the downstream dataset; (3) Reconstructing the trained classification head via any set of user-desired textual concepts encoded by CLIP's text encoder. To reveal potentially missing concepts from users, we further propose to iteratively find the closest concept embedding to the residual parameters during the reconstruction until the residual is small enough. To the best of our knowledge, our "OpenCBM" is the first CBM with concepts of open vocabularies, providing users the unique benefit such as removing, adding, or replacing any desired concept to explain the model's prediction even after a model is trained. Moreover, our model significantly outperforms the previous state-of-the-art CBM by 9% in the classification accuracy on the benchmark dataset CUB-200-2011.

cs.CV↗

arXiv2024

Feedback Reciprocal Graph Collaborative Filtering

Collaborative filtering on user-item interaction graphs has achieved success in the industrial recommendation. However, recommending users' truly fascinated items poses a seesaw dilemma for collaborative filtering models learned from the interaction graph. On the one hand, not all items that users interact with are equally appealing. Some items are genuinely fascinating to users, while others are unfascinated. Training graph collaborative filtering models in the absence of distinction between them can lead to the recommendation of unfascinating items to users. On the other hand, disregarding the interacted but unfascinating items during graph collaborative filtering will result in an incomplete representation of users' interaction intent, leading to a decline in the model's recommendation capabilities. To address this seesaw problem, we propose Feedback Reciprocal Graph Collaborative Filtering (FRGCF), which emphasizes the recommendation of fascinating items while attenuating the recommendation of unfascinating items. Specifically, FRGCF first partitions the entire interaction graph into the Interacted & Fascinated (I&F) graph and the Interacted & Unfascinated (I&U) graph based on the user feedback. Then, FRGCF introduces separate collaborative filtering on the I&F graph and the I&U graph with feedback-reciprocal contrastive learning and macro-level feedback modeling. This enables the I&F graph recommender to learn multi-grained interaction characteristics from the I&U graph without being misdirected by it. Extensive experiments on four benchmark datasets and a billion-scale industrial dataset demonstrate that FRGCF improves the performance by recommending more fascinating items and fewer unfascinating items. Besides, online A/B tests on Taobao's recommender system verify the superiority of FRGCF.

cs.IR↗

arXiv2024

Navigation-grade interferometric air-core antiresonant fibre optic gyroscope with enhanced thermal stability

We present a groundbreaking navigation-grade interferometric air-core fibre optic gyroscope (IFOG) using a quadrupolar-wound coil of four-tube truncated double nested antiresonant nodeless fibre (tDNANF). This state-of-the-art tDNANF simultaneously achieves low loss, low bend loss, single-spatial-mode operation, and exceptional linear polarization purity over a broad wavelength range. Our 469 m tDNANF coil demonstrated a polarization extinction ratio (PER) of ~20 dB when illuminated by an amplified spontaneous emission (ASE) source spanning 1525-1565 nm. Under these conditions, the gyro archives an angular random walk (ARW) of 0.0038 deg h-1/2 and a bias-stability (BS) drift over 8500 s of 0.0014 deg h-1, marking the first instance of navigation-grade performance in air-core FOGs. Additionally, we validated the low thermal sensitivity of air-core FOGs, with reductions of 9.24/10.68/6.82 compared to that of conventional polarization-maintaining solid-core FOGs of the same size across various temperature ranges. These results represent a significant step towards long-standing promise of high-precision inertial navigation applications with superior environmental adaptability.

physics.optics↗

arXiv2024

Absence of BCS-BEC Crossover in FeSe0.45Te0 55 Superconductor

In iron-based superconductor Fe(Se,Te), a flat band-like feature near the Fermi level was observed around the Brillouin zone center in the superconducting state. It is under debate whether this is the evidence on the presence of the BCS-BEC crossover in the superconductor. High-resolution laser-based angle-resolved photoemission measurements are carried out on high quality single crystals of FeSe0.45Te0.55 superconductor to address the issue. By employing different polarization geometries, we have resolved and isolated the dyz band and the topological surface band, making it possible to study their superconducting behaviors separately. The dyz band alone does not form a flat band-like feature in the superconducting state and the measured dispersion can be well described by the BCS picture. We find that the flat band-like feature is formed from the combination of the dyz band and the topological surface state band in the superconducting state. These results reveal the origin of the flat band-like feature and rule out the presence of BCS-BEC crossover in Fe(Se,Te) superconductor.

cond-mat.supr-con↗

arXiv2024

Thermodynamic topology of Kerr-Sen black holes via Rényi statistics

In the present study, we investigate the topological properties of black holes in terms of Rényi statistics as an extension of the Gibbs-Boltzmann (GB) statistics, aiming to characterize the non-Boltzmannian thermodynamic topology of Kerr-Sen and dyonic Kerr-Sen black holes. Through this research, we discover that the topological number derived via Rényi statistics differs from that obtained through GB statistics. Interestingly, although the non-extended parameter $λ$ changes the topological number, the topological classification of the Kerr-Sen and dyonic Kerr-Sen black holes remains consistent under both GB and Rényi statistics. In addition, the topological numbers associated with these two types of black holes without cosmological constant using Rényi entropy processes are the same as the AdS cases of them by considering the GB entropy, as further evidenced by such a study found here. This indicates the cosmological constant has some potential connections the with the nonextensive Rényi parameter from the perspective of thermodynamic topology.

gr-qc↗

arXiv2024

Empowering Embodied Visual Tracking with Visual Foundation Models and Offline RL

Embodied visual tracking is to follow a target object in dynamic 3D environments using an agent's egocentric vision. This is a vital and challenging skill for embodied agents. However, existing methods suffer from inefficient training and poor generalization. In this paper, we propose a novel framework that combines visual foundation models(VFM) and offline reinforcement learning(offline RL) to empower embodied visual tracking. We use a pre-trained VFM, such as "Tracking Anything", to extract semantic segmentation masks with text prompts. We then train a recurrent policy network with offline RL, e.g., Conservative Q-Learning, to learn from the collected demonstrations without online interactions. To further improve the robustness and generalization of the policy network, we also introduce a mask re-targeting mechanism and a multi-level data collection strategy. In this way, we can train a robust policy within an hour on a consumer-level GPU, e.g., Nvidia RTX 3090. We evaluate our agent on several high-fidelity environments with challenging situations, such as distraction and occlusion. The results show that our agent outperforms state-of-the-art methods in terms of sample efficiency, robustness to distractors, and generalization to unseen scenarios and targets. We also demonstrate the transferability of the learned agent from virtual environments to a real-world robot.

cs.CV↗

arXiv2024

VegeDiff: Latent Diffusion Model for Geospatial Vegetation Forecasting

In the context of global climate change and frequent extreme weather events, forecasting future geospatial vegetation states under these conditions is of significant importance. The vegetation change process is influenced by the complex interplay between dynamic meteorological variables and static environmental variables, leading to high levels of uncertainty. Existing deterministic methods are inadequate in addressing this uncertainty and fail to accurately model the impact of these variables on vegetation, resulting in blurry and inaccurate forecasting results. To address these issues, we propose VegeDiff for the geospatial vegetation forecasting task. To our best knowledge, VegeDiff is the first to employ a diffusion model to probabilistically capture the uncertainties in vegetation change processes, enabling the generation of clear and accurate future vegetation states. VegeDiff also separately models the global impact of dynamic meteorological variables and the local effects of static environmental variables, thus accurately modeling the impact of these variables. Extensive experiments on geospatial vegetation forecasting tasks demonstrate the effectiveness of VegeDiff. By capturing the uncertainties in vegetation changes and modeling the complex influence of relevant variables, VegeDiff outperforms existing deterministic methods, providing clear and accurate forecasting results of future vegetation states. Interestingly, we demonstrate the potential of VegeDiff in applications of forecasting future vegetation states from multiple aspects and exploring the impact of meteorological variables on vegetation dynamics. The code of this work will be available at https://github.com/walking-shadow/ Official_VegeDiff.

cs.CV↗

arXiv2024

PromptRobust: Towards Evaluating the Robustness of Large Language Models on Adversarial Prompts

The increasing reliance on Large Language Models (LLMs) across academia and industry necessitates a comprehensive understanding of their robustness to prompts. In response to this vital need, we introduce PromptRobust, a robustness benchmark designed to measure LLMs' resilience to adversarial prompts. This study uses a plethora of adversarial textual attacks targeting prompts across multiple levels: character, word, sentence, and semantic. The adversarial prompts, crafted to mimic plausible user errors like typos or synonyms, aim to evaluate how slight deviations can affect LLM outcomes while maintaining semantic integrity. These prompts are then employed in diverse tasks including sentiment analysis, natural language inference, reading comprehension, machine translation, and math problem-solving. Our study generates 4,788 adversarial prompts, meticulously evaluated over 8 tasks and 13 datasets. Our findings demonstrate that contemporary LLMs are not robust to adversarial prompts. Furthermore, we present a comprehensive analysis to understand the mystery behind prompt robustness and its transferability. We then offer insightful robustness analysis and pragmatic recommendations for prompt composition, beneficial to both researchers and everyday users.

cs.CL↗

arXiv2024

Self-Prompt Tuning: Enable Autonomous Role-Playing in LLMs

Recent advancements in LLMs have showcased their remarkable role-playing capabilities, able to accurately simulate the dialogue styles and cognitive processes of various roles based on different instructions and contexts. Studies indicate that assigning LLMs the roles of experts, a strategy known as role-play prompting, can enhance their performance in the corresponding domains. However, the prompt needs to be manually designed for the given problem, requiring certain expertise and iterative modifications. To this end, we propose self-prompt tuning, making LLMs themselves generate role-play prompts through fine-tuning. Leveraging the LIMA dataset as our foundational corpus, we employ GPT-4 to annotate role-play prompts for each data points, resulting in the creation of the LIMA-Role dataset. We then fine-tune LLMs like Llama-2-7B and Mistral-7B on LIMA-Role. Consequently, the self-prompt tuned LLMs can automatically generate expert role prompts for any given question. We extensively evaluate self-prompt tuned LLMs on widely used NLP benchmarks and open-ended question test. Our empirical results illustrate that self-prompt tuned LLMs outperform standard instruction tuned baselines across most datasets. This highlights the great potential of utilizing fine-tuning to enable LLMs to self-prompt, thereby automating complex prompting strategies. We release the dataset, models, and code at this \href{https://anonymous.4open.science/r/Self-Prompt-Tuning-739E/}{url}.

cs.CL↗