arXiv subjects

Jun Wang

Publications and source records attributed to Jun Wang.

At least 487 records · Page 27Linked to original sources

arXiv2024

AlphaFolding: 4D Diffusion for Dynamic Protein Structure Prediction with Reference and Motion Guidance

Protein structure prediction is pivotal for understanding the structure-function relationship of proteins, advancing biological research, and facilitating pharmaceutical development and experimental design. While deep learning methods and the expanded availability of experimental 3D protein structures have accelerated structure prediction, the dynamic nature of protein structures has received limited attention. This study introduces an innovative 4D diffusion model incorporating molecular dynamics (MD) simulation data to learn dynamic protein structures. Our approach is distinguished by the following components: (1) a unified diffusion model capable of generating dynamic protein structures, including both the backbone and side chains, utilizing atomic grouping and side-chain dihedral angle predictions; (2) a reference network that enhances structural consistency by integrating the latent embeddings of the initial 3D protein structures; and (3) a motion alignment module aimed at improving temporal structural coherence across multiple time steps. To our knowledge, this is the first diffusion-based model aimed at predicting protein trajectories across multiple time steps simultaneously. Validation on benchmark datasets demonstrates that our model exhibits high accuracy in predicting dynamic 3D structures of proteins containing up to 256 amino acids over 32 time steps, effectively capturing both local flexibility in stable states and significant conformational changes. URL: https://fudan-generative-vision.github.io/AlphaFolding/#/

cs.LG↗

arXiv2024

Small Language Models as Effective Guides for Large Language Models in Chinese Relation Extraction

Recently, large language models (LLMs) have been successful in relational extraction (RE) tasks, especially in the few-shot learning. An important problem in the field of RE is long-tailed data, while not much attention is paid to this problem using LLM approaches. Therefore, in this paper, we propose SLCoLM, a model collaboration framework, to mitigate the data long-tail problem. In our framework, we use the ``\textit{Training-Guide-Predict}'' strategy to combine the strengths of small pre-trained language models (SLMs) and LLMs, where a task-specific SLM framework acts as a guider, transfers task knowledge to the LLM and guides the LLM in performing RE tasks. Our experiments on an ancient Chinese RE dataset rich in relation types show that the approach facilitates RE of long-tail relation types.

cs.CL↗

arXiv2024

Covariance Analysis of Impulsive Streaking

We present a comprehensive framework of modeling covariance in angular streaking experiments. Within the impulsive streaking regime, the displacement of electron momentum distribution (MD) provides a tight connection between the dressing-free MD and the dressed MD. Such connection establishes universal structures in the composition of streaking covariance that are common across different MDs, regardless of their exact shape. Building on this robust framework, we have developed methods for retrieving temporal information from angular streaking measurements. By providing a detailed understanding of the covariance structure in angular streaking experiments, our work enables more accurate and robust temporal measurements in a wide range of experimental scenarios.

physics.data-an↗

arXiv2024

Representation of finite order solutions to linear differential equations with exponential sum coefficients

We show a necessary and sufficient condition on the existence of finite order entire solutions of linear differential equations $$ f^{(n)}+a_{n-1}f^{(n-1)}+\cdots+a_1f'+a_0f=0,\eqno(+) $$ where $a_i$ are exponential sums for $i=0,\ldots,n-1$ with all positive (or all negative) rational frequencies and constant coefficients. Moreover, under the condition that there exists a finite order solution of (+) with exponential sum coefficients having rational frequencies and constant coefficients, we give the precise form of all finite order solutions, which are exponential sums. It is a partial answer to Gol'dberg-Ostrovskiǐ Problem and Problem 5 in \cite{HITW2022} since exponential sums are of completely regular growth.

math.CV↗

arXiv2024

STAIR: Manipulating Collaborative and Multimodal Information for E-Commerce Recommendation

While the mining of modalities is the focus of most multimodal recommendation methods, we believe that how to fully utilize both collaborative and multimodal information is pivotal in e-commerce scenarios where, as clarified in this work, the user behaviors are rarely determined entirely by multimodal features. In order to combine the two distinct types of information, some additional challenges are encountered: 1) Modality erasure: Vanilla graph convolution, which proves rather useful in collaborative filtering, however erases multimodal information; 2) Modality forgetting: Multimodal information tends to be gradually forgotten as the recommendation loss essentially facilitates the learning of collaborative information. To this end, we propose a novel approach named STAIR, which employs a novel STepwise grAph convolution to enable a co-existence of collaborative and multimodal Information in e-commerce Recommendation. Besides, it starts with the raw multimodal features as an initialization, and the forgetting problem can be significantly alleviated through constrained embedding updates. As a result, STAIR achieves state-of-the-art recommendation performance on three public e-commerce datasets with minimal computational and memory costs. Our code is available at https://github.com/yhhe2004/STAIR.

cs.IR↗

arXiv2024

MEATRD: Multimodal Anomalous Tissue Region Detection Enhanced with Spatial Transcriptomics

The detection of anomalous tissue regions (ATRs) within affected tissues is crucial in clinical diagnosis and pathological studies. Conventional automated ATR detection methods, primarily based on histology images alone, falter in cases where ATRs and normal tissues have subtle visual differences. The recent spatial transcriptomics (ST) technology profiles gene expressions across tissue regions, offering a molecular perspective for detecting ATRs. However, there is a dearth of ATR detection methods that effectively harness complementary information from both histology images and ST. To address this gap, we propose MEATRD, a novel ATR detection method that integrates histology image and ST data. MEATRD is trained to reconstruct image patches and gene expression profiles of normal tissue spots (inliers) from their multimodal embeddings, followed by learning a one-class classification AD model based on latent multimodal reconstruction errors. This strategy harmonizes the strengths of reconstruction-based and one-class classification approaches. At the heart of MEATRD is an innovative masked graph dual-attention transformer (MGDAT) network, which not only facilitates cross-modality and cross-node information sharing but also addresses the model over-generalization issue commonly seen in reconstruction-based AD methods. Additionally, we demonstrate that modality-specific, task-relevant information is collated and condensed in multimodal bottleneck encoding generated in MGDAT, marking the first theoretical analysis of the informational properties of multimodal bottleneck encoding. Extensive evaluations across eight real ST datasets reveal MEATRD's superior performance in ATR detection, surpassing various state-of-the-art AD methods. Remarkably, MEATRD also proves adept at discerning ATRs that only show slight visual deviations from normal tissues.

cs.CV↗

arXiv2024

Why Not Together? A Multiple-Round Recommender System for Queries and Items

A fundamental technique of recommender systems involves modeling user preferences, where queries and items are widely used as symbolic representations of user interests. Queries delineate user needs at an abstract level, providing a high-level description, whereas items operate on a more specific and concrete level, representing the granular facets of user preference. While practical, both query and item recommendations encounter the challenge of sparse user feedback. To this end, we propose a novel approach named Multiple-round Auto Guess-and-Update System (MAGUS) that capitalizes on the synergies between both types, allowing us to leverage both query and item information to form user interests. This integrated system introduces a recursive framework that could be applied to any recommendation method to exploit queries and items in historical interactions and to provide recommendations for both queries and items in each interaction round. Empirical results from testing 12 different recommendation methods demonstrate that integrating queries into item recommendations via MAGUS significantly enhances the efficiency, with which users can identify their preferred items during multiple-round interactions.

cs.IR↗

arXiv2024

A wonderful triangle in compressed sensing

In order to determine the sparse approximation function which has a direct metric relationship with the $\ell_{0}$ quasi-norm, we introduce a wonderful triangle whose sides are composed of $\Vert \mathbf{x} \Vert_{0}$, $\Vert \mathbf{x} \Vert_{1}$ and $\Vert \mathbf{x} \Vert_{\infty}$ for any non-zero vector $\mathbf{x} \in \mathbb{R}^{n}$ by delving into the iterative soft-thresholding operator in this paper. Based on this triangle, we deduce the ratio $\ell_{1}$ and $\ell_{\infty}$ norms as a sparsity-promoting objective function for sparse signal reconstruction and also try to give the sparsity interval of the signal. Considering the $\ell_{1}/\ell_{\infty}$ minimization from a angle $β$ of the triangle corresponding to the side whose length is $\Vert \mathbf{x} \Vert_{\infty} - \Vert \mathbf{x} \Vert_{1}/\Vert \mathbf{x} \Vert_{0}$, we finally demonstrate the performance of existing $\ell_{1}/\ell_{\infty}$ algorithm by comparing it with $\ell_{1}/\ell_{2}$ algorithm.

cs.IT↗

arXiv2024

Orbital stability of normalized ground states for critical Choquard equation with potential

In this paper, we study the existence of ground state standing waves and orbital stability, of prescribed mass, for the nonlinear critical Choquard equation \begin{equation*} \left\{\begin{array}{l} i \partial_t u+Δu -V(x)u+(I_α\ast|u|^{q})|u|^{q-2}u+(I_α\ast|u|^{2_α^*})|u|^{2_α^*-2}u=0,\ (x, t) \in \mathbb{R}^d \times \mathbb{R}, \\ \left.u\right|_{t=0}=φ\in H ^1(\mathbb{R}^d), \end{array}\right. \end{equation*} where $I_α$ is a Riesz potential of order $α\in(0,d),\ d\geq3,\ 2_α^*=\frac{2d-α}{d-2}$ is the upper critical exponent due to Hardy-Littlewood-Sobolev inequality, $\frac{2d-α}{d}<q<\frac{2d-α+2}{d}$. Under appropriate potential conditions, we obtain new Strichartz estimates and construct the new space to get orbital stability of normalized ground state. To our best knowledge, this is the first orbital stability result for this model. Our method is also applicable to other mixed nonlinear equations with potential.

math.AP↗

arXiv2024

StyleMark: A Robust Watermarking Method for Art Style Images Against Black-Box Arbitrary Style Transfer

Arbitrary Style Transfer (AST) achieves the rendering of real natural images into the painting styles of arbitrary art style images, promoting art communication. However, misuse of unauthorized art style images for AST may infringe on artists' copyrights. One countermeasure is robust watermarking, which tracks image propagation by embedding copyright watermarks into carriers. Unfortunately, AST-generated images lose the structural and semantic information of the original style image, hindering end-to-end robust tracking by watermarks. To fill this gap, we propose StyleMark, the first robust watermarking method for black-box AST, which can be seamlessly applied to art style images achieving precise attribution of artistic styles after AST. Specifically, we propose a new style watermark network that adjusts the mean activations of style features through multi-scale watermark embedding, thereby planting watermark traces into the shared style feature space of style images. Furthermore, we design a distribution squeeze loss, which constrain content statistical feature distortion, forcing the reconstruction network to focus on integrating style features with watermarks, thus optimizing the intrinsic watermark distribution. Finally, based on solid end-to-end training, StyleMark mitigates the optimization conflict between robustness and watermark invisibility through decoder fine-tuning under random noise. Experimental results demonstrate that StyleMark exhibits significant robustness against black-box AST and common pixel-level distortions, while also securely defending against malicious adaptive attacks.

cs.CV↗

arXiv2024

A Fast Adaptive Method for the Heat Equation with Moving or Free Boundaries in One Dimension

We present a fast adaptive method for the evaluation of heat potentials, which plays a key role in the integral equation approach for the solution of the heat equation, especially in a non-stationary domain. The algorithm utilizes a sum-of-exponential based fast Gauss transform that evaluates the convolution of a Gaussian with either discrete or continuous volume distributions. The latest implementation of the algorithm allows for both periodic and free space boundary conditions. The history dependence is overcome by splitting the heat potentials into a smooth history part and a singular local part. We discuss the resolution of the history part on an adaptive volume grid in detail, providing sharp estimates that allow for the construction of an optimal grid, justifying the efficiency of the bootstrapping scheme. While the discussion in this paper is restricted to one spatial dimension, the generalization to two and three dimensions is straightforward. The performance of the algorithm is illustrated via several numerical examples.

math.NA↗

arXiv2024

Advancing global aerosol forecasting with artificial intelligence

Aerosol forecasting is essential for air quality warnings, health risk assessment, and climate change mitigation. However, it is more complex than weather forecasting due to the intricate interactions between aerosol physicochemical processes and atmospheric dynamics, resulting in significant uncertainty and high computational costs. Here, we develop an artificial intelligence-driven global aerosol-meteorology forecasting system (AI-GAMFS), which provides reliable 5-day, 3-hourly forecasts of aerosol optical components and surface concentrations at a 0.5° x 0.625° resolution. AI-GAMFS combines Vision Transformer and U-Net in a backbone network, robustly capturing the complex aerosol-meteorology interactions via global attention and spatiotemporal encoding. Trained on 42 years of advanced aerosol reanalysis data and initialized with GEOS Forward Processing (GEOS-FP) analyses, AI-GAMFS delivers operational 5-day forecasts in one minute. It outperforms the Copernicus Atmosphere Monitoring Service (CAMS) global forecasting system, GEOS-FP forecasts, and several regional dust forecasting systems in forecasting most aerosol variables including aerosol optical depth and dust components. Our results mark a significant step forward in leveraging AI to refine physics-based aerosol forecasting, facilitating more accurate global warnings for aerosol pollution events, such as dust storms and wildfires.

physics.ao-ph↗

arXiv2024

Almost sure well-posedness and orbital stability for Schrödinger equation with potential

In this paper, we study the almost sure well-posedness theory and orbital stability for the nonlinear Schrödinger equation with potential \begin{equation*} \left\{\begin{array}{l} i \partial_t u+Δu-V(x)u+|u|^{2}u=0,\ (x, t) \in \mathbb{R}^4 \times \mathbb{R}, \\ \left.u\right|_{t=0}=f \in H ^s(\mathbb{R}^4), \end{array}\right. \end{equation*} where $\frac{1}{3}<s<1$ and $V(x):\mathbb{R}^4\rightarrow \mathbb{R}$ satisfies appropriate conditions. The main idea in the proofs is based on Strichartz spaces as well as variants of local smoothing, inhomogeneous local smoothing and maximal function spaces. To our best knowledge, this is the first orbital stability result for this model.

math.AP↗

arXiv2024

BAHOP: Similarity-based Basin Hopping for A fast hyper-parameter search in WSI classification

Pre-processing whole slide images (WSIs) can impact classification performance. Our study shows that using fixed hyper-parameters for pre-processing out-of-domain WSIs can significantly degrade performance. Therefore, it is critical to search domain-specific hyper-parameters during inference. However, searching for an optimal parameter set is time-consuming. To overcome this, we propose BAHOP, a novel Similarity-based Basin Hopping optimization for fast parameter tuning to enhance inference performance on out-of-domain data. The proposed BAHOP achieves 5\% to 30\% improvement in accuracy with $\times5$ times faster on average.

cs.CV↗

arXiv2024

Redefining Crowdsourced Test Report Prioritization: An Innovative Approach with Large Language Model

Context: Crowdsourced testing has gained popularity in software testing, especially for mobile app testing, due to its ability to bring diversity and tackle fragmentation issues. However, the openness of crowdsourced testing presents challenges, particularly in the manual review of numerous test reports, which is time-consuming and labor-intensive. Objective: The primary goal of this research is to improve the efficiency of review processes in crowdsourced testing. Traditional approaches to test report prioritization lack a deep understanding of semantic information in textual descriptions of these reports. This paper introduces LLMPrior, a novel approach for prioritizing crowdsourced test reports using large language models (LLMs). Method: LLMPrior leverages LLMs for the analysis and clustering of crowdsourced test reports based on the types of bugs revealed in their textual descriptions. This involves using prompt engineering techniques to enhance the performance of LLMs. Following the clustering, a recurrent selection algorithm is applied to prioritize the reports. Results: Empirical experiments are conducted to evaluate the effectiveness of LLMPrior. The findings indicate that LLMPrior not only surpasses current state-of-the-art approaches in terms of performance but also proves to be more feasible, efficient, and reliable. This success is attributed to the use of prompt engineering techniques and the cluster-based prioritization strategy. Conclusion: LLMPrior represents a significant advancement in crowdsourced test report prioritization. By effectively utilizing large language models and a cluster-based strategy, it addresses the challenges in traditional prioritization approaches, offering a more efficient and reliable solution for app developers dealing with crowdsourced test reports.

cs.SE↗

arXiv2024

Probabilistically Correct Language-based Multi-Robot Planning using Conformal Prediction

This paper addresses task planning problems for language-instructed robot teams. Tasks are expressed in natural language (NL), requiring the robots to apply their capabilities at various locations and semantic objects. Several recent works have addressed similar planning problems by leveraging pre-trained Large Language Models (LLMs) to design effective multi-robot plans. However, these approaches lack performance guarantees. To address this challenge, we introduce a new distributed LLM-based planner, called S-ATLAS for Safe plAnning for Teams of Language-instructed AgentS, that is capable of achieving user-defined mission success rates. This is accomplished by leveraging conformal prediction (CP), a distribution-free uncertainty quantification tool in black-box models. CP allows the proposed multi-robot planner to reason about its inherent uncertainty in a distributed fashion, enabling robots to make individual decisions when they are sufficiently certain and seek help otherwise. We show, both theoretically and empirically, that the proposed planner can achieve user-specified task success rates, assuming successful plan execution, while minimizing the overall number of help requests. We provide comparative experiments against related works showing that our method is significantly more computational efficient and achieves lower help rates. The advantage of our algorithm over baselines becomes more pronounced with increasing robot team size.

cs.RO↗

arXiv2024

BLIP3-KALE: Knowledge Augmented Large-Scale Dense Captions

We introduce BLIP3-KALE, a dataset of 218 million image-text pairs that bridges the gap between descriptive synthetic captions and factual web-scale alt-text. KALE augments synthetic dense image captions with web-scale alt-text to generate factually grounded image captions. Our two-stage approach leverages large vision-language models and language models to create knowledge-augmented captions, which are then used to train a specialized VLM for scaling up the dataset. We train vision-language models on KALE and demonstrate improvements on vision-language tasks. Our experiments show the utility of KALE for training more capable and knowledgeable multimodal models. We release the KALE dataset at https://huggingface.co/datasets/Salesforce/blip3-kale

cs.CV↗

arXiv2024

Adaptive Conditional Expert Selection Network for Multi-domain Recommendation

Mixture-of-Experts (MOE) has recently become the de facto standard in Multi-domain recommendation (MDR) due to its powerful expressive ability. However, such MOE-based method typically employs all experts for each instance, leading to scalability issue and low-discriminability between domains and experts. Furthermore, the design of commonly used domain-specific networks exacerbates the scalability issues. To tackle the problems, We propose a novel method named CESAA consists of Conditional Expert Selection (CES) Module and Adaptive Expert Aggregation (AEA) Module to tackle these challenges. Specifically, CES first combines a sparse gating strategy with domain-shared experts. Then AEA utilizes mutual information loss to strengthen the correlations between experts and specific domains, and significantly improve the distinction between experts. As a result, only domain-shared experts and selected domain-specific experts are activated for each instance, striking a balance between computational efficiency and model performance. Experimental results on both public ranking and industrial retrieval datasets verify the effectiveness of our method in MDR tasks.

cs.LG↗