arXiv subjects

Hao Wu

Publications and source records attributed to Hao Wu.

At least 505 records · Page 28Linked to original sources

arXiv2024

Parameterized Wasserstein Gradient Flow

We develop a fast and scalable numerical approach to solve Wasserstein gradient flows (WGFs), particularly suitable for high-dimensional cases. Our approach is to use general reduced-order models, like deep neural networks, to parameterize the push-forward maps such that they can push a simple reference density to the one solving the given WGF. The new dynamical system is called parameterized WGF (PWGF), and it is defined on the finite-dimensional parameter space equipped with a pullback Wasserstein metric. Our numerical scheme can approximate the solutions of WGFs for general energy functionals effectively, without requiring spatial discretization or nonconvex optimization procedures, thus avoiding some limitations of classical numerical methods and more recent deep-learning-based approaches. A comprehensive analysis of the approximation errors measured by Wasserstein distance is also provided in this work. Numerical experiments show promising computational efficiency and verified accuracy on various WGF examples using our approach.

math.NA↗

arXiv2024

All Nodes are created Not Equal: Node-Specific Layer Aggregation and Filtration for GNN

The ever-designed Graph Neural Networks, though opening a promising path for the modeling of the graph-structure data, unfortunately introduce two daunting obstacles to their deployment on devices. (I) Most of existing GNNs are shallow, due mostly to the over-smoothing and gradient-vanish problem as they go deeper as convolutional architectures. (II) The vast majority of GNNs adhere to the homophily assumption, where the central node and its adjacent nodes share the same label. This assumption often poses challenges for many GNNs working with heterophilic graphs. Addressing the aforementioned issue has become a looming challenge in enhancing the robustness and scalability of GNN applications. In this paper, we take a comprehensive and systematic approach to overcoming the two aforementioned challenges for the first time. We propose a Node-Specific Layer Aggregation and Filtration architecture, termed NoSAF, a framework capable of filtering and processing information from each individual nodes. NoSAF introduces the concept of "All Nodes are Created Not Equal" into every layer of deep networks, aiming to provide a reliable information filter for each layer's nodes to sieve out information beneficial for the subsequent layer. By incorporating a dynamically updated codebank, NoSAF dynamically optimizes the optimal information outputted downwards at each layer. This effectively overcomes heterophilic issues and aids in deepening the network. To compensate for the information loss caused by the continuous filtering in NoSAF, we also propose NoSAF-D (Deep), which incorporates a compensation mechanism that replenishes information in every layer of the model, allowing NoSAF to perform meaningful computations even in very deep layers.

cs.LG↗

arXiv2024

Observation of diamagnetic strange-metal phase in sulfur-copper codoped lead apatite

By codoping sulfur and copper into lead apatite, the crystal grains are directionally stacked and the room-temperature resistivity is reduced from insulating to $2\times10^{-5}~Ω\cdot$m. The resistance-temperature curve exhibits a nearly linear relationship at low temperature suggesting the presence of strange-metal phase, and a second-order phase transition is then observed at around 230~K during cooling the samples. A possible Meissner effect is present in dc magnetic measurements. Further hydrothermal lead-free synthesis results in smaller resistance and stronger diamagnetism, demonstrating the essential component might be sulfur-substituted copper apatite and the alkalis matter as well. A clear pathway towards superconductivity in this material is subsequently benchmarked.

cond-mat.supr-con↗

arXiv2024

COM3D: Leveraging Cross-View Correspondence and Cross-Modal Mining for 3D Retrieval

In this paper, we investigate an open research task of cross-modal retrieval between 3D shapes and textual descriptions. Previous approaches mainly rely on point cloud encoders for feature extraction, which may ignore key inherent features of 3D shapes, including depth, spatial hierarchy, geometric continuity, etc. To address this issue, we propose COM3D, making the first attempt to exploit the cross-view correspondence and cross-modal mining to enhance the retrieval performance. Notably, we augment the 3D features through a scene representation transformer, to generate cross-view correspondence features of 3D shapes, which enrich the inherent features and enhance their compatibility with text matching. Furthermore, we propose to optimize the cross-modal matching process based on the semi-hard negative example mining method, in an attempt to improve the learning efficiency. Extensive quantitative and qualitative experiments demonstrate the superiority of our proposed COM3D, achieving state-of-the-art results on the Text2Shape dataset.

cs.CV↗

arXiv2024

Exploring the potential of synthesizing unknown superheavy isotopes via cold-fusion reactions based on the dinuclear system model

To assess the potential of cold-fusion for synthesizing superheavy nuclei (SHN) with proton numbers 104-113, we systematically calculated 145 naturally occurring projectile-target combinations within the DNS model. Reactions predominantly show maximum cross-sections in the 1n to 2n channels, peaking near the Coulomb barrier with a sum of barrier and Q-value within 30 MeV. The maximum cross-section occurs below the Bass barrier, suggesting either the Bass model's limitation or significant deformation reducing the effective Coulomb barrier. Our calculations align well with experimental data, revealing that more neutron-rich projectiles slightly enhance fusion, though the effect is minor. For fixed targets (Pb, Bi), evaporation residue cross-sections decrease linearly with increasing projectile proton number, attributed to reduced fusion probability and lower fission barriers in heavier SHN. The touching potential $V_{\rm in}$ shows a linear trend with the product of projectile-target proton numbers, with neutron-rich systems exhibiting lower $V_{\rm in}$. Some reactions with $V_{\rm in} < V_{\rm S}$ may involve nucleon transfer before capture. Based on the DNS model, we identified optimal combinations and collision energies for synthesizing SHN with significant cross-sections. Collectively, our findings indicate that cold fusion is a promising avenue for creating proton-rich SHN around the drip line in the Z=104-113 region, offering distinct advantages over alternative mechanisms.

nucl-th↗

arXiv2024

On Convergence of Discrete Schemes for Computing the Rate-Distortion Function of Continuous Source

Computing the rate-distortion function for continuous sources is commonly regarded as a standard continuous optimization problem. When numerically addressing this problem, a typical approach involves discretizing the source space and subsequently solving the associated discrete problem. However, existing literature has predominantly concentrated on the convergence analysis of solving discrete problems, usually neglecting the convergence relationship between the original continuous optimization and its associated discrete counterpart. This neglect is not rigorous, since the solution of a discrete problem does not necessarily imply convergence to the solution of the original continuous problem, especially for non-linear problems. To address this gap, our study employs rigorous mathematical analysis, which constructs a series of finite-dimensional spaces approximating the infinite-dimensional space of the probability measure, establishing that solutions from discrete schemes converge to those from the continuous problems.

cs.IT↗

arXiv2024

A Double Maximization Approach for Optimizing the LM Rate of Mismatched Decoding

An approach is established for maximizing the Lower bound on the Mismatch capacity (hereafter abbreviated as LM rate), a key performance bound in mismatched decoding, by optimizing the channel input probability distribution. Under a fixed channel input probability distribution, the computation of the corresponding LM rate is a convex optimization problem. When optimizing the channel input probability distribution, however, the corresponding optimization problem adopts a max-min formulation, which is generally non-convex and is intractable with standard approaches. To solve this problem, a novel dual form of the LM rate is proposed, thereby transforming the max-min formulation into an equivalent double maximization formulation. This new formulation leads to a maximization problem setup wherein each individual optimization direction is convex. Consequently, an alternating maximization algorithm is established to solve the resultant maximization problem setup. Each step of the algorithm only involves a closed-form iteration, which is efficiently implemented with standard optimization procedures. Numerical experiments show the proposed approach for optimizing the LM rate leads to noticeable rate gains.

cs.IT↗

arXiv2024

An Expectation-Maximization Relaxed Method for Privacy Funnel

The privacy funnel (PF) gives a framework of privacy-preserving data release, where the goal is to release useful data while also limiting the exposure of associated sensitive information. This framework has garnered significant interest due to its broad applications in characterization of the privacy-utility tradeoff. Hence, there is a strong motivation to develop numerical methods with high precision and theoretical convergence guarantees. In this paper, we propose a novel relaxation variant based on Jensen's inequality of the objective function for the computation of the PF problem. This model is proved to be equivalent to the original in terms of optimal solutions and optimal values. Based on our proposed model, we develop an accurate algorithm which only involves closed-form iterations. The convergence of our algorithm is theoretically guaranteed through descent estimation and Pinsker's inequality. Numerical results demonstrate the effectiveness of our proposed algorithm.

cs.IT↗

arXiv2024

QML-IB: Quantized Collaborative Intelligence between Multiple Devices and the Mobile Network

The integration of artificial intelligence (AI) and mobile networks is regarded as one of the most important scenarios for 6G. In 6G, a major objective is to realize the efficient transmission of task-relevant data. Then a key problem arises, how to design collaborative AI models for the device side and the network side, so that the transmitted data between the device and the network is efficient enough, which means the transmission overhead is low but the AI task result is accurate. In this paper, we propose the multi-link information bottleneck (ML-IB) scheme for such collaborative models design. We formulate our problem based on a novel performance metric, which can evaluate both task accuracy and transmission overhead. Then we introduce a quantizer that is adjustable in the quantization bit depth, amplitudes, and breakpoints. Given the infeasibility of calculating our proposed metric on high-dimensional data, we establish a variational upper bound for this metric. However, due to the incorporation of quantization, the closed form of the variational upper bound remains uncomputable. Hence, we employ the Log-Sum Inequality to derive an approximation and provide a theoretical guarantee. Based on this, we devise the quantized multi-link information bottleneck (QML-IB) algorithm for collaborative AI models generation. Finally, numerical experiments demonstrate the superior performance of our QML-IB algorithm compared to the state-of-the-art algorithm.

cs.IT↗

arXiv2024

A nearly-$4\log n$ depth lower bound for formulas with restriction on top

One of the major open problems in complexity theory is to demonstrate an explicit function which requires super logarithmic depth, a.k.a, the $\mathbf{P}$ versus $\mathbf{NC^1}$ problem. The current best depth lower bound is $(3-o(1))\cdot \log n$, and it is widely open how to prove a super-$3\log n$ depth lower bound. Recently Mihajlin and Sofronova (CCC'22) show if considering formulas with restriction on top, we can break the $3\log n$ barrier. Formally, they prove there exist two functions $f:\{0,1\}^n \rightarrow \{0,1\},g:\{0,1\}^n \rightarrow \{0,1\}^n$, such that for any constant $0<α<0.4$ and constant $0<ε<α/2$, their XOR composition $f(g(x)\oplus y)$ is not computable by an AND of $2^{(α-ε)n}$ formulas of size at most $2^{(1-α/2-ε)n}$. This implies a modified version of Andreev function is not computable by any circuit of depth $(3.2-ε)\log n$ with the restriction that top $0.4-ε$ layers only consist of AND gates for any small constant $ε>0$. They ask whether the parameter $α$ can be push up to nearly $1$ thus implying a nearly-$3.5\log n$ depth lower bound. In this paper, we provide a stronger answer to their question. We show there exist two functions $f:\{0,1\}^n \rightarrow \{0,1\},g:\{0,1\}^n \rightarrow \{0,1\}^n$, such that for any constant $0<α<2-o(1)$, their XOR composition $f(g(x)\oplus y)$ is not computable by an AND of $2^{αn}$ formulas of size at most $2^{(1-α/2-o(1))n}$. This implies a $(4-o(1))\log n$ depth lower bound with the restriction that top $2-o(1)$ layers only consist of AND gates. We prove it by observing that one crucial component in Mihajlin and Sofronova's work, called the well-mixed set of functions, can be significantly simplified thus improved. Then with this observation and a more careful analysis, we obtain these nearly tight results.

cs.CC↗

arXiv2024

emrQA-msquad: A Medical Dataset Structured with the SQuAD V2.0 Framework, Enriched with emrQA Medical Information

Machine Reading Comprehension (MRC) holds a pivotal role in shaping Medical Question Answering Systems (QAS) and transforming the landscape of accessing and applying medical information. However, the inherent challenges in the medical field, such as complex terminology and question ambiguity, necessitate innovative solutions. One key solution involves integrating specialized medical datasets and creating dedicated datasets. This strategic approach enhances the accuracy of QAS, contributing to advancements in clinical decision-making and medical research. To address the intricacies of medical terminology, a specialized dataset was integrated, exemplified by a novel Span extraction dataset derived from emrQA but restructured into 163,695 questions and 4,136 manually obtained answers, this new dataset was called emrQA-msquad dataset. Additionally, for ambiguous questions, a dedicated medical dataset for the Span extraction task was introduced, reinforcing the system's robustness. The fine-tuning of models such as BERT, RoBERTa, and Tiny RoBERTa for medical contexts significantly improved response accuracy within the F1-score range of 0.75 to 1.00 from 10.1% to 37.4%, 18.7% to 44.7% and 16.0% to 46.8%, respectively. Finally, emrQA-msquad dataset is publicy available at https://huggingface.co/datasets/Eladio/emrqa-msquad.

cs.CL↗

arXiv2024

Fast Gradient Computation for Gromov-Wasserstein Distance

The Gromov-Wasserstein distance is a notable extension of optimal transport. In contrast to the classic Wasserstein distance, it solves a quadratic assignment problem that minimizes the pair-wise distance distortion under the transportation of distributions and thus could apply to distributions in different spaces. These properties make Gromov-Wasserstein widely applicable to many fields, such as computer graphics and machine learning. However, the computation of the Gromov-Wasserstein distance and transport plan is expensive. The well-known Entropic Gromov-Wasserstein approach has a cubic complexity since the matrix multiplication operations need to be repeated in computing the gradient of Gromov-Wasserstein loss. This becomes a key bottleneck of the method. Currently, existing methods accelerate the computation focus on sampling and approximation, which leads to low accuracy or incomplete transport plan. In this work, we propose a novel method to accelerate accurate gradient computation by dynamic programming techniques, reducing the complexity from cubic to quadratic. In this way, the original computational bottleneck is broken and the new entropic solution can be obtained with total quadratic time, which is almost optimal complexity. Furthermore, it can be extended to some variants easily. Extensive experiments validate the efficiency and effectiveness of our method.

cs.LG↗

arXiv2024

Searching for Hyper-compact star clusters in the Milky Way using LAMOST and Gaia

During the early merger of the Milky Way, intermediate-mass black holes in merged dwarf galaxies may have been ejected from the center of their host galaxies due to gravitational waves, carrying some central stars along. This process can lead to the formation of hyper-compact star clusters, potentially hosting black holes in the mass range of $10^4$ to $10^5$ solar masses. These clusters are crucial targets for identifying and investigating intermediate-mass black holes. However, no hyper-compact star clusters in the Milky Way have been identified so far. In this paper, taking advantage of the high spatial resolution power of Gaia, we used data from Gaia EDR3 and LAMOST DR7, along with additional data from Pan-STARRS and SDSS, to conduct an initial screening of 6,138,049 sources using various parameters of Gaia EDR3. A total of 4,786 sources were selected for in-depth analysis. Each of these sources was meticulously scrutinized by examining their images, spectra, and nearby celestial objects to exclude various false positives, such as contaminations, galaxies, wide binaries, or wrong matches. We finally identified one likely hyper-compact star cluster candidate in the Milky Way, laying the foundation for further high-resolution imaging and spectral verification.

astro-ph.GA↗

arXiv2024

Efficient and Provably Convergent Computation of Information Bottleneck: A Semi-Relaxed Approach

Information Bottleneck (IB) is a technique to extract information about one target random variable through another relevant random variable. This technique has garnered significant interest due to its broad applications in information theory and deep learning. Hence, there is a strong motivation to develop efficient numerical methods with high precision and theoretical convergence guarantees. In this paper, we propose a semi-relaxed IB model, where the Markov chain and transition probability condition are relaxed from the relevance-compression function. Based on the proposed model, we develop an algorithm, which recovers the relaxed constraints and involves only closed-form iterations. Specifically, the algorithm is obtained by analyzing the Lagrangian of the relaxed model with alternating minimization in each direction. The convergence property of the proposed algorithm is theoretically guaranteed through descent estimation and Pinsker's inequality. Numerical experiments across classical and discrete distributions corroborate the analysis. Moreover, our proposed algorithm demonstrates notable advantages in terms of computational efficiency, evidenced by significantly reduced run times compared to existing methods with comparable accuracy.

cs.IT↗

arXiv2024

Optimal Distributed Control for a Cahn-Hilliard-Darcy System with Mass Sources, Unmatched Viscosities and Singular Potential

We study a Cahn-Hilliard-Darcy system with mass sources, which can be considered as a basic, though simplified, diffuse interface model for the evolution of tumor growth. This system is equipped with an impermeability condition for the (volume) averaged velocity $\mathbf{u}$ as well as homogeneous Neumann boundary conditions for the phase function $φ$ and the chemical potential $μ$. The source term in the convective Cahn-Hilliard equation contains a control $R$ that can be thought, for instance, as a drug or a nutrient in applications. Our goal is to study a distributed optimal control problem in the two dimensional setting with a cost functional of tracking-type. In the physically relevant case with unmatched viscosities for the binary fluid mixtures and a singular potential, we first prove the existence and uniqueness of a global strong solution with $φ$ being strictly separated from the pure phases $\pm 1$. This well-posedness result enables us to characterize the control-to-state mapping $\mathcal{S}:R \mapsto φ$. Then we obtain the existence of an optimal control, the Fréchet differentiability of $\mathcal{S}$ and first-order necessary optimality conditions expressed through a suitable variational inequality for the adjoint variables. Finally, we prove the differentiability of the control-to-costate operator and establish a second-order sufficient condition for the strict local optimality.

math.OC↗

arXiv2024

DIBS: Enhancing Dense Video Captioning with Unlabeled Videos via Pseudo Boundary Enrichment and Online Refinement

We present Dive Into the BoundarieS (DIBS), a novel pretraining framework for dense video captioning (DVC), that elaborates on improving the quality of the generated event captions and their associated pseudo event boundaries from unlabeled videos. By leveraging the capabilities of diverse large language models (LLMs), we generate rich DVC-oriented caption candidates and optimize the corresponding pseudo boundaries under several meticulously designed objectives, considering diversity, event-centricity, temporal ordering, and coherence. Moreover, we further introduce a novel online boundary refinement strategy that iteratively improves the quality of pseudo boundaries during training. Comprehensive experiments have been conducted to examine the effectiveness of the proposed technique components. By leveraging a substantial amount of unlabeled video data, such as HowTo100M, we achieve a remarkable advancement on standard DVC datasets like YouCook2 and ActivityNet. We outperform the previous state-of-the-art Vid2Seq across a majority of metrics, achieving this with just 0.4% of the unlabeled video data used for pre-training by Vid2Seq.

cs.CV↗

arXiv2024

SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models

Large language models (LLMs) show excellent performance but are compute- and memory-intensive. Quantization can reduce memory and accelerate inference. However, existing methods cannot maintain accuracy and hardware efficiency at the same time. We propose SmoothQuant, a training-free, accuracy-preserving, and general-purpose post-training quantization (PTQ) solution to enable 8-bit weight, 8-bit activation (W8A8) quantization for LLMs. Based on the fact that weights are easy to quantize while activations are not, SmoothQuant smooths the activation outliers by offline migrating the quantization difficulty from activations to weights with a mathematically equivalent transformation. SmoothQuant enables an INT8 quantization of both weights and activations for all the matrix multiplications in LLMs, including OPT, BLOOM, GLM, MT-NLG, Llama-1/2, Falcon, Mistral, and Mixtral models. We demonstrate up to 1.56x speedup and 2x memory reduction for LLMs with negligible loss in accuracy. SmoothQuant enables serving 530B LLM within a single node. Our work offers a turn-key solution that reduces hardware costs and democratizes LLMs. Code is available at https://github.com/mit-han-lab/smoothquant.

cs.CL↗

arXiv2024

SLFNet: Generating Semantic Logic Forms from Natural Language Using Semantic Probability Graphs

Building natural language interfaces typically uses a semantic parser to parse the user's natural language and convert it into structured \textbf{S}emantic \textbf{L}ogic \textbf{F}orms (SLFs). The mainstream approach is to adopt a sequence-to-sequence framework, which requires that natural language commands and SLFs must be represented serially. Since a single natural language may have multiple SLFs or multiple natural language commands may have the same SLF, training a sequence-to-sequence model is sensitive to the choice among them, a phenomenon recorded as "order matters". To solve this problem, we propose a novel neural network, SLFNet, which firstly incorporates dependent syntactic information as prior knowledge and can capture the long-range interactions between contextual information and words. Secondly construct semantic probability graphs to obtain local dependencies between predictor variables. Finally we propose the Multi-Head SLF Attention mechanism to synthesize SLFs from natural language commands based on Sequence-to-Slots. Experiments show that SLFNet achieves state-of-the-art performance on the ChineseQCI-TS and Okapi datasets, and competitive performance on the ATIS dataset.

cs.CL↗