arXiv subjects

Zhiming Ma

Publications and source records attributed to Zhiming Ma.

At least 55 records · Page 3Linked to original sources

arXiv2024

New Partial Orders of Polar Codes for BMSC

In this paper, we define partial orders (POs) of polar codes based on the Bhattacharyya parameter and the bit-error probability, respectively. These POs are applicable to arbitrary binary memoryless symmetric channel (BMSC). Leveraging the extremal inequalities of polarization transformation, we derive new POs for BMSC based on the corresponding POs observed in the Binary Erasure Channel (BEC). %Additionally, we discover more special POs in the Binary Symmetric Channel (BSC). We provide examples that demonstrate the inability of existing POs to deduce these novel POs. Furthermore, we establish upper bounds for the expansion parameter $β$ if the polar codes constructed by $β$-expansion method obey these POs.

cs.IT↗

arXiv2024

ProFSA: Self-supervised Pocket Pretraining via Protein Fragment-Surroundings Alignment

Pocket representations play a vital role in various biomedical applications, such as druggability estimation, ligand affinity prediction, and de novo drug design. While existing geometric features and pretrained representations have demonstrated promising results, they usually treat pockets independent of ligands, neglecting the fundamental interactions between them. However, the limited pocket-ligand complex structures available in the PDB database (less than 100 thousand non-redundant pairs) hampers large-scale pretraining endeavors for interaction modeling. To address this constraint, we propose a novel pocket pretraining approach that leverages knowledge from high-resolution atomic protein structures, assisted by highly effective pretrained small molecule representations. By segmenting protein structures into drug-like fragments and their corresponding pockets, we obtain a reasonable simulation of ligand-receptor interactions, resulting in the generation of over 5 million complexes. Subsequently, the pocket encoder is trained in a contrastive manner to align with the representation of pseudo-ligand furnished by some pretrained small molecule encoders. Our method, named ProFSA, achieves state-of-the-art performance across various tasks, including pocket druggability prediction, pocket matching, and ligand binding affinity prediction. Notably, ProFSA surpasses other pretraining methods by a substantial margin. Moreover, our work opens up a new avenue for mitigating the scarcity of protein-ligand complex data through the utilization of high-quality and diverse protein structure databases.

cs.LG↗

arXiv2024

Achieving the Fundamental Limit of Lossless Analog Compression via Polarization

In this paper, we study the lossless analog compression for i.i.d. nonsingular signals via the polarization-based framework. We prove that for nonsingular source, the error probability of maximum a posteriori (MAP) estimation polarizes under the Hadamard transform, which extends the polarization phenomenon to analog domain. Building on this insight, we propose partial Hadamard compression and develop the corresponding analog successive cancellation (SC) decoder. The proposed scheme consists of deterministic measurement matrices and non-iterative reconstruction algorithm, providing benefits in both space and computational complexity. Using the polarization of error probability, we prove that our approach achieves the information-theoretical limit for lossless analog compression developed by Wu and Verdu.

cs.IT↗

arXiv2023

Molecule Joint Auto-Encoding: Trajectory Pretraining with 2D and 3D Diffusion

Recently, artificial intelligence for drug discovery has raised increasing interest in both machine learning and chemistry domains. The fundamental building block for drug discovery is molecule geometry and thus, the molecule's geometrical representation is the main bottleneck to better utilize machine learning techniques for drug discovery. In this work, we propose a pretraining method for molecule joint auto-encoding (MoleculeJAE). MoleculeJAE can learn both the 2D bond (topology) and 3D conformation (geometry) information, and a diffusion process model is applied to mimic the augmented trajectories of such two modalities, based on which, MoleculeJAE will learn the inherent chemical structure in a self-supervised manner. Thus, the pretrained geometrical representation in MoleculeJAE is expected to benefit downstream geometry-related tasks. Empirically, MoleculeJAE proves its effectiveness by reaching state-of-the-art performance on 15 out of 20 tasks by comparing it with 12 competitive baselines.

cs.LG↗

arXiv2023

Elastic Information Bottleneck

Information bottleneck is an information-theoretic principle of representation learning that aims to learn a maximally compressed representation that preserves as much information about labels as possible. Under this principle, two different methods have been proposed, i.e., information bottleneck (IB) and deterministic information bottleneck (DIB), and have gained significant progress in explaining the representation mechanisms of deep learning algorithms. However, these theoretical and empirical successes are only valid with the assumption that training and test data are drawn from the same distribution, which is clearly not satisfied in many real-world applications. In this paper, we study their generalization abilities within a transfer learning scenario, where the target error could be decomposed into three components, i.e., source empirical error, source generalization gap (SG), and representation discrepancy (RD). Comparing IB and DIB on these terms, we prove that DIB's SG bound is tighter than IB's while DIB's RD is larger than IB's. Therefore, it is difficult to tell which one is better. To balance the trade-off between SG and the RD, we propose an elastic information bottleneck (EIB) to interpolate between the IB and DIB regularizers, which guarantees a Pareto frontier within the IB framework. Additionally, simulations and real data experiments show that EIB has the ability to achieve better domain adaptation results than IB and DIB, which validates the correctness of our theories.

cs.IT↗

arXiv2023

On the Weight Spectrum Improvement of Pre-transformed Reed-Muller Codes and Polar Codes

Pre-transformation with an upper-triangular matrix (including cyclic redundancy check (CRC), parity-check (PC) and polarization-adjusted convolutional (PAC) codes) improves the weight spectrum of Reed-Muller (RM) codes and polar codes significantly. However, a theoretical analysis to quantify the improvement is missing. In this paper, we provide asymptotic analysis on the number of low-weight codewords of the original and pre-transformed RM codes respectively, and prove that pre-transformation significantly reduces low-weight codewords, even in the order sense. For polar codes, we prove that the average number of minimum-weight codewords does not increase after pre-transformation. Both results confirm the advantages of pre-transformation.

cs.IT↗

arXiv2023

Theoretical Bounds for the Size of Elementary Trapping Sets by Graphic Methods

Elementary trapping sets (ETSs) are the main culprits for the performance of LDPC codes in the error floor region. Due to the large quantity, complex structures, and computational difficulties of ETSs, how to eliminate dominant ETSs in designing LDPC codes becomes a pivotal issue to improve the error floor behavior. In practice, researchers commonly address this problem by avoiding some special graph structures to free specific ETSs in Tanner graph. In this paper, we deduce the accurate Turán number of $θ(1,2,2)$ and prove that all $(a,b)$-ETSs in Tanner graph with variable-regular degree $d_L(v)=γ$ must satisfy the bound $b\geq aγ-\frac{1}{2}a^2$, which improves the lower bound obtained by Amirzade when the girth is 6. For the case of girth 8, by limiting the relation between any two 8-cycles in the Tanner graph, we prove a similar inequality $b\geq aγ-\frac{a(\sqrt{8a-7}-1)}{2}$. The simulation results show that the designed codes have good performance with lower error floor over additive white Gaussian noise channels.

cs.IT↗

arXiv2023

Symmetry-Informed Geometric Representation for Molecules, Proteins, and Crystalline Materials

Artificial intelligence for scientific discovery has recently generated significant interest within the machine learning and scientific communities, particularly in the domains of chemistry, biology, and material discovery. For these scientific problems, molecules serve as the fundamental building blocks, and machine learning has emerged as a highly effective and powerful tool for modeling their geometric structures. Nevertheless, due to the rapidly evolving process of the field and the knowledge gap between science (e.g., physics, chemistry, & biology) and machine learning communities, a benchmarking study on geometrical representation for such data has not been conducted. To address such an issue, in this paper, we first provide a unified view of the current symmetry-informed geometric methods, classifying them into three main categories: invariance, equivariance with spherical frame basis, and equivariance with vector frame basis. Then we propose a platform, coined Geom3D, which enables benchmarking the effectiveness of geometric strategies. Geom3D contains 16 advanced symmetry-informed geometric representation models and 14 geometric pretraining methods over 46 diverse datasets, including small molecules, proteins, and crystalline materials. We hope that Geom3D can, on the one hand, eliminate barriers for machine learning researchers interested in exploring scientific problems; and, on the other hand, provide valuable guidance for researchers in computational chemistry, structural biology, and materials science, aiding in the informed selection of representation techniques for specific applications.

cs.LG↗

arXiv2023

Explore and Exploit the Diverse Knowledge in Model Zoo for Domain Generalization

The proliferation of pretrained models, as a result of advancements in pretraining techniques, has led to the emergence of a vast zoo of publicly available models. Effectively utilizing these resources to obtain models with robust out-of-distribution generalization capabilities for downstream tasks has become a crucial area of research. Previous research has primarily focused on identifying the most powerful models within the model zoo, neglecting to fully leverage the diverse inductive biases contained within. This paper argues that the knowledge contained in weaker models is valuable and presents a method for leveraging the diversity within the model zoo to improve out-of-distribution generalization capabilities. Specifically, we investigate the behaviors of various pretrained models across different domains of downstream tasks by characterizing the variations in their encoded representations in terms of two dimensions: diversity shift and correlation shift. This characterization enables us to propose a new algorithm for integrating diverse pretrained models, not limited to the strongest models, in order to achieve enhanced out-of-distribution generalization performance. Our proposed method demonstrates state-of-the-art empirical results on a variety of datasets, thus validating the benefits of utilizing diverse knowledge.

cs.LG↗

arXiv2023

Complex-valued neural operator assisted soliton identification

The numerical determination of solitary states is an important topic for such research areas as Bose-Einstein condensates, nonlinear optics, plasma physics, etc. In this paper, we propose a data-driven approach for identifying solitons based on dynamical solutions of real-time differential equations. Our approach combines a machine-learning architecture called the complex-valued neural operator (CNO) with an energy-restricted gradient optimization. The former serves as a generalization of the traditional neural operator to the complex domain, and constructs a smooth mapping between the initial and final states; the latter facilitates the search for solitons by constraining the energy space. We concretely demonstrate this approach on the quasi-one-dimensional Bose-Einstein condensate with homogeneous and inhomogeneous nonlinearities. Our work offers a new idea for data-driven effective modeling and studies of solitary waves in nonlinear physical systems.

cond-mat.quant-gas↗

arXiv2023

Improving the Gilbert-Varshamov Bound by Graph Spectral Method

We improve Gilbert-Varshamov bound by graph spectral method. Gilbert graph $G_{q,n,d}$ is a graph with all vectors in $\mathbb{F}_q^n$ as vertices where two vertices are adjacent if their Hamming distance is less than $d$. In this paper, we calculate the eigenvalues and eigenvectors of $G_{q,n,d}$ using the properties of Cayley graph. The improved bound is associated with the minimum eigenvalue of the graph. Finally we give an algorithm to calculate the bound and linear codes which satisfy the bound.

cs.IT↗

arXiv2023

On the Message Passing Efficiency of Polar and Low-Density Parity-Check Decoders

This study focuses on the efficiency of message-passing-based decoding algorithms for polar and low-density parity-check (LDPC) codes. Both successive cancellation (SC) and belief propagation (BP) decoding algorithms are studied {in} the message-passing framework. Counter-intuitively, SC decoding demonstrates the highest decoding efficiency, although it was considered a weak decoder {in terms of} error-correction performance. We analyze the complexity-performance tradeoff to dynamically track the decoding efficiency, where the complexity is measured by the number of messages passed (NMP), and the performance is measured by the statistical distance to the maximum a posteriori (MAP) estimate. This study offers a new insight into the contribution of each message passed in decoding, and compares various decoding algorithms on a message-by-message level. The analysis corroborates recent results on terabits-per-second polar SC decoders, and might shed light on better scheduling strategies.

cs.IT↗

arXiv2022

When Does Group Invariant Learning Survive Spurious Correlations?

By inferring latent groups in the training data, recent works introduce invariant learning to the case where environment annotations are unavailable. Typically, learning group invariance under a majority/minority split is empirically shown to be effective in improving out-of-distribution generalization on many datasets. However, theoretical guarantee for these methods on learning invariant mechanisms is lacking. In this paper, we reveal the insufficiency of existing group invariant learning methods in preventing classifiers from depending on spurious correlations in the training set. Specifically, we propose two criteria on judging such sufficiency. Theoretically and empirically, we show that existing methods can violate both criteria and thus fail in generalizing to spurious correlation shifts. Motivated by this, we design a new group invariant learning method, which constructs groups with statistical independence tests, and reweights samples by group label proportion to meet the criteria. Experiments on both synthetic and real data demonstrate that the new method significantly outperforms existing group invariant learning methods in generalizing to spurious correlation shifts.

cs.LG↗

arXiv2022

PEMP: Leveraging Physics Properties to Enhance Molecular Property Prediction

Molecular property prediction is essential for drug discovery. In recent years, deep learning methods have been introduced to this area and achieved state-of-the-art performances. However, most of existing methods ignore the intrinsic relations between molecular properties which can be utilized to improve the performances of corresponding prediction tasks. In this paper, we propose a new approach, namely Physics properties Enhanced Molecular Property prediction (PEMP), to utilize relations between molecular properties revealed by previous physics theory and physical chemistry studies. Specifically, we enhance the training of the chemical and physiological property predictors with related physics property prediction tasks. We design two different methods for PEMP, respectively based on multi-task learning and transfer learning. Both methods include a model-agnostic molecule representation module and a property prediction module. In our implementation, we adopt both the state-of-the-art molecule embedding models under the supervised learning paradigm and the pretraining paradigm as the molecule representation module of PEMP, respectively. Experimental results on public benchmark MoleculeNet show that the proposed methods have the ability to outperform corresponding state-of-the-art models.

q-bio.BM↗

arXiv2022

Deterministic Identification over Channels without CSI

Identification capacities of randomized and deterministic identification were proved to exceed channel capacity for Gaussian channels \emph{with} channel side information (CSI). In this work, we extend deterministic identification to the block fading channels without CSI by applying identification codes for both channel estimation and user identification. We prove that identification capacity is asymptotically higher than transmission capacity even in the absence of CSI. And we also analyze the finite-length performance theoretically and numerically. The simulation results verify the feasibility of the proposed blind deterministic identification in finite blocklength regime.

cs.IT↗

arXiv2022

The Complete SC-invariant Affine Automorphisms of Polar Codes

Automorphism ensemble (AE) decoding for polar codes was proposed by decoding permuted codewords with successive cancellation (SC) decoders in parallel and hence has lower latency compared to that of successive cancellation list (SCL) decoding. However, some automorphisms are SC-invariant, thus are redundant in AE decoding. In this paper, we find a necessary and sufficient condition related to the block lower-triangular structure of transformation matrices to identify SC-invariant automorphisms. Furthermore, we provide an algorithm to determine the complete SC-invariant affine automorphisms under a specific polar code construction.

cs.IT↗

arXiv2021

On the Weight Spectrum of Pre-Transformed Polar Codes

Polar codes are the first class of channel codes achieving the symmetric capacity of the binary-input discrete memoryless channels with efficient encoding and decoding algorithms. But the weight spectrum of Polar codes is relatively poor compared to RM codes, which degrades their ML performance. Pre-transformation with an upper-triangular matrix (including cyclic redundancy check (CRC), parity-check (PC) and polarization-adjusted convolutional (PAC) codes), improves weight spectrum while retaining polarization. In this paper, the weight spectrum of upper-triangular pre-transformed Polar Codes is mathematically analyzed. In particular, we focus on calculating the number of low-weight codewords due to their impact on error-correction performance. Simulation results verify the accuracy of the analysis.

cs.IT↗

arXiv2021

The Complete Affine Automorphism Group of Polar Codes

Recently, a permutation-based successive cancellation (PSC) decoding framework for polar codes attaches much attention. It decodes several permuted codewords with independent successive cancellation (SC) decoders. Its latency thus can be reduced to that of SC decoding. However, the PSC framework is ineffective for permutations falling into the lower-triangular affine (LTA) automorphism group, as they are invariant under SC decoding. As such, a larger block lower-triangular affine (BLTA) group that contains SC-variant permutations was discovered for decreasing polar codes. But it was unknown whether BLTA equals the complete automorphism group. In this paper, we prove that BLTA equals the complete automorphisms of decreasing polar codes that can be formulated as affine trasformations.

cs.IT↗