Searcharxiv⌕ Search

arXiv subjects

Jun Zhang

Publications and source records attributed to Jun Zhang.

At least 613 records · Page 34Linked to original sources
arXiv2023

Unsupervisedly Prompting AlphaFold2 for Few-Shot Learning of Accurate Folding Landscape and Protein Structure Prediction

Data-driven predictive methods which can efficiently and accurately transform protein sequences into biologically active structures are highly valuable for scientific research and medical development. Determining accurate folding landscape using co-evolutionary information is fundamental to the success of modern protein structure prediction methods. As the state of the art, AlphaFold2 has dramatically raised the accuracy without performing explicit co-evolutionary analysis. Nevertheless, its performance still shows strong dependence on available sequence homologs. Based on the interrogation on the cause of such dependence, we presented EvoGen, a meta generative model, to remedy the underperformance of AlphaFold2 for poor MSA targets. By prompting the model with calibrated or virtually generated homologue sequences, EvoGen helps AlphaFold2 fold accurately in low-data regime and even achieve encouraging performance with single-sequence predictions. Being able to make accurate predictions with few-shot MSA not only generalizes AlphaFold2 better for orphan sequences, but also democratizes its use for high-throughput applications. Besides, EvoGen combined with AlphaFold2 yields a probabilistic structure generation method which could explore alternative conformations of protein sequences, and the task-aware differentiable algorithm for sequence generation will benefit other related tasks including protein design.

cs.LG↗
arXiv2023

L-Eval: Instituting Standardized Evaluation for Long Context Language Models

Recently, there has been growing interest in extending the context length of large language models (LLMs), aiming to effectively process long inputs of one turn or conversations with more extensive histories. While proprietary models such as GPT-4 and Claude can largely preserve the reasoning ability in an extended context, open-source models are still progressing through the early stages of development. To bridge this gap, we propose L-Eval to institute a more standardized evaluation for long context language models (LCLMs) addressing two key aspects: dataset construction and evaluation metrics. On the one hand, we build a new evaluation suite containing 20 sub-tasks, 508 long documents, and over 2,000 human-labeled query-response pairs encompassing diverse question styles, domains, and input length (3k$\sim$200k tokens). On the other hand, we investigate the effectiveness in evalution metrics for LCLMs. Results show that popular n-gram matching metrics generally can not correlate well with human judgment, and thus we strongly advocate for length-instruction-enhanced (LIE) evaluation and employing LLM judges. We conducted a comprehensive study of 4 popular commercial LLMs and 12 open-source counterparts using the L-Eval benchmark. Our empirical findings offer useful insights into the study of LCLMs and lay the groundwork for the development of more principled evaluation of these models.

cs.CL↗
arXiv2023

Mode Connectivity and Data Heterogeneity of Federated Learning

Federated learning (FL) enables multiple clients to train a model while keeping their data private collaboratively. Previous studies have shown that data heterogeneity between clients leads to drifts across client updates. However, there are few studies on the relationship between client and global modes, making it unclear where these updates end up drifting. We perform empirical and theoretical studies on this relationship by utilizing mode connectivity, which measures performance change (i.e., connectivity) along parametric paths between different modes. Empirically, reducing data heterogeneity makes the connectivity on different paths more similar, forming more low-error overlaps between client and global modes. We also find that a barrier to connectivity occurs when linearly connecting two global modes, while it disappears with considering non-linear mode connectivity. Theoretically, we establish a quantitative bound on the global-mode connectivity using mean-field theory or dropout stability. The bound demonstrates that the connectivity improves when reducing data heterogeneity and widening trained models. Numerical results further corroborate our analytical findings.

cs.LG↗
arXiv2023

Holographic renormalized Entanglement and entropic $c-$function

We compute holographic entanglement entropy (EE) and the renormalized EE in AdS solitons with gauge potential for various dimensions. The renormalized EE is a cutoff-independent universal component of EE. Via Kaluza-Klein compactification of $S^1$ and considering the low-energy regime, we deduce the $(d-1)$-dimensional renormalized EE from the odd-dimensional counterpart. This corresponds to the shrinking circle of AdS solitons, probed at large $l$. The minimal surface transitions from disk to cylinder dominance as $l$ increases. The quantum phase transition occurs at a critical subregion size, with renormalized EE showing non-monotonic behavior around this size. Across dimensions, massive modes decouple at lower energy, while degrees of freedom with Wilson lines contribute at smaller energy scales.

hep-th↗
arXiv2023

Semantic Communications using Foundation Models: Design Approaches and Open Issues

Foundation models (FMs), including large language models, have become increasingly popular due to their wide-ranging applicability and ability to understand human-like semantics. While previous research has explored the use of FMs in semantic communications to improve semantic extraction and reconstruction, the impact of these models on different system levels, considering computation and memory complexity, requires further analysis. This study focuses on integrating FMs at the effectiveness, semantic, and physical levels, using universal knowledge to profoundly transform system design. Additionally, it examines the use of compact models to balance performance and complexity, comparing three separate approaches that employ FMs. Ultimately, the study highlights unresolved issues in the field that need addressing.

eess.SP↗
arXiv2023

Towards Real-Time Neural Video Codec for Cross-Platform Application Using Calibration Information

The state-of-the-art neural video codecs have outperformed the most sophisticated traditional codecs in terms of RD performance in certain cases. However, utilizing them for practical applications is still challenging for two major reasons. 1) Cross-platform computational errors resulting from floating point operations can lead to inaccurate decoding of the bitstream. 2) The high computational complexity of the encoding and decoding process poses a challenge in achieving real-time performance. In this paper, we propose a real-time cross-platform neural video codec, which is capable of efficiently decoding of 720P video bitstream from other encoding platforms on a consumer-grade GPU. First, to solve the problem of inconsistency of codec caused by the uncertainty of floating point calculations across platforms, we design a calibration transmitting system to guarantee the consistent quantization of entropy parameters between the encoding and decoding stages. The parameters that may have transboundary quantization between encoding and decoding are identified in the encoding stage, and their coordinates will be delivered by auxiliary transmitted bitstream. By doing so, these inconsistent parameters can be processed properly in the decoding stage. Furthermore, to reduce the bitrate of the auxiliary bitstream, we rectify the distribution of entropy parameters using a piecewise Gaussian constraint. Second, to match the computational limitations on the decoding side for real-time video codec, we design a lightweight model. A series of efficiency techniques enable our model to achieve 25 FPS decoding speed on NVIDIA RTX 2080 GPU. Experimental results demonstrate that our model can achieve real-time decoding of 720P videos while encoding on another platform. Furthermore, the real-time model brings up to a maximum of 24.2\% BD-rate improvement from the perspective of PSNR with the anchor H.265.

cs.CV↗
arXiv2023

Orbits and tsectors in irregular exceptional directions of full-null degenerate singular point

Near full-null degenerate singular points of analytic vector fields, asymptotic behaviors of orbits are not given by eigenvectors but totally decided by nonlinearities. Especially, in the case of high full-null degeneracy, i.e., the lowest degree of nonlinearities is high, such a singular point may have irregular exceptional directions and the blow-up technique can be hardly applied, which leaves a problem how to determine numbers of orbits and (elliptic, hyperbolic and parabolic) tangential sectors in this case. In this paper we work on this problem. Using Newton polygons to decompose nonlinearities into principal parts and remainder parts, we convert the problem to the numbers of nonzero real roots of edge-polynomials of principal parts. Computing Newton polygons for multiplication and differentiation of analytic functions and giving Newton polygons for addition, which was not found in literatures, we determine semi-definiteness of the Lie-bracket of principal parts and therefore obtain criteria for those numbers.

math.DS↗
arXiv2023

Orientation-Dependent Atomic-Scale Mechanism of $β$-$\mathrm{Ga}_{2}\mathrm{O}_{3}$ Thin Film Epitaxial Growth

$β$-$\mathrm{Ga}_{2}\mathrm{O}_{3}$ has gained intensive interests of research and application as an ultrawide bandgap semiconductor. Epitaxial growth technique of the $β$-$\mathrm{Ga}_{2}\mathrm{O}_{3}$ thin film possesses a fundamental and vital role in the $\mathrm{Ga}_{2}\mathrm{O}_{3}$-based device fabrication. In this work, epitaxial growth mechanisms of $β$-$\mathrm{Ga}_{2}\mathrm{O}_{3}$ with four low Miller-index facets, namely (100), (010), (001), and ($\overline{2}$01), are systematically explored using large-scale machine-learning molecular dynamics simulations at the atomic scale. The simulations reveal that the migration of the face-centered cubic stacking O sublattice plays a predominant role in rationalizing the different growth mechanisms between (100)/(010)/(001) and ($\overline{2}$01) orientations. The resultant complex combinations of the stacking faults and twin boundaries are carefully identified, and shows a good agreement with the experimental observation and ab initio calculation. Our results provide useful insights into the gas-phase epitaxial growth of the $β$-$\mathrm{Ga}_{2}\mathrm{O}_{3}$ thin films and suggest possible ways to tailor its properties for specific applications.

cond-mat.mtrl-sci↗
arXiv2023

A Synergy between On- and Off-Chip Data Reuse for GPU-based Out-of-Core Stencil Computation

Stencil computation is an extensively-utilized class of scientific-computing applications that can be efficiently accelerated by graphics processing units (GPUs). Out-of-core approaches enable a GPU to handle large stencil codes whose data size is beyond the memory capacity of the GPU. However, current research on out-of-core stencil computation primarily focus on minimizing the amount of data transferred between the CPU and GPU. Few studies consider simultaneously optimizing data transfer and kernel execution. To fill the research gap, this work presents a synergy between on- and off-chip data reuse for out-of-core stencil codes, termed SO2DR. First, overlapping regions between data chunks are shared in the off-chip memory to eliminate redundant CPU-GPU data transfer. Secondly, redundant computation at the off-chip memory level is intentionally introduced to decouple kernel execution from region sharing, hence enabling data reuse in the on-chip memory. Experimental results demonstrate that SO2DR significantly enhances the kernel-execution performance while reducing the CPU-GPU data-transfer time. Specifically, SO2DR achieves average speedups of 2.78x and 1.14x for five stencil benchmarks, compared to an out-of-core stencil code which is free of redundant transfer and computation, and an in-core stencil code which is free of data transfer, respectively.

cs.DC↗
arXiv2023

Client-side Gradient Inversion Against Federated Learning from Poisoning

Federated Learning (FL) enables distributed participants (e.g., mobile devices) to train a global model without sharing data directly to a central server. Recent studies have revealed that FL is vulnerable to gradient inversion attack (GIA), which aims to reconstruct the original training samples and poses high risk against the privacy of clients in FL. However, most existing GIAs necessitate control over the server and rely on strong prior knowledge including batch normalization and data distribution information. In this work, we propose Client-side poisoning Gradient Inversion (CGI), which is a novel attack method that can be launched from clients. For the first time, we show the feasibility of a client-side adversary with limited knowledge being able to recover the training samples from the aggregated global model. We take a distinct approach in which the adversary utilizes a malicious model that amplifies the loss of a specific targeted class of interest. When honest clients employ the poisoned global model, the gradients of samples belonging to the targeted class are magnified, making them the dominant factor in the aggregated update. This enables the adversary to effectively reconstruct the private input belonging to other clients using the aggregated update. In addition, our CGI also features its ability to remain stealthy against Byzantine-robust aggregation rules (AGRs). By optimizing malicious updates and blending benign updates with a malicious replacement vector, our method remains undetected by these defense mechanisms. To evaluate the performance of CGI, we conduct experiments on various benchmark datasets, considering representative Byzantine-robust AGRs, and exploring diverse FL settings with different levels of adversary knowledge about the data. Our results demonstrate that CGI consistently and successfully extracts training input in all tested scenarios.

cs.CR↗
arXiv2023

Task-Oriented Communication for Multi-Device Cooperative Edge Inference

This paper investigates task-oriented communication for multi-device cooperative edge inference, where a group of distributed low-end edge devices transmit the extracted features of local samples to a powerful edge server for inference. While cooperative edge inference can overcome the limited sensing capability of a single device, it substantially increases the communication overhead and may incur excessive latency. To enable low-latency cooperative inference, we propose a learning-based communication scheme that optimizes local feature extraction and distributed feature encoding in a task-oriented manner, i.e., to remove data redundancy and transmit information that is essential for the downstream inference task rather than reconstructing the data samples at the edge server. Specifically, we leverage an information bottleneck (IB) principle to extract the task-relevant feature at each edge device and adopt a distributed information bottleneck (DIB) framework to formalize a single-letter characterization of the optimal rate-relevance tradeoff for distributed feature encoding. To admit flexible control of the communication overhead, we extend the DIB framework to a distributed deterministic information bottleneck (DDIB) objective that explicitly incorporates the representational costs of the encoded features. As the IB-based objectives are computationally prohibitive for high-dimensional data, we adopt variational approximations to make the optimization problems tractable. To compensate the potential performance loss due to the variational approximations, we also develop a selective retransmission (SR) mechanism to identify the redundancy in the encoded features of multiple edge devices to attain additional communication overhead reduction. Extensive experiments evidence that the proposed task-oriented communication scheme achieves a better rate-relevance tradeoff than baseline methods.

eess.SP↗
arXiv2023

Observation of Flat Band and Van Hove Singularity in Non-superconducting Nitrogen-doped Lutetium Hydride

Hydrogen-rich materials offer a compelling avenue towards room temperature superconductivity, albeit under ultra-high pressure. However, the experimental investigation of the electronic band structure remains elusive, due to the inherent instability of most of the hydrogen-rich materials upon pressure release. Very recently, nitrogen-doped lutetium hydride was claimed to host room temperature superconductivity under near ambient pressure but was disproven by following works. Upon decompression, nitrogen doped lutetium hydride manifests a stable metallic phase with dark blue color. Moreover, high temperature superconductivity has been reported in lutetium hydrides Lu4H23 (~71 K) under around 200 GPa. These properties engender an unprecedented opportunity, allowing for the experimental investigation of the electronic band structure intrinsic to hydrogen-rich material. In this work, using angle resolved photoemission spectroscopy to investigate the non-superconducting nitrogen doped lutetium hydride, we observed significant flat band and Van Hove singularity marginally below the Fermi level. These salient features, identified as critical elements, proffer potential amplifiers for the realization of heightened superconductivity, as evidenced by prior research. Our results not only unveil a confluence of potent strong correlation effects and anisotropy within the Lu-H-N compound, but also provide a prospect for engineering high temperature superconductivity through the strategic manipulation of flat band and the VHS, effectively tailoring their alignment with the Fermi energy.

cond-mat.supr-con↗
arXiv2023

Stress-shape misalignment in confluent cell layers

This study investigates the relationship between cell shape and cell-generated stresses in confluent cell layers. Using simultaneous measurements of cell shape orientation and cell-generated contractile forces in MDCK and LP-9 colonies, we report the emergence of correlated, dynamic domains in which misalignment between the directors defined by cell shape and by contractile forces reaches up to 90$^o$, effectively creating extensile domains in a monolayer of contractile cells. To understand this misalignment, we develop a continuum model that decouples the orientation of cell-generated active forces from the orientation of the cell shapes. This challenges the prevailing understanding that cells throughout a tissue create either contractile or extensile forces, and the validity of the usual active nematic models of cell motility where active forces are strictly slaved to cell shape orientation.

physics.bio-ph↗
arXiv2023

NTU4DRadLM: 4D Radar-centric Multi-Modal Dataset for Localization and Mapping

Simultaneous Localization and Mapping (SLAM) is moving towards a robust perception age. However, LiDAR- and visual- SLAM may easily fail in adverse conditions (rain, snow, smoke and fog, etc.). In comparison, SLAM based on 4D Radar, thermal camera and IMU can work robustly. But only a few literature can be found. A major reason is the lack of related datasets, which seriously hinders the research. Even though some datasets are proposed based on 4D radar in past four years, they are mainly designed for object detection, rather than SLAM. Furthermore, they normally do not include thermal camera. Therefore, in this paper, NTU4DRadLM is presented to meet this requirement. The main characteristics are: 1) It is the only dataset that simultaneously includes all 6 sensors: 4D radar, thermal camera, IMU, 3D LiDAR, visual camera and RTK GPS. 2) Specifically designed for SLAM tasks, which provides fine-tuned ground truth odometry and intentionally formulated loop closures. 3) Considered both low-speed robot platform and fast-speed unmanned vehicle platform. 4) Covered structured, unstructured and semi-structured environments. 5) Considered both middle- and large- scale outdoor environments, i.e., the 6 trajectories range from 246m to 6.95km. 6) Comprehensively evaluated three types of SLAM algorithms. Totally, the dataset is around 17.6km, 85mins, 50GB and it will be accessible from this link: https://github.com/junzhang2016/NTU4DRadLM

cs.RO↗
arXiv2023

Quantitative characterization in contact Hamiltonian dynamics -- I

Based on the contact Hamiltonian Floer theory established by Will J. Merry and the second author that applies to any admissible contact Hamiltonian system $(M, ξ= \ker α, h)$, where $h$ is a contact Hamiltonian function on a Liouville fillable contact manifold $(M, ξ= \ker α)$, we associate a persistence module to $(M, ξ, h)$, called a gapped module, that is parametrized only by a partially ordered set. It enables us to define various numerical Floer-theoretic invariants. In this paper, we focus on the contact spectral invariants and their applications. Several key properties are proved, which include stability with respect to the Shelukhin-Hofer norm in contact geometry and a triangle inequality of contact spectral invariants. In particular, our stability property does not involve any conformal factors; our triangle inequality is derived from a novel analysis on pair-of-pants in the contact Hamiltonian Floer homology. While this paper was nearing completion, the authors were made aware of upcoming work by Dylan Cant, where a similar persistence module for contact Hamiltonian dynamics was constructed.

math.SG↗
arXiv2023

A multinode quantum network over a metropolitan area

Towards realizing the future quantum internet, a pivotal milestone entails the transition from two-node proof-of-principle experiments conducted in laboratories to comprehensive, multi-node setups on large scales. Here, we report on the debut implementation of a multi-node entanglement-based quantum network over a metropolitan area. We equipped three quantum nodes with atomic quantum memories and their telecom interfaces, and combined them into a scalable phase-stabilized architecture through a server node. We demonstrated heralded entanglement generation between two quantum nodes situated 12.5 km apart, and the storage of entanglement exceeding the round-trip communication time. We also showed the concurrent entanglement generation on three links. Our work provides a metropolitan-scale testbed for the evaluation and exploration of multi-node quantum network protocols and starts a new stage of quantum internet research.

quant-ph↗
arXiv2023

FedCiR: Client-Invariant Representation Learning for Federated Non-IID Features

Federated learning (FL) is a distributed learning paradigm that maximizes the potential of data-driven models for edge devices without sharing their raw data. However, devices often have non-independent and identically distributed (non-IID) data, meaning their local data distributions can vary significantly. The heterogeneity in input data distributions across devices, commonly referred to as the feature shift problem, can adversely impact the training convergence and accuracy of the global model. To analyze the intrinsic causes of the feature shift problem, we develop a generalization error bound in FL, which motivates us to propose FedCiR, a client-invariant representation learning framework that enables clients to extract informative and client-invariant features. Specifically, we improve the mutual information term between representations and labels to encourage representations to carry essential classification knowledge, and diminish the mutual information term between the client set and representations conditioned on labels to promote representations of clients to be client-invariant. We further incorporate two regularizers into the FL framework to bound the mutual information terms with an approximate global representation distribution to compensate for the absence of the ground-truth global representation distribution, thus achieving informative and client-invariant feature extraction. To achieve global representation distribution approximation, we propose a data-free mechanism performed by the server without compromising privacy. Extensive experiments demonstrate the effectiveness of our approach in achieving client-invariant representation learning and solving the data heterogeneity issue.

cs.LG↗
arXiv2023

Coarse distance from dynamically convex to convex

Chaidez and Edtmair have recently found the first example of dynamically convex domains in $\mathbb R^4$ that are not symplectomorphic to convex domains (called symplectically convex domains), answering a long-standing open question. In this paper, we discover new examples of such domains without referring to Chaidez-Edtmair's criterion. We also show that these domains are arbitrarily far from the set of symplectically convex domains in $\mathbb R^4$ with respect to the coarse symplectic Banach-Mazur distance by using an explicit numerical criterion for symplectic non-convexity.

math.SG↗