arXiv subjects

Jiantao Wu

Publications and source records attributed to Jiantao Wu.

18 recordsLinked to original sources

arXiv2026

DegradeQuery: Counterfactual Tuple Pretraining for Context-Aware PROTAC Degradation Prediction

Proteolysis-targeting chimeras (PROTACs) induce protein degradation by recruiting a target protein to an E3 ubiquitin ligase, making degradation a joint outcome of the degrader molecule and its biological context. Although public databases contain thousands of structured molecule-target-E3 records, degradation measurements are available for only a small fraction of them. Existing supervised approaches therefore leave most recorded chemical-biological relationships unused. We introduce DegradeQuery, a context-aware prediction framework that converts these label-missing records into a pretraining signal. Its counterfactual tuple pretraining objective contrasts recorded tuples with alternatives formed by replacing the target, the E3 ligase, or both, enabling the model to learn contextual associations without assigning activity pseudo-labels. The resulting representation is then fine-tuned to predict degradation from the complete molecule-target-E3 context. On the official PROTAC-8K benchmark, DegradeQuery achieves an area under the receiver operating characteristic curve of 0.9065 and an accuracy of 0.8500, outperforming the compared methods. Controlled analyses further show that the improvement is primarily attributable to tuple-level pretraining, can be recovered using only label-missing records, and remains complementary to protein language model representations. These findings demonstrate that incompletely labeled PROTAC databases contain useful relational supervision and provide a practical route for learning context-aware degradation predictors from scarce experimental labels.

q-bio.BM↗

arXiv2026

Rethinking Drug-Drug Interaction Modeling as Generalizable Relation Learning

Drug-drug interaction (DDI) prediction is central to drug discovery and clinical development, particularly in the context of increasingly prevalent polypharmacy. Although existing computational methods achieve strong performance on standard benchmarks, they often fail to generalize to realistic deployment scenarios, where most candidate drug pairs involve previously unseen drugs and validated interactions are scarce. We demonstrate that proximity in the embedding spaces of prevailing molecule-centric DDI models does not reliably correspond to interaction labels, and that simply scaling up model capacity therefore fails to improve generalization. To address these limitations, we propose GenRel-DDI, a generalizable relation learning framework that reformulates DDI prediction as a relation-centric learning problem, in which interaction representations are learned independently of drug identities. This relation-level abstraction enables the capture of transferable interaction patterns that generalize to unseen drugs and novel drug pairs. Extensive experiments across multiple benchmark demonstrate that GenRel-DDI consistently and significantly outperforms state-of-the-art methods, with particularly large gains on strict entity-disjoint evaluations, highlighting the effectiveness and practical utility of relation learning for robust DDI prediction. The code is available at https://github.com/SZU-ADDG/GenRel-DDI.

cs.LG↗

arXiv2025

Rethinking Positive Pairs in Contrastive Learning

The training methods in AI do involve semantically distinct pairs of samples. However, their role typically is to enhance the between class separability. The actual notion of similarity is normally learned from semantically identical pairs. This paper presents SimLAP: a simple framework for learning visual representation from arbitrary pairs. SimLAP explores the possibility of learning similarity from semantically distinct sample pairs. The approach is motivated by the observation that for any pair of classes there exists a subspace in which semantically distinct samples exhibit similarity. This phenomenon can be exploited for a novel method of learning, which optimises the similarity of an arbitrary pair of samples, while simultaneously learning the enabling subspace. The feasibility of the approach will be demonstrated experimentally and its merits discussed.

cs.CV↗

arXiv2024

DailyMAE: Towards Pretraining Masked Autoencoders in One Day

Recently, masked image modeling (MIM), an important self-supervised learning (SSL) method, has drawn attention for its effectiveness in learning data representation from unlabeled data. Numerous studies underscore the advantages of MIM, highlighting how models pretrained on extensive datasets can enhance the performance of downstream tasks. However, the high computational demands of pretraining pose significant challenges, particularly within academic environments, thereby impeding the SSL research progress. In this study, we propose efficient training recipes for MIM based SSL that focuses on mitigating data loading bottlenecks and employing progressive training techniques and other tricks to closely maintain pretraining performance. Our library enables the training of a MAE-Base/16 model on the ImageNet 1K dataset for 800 epochs within just 18 hours, using a single machine equipped with 8 A100 GPUs. By achieving speed gains of up to 5.8 times, this work not only demonstrates the feasibility of conducting high-efficiency SSL training but also paves the way for broader accessibility and promotes advancement in SSL research particularly for prototyping and initial testing of SSL ideas. The code is available in https://github.com/erow/FastSSL.

cs.LG↗

arXiv2023

Beyond Accuracy: Statistical Measures and Benchmark for Evaluation of Representation from Self-Supervised Learning

Recently, self-supervised metric learning has raised attention for the potential to learn a generic distance function. It overcomes the limitations of conventional supervised one, e.g., scalability and label biases. Despite progress in this domain, current benchmarks, incorporating a narrow scope of classes, stop the nuanced evaluation of semantic representations. To bridge this gap, we introduce a large-scale benchmark with diversity and granularity of classes, Statistical Metric Learning Benchmark (SMLB) built upon ImageNet-21K and WordNet. SMLB is designed to rigorously evaluate the discriminative discernment and generalizability across more than 14M images, 20K classes, and 16K taxonomic nodes. Alongside, we propose novel evaluation metrics -- `overlap' for separability and `aSTD' for consistency -- to measure distance statistical information, which are efficient and robust to the change of class number. Our benchmark offers a novel perspective of evaluating the quality of representations beyond accuracy. Our findings reveal the limitations of supervised learning and the class bias inherent in SSL models, offering insights into potential areas for future model enhancement.

cs.CV↗

arXiv2023

Variantional autoencoder with decremental information bottleneck for disentanglement

One major challenge of disentanglement learning with variational autoencoders is the trade-off between disentanglement and reconstruction fidelity. Previous studies, which increase the information bottleneck during training, tend to lose the constraint of disentanglement, leading to the information diffusion problem. In this paper, we present a novel framework for disentangled representation learning, DeVAE, which utilizes hierarchical latent spaces with decreasing information bottlenecks across these spaces. The key innovation of our approach lies in connecting the hierarchical latent spaces through disentanglement-invariant transformations, allowing the sharing of disentanglement properties among spaces while maintaining an acceptable level of reconstruction performance. We demonstrate the effectiveness of DeVAE in achieving a balance between disentanglement and reconstruction through a series of experiments and ablation studies on dSprites and Shapes3D datasets. Code is available at https://github.com/erow/disentanglement_lib/tree/pytorch#devae.

cs.LG↗

arXiv2023

Masked Momentum Contrastive Learning for Zero-shot Semantic Understanding

Self-supervised pretraining (SSP) has emerged as a popular technique in machine learning, enabling the extraction of meaningful feature representations without labelled data. In the realm of computer vision, pretrained vision transformers (ViTs) have played a pivotal role in advancing transfer learning. Nonetheless, the escalating cost of finetuning these large models has posed a challenge due to the explosion of model size. This study endeavours to evaluate the effectiveness of pure self-supervised learning (SSL) techniques in computer vision tasks, obviating the need for finetuning, with the intention of emulating human-like capabilities in generalisation and recognition of unseen objects. To this end, we propose an evaluation protocol for zero-shot segmentation based on a prompting patch. Given a point on the target object as a prompt, the algorithm calculates the similarity map between the selected patch and other patches, upon that, a simple thresholding is applied to segment the target. Another evaluation is intra-object and inter-object similarity to gauge discriminatory ability of SSP ViTs. Insights from zero-shot segmentation from prompting and discriminatory abilities of SSP led to the design of a simple SSP approach, termed MMC. This approaches combines Masked image modelling for encouraging similarity of local features, Momentum based self-distillation for transferring semantics from global to local features, and global Contrast for promoting semantics of global features, to enhance discriminative representations of SSP ViTs. Consequently, our proposed method significantly reduces the overlap of intra-object and inter-object similarities, thereby facilitating effective object segmentation within an image. Our experiments reveal that MMC delivers top-tier results in zero-shot semantic segmentation across various datasets.

cs.CV↗

arXiv2022

Link Climate: An Interoperable Knowledge Graph Platform for Climate Data

Climate science has become more ambitious in recent years as global awareness about the environment has grown. To better understand climate, historical climate (e.g. archived meteorological variables such as temperature, wind, water, etc.) and climate-related data (e.g. geographical features and human activities) are widely used by today's climate research to derive models for an explainable climate change and its effects. However, such data sources are often dispersed across a multitude of disconnected data silos on the Web. Moreover, there is a lack of advanced climate data platforms to enable multi-source heterogeneous climate data analysis, therefore, researchers must face a stern challenge in collecting and analyzing multi-source data. In this paper, we address this problem by proposing a climate knowledge graph for the integration of multiple climate data and other data sources into one service, leveraging Web technologies (e.g. HTTP) for multi-source climate data analysis. The proposed knowledge graph is primarily composed of data from the National Oceanic and Atmospheric Administration's daily climate summaries, OpenStreetMap, and Wikidata, and it supports joint data queries on these widely used databases. This paper shows, with a use case in Ireland and the United Kingdom, how climate researchers could benefit from this platform as it allows them to easily integrate datasets from different domains and geographical locations.

cs.DB↗

arXiv2022

A semantic web approach to uplift decentralized household energy data

In a decentralized household energy system comprised of various devices such as home appliances, electric vehicles, and solar panels, end-users are able to dig deeper into the system's details and further achieve energy sustainability if they are presented with data on the electric energy consumption and production at the granularity of the device. However, many databases in this field are siloed from other domains, including solely information pertaining to energy. This may result in the loss of information (e.g. weather) on each device's energy use. Meanwhile, a large number of these datasets have been extensively used in computational modeling techniques such as machine learning models. While such computational approaches achieve great accuracy and performance by concentrating only on a local view of datasets, model reliability cannot be guaranteed since such models are very vulnerable to data input fluctuations when information omission is taken into account. This article tackles the data isolation issue in the field of smart energy systems by examining Semantic Web methods on top of a household energy system. We offer an ontology-based approach for managing decentralized data at the device-level resolution in a system. As a consequence, the scope of the data associated with each device may easily be expanded in an interoperable manner throughout the Web, and additional information, such as weather, can be obtained from the Web, provided that the data is organized according to W3C standards.

cs.AI↗

arXiv2022

Analyzing Air Pollutant Concentrations in New Delhi, India

Air pollutants have long been known to cause major health problems across humans and all living organisms. Apart from that, they also play a crucial role in temperature inversion situations in the atmospheric layers thereby seriously impacting the radio communications, increased fog levels and decreased visibility. Appreciating the seriousness of these pollutants, this paper attempts to analyze and create a publicly available and easily accessible dataset of seven different pollutants for New Delhi region in India. This analysis and pre-processing is done to assist the researchers who wish to use the dataset for further studies like pollutant forecasting or correlation analysis, thereby promoting the research in the domain.

physics.ao-ph↗

arXiv2022

Object-wise Masked Autoencoders for Fast Pre-training

Self-supervised pre-training for images without labels has recently achieved promising performance in image classification. The success of transformer-based methods, ViT and MAE, draws the community's attention to the design of backbone architecture and self-supervised task. In this work, we show that current masked image encoding models learn the underlying relationship between all objects in the whole scene, instead of a single object representation. Therefore, those methods bring a lot of compute time for self-supervised pre-training. To solve this issue, we introduce a novel object selection and division strategy to drop non-object patches for learning object-wise representations by selective reconstruction with interested region masks. We refer to this method ObjMAE. Extensive experiments on four commonly-used datasets demonstrate the effectiveness of our model in reducing the compute cost by 72% while achieving competitive performance. Furthermore, we investigate the inter-object and intra-object relationship and find that the latter is crucial for self-supervised pre-training.

cs.CV↗

arXiv2021

Automated Climate Analyses Using Knowledge Graph

The FAIR (Findable, Accessible, Interoperable, Reusable) data principles are fundamental for climate researchers and all stakeholders in the current digital ecosystem. In this paper, we demonstrate how relational climate data can be "FAIR" and modeled using RDF, in line with Semantic Web technologies and our Climate Analysis ontology. Thus, heterogeneous climate data can be stored in graph databases and offered as Linked Data on the Web. As a result, climate researchers will be able to use the standard SPARQL query language to query these sources directly on the Web. In this paper, we demonstrate the usefulness of our SPARQL endpoint for automated climate analytics. We illustrate two sample use cases that establish the advantage of representing climate data as knowledge graphs.

cs.DB↗

arXiv2021

Analyzing the Impact of Meteorological Parameters on Rainfall Prediction

Rainfall is a climatic factor that affects many human activities like agriculture, construction, and forestry. Rainfall is dependent on various meteorological features and its prediction is a very complex task due to the dynamic climatic nature. A detailed study of different climatic features associated with the occurrence of rainfall should be made in order to understand the influence of each parameter in the context of rainfall. In this paper, we propose a methodical approach to analyze the affect of various parameters on rainfall. Our study uses $5$ years of meteorological data from a weather station located in the United States. The correlation and interdependence among the collected meteorological features were obtained. Additionally, we identified the most important meteorological features for rainfall prediction using a machine learning-based feature selection technique.

physics.ao-ph↗

arXiv2021

An Interoperable Open Data Portal for Climate Analysis

This work proposes an open interoperable data portal that offers access to a Web-wide climate domain knowledge graph created for Ireland and England's NOAA climate daily data. There are three main components contributing to this data portal: the first is the upper layer schema of the knowledge graph -- the climate analysis (CA) ontology -- the second is an ad hoc SPARQL server by which to store the graph data and provide public Web access, the last is a dereferencing engine deployed to resolve URIs for entity information. Our knowledge graph form of NOAA climate data facilitates the supply of semantic climate information to researchers and offers a variety of semantic applications that can be built on top of it.

cs.DB↗

arXiv2021

Ontology Modeling for Decentralized Household Energy Systems

In a decentralized household energy system consisting of various devices such as washing machines, heat pumps, and solar panels, understanding the electric energy consumption and production data at the granularity of the device helps end-users be closer to the system and further achieve the sustainability of energy use. However, many datasets in this area are isolated from other domains with records of only energy-related data. This may raise a loss of information (e.g. weather) that is relevant to the energy use of each device. A noticeable disadvantage is that many of those datasets have to be used in computational modeling approaches such as machine learning models, which are vulnerable to the data feed, to advance the understanding of energy consumption and production. Although such computational methods have achieved a high benchmark merely through a local view of datasets, the reusability cannot be firmly guaranteed when the information omission is taken into account. This paper addresses the data isolation problem in the smart energy systems area by exploring Semantic Web techniques on top of a household energy system. We propose an ontology modeling solution for the management of decentralized data at the resolution of a device in the system. As a result, the scope of the data concerning each device can be easily extended to be wider across the web and more information that may be of interest such as weather can be retrieved from the Web if the data are structured by the ontology.

cs.DB↗

arXiv2021

An Ontology Model for Climatic Data Analysis

Recently ontologies have been exploited in a wide range of research areas for data modeling and data management. They greatly assists in defining the semantic model of the underlying data combined with domain knowledge. In this paper, we propose the Climate Analysis (CA) Ontology to model climate datasets used by remote sensing analysts. We use the data published by National Oceanic and Atmospheric Administration (NOAA) to further explore how ontology modeling can be used to facilitate the field of climatic data processing. The idea of this work is to convert relational climate data to the Resource Description Framework (RDF) data model, so that it can be stored in a graph database and easily accessed through the Web as Linked Data. Typically, this provides climate researchers, who are interested in datasets such as NOAA, with the potential of enriching and interlinking with other databases. As a result, our approach facilitates data integration and analysis of diverse climatic data sources and allows researchers to interrogate these sources directly on the Web using the standard SPARQL query language.

cs.DB↗

arXiv2021

DEFT: Distilling Entangled Factors by Preventing Information Diffusion

Disentanglement is a highly desirable property of representation owing to its similarity to human understanding and reasoning. Many works achieve disentanglement upon information bottlenecks (IB). Despite their elegant mathematical foundations, the IB branch usually exhibits lower performance. In order to provide an insight into the problem, we develop an annealing test to calculate the information freezing point (IFP), which is a transition state to freeze information into the latent variables. We also explore these clues or inductive biases for separating the entangled factors according to the differences in the IFP distributions. We found the existing approaches suffer from the information diffusion problem, according to which the increased information diffuses in all latent variables. Based on this insight, we propose a novel disentanglement framework, termed the distilling entangled factor (DEFT), to address the information diffusion problem by scaling backward information. DEFT applies a multistage training strategy, including multigroup encoders with different learning rates and piecewise disentanglement pressure, to disentangle the factors stage by stage. We evaluate DEFT on three variants of dSprite and SmallNORB, which show low-variance and high-level disentanglement scores. Furthermore, the experiment under the correlative factors shows incapable of TC-based approaches. DEFT also exhibits a competitive performance in the unsupervised setting.

cs.LG↗

arXiv2020

Disentangling Action Sequences: Discovering Correlated Samples

Disentanglement is a highly desirable property of representation due to its similarity with human's understanding and reasoning. This improves interpretability, enables the performance of down-stream tasks, and enables controllable generative models. However, this domain is challenged by the abstract notion and incomplete theories to support unsupervised disentanglement learning. We demonstrate the data itself, such as the orientation of images, plays a crucial role in disentanglement and instead of the factors, and the disentangled representations align the latent variables with the action sequences. We further introduce the concept of disentangling action sequences which facilitates the description of the behaviours of the existing disentangling approaches. An analogy for this process is to discover the commonality between the things and categorizing them. Furthermore, we analyze the inductive biases on the data and find that the latent information thresholds are correlated with the significance of the actions. For the supervised and unsupervised settings, we respectively introduce two methods to measure the thresholds. We further propose a novel framework, fractional variational autoencoder (FVAE), to disentangle the action sequences with different significance step-by-step. Experimental results on dSprites and 3D Chairs show that FVAE improves the stability of disentanglement.

cs.LG↗