arXiv subjects

Tianyu Zhu

Publications and source records attributed to Tianyu Zhu.

At least 37 records · Page 2Linked to original sources

arXiv2025

Generalized Contrastive Learning for Multi-Modal Retrieval and Ranking

Contrastive learning has gained widespread adoption for retrieval tasks due to its minimal requirement for manual annotations. However, popular training frameworks typically learn from binary (positive/negative) relevance, making them ineffective at incorporating desired rankings. As a result, the poor ranking performance of these models forces systems to employ a re-ranker, which increases complexity, maintenance effort and inference time. To address this, we introduce Generalized Contrastive Learning (GCL), a training framework designed to learn from continuous ranking scores beyond binary relevance. GCL encodes both relevance and ranking information into a unified embedding space by applying ranking scores to the loss function. This enables a single-stage retrieval system. In addition, during our research, we identified a lack of public multi-modal datasets that benchmark both retrieval and ranking capabilities. To facilitate this and future research for ranked retrieval, we curated a large-scale MarqoGS-10M dataset using GPT-4 and Google Shopping, providing ranking scores for each of the 10 million query-document pairs. Our results show that GCL achieves a 29.3% increase in NDCG@10 for in-domain evaluations and 6.0% to 10.0% increases for cold-start evaluations compared to the finetuned CLIP baseline with MarqoGS-10M. Additionally, we evaluated GCL offline on a proprietary user interaction data. GCL shows an 11.2% gain for in-domain evaluations. The dataset and the method are available at: https://github.com/marqo-ai/GCL.

cs.IR↗

arXiv2025

Energy-Specific Bethe-Salpeter Equation Implementation for Efficient Optical Spectrum Calculations

We present an energy-specific Bethe-Salpeter equation (BSE) implementation for efficient core and valence optical spectrum calculations. In energy-specific BSE, high-lying excitation energies are obtained by constructing trial vectors and expanding the subspace targeting excitation energies above the predefined energy threshold in the Davidson algorithm. To calculate optical spectra over a wide energy range, energy-specific BSE can be applied to multiple consecutive small energy windows, where trial vectors for each subsequent energy window are made orthogonal to the subspace of preceding windows to accelerate the convergence of the Davidson algorithm. For seven small molecules, energy-specific BSE combined with $G_0W_0$ provides small errors around 0.8 eV for absolute and relative $K$-edge excitation energies when starting from a hybrid PBEh solution with 45% exact exchange. We further showcase the computational efficiency of this approach by simulating the N $1s$ $K$-edge excitation spectrum of the porphine molecule and the valence optical spectrum of silicon nanoclusters involving 6,000 excited states using $G_0W_0$-BSE. This work expands the applicability of the $GW$-BSE formalism for investigating high-energy excited states of large systems.

cond-mat.mtrl-sci↗

arXiv2024

Video Quality Assessment: A Comprehensive Survey

Video quality assessment (VQA) is an important processing task, aiming at predicting the quality of videos in a manner highly consistent with human judgments of perceived quality. Traditional VQA models based on natural image and/or video statistics, which are inspired both by models of projected images of the real world and by dual models of the human visual system, deliver only limited prediction performances on real-world user-generated content (UGC), as exemplified in recent large-scale VQA databases containing large numbers of diverse video contents crawled from the web. Fortunately, recent advances in deep neural networks and Large Multimodality Models (LMMs) have enabled significant progress in solving this problem, yielding better results than prior handcrafted models. Numerous deep learning-based VQA models have been developed, with progress in this direction driven by the creation of content-diverse, large-scale human-labeled databases that supply ground truth psychometric video quality data. Here, we present a comprehensive survey of recent progress in the development of VQA algorithms and the benchmarking studies and databases that make them possible. We also analyze open research directions on study design and VQA algorithm architectures. Github link: https://github.com/taco-group/Video-Quality-Assessment-A-Comprehensive-Survey.

eess.IV↗

arXiv2024

Accurate and Efficient Prediction of Double Excitation Energies Using the Particle-Particle Random Phase Approximation

Double excitations are crucial to understanding numerous chemical, physical, and biological processes, but accurately predicting them remains a challenge. In this work, we explore the particle-particle random phase approximation (ppRPA) as an efficient and accurate approach for computing double excitation energies. We benchmark ppRPA using various exchange-correlation functionals for 21 molecular systems and two point defect systems. Our results show that ppRPA with functionals containing appropriate amounts of exact exchange provides accuracy comparable to high-level wave function methods such as CCSDT and CASPT2, with significantly reduced computational cost. Furthermore, we demonstrate the use of ppRPA starting from an excited ($N-2$)-electron state calculated by $Δ$SCF for the first time, as well as its application to double excitations in bulk periodic systems. These findings suggest that ppRPA is a promising tool for the efficient calculation of double and partial double excitation energies in both molecular and bulk systems.

physics.chem-ph↗

arXiv2024

High-Order Spectral Simulation of Dispersive Two-Dimensional Materials

Over the past twenty years, the field of plasmonics has been revolutionized with the isolation and utilization of two--dimensional materials, particularly graphene. Consequently there is significant interest in rapid, robust, and highly accurate computational schemes which can incorporate such materials. Standard volumetric approaches can be contemplated, but these require huge computational resources. Here we describe an algorithm which addresses this issue for nonlocal models of the electromagnetic response of graphene. Our methodology not only approximates the graphene layer with a surface current, but also reformulates the governing volumetric equations in terms of surface quantities using Dirichlet--Neumann Operators. We have recently shown how these surface equations can be numerically simulated in an efficient, stable, and accurate fashion using a High--Order Perturbation of Envelopes methodology. We extend these results to the nonlocal model mentioned above, and using an implementation of this algorithm, we study absorbance spectra of TM polarized plane--waves scattered by a periodic grid of graphene ribbons.

math.NA↗

arXiv2024

Interacting-bath dynamical embedding for capturing non-local electron correlation in solids

Quantitative simulation of electronic structure of solids requires treating local and non-local electron correlations on an equal footing. We present a new ab initio formulation of Green's function embedding which, unlike dynamical mean-field theory that uses non-interacting bath, derives bath representation with general two-particle interactions in a systematically improvable manner. The resulting interacting-bath dynamical embedding theory (ibDET) utilizes an efficient real-axis coupled-cluster solver to compute the self-energy, approaching the full system limit at much reduced cost. When combined with the GW theory, GW+ibDET achieves good agreement with experimental spectral properties across a range of semiconducting, insulating and metallic materials. Our approach also enables quantifying the role of non-local electron correlation in determining material properties and addressing the long-standing debate on the bandwidth narrowing of metallic sodium.

cond-mat.mtrl-sci↗

arXiv2024

Particle-Particle Random Phase Approximation for Predicting Correlated Excited States of Point Defects

The particle-particle random phase approximation (ppRPA) within the hole-hole channel was recently proposed as an efficient tool for computing excitation energies of point defects in solids [J. Phys. Chem. Lett. 2024, 15, 2757-2764]. In this work, we investigate the application of ppRPA within the particle-particle channel for predicting correlated excited states of point defects, including the carbon-vacancy (VC) in diamond, the oxygen-vacancy (VO) in magnesium oxide (MgO), and the carbon dimer defect (C$_{\text{B}}$C$_{\text{N}}$) in two-dimensional hexagonal boron nitride (h-BN). Starting from a density functional theory calculation of the ($N-2$)-electron ground state, vertical excitation energies of the $N$-electron system are obtained as the differences between the two-electron addition energies. We show that active-space ppRPA with the B3LYP functional yields accurate excitation energies, with errors mostly smaller than 0.1 eV for tested systems compared to available experimental values. We further develop a natural transition orbital scheme within ppRPA, which provides insights into the multireference character of defect states. This study, together with our previous work, establishes ppRPA as a low-cost and accurate method for investigating excited-state properties of point defect systems.

physics.chem-ph↗

arXiv2024

Unifying Graph Convolution and Contrastive Learning in Collaborative Filtering

Graph-based models and contrastive learning have emerged as prominent methods in Collaborative Filtering (CF). While many existing models in CF incorporate these methods in their design, there seems to be a limited depth of analysis regarding the foundational principles behind them. This paper bridges graph convolution, a pivotal element of graph-based models, with contrastive learning through a theoretical framework. By examining the learning dynamics and equilibrium of the contrastive loss, we offer a fresh lens to understand contrastive learning via graph theory, emphasizing its capability to capture high-order connectivity. Building on this analysis, we further show that the graph convolutional layers often used in graph-based models are not essential for high-order connectivity modeling and might contribute to the risk of oversmoothing. Stemming from our findings, we introduce Simple Contrastive Collaborative Filtering (SCCF), a simple and effective algorithm based on a naive embedding model and a modified contrastive loss. The efficacy of the algorithm is demonstrated through extensive experiments across four public datasets. The experiment code is available at \url{https://github.com/wu1hong/SCCF}. \end{abstract}

cs.IR↗

arXiv2024

History-Aware Conversational Dense Retrieval

Conversational search facilitates complex information retrieval by enabling multi-turn interactions between users and the system. Supporting such interactions requires a comprehensive understanding of the conversational inputs to formulate a good search query based on historical information. In particular, the search query should include the relevant information from the previous conversation turns. However, current approaches for conversational dense retrieval primarily rely on fine-tuning a pre-trained ad-hoc retriever using the whole conversational search session, which can be lengthy and noisy. Moreover, existing approaches are limited by the amount of manual supervision signals in the existing datasets. To address the aforementioned issues, we propose a History-Aware Conversational Dense Retrieval (HAConvDR) system, which incorporates two ideas: context-denoised query reformulation and automatic mining of supervision signals based on the actual impact of historical turns. Experiments on two public conversational search datasets demonstrate the improved history modeling capability of HAConvDR, in particular for long conversations with topic shifts.

cs.IR↗

arXiv2024

Attack and Defense Analysis of Learned Image Compression

Learned image compression (LIC) is becoming more and more popular these years with its high efficiency and outstanding compression quality. Still, the practicality against modified inputs added with specific noise could not be ignored. White-box attacks such as FGSM and PGD use only gradient to compute adversarial images that mislead LIC models to output unexpected results. Our experiments compare the effects of different dimensions such as attack methods, models, qualities, and targets, concluding that in the worst case, there is a 61.55% decrease in PSNR or a 19.15 times increase in bpp under the PGD attack. To improve their robustness, we conduct adversarial training by adding adversarial images into the training datasets, which obtains a 95.52% decrease in the R-D cost of the most vulnerable LIC model. We further test the robustness of H.266, whose better performance on reconstruction quality extends its possibility to defend one-step or iterative adversarial attacks.

eess.IV↗

arXiv2024

Accurate Excitation Energies of Point Defects from Fast Particle-Particle Random Approximation Calculations

We present an efficient particle-particle random phase approximation (ppRPA) approach that predicts accurate excitation energies of point defects, including the nitrogen-vacancy (NV$^-$) and the silicon-vacancy (SiV$^0$) centers in diamond and the divacancy center (VV$^0$) in 4H silicon carbide, with errors within 0.2 eV compared with experimental values. Starting from the ($N+2$)-electron ground state calculated with the density functional theory (DFT), the ppRPA excitation energies of the $N$-electron system are calculated as the differences between the two-electron removal energies of the ($N+2$)-electron system. We demonstrate that the ppRPA excitation energies converge rapidly with a few hundred of canonical active-space orbitals. We also show that active-space ppRPA has weak DFT starting-point dependence and is significantly cheaper than the corresponding ground-state DFT calculation. This work establishes ppRPA as an accurate and low-cost tool for investigating excited-state properties of point defects and opens up new opportunities for applications of ppRPA to periodic bulk materials.

physics.chem-ph↗

arXiv2023

Collaboration and Transition: Distilling Item Transitions into Multi-Query Self-Attention for Sequential Recommendation

Modern recommender systems employ various sequential modules such as self-attention to learn dynamic user interests. However, these methods are less effective in capturing collaborative and transitional signals within user interaction sequences. First, the self-attention architecture uses the embedding of a single item as the attention query, making it challenging to capture collaborative signals. Second, these methods typically follow an auto-regressive framework, which is unable to learn global item transition patterns. To overcome these limitations, we propose a new method called Multi-Query Self-Attention with Transition-Aware Embedding Distillation (MQSA-TED). First, we propose an $L$-query self-attention module that employs flexible window sizes for attention queries to capture collaborative signals. In addition, we introduce a multi-query self-attention method that balances the bias-variance trade-off in modeling user preferences by combining long and short-query self-attentions. Second, we develop a transition-aware embedding distillation module that distills global item-to-item transition patterns into item embeddings, which enables the model to memorize and leverage transitional signals and serves as a calibrator for collaborative signals. Experimental results on four real-world datasets demonstrate the effectiveness of the proposed modules.

cs.IR↗

arXiv2023

Machine Learning Many-Body Green's Functions for Molecular Excitation Spectra

We present a machine learning (ML) framework for predicting Green's functions of molecular systems, from which photoemission spectra and quasiparticle energies at quantum many-body level can be obtained. Kernel ridge regression is adopted to predict self-energy matrix elements on compact imaginary frequency grids from static and dynamical mean-field electronic features, which gives direct access to real-frequency many-body Green's functions through analytic continuation and Dyson's equation. Feature and self-energy matrices are represented in a symmetry-adapted intrinsic atomic orbital plus projected atomic orbital basis to enforce rotational invariance. We demonstrate good transferability and high data efficiency of proposed ML method across molecular sizes and chemical species by showing accurate predictions of density of states (DOS) and quasiparticle energies at the level of many-body perturbation theory (GW) or full configuration interaction. For the ML model trained on 48 out of 1995 molecules randomly sampled from the QM7 and QM9 datasets, we report the mean absolute errors of ML-predicted HOMO and LUMO energies to be 0.13 eV and 0.10 eV compared to GW@PBE0. We further showcase the capability of this method by applying the same ML model to predict DOS for significantly larger organic molecules with up to 44 heavy atoms.

physics.chem-ph↗

arXiv2023

Block2: a comprehensive open source framework to develop and apply state-of-the-art DMRG algorithms in electronic structure and beyond

Block2 is an open source framework to implement and perform density matrix renormalization group and matrix product state algorithms. Out-of-the-box it supports the eigenstate, time-dependent, response, and finite-temperature algorithms. In addition, it carries special optimizations for ab initio electronic structure Hamiltonians and implements many quantum chemistry extensions to the density matrix renormalization group, such as dynamical correlation theories. The code is designed with an emphasis on flexibility, extensibility, and efficiency, and to support integration with external numerical packages. Here we explain the design principles and currently supported features and present numerical examples in a range of applications.

physics.chem-ph↗

arXiv2023

Knowledge Combination to Learn Rotated Detection Without Rotated Annotation

Rotated bounding boxes drastically reduce output ambiguity of elongated objects, making it superior to axis-aligned bounding boxes. Despite the effectiveness, rotated detectors are not widely employed. Annotating rotated bounding boxes is such a laborious process that they are not provided in many detection datasets where axis-aligned annotations are used instead. In this paper, we propose a framework that allows the model to predict precise rotated boxes only requiring cheaper axis-aligned annotation of the target dataset 1. To achieve this, we leverage the fact that neural networks are capable of learning richer representation of the target domain than what is utilized by the task. The under-utilized representation can be exploited to address a more detailed task. Our framework combines task knowledge of an out-of-domain source dataset with stronger annotation and domain knowledge of the target dataset with weaker annotation. A novel assignment process and projection loss are used to enable the co-training on the source and target datasets. As a result, the model is able to solve the more detailed task in the target domain, without additional computation overhead during inference. We extensively evaluate the method on various target datasets including fresh-produce dataset, HRSC2016 and SSDD. Results show that the proposed method consistently performs on par with the fully supervised approach.

cs.CV↗

arXiv2023

Efficient Routing Algorithm Design for Large DetNet

Deterministic Networking (DetNet) is a rising technology that offers deterministic delay \& jitter and zero packet loss regardless of failures in large IP networks. In order to support DetNet, we must be able to find a set of low-cost routing paths for a given node pair subject to delay-range constraints. Unfortunately, the \textbf{Delay-Range} Constrained Routing (DRCR) problem is NP-Complete. Existing routing approaches either cannot support the delay-range constraints, or incur extremely high computational complexity. We propose Pulse$+$, a highly scalable and efficient DRCR problem solver. Pulse$+$ adopts a branch-and-bound methodology and optimizes its pruning strategies for higher efficiency. We also integrate Pulse$+$ with a divide-and-conquer approach and propose CoSE-Pulse$+$ to find a pair of active/backup paths that meet DetNet's delay-range and delay-diff constraints. Both Pulse$+$ and CoSE-Pulse$+$ offer optimality guarantee. Notably, although Pulse$+$ and CoSE-Pulse$+$ do not have a polynomial worst-case time complexity, their empirical performance is superior. We evaluate Pulse$+$ and CoSE-Pulse$+$ against the K-Shorst-Path and Lagrangian-dual based algorithms using synthetic test cases generated over networks with thousands of nodes and links. Both Pulse$+$ and CoSE-Pulse$+$ achieve significant speedup. To enable reproduction, we open source our code and test cases at [1].

cs.NI↗

arXiv2022

Gaussian-Based Quasiparticle Self-Consistent $GW$ for Periodic Systems

We present a quasiparticle self-consistent $GW$ (QSGW) implementation for periodic systems based on crystalline Gaussian basis sets. Our QSGW approach is based on a full-frequency analytic continuation GW scheme with Brillouin zone sampling and employs the Gaussian density fitting technique. We benchmark our QSGW implementation on a set of weakly-correlated semiconductors and insulators as well as strongly correlated transition metal oxides including MnO, FeO, CoO, and NiO. Band gap, band structure, and density of states are evaluated using finite size corrected QSGW. We find that although QSGW systematically overestimates band gaps of tested semiconductors and transition metal oxides, it completely removes the dependence on the choice of density functionals and provides more consistent prediction of spectral properties than $G_0W_0$ across a wide range of solids. This work paves the way for utilizing QSGW in ab initio quantum embedding for solids.

physics.chem-ph↗

arXiv2022

Looking Beyond Two Frames: End-to-End Multi-Object Tracking Using Spatial and Temporal Transformers

Tracking a time-varying indefinite number of objects in a video sequence over time remains a challenge despite recent advances in the field. Most existing approaches are not able to properly handle multi-object tracking challenges such as occlusion, in part because they ignore long-term temporal information. To address these shortcomings, we present MO3TR: a truly end-to-end Transformer-based online multi-object tracking (MOT) framework that learns to handle occlusions, track initiation and termination without the need for an explicit data association module or any heuristics. MO3TR encodes object interactions into long-term temporal embeddings using a combination of spatial and temporal Transformers, and recursively uses the information jointly with the input data to estimate the states of all tracked objects over time. The spatial attention mechanism enables our framework to learn implicit representations between all the objects and the objects to the measurements, while the temporal attention mechanism focuses on specific parts of past information, allowing our approach to resolve occlusions over multiple frames. Our experiments demonstrate the potential of this new approach, achieving results on par with or better than the current state-of-the-art on multiple MOT metrics for several popular multi-object tracking benchmarks.

cs.CV↗