Searcharxiv⌕ Search

arXiv subjects

Yu Li

Publications and source records attributed to Yu Li.

At least 397 records · Page 22Linked to original sources
arXiv2023

On the Opportunities of Green Computing: A Survey

Artificial Intelligence (AI) has achieved significant advancements in technology and research with the development over several decades, and is widely used in many areas including computing vision, natural language processing, time-series analysis, speech synthesis, etc. During the age of deep learning, especially with the arise of Large Language Models, a large majority of researchers' attention is paid on pursuing new state-of-the-art (SOTA) results, resulting in ever increasing of model size and computational complexity. The needs for high computing power brings higher carbon emission and undermines research fairness by preventing small or medium-sized research institutions and companies with limited funding in participating in research. To tackle the challenges of computing resources and environmental impact of AI, Green Computing has become a hot research topic. In this survey, we give a systematic overview of the technologies used in Green Computing. We propose the framework of Green Computing and devide it into four key components: (1) Measures of Greenness, (2) Energy-Efficient AI, (3) Energy-Efficient Computing Systems and (4) AI Use Cases for Sustainability. For each components, we discuss the research progress made and the commonly used techniques to optimize the AI efficiency. We conclude that this new research direction has the potential to address the conflicts between resource constraints and AI development. We encourage more researchers to put attention on this direction and make AI more environmental friendly.

cs.AI↗
arXiv2023

L-CAD: Language-based Colorization with Any-level Descriptions using Diffusion Priors

Language-based colorization produces plausible and visually pleasing colors under the guidance of user-friendly natural language descriptions. Previous methods implicitly assume that users provide comprehensive color descriptions for most of the objects in the image, which leads to suboptimal performance. In this paper, we propose a unified model to perform language-based colorization with any-level descriptions. We leverage the pretrained cross-modality generative model for its robust language understanding and rich color priors to handle the inherent ambiguity of any-level descriptions. We further design modules to align with input conditions to preserve local spatial structures and prevent the ghosting effect. With the proposed novel sampling strategy, our model achieves instance-aware colorization in diverse and complex scenarios. Extensive experimental results demonstrate our advantages of effectively handling any-level descriptions and outperforming both language-based and automatic colorization methods. The code and pretrained models are available at: https://github.com/changzheng123/L-CAD.

cs.CV↗
arXiv2023

Large-Scale and Multi-Perspective Opinion Summarization with Diverse Review Subsets

Opinion summarization is expected to digest larger review sets and provide summaries from different perspectives. However, most existing solutions are deficient in epitomizing extensive reviews and offering opinion summaries from various angles due to the lack of designs for information selection. To this end, we propose SUBSUMM, a supervised summarization framework for large-scale multi-perspective opinion summarization. SUBSUMM consists of a review sampling strategy set and a two-stage training scheme. The sampling strategies take sentiment orientation and contrastive information value into consideration, with which the review subsets from different perspectives and quality levels can be selected. Subsequently, the summarizer is encouraged to learn from the sub-optimal and optimal subsets successively in order to capitalize on the massive input. Experimental results on AmaSum and Rotten Tomatoes datasets demonstrate that SUBSUMM is adept at generating pros, cons, and verdict summaries from hundreds of input reviews. Furthermore, our in-depth analysis verifies that the advanced selection of review subsets and the two-stage training scheme are vital to boosting the summarization performance.

cs.CL↗
arXiv2023

Consistent123: Improve Consistency for One Image to 3D Object Synthesis

Large image diffusion models enable novel view synthesis with high quality and excellent zero-shot capability. However, such models based on image-to-image translation have no guarantee of view consistency, limiting the performance for downstream tasks like 3D reconstruction and image-to-3D generation. To empower consistency, we propose Consistent123 to synthesize novel views simultaneously by incorporating additional cross-view attention layers and the shared self-attention mechanism. The proposed attention mechanism improves the interaction across all synthesized views, as well as the alignment between the condition view and novel views. In the sampling stage, such architecture supports simultaneously generating an arbitrary number of views while training at a fixed length. We also introduce a progressive classifier-free guidance strategy to achieve the trade-off between texture and geometry for synthesized object views. Qualitative and quantitative experiments show that Consistent123 outperforms baselines in view consistency by a large margin. Furthermore, we demonstrate a significant improvement of Consistent123 on varying downstream tasks, showing its great potential in the 3D generation field. The project page is available at consistent-123.github.io.

cs.CV↗
arXiv2023

Can ChatGPT Replace Traditional KBQA Models? An In-depth Analysis of the Question Answering Performance of the GPT LLM Family

ChatGPT is a powerful large language model (LLM) that covers knowledge resources such as Wikipedia and supports natural language question answering using its own knowledge. Therefore, there is growing interest in exploring whether ChatGPT can replace traditional knowledge-based question answering (KBQA) models. Although there have been some works analyzing the question answering performance of ChatGPT, there is still a lack of large-scale, comprehensive testing of various types of complex questions to analyze the limitations of the model. In this paper, we present a framework that follows the black-box testing specifications of CheckList proposed by Ribeiro et. al. We evaluate ChatGPT and its family of LLMs on eight real-world KB-based complex question answering datasets, which include six English datasets and two multilingual datasets. The total number of test cases is approximately 190,000. In addition to the GPT family of LLMs, we also evaluate the well-known FLAN-T5 to identify commonalities between the GPT family and other LLMs. The dataset and code are available at https://github.com/tan92hl/Complex-Question-Answering-Evaluation-of-GPT-family.git

cs.CL↗
arXiv2023

A Simple Solvable Model for Heavy Fermion Superconductivity from the Two-Fluid Normal State

We propose an exactly solvable momentum-space Kondo-BCS model to study heavy fermion superconductivity. The Kondo interaction is local in momentum space, which can be derived from an Anderson lattice with a Hatsugai-Kohmoto interaction between $f$-electrons. By increasing the Kondo interaction, the model exhibits a crossover from a weak-coupling BCS superconductor to a strong-coupling heavy fermion superconductor featured with a large gap ratio and a large specific heat jump anomaly. Accordingly, the normal state evolves from a Fermi liquid above the BCS superconductor to a non-Fermi liquid two-fluid state above the heavy fermion superconductor. The two-fluid normal state also leads to two types of Cooper pairs, one between conduction electrons, the other between composite fermions formed by conduction electrons and $f$-spins, which is responsible for the strong coupling behaviors of heavy fermion superconductivity.

cond-mat.str-el↗
arXiv2023

Effective Whole-body Pose Estimation with Two-stages Distillation

Whole-body pose estimation localizes the human body, hand, face, and foot keypoints in an image. This task is challenging due to multi-scale body parts, fine-grained localization for low-resolution regions, and data scarcity. Meanwhile, applying a highly efficient and accurate pose estimator to widely human-centric understanding and generation tasks is urgent. In this work, we present a two-stage pose \textbf{D}istillation for \textbf{W}hole-body \textbf{P}ose estimators, named \textbf{DWPose}, to improve their effectiveness and efficiency. The first-stage distillation designs a weight-decay strategy while utilizing a teacher's intermediate feature and final logits with both visible and invisible keypoints to supervise the student from scratch. The second stage distills the student model itself to further improve performance. Different from the previous self-knowledge distillation, this stage finetunes the student's head with only 20% training time as a plug-and-play training strategy. For data limitations, we explore the UBody dataset that contains diverse facial expressions and hand gestures for real-life applications. Comprehensive experiments show the superiority of our proposed simple yet effective methods. We achieve new state-of-the-art performance on COCO-WholeBody, significantly boosting the whole-body AP of RTMPose-l from 64.8% to 66.5%, even surpassing RTMPose-x teacher with 65.3% AP. We release a series of models with different sizes, from tiny to large, for satisfying various downstream tasks. Our codes and models are available at https://github.com/IDEA-Research/DWPose.

cs.CV↗
arXiv2023

ProNet DB: A proteome-wise database for protein surface property representations and RNA-binding profiles

The rapid growth in the number of experimental and predicted protein structures and more complicated protein structures challenge users in computational biology for utilizing the structural information and protein surface property representation. Recently, AlphaFold2 released the comprehensive proteome of various species, and protein surface property representation plays a crucial role in protein-molecule interaction prediction such as protein-protein interaction, protein-nucleic acid interaction, and protein-compound interaction. Here, we proposed the first comprehensive database, namely ProNet DB, which incorporates multiple protein surface representations and RNA-binding landscape for more than 326,175 protein structures covering 16 model organism proteomes from AlphaFold Protein Structure Database (AlphaFold DB) and experimentally validated protein structures deposited in Protein Data Bank (PDB). For each protein, we provided the original protein structure, surface property representation including hydrophobicity, charge distribution, hydrogen bond, interacting face, and RNA-binding landscape such as RNA binding sites and RNA binding preference. To interpret protein surface property representation and RNA binding landscape intuitively, we also integrate Mol* and Online 3D Viewer to visualize the representation on the protein surface. The pre-computed features are available for the users instantaneously and boost computational biology development including molecular mechanism exploration, geometry-based drug discovery and novel therapeutics development. The server is now available on https://proj.cse.cuhk.edu.hk/aihlab/pronet/.

q-bio.QM↗
arXiv2023

MODA: Mapping-Once Audio-driven Portrait Animation with Dual Attentions

Audio-driven portrait animation aims to synthesize portrait videos that are conditioned by given audio. Animating high-fidelity and multimodal video portraits has a variety of applications. Previous methods have attempted to capture different motion modes and generate high-fidelity portrait videos by training different models or sampling signals from given videos. However, lacking correlation learning between lip-sync and other movements (e.g., head pose/eye blinking) usually leads to unnatural results. In this paper, we propose a unified system for multi-person, diverse, and high-fidelity talking portrait generation. Our method contains three stages, i.e., 1) Mapping-Once network with Dual Attentions (MODA) generates talking representation from given audio. In MODA, we design a dual-attention module to encode accurate mouth movements and diverse modalities. 2) Facial composer network generates dense and detailed face landmarks, and 3) temporal-guided renderer syntheses stable videos. Extensive evaluations demonstrate that the proposed system produces more natural and realistic video portraits compared to previous methods.

cs.CV↗
arXiv2023

From Knowledge Distillation to Self-Knowledge Distillation: A Unified Approach with Normalized Loss and Customized Soft Labels

Knowledge Distillation (KD) uses the teacher's prediction logits as soft labels to guide the student, while self-KD does not need a real teacher to require the soft labels. This work unifies the formulations of the two tasks by decomposing and reorganizing the generic KD loss into a Normalized KD (NKD) loss and customized soft labels for both target class (image's category) and non-target classes named Universal Self-Knowledge Distillation (USKD). We decompose the KD loss and find the non-target loss from it forces the student's non-target logits to match the teacher's, but the sum of the two non-target logits is different, preventing them from being identical. NKD normalizes the non-target logits to equalize their sum. It can be generally used for KD and self-KD to better use the soft labels for distillation loss. USKD generates customized soft labels for both target and non-target classes without a teacher. It smooths the target logit of the student as the soft target label and uses the rank of the intermediate feature to generate the soft non-target labels with Zipf's law. For KD with teachers, our NKD achieves state-of-the-art performance on CIFAR-100 and ImageNet datasets, boosting the ImageNet Top-1 accuracy of ResNet18 from 69.90% to 71.96% with a ResNet-34 teacher. For self-KD without teachers, USKD is the first self-KD method that can be effectively applied to both CNN and ViT models with negligible additional time and memory cost, resulting in new state-of-the-art results, such as 1.17% and 0.55% accuracy gains on ImageNet for MobileNet and DeiT-Tiny, respectively. Our codes are available at https://github.com/yzd-v/cls_KD.

cs.CV↗
arXiv2023

Inflationary magnetogenesis of primordial magnetic fields with multiple vector fields

In this paper, we discussed the multiple vector fields during the inflation era and the inflationary magnetogenesis with multiple vector fields. Instead of a single coupling function in single vector field models, the coupling matrix between vector fields and scalar field which drive the inflation is introduced. The dynamical equations for multiple vector fields are obtained and applied to the inflation era. We discussed three cases for the double-field model. In no mutual-coupling case, one can find that both electric and magnetic spectrum can be scale-invariant at the end of inflation, meanwhile, the strong coupling problem can be avoided. The effect of mutual-coupling between different vector fields is also discussed. We found that weak mutual-coupling can lead to the slightly blue spectrum of the magnetic field. On the other hand, in the strong mutual-coupling case, the scale-invariant magnetic spectrum can also be obtained but the energy density of electromagnetic fields either lead to the backreaction problem or is diluted by inflation.

astro-ph.CO↗
arXiv2023

Audio-Driven 3D Facial Animation from In-the-Wild Videos

Given an arbitrary audio clip, audio-driven 3D facial animation aims to generate lifelike lip motions and facial expressions for a 3D head. Existing methods typically rely on training their models using limited public 3D datasets that contain a restricted number of audio-3D scan pairs. Consequently, their generalization capability remains limited. In this paper, we propose a novel method that leverages in-the-wild 2D talking-head videos to train our 3D facial animation model. The abundance of easily accessible 2D talking-head videos equips our model with a robust generalization capability. By combining these videos with existing 3D face reconstruction methods, our model excels in generating consistent and high-fidelity lip synchronization. Additionally, our model proficiently captures the speaking styles of different individuals, allowing it to generate 3D talking-heads with distinct personal styles. Extensive qualitative and quantitative experimental results demonstrate the superiority of our method.

cs.CV↗
arXiv2023

Enhancing the Protein Tertiary Structure Prediction by Multiple Sequence Alignment Generation

The field of protein folding research has been greatly advanced by deep learning methods, with AlphaFold2 (AF2) demonstrating exceptional performance and atomic-level precision. As co-evolution is integral to protein structure prediction, AF2's accuracy is significantly influenced by the depth of multiple sequence alignment (MSA), which requires extensive exploration of a large protein database for similar sequences. However, not all protein sequences possess abundant homologous families, and consequently, AF2's performance can degrade on such queries, at times failing to produce meaningful results. To address this, we introduce a novel generative language model, MSA-Augmenter, which leverages protein-specific attention mechanisms and large-scale MSAs to generate useful, novel protein sequences not currently found in databases. These sequences supplement shallow MSAs, enhancing the accuracy of structural property predictions. Our experiments on CASP14 demonstrate that MSA-Augmenter can generate de novo sequences that retain co-evolutionary information from inferior MSAs, thereby improving protein structure prediction quality on top of strong AF2.

q-bio.QM↗
arXiv2023

Effects of spin orbit coupling on proximity induced superconductivity

We investigate the effect of spin orbit coupling on proximity induced superconductivity in a normal metal attached to a superconductor. Specifically, we consider a heterostructure where the presence of interfaces gives rise to a Rashba spin orbit coupling. The properties of the induced superconductivity in these systems are addressed within the tunneling Hamiltonian formalism. We find that the spin orbit coupling induces a mixture of singlet and triplet pairing and, under specific circumstances, an odd frequency, even parity, spin triplet pairs can arise. We also address the effect of impurity scattering on the induced pairs, and discuss our results in context of heterostructures consisting of materials with spin-momentum locking.

cond-mat.supr-con↗
arXiv2023

DIONYSUS: A Pre-trained Model for Low-Resource Dialogue Summarization

Dialogue summarization has recently garnered significant attention due to its wide range of applications. However, existing methods for summarizing dialogues have limitations because they do not take into account the inherent structure of dialogue and rely heavily on labeled data, which can lead to poor performance in new domains. In this work, we propose DIONYSUS (dynamic input optimization in pre-training for dialogue summarization), a pre-trained encoder-decoder model for summarizing dialogues in any new domain. To pre-train DIONYSUS, we create two pseudo summaries for each dialogue example: one is produced by a fine-tuned summarization model, and the other is a collection of dialogue turns that convey important information. We then choose one of these pseudo summaries based on the difference in information distribution across different types of dialogues. This selected pseudo summary serves as the objective for pre-training DIONYSUS using a self-supervised approach on a large dialogue corpus. Our experiments show that DIONYSUS outperforms existing methods on six datasets, as demonstrated by its ROUGE scores in zero-shot and few-shot settings.

cs.CL↗
arXiv2023

Encoding and multiplexing information signals in magnetic multilayers with fractional skyrmion tubes

Tailored magnetic multilayers (MMLs) provide skyrmions with enhanced thermal stability, leading to the possibility of skyrmion-based devices for room temperature applications. At the same time, the search for additional stable topological spin textures has been under intense research focus. Besides their fundamental importance, such textures may expand the information encoding capability of spintronic devices. However, fractional spin texture states within MMLs in the vertical dimension have yet to be investigated. In this work, we demonstrate numerically fractional skyrmion tubes (FSTs) in a tailored MML system. We subsequently propose to encode sequences of information signals with fractional skyrmion tubes (FSTs) as information bits in a tailored MML device. Micromagnetic simulations and theoretical calculations are used to verify the feasibility of hosting distinct FST states within a single device, and their thermal stability is investigated. A multilayer multiplexing device is proposed, where multiple sequences of the information signals can be encoded and transmitted based on the nucleation and propagation of packets of FSTs. Finally, pipelined information transmission and automatic demultiplexing is demonstrated by exploiting the skyrmion Hall effect and introducing voltage-controlled synchronizers and width-based track selectors. The findings indicate that FSTs can be potential candidates as information carriers for future spintronic applications.

cond-mat.mes-hall↗
arXiv2023

Estimating ensemble likelihoods for the Sentinel-1 based Global Flood Monitoring product of the Copernicus Emergency Management Service

The Global Flood Monitoring (GFM) system of the Copernicus Emergency Management Service (CEMS) addresses the challenges and impacts that are caused by flooding. The GFM system provides global, near-real time flood extent masks for each newly acquired Sentinel-1 Interferometric Wide Swath Synthetic Aperture Radar (SAR) image, as well as flood information from the whole Sentinel-1 archive from 2015 on. The GFM flood extent is an ensemble product based on a combination of three independently developed flood mapping algorithms that individually derive the flood information from Sentinel-1 data. Each flood algorithm also provides classification uncertainty information that is aggregated into the GFM ensemble likelihood product as the mean of the individual classification likelihoods. As the flood detection algorithms derive uncertainty information with different methods, the value range of the three input likelihoods must be harmonized to a range from low [0] to high [100] flood likelihood. The ensemble likelihood is evaluated on two test sites in Myanmar and Somalia, showcasing the performance during an actual flood event and an area with challenging conditions for SAR-based flood detection. The Myanmar use case demonstrates the robustness if flood detections in the ensemble step disagree and how that information is communicated to the end-user. The Somalia use case demonstrates a setting where misclassifications are likely, how the ensemble process mitigates false detections and how the flood likelihoods can be interpreted to use such results with adequate caution.

physics.geo-ph↗