arXiv subjects

Feng Gao

Publications and source records attributed to Feng Gao.

At least 163 records · Page 9Linked to original sources

arXiv2024

PKU-DyMVHumans: A Multi-View Video Benchmark for High-Fidelity Dynamic Human Modeling

High-quality human reconstruction and photo-realistic rendering of a dynamic scene is a long-standing problem in computer vision and graphics. Despite considerable efforts invested in developing various capture systems and reconstruction algorithms, recent advancements still struggle with loose or oversized clothing and overly complex poses. In part, this is due to the challenges of acquiring high-quality human datasets. To facilitate the development of these fields, in this paper, we present PKU-DyMVHumans, a versatile human-centric dataset for high-fidelity reconstruction and rendering of dynamic human scenarios from dense multi-view videos. It comprises 8.2 million frames captured by more than 56 synchronized cameras across diverse scenarios. These sequences comprise 32 human subjects across 45 different scenarios, each with a high-detailed appearance and realistic human motion. Inspired by recent advancements in neural radiance field (NeRF)-based scene representations, we carefully set up an off-the-shelf framework that is easy to provide those state-of-the-art NeRF-based implementations and benchmark on PKU-DyMVHumans dataset. It is paving the way for various applications like fine-grained foreground/background decomposition, high-quality human reconstruction and photo-realistic novel view synthesis of a dynamic scene. Extensive studies are performed on the benchmark, demonstrating new observations and challenges that emerge from using such high-fidelity dynamic data.

cs.CV↗

arXiv2024

Modeling Output-Level Task Relatedness in Multi-Task Learning with Feedback Mechanism

Multi-task learning (MTL) is a paradigm that simultaneously learns multiple tasks by sharing information at different levels, enhancing the performance of each individual task. While previous research has primarily focused on feature-level or parameter-level task relatedness, and proposed various model architectures and learning algorithms to improve learning performance, we aim to explore output-level task relatedness. This approach introduces a posteriori information into the model, considering that different tasks may produce correlated outputs with mutual influences. We achieve this by incorporating a feedback mechanism into MTL models, where the output of one task serves as a hidden feature for another task, thereby transforming a static MTL model into a dynamic one. To ensure the training process converges, we introduce a convergence loss that measures the trend of a task's outputs during each iteration. Additionally, we propose a Gumbel gating mechanism to determine the optimal projection of feedback signals. We validate the effectiveness of our method and evaluate its performance through experiments conducted on several baseline models in spoken language understanding.

cs.LG↗

arXiv2024

Multi-Objective Trajectory Planning with Dual-Encoder

Time-jerk optimal trajectory planning is crucial in advancing robotic arms' performance in dynamic tasks. Traditional methods rely on solving complex nonlinear programming problems, bringing significant delays in generating optimized trajectories. In this paper, we propose a two-stage approach to accelerate time-jerk optimal trajectory planning. Firstly, we introduce a dual-encoder based transformer model to establish a good preliminary trajectory. This trajectory is subsequently refined through sequential quadratic programming to improve its optimality and robustness. Our approach outperforms the state-of-the-art by up to 79.72\% in reducing trajectory planning time. Compared with existing methods, our method shrinks the optimality gap with the objective function value decreasing by up to 29.9\%.

cs.RO↗

arXiv2024

Large-scale Array for Radio Astronomy on the Farside

At the Royal Society meeting in 2023, we have mainly presented our lunar orbit array concept called DSL, and also briefly introduced a concept of a lunar surface array, LARAF. As the DSL concept had been presented before, in this article we introduce the LARAF. We propose to build an array in the far side of the Moon, with a master station which handles the data collection and processing, and 20 stations with maximum baseline of 10 km. Each station consists 12 membrane antenna units, and the stations are connected to the master station by power line and optical fiber. The array will make interferometric observation in the 0.1-50 MHz band during the lunar night, powered by regenerated fuel cells (RFCs). The whole array can be carried to the lunar surface with a heavy rocket mission, and deployed with a rover in 8 months. Such an array would be an important step in the long term development of lunar based ultralong wavelength radio astronomy. It has a sufficiently high sensitivity to observe many radio sources in the sky, though still short of the dark age fluctuations. We discuss the possible options in the power supply, data communication, deployment, etc.

astro-ph.IM↗

arXiv2024

Annotation-Efficient Polyp Segmentation via Active Learning

Deep learning-based techniques have proven effective in polyp segmentation tasks when provided with sufficient pixel-wise labeled data. However, the high cost of manual annotation has created a bottleneck for model generalization. To minimize annotation costs, we propose a deep active learning framework for annotation-efficient polyp segmentation. In practice, we measure the uncertainty of each sample by examining the similarity between features masked by the prediction map of the polyp and the background area. Since the segmentation model tends to perform weak in samples with indistinguishable features of foreground and background areas, uncertainty sampling facilitates the fitting of under-learning data. Furthermore, clustering image-level features weighted by uncertainty identify samples that are both uncertain and representative. To enhance the selectivity of the active selection strategy, we propose a novel unsupervised feature discrepancy learning mechanism. The selection strategy and feature optimization work in tandem to achieve optimal performance with a limited annotation budget. Extensive experimental results have demonstrated that our proposed method achieved state-of-the-art performance compared to other competitors on both a public dataset and a large-scale in-house dataset.

cs.CV↗

arXiv2024

Multifrequency and multimode topological waveguides in a Stampfli-triangle photonic crystal with large valley Chern numbers

The multifrequency quantum valley Hall effect (QVHE) with a large valley Chern number has been realized to significantly improve the transmission capacity of topological waveguides and increase the mode density of topological waveguides. However, multifrequency and multimode QVHEs have not been realized simultaneously. In this work, using tight-binding model calculations and numerical simulations, a valley photonic crystal (VPC) consisting of a Stampfli-triangle photonic crystal is constructed, and its multiple degeneracies in the low-frequency and high-frequency bands split simultaneously to realize the QVHE with multiple topological edge states (TESs). The multifrequency and multimode topological transmission with two low-frequency modes and four high-frequency modes is realized by means of simulations and experiments through a Z-shaped waveguide constructed using two VPCs with opposite valley Chern numbers to prove the realization of a large valley Chern number in the two frequency bands. The two low-frequency modes are successfully distinguished with position-dependent selective excitations, which experimentally demonstrates the occurrence of a large valley Chern number. A frequency-dependent multimode beam splitter is theoretically proposed for high-performance integrated photonic device applications. These results provide new ideas for high-efficiency and high-capacity optical transmission and communication devices and their integration; furthermore, they broaden the application range of TESs.

physics.optics↗

arXiv2024

Vision-Language Navigation with Embodied Intelligence: A Survey

As a long-term vision in the field of artificial intelligence, the core goal of embodied intelligence is to improve the perception, understanding, and interaction capabilities of agents and the environment. Vision-language navigation (VLN), as a critical research path to achieve embodied intelligence, focuses on exploring how agents use natural language to communicate effectively with humans, receive and understand instructions, and ultimately rely on visual information to achieve accurate navigation. VLN integrates artificial intelligence, natural language processing, computer vision, and robotics. This field faces technical challenges but shows potential for application such as human-computer interaction. However, due to the complex process involved from language understanding to action execution, VLN faces the problem of aligning visual information and language instructions, improving generalization ability, and many other challenges. This survey systematically reviews the research progress of VLN and details the research direction of VLN with embodied intelligence. After a detailed summary of its system architecture and research based on methods and commonly used benchmark datasets, we comprehensively analyze the problems and challenges faced by current research and explore the future development direction of this field, aiming to provide a practical reference for researchers.

cs.RO↗

arXiv2024

Hybrid Convolutional and Attention Network for Hyperspectral Image Denoising

Hyperspectral image (HSI) denoising is critical for the effective analysis and interpretation of hyperspectral data. However, simultaneously modeling global and local features is rarely explored to enhance HSI denoising. In this letter, we propose a hybrid convolution and attention network (HCANet), which leverages both the strengths of convolution neural networks (CNNs) and Transformers. To enhance the modeling of both global and local features, we have devised a convolution and attention fusion module aimed at capturing long-range dependencies and neighborhood spectral correlations. Furthermore, to improve multi-scale information aggregation, we design a multi-scale feed-forward network to enhance denoising performance by extracting features at different scales. Experimental results on mainstream HSI datasets demonstrate the rationality and effectiveness of the proposed HCANet. The proposed model is effective in removing various types of complex noise. Our codes are available at \url{https://github.com/summitgao/HCANet}.

eess.IV↗

arXiv2024

In Defense and Revival of Bayesian Filtering for Thermal Infrared Object Tracking

Deep learning-based methods monopolize the latest research in the field of thermal infrared (TIR) object tracking. However, relying solely on deep learning models to obtain better tracking results requires carefully selecting feature information that is beneficial to representing the target object and designing a reasonable template update strategy, which undoubtedly increases the difficulty of model design. Thus, recent TIR tracking methods face many challenges in complex scenarios. This paper introduces a novel Deep Bayesian Filtering (DBF) method to enhance TIR tracking in these challenging situations. DBF is distinctive in its dual-model structure: the system and observation models. The system model leverages motion data to estimate the potential positions of the target object based on two-dimensional Brownian motion, thus generating a prior probability. Following this, the observation model comes into play upon capturing the TIR image. It serves as a classifier and employs infrared information to ascertain the likelihood of these estimated positions, creating a likelihood probability. According to the guidance of the two models, the position of the target object can be determined, and the template can be dynamically updated. Experimental analysis across several benchmark datasets reveals that DBF achieves competitive performance, surpassing most existing TIR tracking methods in complex scenarios.

cs.CV↗

arXiv2024

Medical Knowledge Graph QA for Drug-Drug Interaction Prediction based on Multi-hop Machine Reading Comprehension

Drug-drug interaction prediction is a crucial issue in molecular biology. Traditional methods of observing drug-drug interactions through medical experiments require significant resources and labor. This paper presents a medical knowledge graph question answering model, dubbed MedKGQA, that predicts drug-drug interaction by employing machine reading comprehension from closed-domain literature and constructing a knowledge graph of drug-protein triplets from open-domain documents. The model vectorizes the drug-protein target attributes in the graph using entity embeddings and establishes directed connections between drug and protein entities based on the metabolic interaction pathways of protein targets in the human body. This aligns multiple external knowledge and applies it to learn the graph neural network. Without bells and whistles, the proposed model achieved a 4.5% improvement in terms of drug-drug interaction prediction accuracy compared to previous state-of-the-art models on the Qangaroo MedHop dataset. Experimental results demonstrate the efficiency and effectiveness of the model and verify the feasibility of integrating external knowledge in machine reading comprehension tasks.

cs.CL↗

arXiv2024

Magnetic moments and axial charges of the octet hidden-charm molecular pentaquark family

In this work, we calculate the magnetic moment and axial charge of the octet hidden-charm molecular pentaquark family in quark model. The Coleman-Glashow sum-rule for the magnetic moments of pentaquark family is always fulfilled independently of SU(3) symmetry breaking. In the $ 8_{2f} $ flavor representation, the magnetic moments of hidden-charm molecular pentaquark states with spin configuration $J^{P}=\frac{1}{2}^{-}(\frac{1}{2}^{+}\otimes0^{-})$ are all equal to $μ_{c}=0.38μ_{N}$. The axial charges of octet hidden-charm molecular pentaquark states are quite small compared to the axial charge of nucleon. The axial charges of the pentaquark states with the spin configuration $J^{P}=\frac{1}{2}^{-}(\frac{1}{2}^{+}\otimes0^{-})$ in $8_{2f}$ flavor representation are all zero.

hep-ph↗

arXiv2024

Ranking-based Adaptive Query Generation for DETRs in Crowded Pedestrian Detection

DEtection TRansformer (DETR) and its variants (DETRs) have been successfully applied to crowded pedestrian detection, which achieved promising performance. However, we find that, in different degrees of crowded scenes, the number of DETRs' queries must be adjusted manually, otherwise, the performance would degrade to varying degrees. In this paper, we first analyze the two current query generation methods and summarize four guidelines for designing the adaptive query generation method. Then, we propose Rank-based Adaptive Query Generation (RAQG) to alleviate the problem. Specifically, we design a rank prediction head that can predict the rank of the lowest confidence positive training sample produced by the encoder. Based on the predicted rank, we design an adaptive selection method that can adaptively select coarse detection results produced by the encoder to generate queries. Moreover, to train the rank prediction head better, we propose Soft Gradient L1 Loss. The gradient of Soft Gradient L1 Loss is continuous, which can describe the relationship between the loss value and the updated value of model parameters granularly. Our method is simple and effective, which can be plugged into any DETRs to make it query-adaptive in theory. The experimental results on Crowdhuman dataset and Citypersons dataset show that our method can adaptively generate queries for DETRs and achieve competitive results. Especially, our method achieves state-of-the-art 39.4% MR on Crowdhuman dataset.

cs.CV↗

arXiv2023

Moirai: Towards Optimal Placement for Distributed Inference on Heterogeneous Devices

The escalating size of Deep Neural Networks (DNNs) has spurred a growing research interest in hosting and serving DNN models across multiple devices. A number of studies have been reported to partition a DNN model across devices, providing device placement solutions. The methods appeared in the literature, however, either suffer from poor placement performance due to the exponential search space or miss an optimal placement as a consequence of the reduced search space with limited heuristics. Moreover, these methods have ignored the runtime inter-operator optimization of a computation graph when coarsening the graph, which degrades the end-to-end inference performance. This paper presents Moirai that better exploits runtime inter-operator fusion in a model to render a coarsened computation graph, reducing the search space while maintaining the inter-operator optimization provided by inference backends. Moirai also generalizes the device placement algorithm from multiple perspectives by considering inference constraints and device heterogeneity.Extensive experimental evaluation with 11 large DNNs demonstrates that Moirai outperforms the state-of-the-art counterparts, i.e., Placeto, m-SCT, and GETF, up to 4.28$\times$ in reduction of the end-to-end inference latency. Moirai code is anonymously released at \url{https://github.com/moirai-placement/moirai}.

cs.DC↗

arXiv2023

Second-harmonic generation with a 440,000% W-1 conversion efficiency in a lithium niobate microcavity without periodic poling

Thin-film lithium niobate (TFLN) enables extremely high-efficiency second-order nonlinear optical effects due to large nonlinear coefficient d33 and strong optical field localization. Here, we first designed and fabricated a pulley-waveguide-coupled microring resonator with an intrinsic quality factor above 9.4 x10^5 on the reverse-polarized double-layer X-cut TFLN. In such a TFLN resonator without fine domain structures, second harmonic generation with an absolute (normalized) conversion efficiency of 30% (440,000% W-1), comparable to that in periodically poled lithium niobate (PPLN) microring resonators, was realized with a sub-microwatt continuous pump. This work reduces the dependence of high-efficiency nonlinear frequency conversion on PPLN microcavities that are difficult to prepare.

physics.optics↗

arXiv2023

Planning as In-Painting: A Diffusion-Based Embodied Task Planning Framework for Environments under Uncertainty

Task planning for embodied AI has been one of the most challenging problems where the community does not meet a consensus in terms of formulation. In this paper, we aim to tackle this problem with a unified framework consisting of an end-to-end trainable method and a planning algorithm. Particularly, we propose a task-agnostic method named 'planning as in-painting'. In this method, we use a Denoising Diffusion Model (DDM) for plan generation, conditioned on both language instructions and perceptual inputs under partially observable environments. Partial observation often leads to the model hallucinating the planning. Therefore, our diffusion-based method jointly models both state trajectory and goal estimation to improve the reliability of the generated plan, given the limited available information at each step. To better leverage newly discovered information along the plan execution for a higher success rate, we propose an on-the-fly planning algorithm to collaborate with the diffusion-based planner. The proposed framework achieves promising performances in various embodied AI tasks, including vision-language navigation, object manipulation, and task planning in a photorealistic virtual environment. The code is available at: https://github.com/joeyy5588/planning-as-inpainting.

cs.CV↗

arXiv2023

Networked Time Series Prediction with Incomplete Data via Generative Adversarial Network

A networked time series (NETS) is a family of time series on a given graph, one for each node. It has a wide range of applications from intelligent transportation, environment monitoring to smart grid management. An important task in such applications is to predict the future values of a NETS based on its historical values and the underlying graph. Most existing methods require complete data for training. However, in real-world scenarios, it is not uncommon to have missing data due to sensor malfunction, incomplete sensing coverage, etc. In this paper, we study the problem of NETS prediction with incomplete data. We propose NETS-ImpGAN, a novel deep learning framework that can be trained on incomplete data with missing values in both history and future. Furthermore, we propose Graph Temporal Attention Networks, which incorporate the attention mechanism to capture both inter-time series and temporal correlations. We conduct extensive experiments on four real-world datasets under different missing patterns and missing rates. The experimental results show that NETS-ImpGAN outperforms existing methods, reducing the MAE by up to 25%.

cs.LG↗

arXiv2023

Roadmap on Perovskite Light-Emitting Diodes

In recent years, the field of metal-halide perovskite emitters has rapidly emerged as a new community in solid-state lighting. Their exceptional optoelectronic properties have contributed to the rapid rise in external quantum efficiencies (EQEs) in perovskite light-emitting diodes (PeLEDs) from <1% (in 2014) to approaching 30% (in 2023) across a wide range of wavelengths. However, several challenges still hinder their commercialization, including the relatively low EQEs of blue/white devices, limited EQEs in large-area devices, poor device stability, as well as the toxicity of the easily accessible lead components and the solvents used in the synthesis and processing of PeLEDs. This roadmap addresses the current and future challenges in PeLEDs across fundamental and applied research areas, by sharing the community's perspectives. This work will provide the field with practical guidelines to advance PeLED development and facilitate more rapid commercialization.

physics.optics↗

arXiv2023

Tentative detection of cyanoformamide NCCONH2 in space

The peptide-like molecules, cyanoformamide (NCCONH2), is the cyano (CN) derivative of formamide (NH2CHO). It is known to play a role in the synthesis of nucleic acid precursors under prebiotic conditions. In this paper, we present a tentative detection of NCCONH2 in the interstellar medium (ISM) with the Atacama Large Millimeter/submillimeter Array (ALMA) archive data. Ten unblended lines of NCCONH2 were seen around 3sigma noise levels toward Sagittarius B2(N1E), a position that is slightly offset from the continuum peak. The column density of NCCONH2 was estimated to be 2.4\times 10^15 cm ^-2, and the fractional abundance of NCCONH2 toward Sgr B2(N1E) was 6.9\times10^-10. The abundance ratio between NCCONH2 and NH2CHO is estimated to be ~0.01. We also searched for other peptide-like molecules toward Sgr B2(N1E). The abundances of NH2CHO, CH3NCO and CH3NHCHO toward Sgr B2(N1E) were about one tenth of those toward Sgr B2(N1S), while the abundances of CH3CONH2 was only one twentieth of that toward Sgr B2(N1S).

astro-ph.GA↗