arXiv subjects

Ning Zhang

Publications and source records attributed to Ning Zhang.

At least 199 records · Page 11Linked to original sources

arXiv2023

Channel-Aware Distillation Transformer for Depth Estimation on Nano Drones

Autonomous navigation of drones using computer vision has achieved promising performance. Nano-sized drones based on edge computing platforms are lightweight, flexible, and cheap, thus suitable for exploring narrow spaces. However, due to their extremely limited computing power and storage, vision algorithms designed for high-performance GPU platforms cannot be used for nano drones. To address this issue this paper presents a lightweight CNN depth estimation network deployed on nano drones for obstacle avoidance. Inspired by Knowledge Distillation (KD), a Channel-Aware Distillation Transformer (CADiT) is proposed to facilitate the small network to learn knowledge from a larger network. The proposed method is validated on the KITTI dataset and tested on a nano drone Crazyflie, with an ultra-low power microprocessor GAP8.

cs.RO↗

arXiv2023

Lite-Mono: A Lightweight CNN and Transformer Architecture for Self-Supervised Monocular Depth Estimation

Self-supervised monocular depth estimation that does not require ground truth for training has attracted attention in recent years. It is of high interest to design lightweight but effective models so that they can be deployed on edge devices. Many existing architectures benefit from using heavier backbones at the expense of model sizes. This paper achieves comparable results with a lightweight architecture. Specifically, the efficient combination of CNNs and Transformers is investigated, and a hybrid architecture called Lite-Mono is presented. A Consecutive Dilated Convolutions (CDC) module and a Local-Global Features Interaction (LGFI) module are proposed. The former is used to extract rich multi-scale local features, and the latter takes advantage of the self-attention mechanism to encode long-range global information into the features. Experiments demonstrate that Lite-Mono outperforms Monodepth2 by a large margin in accuracy, with about 80% fewer trainable parameters.

cs.CV↗

arXiv2023

Que2Engage: Embedding-based Retrieval for Relevant and Engaging Products at Facebook Marketplace

Embedding-based Retrieval (EBR) in e-commerce search is a powerful search retrieval technique to address semantic matches between search queries and products. However, commercial search engines like Facebook Marketplace Search are complex multi-stage systems optimized for multiple business objectives. At Facebook Marketplace, search retrieval focuses on matching search queries with relevant products, while search ranking puts more emphasis on contextual signals to up-rank the more engaging products. As a result, the end-to-end searcher experience is a function of both relevance and engagement, and the interaction between different stages of the system. This presents challenges to EBR systems in order to optimize for better searcher experiences. In this paper we presents Que2Engage, a search EBR system built towards bridging the gap between retrieval and ranking for end-to-end optimizations. Que2Engage takes a multimodal & multitask approach to infuse contextual information into the retrieval stage and to balance different business objectives. We show the effectiveness of our approach via a multitask evaluation framework and thorough baseline comparisons and ablation studies. Que2Engage is deployed on Facebook Marketplace Search and shows significant improvements in searcher engagement in two weeks of A/B testing.

cs.IR↗

arXiv2023

HSTFormer: Hierarchical Spatial-Temporal Transformers for 3D Human Pose Estimation

Transformer-based approaches have been successfully proposed for 3D human pose estimation (HPE) from 2D pose sequence and achieved state-of-the-art (SOTA) performance. However, current SOTAs have difficulties in modeling spatial-temporal correlations of joints at different levels simultaneously. This is due to the poses' spatial-temporal complexity. Poses move at various speeds temporarily with various joints and body-parts movement spatially. Hence, a cookie-cutter transformer is non-adaptable and can hardly meet the "in-the-wild" requirement. To mitigate this issue, we propose Hierarchical Spatial-Temporal transFormers (HSTFormer) to capture multi-level joints' spatial-temporal correlations from local to global gradually for accurate 3D HPE. HSTFormer consists of four transformer encoders (TEs) and a fusion module. To the best of our knowledge, HSTFormer is the first to study hierarchical TEs with multi-level fusion. Extensive experiments on three datasets (i.e., Human3.6M, MPI-INF-3DHP, and HumanEva) demonstrate that HSTFormer achieves competitive and consistent performance on benchmarks with various scales and difficulties. Specifically, it surpasses recent SOTAs on the challenging MPI-INF-3DHP dataset and small-scale HumanEva dataset, with a highly generalized systematic approach. The code is available at: https://github.com/qianxiaoye825/HSTFormer.

cs.CV↗

arXiv2023

Artificial intelligence for diagnosing and predicting survival of patients with renal cell carcinoma: Retrospective multi-center study

Background: Clear cell renal cell carcinoma (ccRCC) is the most common renal-related tumor with high heterogeneity. There is still an urgent need for novel diagnostic and prognostic biomarkers for ccRCC. Methods: We proposed a weakly-supervised deep learning strategy using conventional histology of 1752 whole slide images from multiple centers. Our study was demonstrated through internal cross-validation and external validations for the deep learning-based models. Results: Automatic diagnosis for ccRCC through intelligent subtyping of renal cell carcinoma was proved in this study. Our graderisk achieved aera the curve (AUC) of 0.840 (95% confidence interval: 0.805-0.871) in the TCGA cohort, 0.840 (0.805-0.871) in the General cohort, and 0.840 (0.805-0.871) in the CPTAC cohort for the recognition of high-grade tumor. The OSrisk for the prediction of 5-year survival status achieved AUC of 0.784 (0.746-0.819) in the TCGA cohort, which was further verified in the independent General cohort and the CPTAC cohort, with AUC of 0.774 (0.723-0.820) and 0.702 (0.632-0.765), respectively. Cox regression analysis indicated that graderisk, OSrisk, tumor grade, and tumor stage were found to be independent prognostic factors, which were further incorporated into the competing-risk nomogram (CRN). Kaplan-Meier survival analyses further illustrated that our CRN could significantly distinguish patients with high survival risk, with hazard ratio of 5.664 (3.893-8.239, p < 0.0001) in the TCGA cohort, 35.740 (5.889-216.900, p < 0.0001) in the General cohort and 6.107 (1.815 to 20.540, p < 0.0001) in the CPTAC cohort. Comparison analyses conformed that our CRN outperformed current prognosis indicators in the prediction of survival status, with higher concordance index for clinical prognosis.

eess.IV↗

arXiv2023

Non-collinear density functional theory

An approach to generalize any kind of collinear functionals in density functional theory to non-collinear functionals is proposed. This approach, for the very first time, satisfies the correct collinear limit for any kind of functionals, guaranteeing that the exact collinear functional after generalized is still exact for collinear spins. Besides, it has well-defined and numerically stable functional derivatives, a desired feature for non-collinear and spin-flip time-dependent density functional theory. Furthermore, it provides local torque, hinting at its applications in spin dynamics.

quant-ph↗

arXiv2023

Performance Analysis and Enhancement of Beamforming Training in 802.11ad

Beamforming (BF) training is crucial to establishing reliable millimeter-wave communication connections between stations (STAs) and an access point. In IEEE 802.11ad BF training protocol, all STAs contend for limited BF training opportunities, i.e., associated BF training (A-BFT) slots, which results in severe collisions and significant BF training latency, especially in dense user scenarios. In this paper, we first develop an analytical model to evaluate the BF training protocol performance. Our analytical model accounts for various protocol components, including user density, the number of A-BFT slots, and protocol parameters, i.e., retry limit and contention window size. We then derive the average successful BF training probability, the BF training efficiency and latency. Since the derived BF training efficiency is an implicit function, to reveal the relationship between system parameters and BF training performance, we also derive an approximate expression of BF training efficiency. Theoretical analysis indicates that the BF training efficiency degrades drastically in dense user scenarios. To address this issue, we propose an enhancement scheme which adaptively adjusts the protocol parameters in tune with user density, to improve the BF training performance in dense user scenarios. Extensive simulations are carried out to validate the accuracy of the developed analytical model. In addition, simulation results show that the proposed enhancement scheme can improve the BF training efficiency by 35% in dense user scenarios.

eess.SY↗

arXiv2023

Beef up mmWave Dense Cellular Networks with D2D-Assisted Cooperative Edge Caching

Edge caching is emerging as the most promising solution to reduce the content retrieval delay and relieve the huge burden on the backhaul links in the ultra-dense networks by proactive caching popular contents in the small base station (SBS). However, constraint cache resource of individual SBSs significantly throttles the performance of edge caching. In this paper, we propose a device-to-device (D2D) assisted cooperative edge caching (DCEC) policy for millimeter (mmWave) dense networks, which cooperatively utilizes the cache resource of users and SBSs in proximity. In the proposed DCEC policy, a content can be cached in either users' devices or SBSs according to the content popularity, and a user can retrieve the requested content from neighboring users via D2D links or the neighboring SBSs via cellular links to efficiently exploit the cache diversity. Unlike existing cooperative caching policies in the lower frequency bands that require complex interference management techniques to suppress interference, we take advantage of directional antenna in mmWave systems to ensure high transmission rate whereas mitigating interference footprint. Taking the practical directional antenna model and the network density into consideration, we derive closed-form expressions of the backhaul offloading performance and content retrieval delay based on the stochastic information of network topology. In addition, analytical results indicate that, with the increase of the network density, the content retrieval delay via D2D links increases significantly while that via cellular links increases slightly. Comprehensive simulations validate our theoretical analysis and demonstrate that the proposed policy can achieve higher performance in offloading the backhaul traffic and reducing the content retrieval delay compared with the state-of-the-art most popular caching (MPC) policy.

eess.SY↗

arXiv2022

Cost-Effective Two-Stage Network Slicing for Edge-Cloud Orchestrated Vehicular Networks

In this paper, we study a network slicing problem for edge-cloud orchestrated vehicular networks, in which the edge and cloud servers are orchestrated to process computation tasks for reducing network slicing cost while satisfying the quality of service requirements. We propose a two-stage network slicing framework, which consists of 1) network planning stage in a large timescale to perform slice deployment, edge resource provisioning, and cloud resource provisioning, and 2) network operation stage in a small timescale to perform resource allocation and task dispatching. Particularly, we formulate the network slicing problem as a two-timescale stochastic optimization problem to minimize the network slicing cost. Since the problem is NP-hard due to coupled network planning and network operation stages, we develop a Two timescAle netWork Slicing (TAWS) algorithm by collaboratively integrating reinforcement learning (RL) and optimization methods, which can jointly make network planning and operation decisions. Specifically, by leveraging the timescale separation property of decisions, we decouple the problem into a large-timescale network planning subproblem and a small-timescale network operation subproblem. The former is solved by an RL method, and the latter is solved by an optimization method. Simulation results based on real-world vehicle traffic traces show that the TAWS can effectively reduce the network slicing cost as compared to the benchmark scheme.

cs.NI↗

arXiv2022

Controllable 3D Face Synthesis with Conditional Generative Occupancy Fields

Capitalizing on the recent advances in image generation models, existing controllable face image synthesis methods are able to generate high-fidelity images with some levels of controllability, e.g., controlling the shapes, expressions, textures, and poses of the generated face images. However, these methods focus on 2D image generative models, which are prone to producing inconsistent face images under large expression and pose changes. In this paper, we propose a new NeRF-based conditional 3D face synthesis framework, which enables 3D controllability over the generated face images by imposing explicit 3D conditions from 3D face priors. At its core is a conditional Generative Occupancy Field (cGOF) that effectively enforces the shape of the generated face to commit to a given 3D Morphable Model (3DMM) mesh. To achieve accurate control over fine-grained 3D face shapes of the synthesized image, we additionally incorporate a 3D landmark loss as well as a volume warping loss into our synthesis algorithm. Experiments validate the effectiveness of the proposed method, which is able to generate high-fidelity face images and shows more precise 3D controllability than state-of-the-art 2D-based controllable face synthesis methods. Find code and demo at https://keqiangsun.github.io/projects/cgof.

cs.CV↗

arXiv2022

UAV Assisted Data Collection for Internet of Things: A Survey

Thanks to the advantages of flexible deployment and high mobility, unmanned aerial vehicles (UAVs) have been widely applied in the areas of disaster management, agricultural plant protection, environment monitoring and so on. With the development of UAV and sensor technologies, UAV assisted data collection for Internet of Things (IoT) has attracted increasing attentions. In this article, the scenarios and key technologies of UAV assisted data collection are comprehensively reviewed. First, we present the system model including the network model and mathematical model of UAV assisted data collection for IoT. Then, we review the key technologies including clustering of sensors, UAV data collection mode as well as joint path planning and resource allocation. Finally, the open problems are discussed from the perspectives of efficient multiple access as well as joint sensing and data collection. This article hopefully provides some guidelines and insights for researchers in the area of UAV assisted data collection for IoT.

cs.RO↗

arXiv2022

Prior-enhanced Temporal Action Localization using Subject-aware Spatial Attention

Temporal action localization (TAL) aims to detect the boundary and identify the class of each action instance in a long untrimmed video. Current approaches treat video frames homogeneously, and tend to give background and key objects excessive attention. This limits their sensitivity to localize action boundaries. To this end, we propose a prior-enhanced temporal action localization method (PETAL), which only takes in RGB input and incorporates action subjects as priors. This proposal leverages action subjects' information with a plug-and-play subject-aware spatial attention module (SA-SAM) to generate an aggregated and subject-prioritized representation. Experimental results on THUMOS-14 and ActivityNet-1.3 datasets demonstrate that the proposed PETAL achieves competitive performance using only RGB features, e.g., boosting mAP by 2.41% or 0.25% over the state-of-the-art approach that uses RGB features or with additional optical flow features on the THUMOS-14 dataset.

cs.CV↗

arXiv2022

FaD-VLP: Fashion Vision-and-Language Pre-training towards Unified Retrieval and Captioning

Multimodal tasks in the fashion domain have significant potential for e-commerce, but involve challenging vision-and-language learning problems - e.g., retrieving a fashion item given a reference image plus text feedback from a user. Prior works on multimodal fashion tasks have either been limited by the data in individual benchmarks, or have leveraged generic vision-and-language pre-training but have not taken advantage of the characteristics of fashion data. Additionally, these works have mainly been restricted to multimodal understanding tasks. To address these gaps, we make two key contributions. First, we propose a novel fashion-specific pre-training framework based on weakly-supervised triplets constructed from fashion image-text pairs. We show the triplet-based tasks are an effective addition to standard multimodal pre-training tasks. Second, we propose a flexible decoder-based model architecture capable of both fashion retrieval and captioning tasks. Together, our model design and pre-training approach are competitive on a diverse set of fashion tasks, including cross-modal retrieval, image retrieval with text feedback, image captioning, relative image captioning, and multimodal categorization.

cs.CV↗

arXiv2022

CFAR based NOMP for Line Spectral Estimation and Detection

The line spectrum estimation problem is considered in this paper. We propose a CFAR-based Newtonized OMP (NOMP-CFAR) method which can maintain a desired false alarm rate without the knowledge of the noise variance. The NOMP-CFAR consists of two steps, namely, an initialization step and a detection step. In the initialization step, NOMP is employed to obtain candidate sinusoidal components. In the detection step, CFAR detector is applied to detect each candidate frequency, and remove the most unlikely frequency component. Then, the Newton refinements are used to refine the remaining parameters. The relationship between the false alarm rate and the required threshold is established. By comparing with the NOMP, NOMP-CFAR has only $1$ dB performance loss in additive white Gaussian noise scenario with false alarm probability $10^{-2}$ and detection probability $0.8$ without knowledge of noise variance. For varied noise variance scenario, NOMP-CFAR still preserves its CFAR property, while NOMP violates the CFAR. Besides, real experiments are also conducted to demonstrate the detection performance of NOMP-CFAR, compared to CFAR and NOMP.

cs.IT↗

arXiv2022

A Survey on Metaverse: Fundamentals, Security, and Privacy

Metaverse, as an evolving paradigm of the next-generation Internet, aims to build a fully immersive, hyper spatiotemporal, and self-sustaining virtual shared space for humans to play, work, and socialize. Driven by recent advances in emerging technologies such as extended reality, artificial intelligence, and blockchain, metaverse is stepping from science fiction to an upcoming reality. However, severe privacy invasions and security breaches (inherited from underlying technologies or emerged in the new digital ecology) of metaverse can impede its wide deployment. At the same time, a series of fundamental challenges (e.g., scalability and interoperability) can arise in metaverse security provisioning owing to the intrinsic characteristics of metaverse, such as immersive realism, hyper spatiotemporality, sustainability, and heterogeneity. In this paper, we present a comprehensive survey of the fundamentals, security, and privacy of metaverse. Specifically, we first investigate a novel distributed metaverse architecture and its key characteristics with ternary-world interactions. Then, we discuss the security and privacy threats, present the critical challenges of metaverse systems, and review the state-of-the-art countermeasures. Finally, we draw open research directions for building future metaverse systems.

cs.CR↗

arXiv2022

Reward Delay Attacks on Deep Reinforcement Learning

Most reinforcement learning algorithms implicitly assume strong synchrony. We present novel attacks targeting Q-learning that exploit a vulnerability entailed by this assumption by delaying the reward signal for a limited time period. We consider two types of attack goals: targeted attacks, which aim to cause a target policy to be learned, and untargeted attacks, which simply aim to induce a policy with a low reward. We evaluate the efficacy of the proposed attacks through a series of experiments. Our first observation is that reward-delay attacks are extremely effective when the goal is simply to minimize reward. Indeed, we find that even naive baseline reward-delay attacks are also highly successful in minimizing the reward. Targeted attacks, on the other hand, are more challenging, although we nevertheless demonstrate that the proposed approaches remain highly effective at achieving the attacker's targets. In addition, we introduce a second threat model that captures a minimal mitigation that ensures that rewards cannot be used out of sequence. We find that this mitigation remains insufficient to ensure robustness to attacks that delay, but preserve the order, of rewards.

cs.LG↗

arXiv2022

Reconfigurable Intelligent Surface With Energy Harvesting Assisted Cooperative Ambient Backscatter Communications

The performance of cooperative ambient backscatter communications (CABC) can be enhanced by employing reconfigurable intelligent surface (RIS) to assist backscatter transmitters. Since the RIS power consumption is a non-negligible issue, we consider a RIS assisted CABC system where the RIS with energy harvesting circuit can not only reflect signal but also harvest wireless energy. We study a transmission design problem to minimize the RIS power consumption with the quality of service constraints for both active and backscatter transmissions. The optimization problem is a mixed-integer non-convex programming problem which is NP-hard. To tackle it, an algorithm is proposed by employing the block coordinate descent, semidefinite relaxation and alternating direction method of multipliers techniques. Simulation results demonstrate the effectiveness of the proposed algorithm.

cs.IT↗

arXiv2022

Accurate and Robust Lesion RECIST Diameter Prediction and Segmentation with Transformers

Automatically measuring lesion/tumor size with RECIST (Response Evaluation Criteria In Solid Tumors) diameters and segmentation is important for computer-aided diagnosis. Although it has been studied in recent years, there is still space to improve its accuracy and robustness, such as (1) enhancing features by incorporating rich contextual information while keeping a high spatial resolution and (2) involving new tasks and losses for joint optimization. To reach this goal, this paper proposes a transformer-based network (MeaFormer, Measurement transFormer) for lesion RECIST diameter prediction and segmentation (LRDPS). It is formulated as three correlative and complementary tasks: lesion segmentation, heatmap prediction, and keypoint regression. To the best of our knowledge, it is the first time to use keypoint regression for RECIST diameter prediction. MeaFormer can enhance high-resolution features by employing transformers to capture their long-range dependencies. Two consistency losses are introduced to explicitly build relationships among these tasks for better optimization. Experiments show that MeaFormer achieves the state-of-the-art performance of LRDPS on the large-scale DeepLesion dataset and produces promising results of two downstream clinic-relevant tasks, i.e., 3D lesion segmentation and RECIST assessment in longitudinal studies.

eess.IV↗