Searcharxiv⌕ Search

arXiv subjects

Zhan Li

Publications and source records attributed to Zhan Li.

At least 55 records · Page 3Linked to original sources
arXiv2024

Membership Inference Attacks against Large Vision-Language Models

Large vision-language models (VLLMs) exhibit promising capabilities for processing multi-modal tasks across various application scenarios. However, their emergence also raises significant data security concerns, given the potential inclusion of sensitive information, such as private photos and medical records, in their training datasets. Detecting inappropriately used data in VLLMs remains a critical and unresolved issue, mainly due to the lack of standardized datasets and suitable methodologies. In this study, we introduce the first membership inference attack (MIA) benchmark tailored for various VLLMs to facilitate training data detection. Then, we propose a novel MIA pipeline specifically designed for token-level image detection. Lastly, we present a new metric called MaxRényi-K%, which is based on the confidence of the model output and applies to both text and image data. We believe that our work can deepen the understanding and methodology of MIAs in the context of VLLMs. Our code and datasets are available at https://github.com/LIONS-EPFL/VL-MIA.

cs.CV↗
arXiv2024

An Animation-based Augmentation Approach for Action Recognition from Discontinuous Video

Action recognition, an essential component of computer vision, plays a pivotal role in multiple applications. Despite significant improvements brought by Convolutional Neural Networks (CNNs), these models suffer performance declines when trained with discontinuous video frames, which is a frequent scenario in real-world settings. This decline primarily results from the loss of temporal continuity, which is crucial for understanding the semantics of human actions. To overcome this issue, we introduce the 4A (Action Animation-based Augmentation Approach) pipeline, which employs a series of sophisticated techniques: starting with 2D human pose estimation from RGB videos, followed by Quaternion-based Graph Convolution Network for joint orientation and trajectory prediction, and Dynamic Skeletal Interpolation for creating smoother, diversified actions using game engine technology. This innovative approach generates realistic animations in varied game environments, viewed from multiple viewpoints. In this way, our method effectively bridges the domain gap between virtual and real-world data. In experimental evaluations, the 4A pipeline achieves comparable or even superior performance to traditional training approaches using real-world data, while requiring only 10% of the original data volume. Additionally, our approach demonstrates enhanced performance on In-the-wild videos, marking a significant advancement in the field of action recognition.

cs.CV↗
arXiv2024

Quater-GCN: Enhancing 3D Human Pose Estimation with Orientation and Semi-supervised Training

3D human pose estimation is a vital task in computer vision, involving the prediction of human joint positions from images or videos to reconstruct a skeleton of a human in three-dimensional space. This technology is pivotal in various fields, including animation, security, human-computer interaction, and automotive safety, where it promotes both technological progress and enhanced human well-being. The advent of deep learning significantly advances the performance of 3D pose estimation by incorporating temporal information for predicting the spatial positions of human joints. However, traditional methods often fall short as they primarily focus on the spatial coordinates of joints and overlook the orientation and rotation of the connecting bones, which are crucial for a comprehensive understanding of human pose in 3D space. To address these limitations, we introduce Quater-GCN (Q-GCN), a directed graph convolutional network tailored to enhance pose estimation by orientation. Q-GCN excels by not only capturing the spatial dependencies among node joints through their coordinates but also integrating the dynamic context of bone rotations in 2D space. This approach enables a more sophisticated representation of human poses by also regressing the orientation of each bone in 3D space, moving beyond mere coordinate prediction. Furthermore, we complement our model with a semi-supervised training strategy that leverages unlabeled data, addressing the challenge of limited orientation ground truth data. Through comprehensive evaluations, Q-GCN has demonstrated outstanding performance against current state-of-the-art methods.

cs.CV↗
arXiv2024

ATDM:An Anthropomorphic Aerial Tendon-driven Manipulator with Low-Inertia and High-Stiffness

Aerial Manipulator Systems (AMS) have garnered significant interest for their utility in aerial operations. Nonetheless, challenges related to the manipulator's limited stiffness and the coupling disturbance with manipulator movement persist. This paper introduces the Aerial Tendon-Driven Manipulator (ATDM), an innovative AMS that integrates a hexrotor Unmanned Aerial Vehicle (UAV) with a 4-degree-of-freedom (4-DOF) anthropomorphic tendon-driven manipulator. The design of the manipulator is anatomically inspired, emulating the human arm anatomy from the shoulder joint downward. To enhance the structural integrity and performance, finite element topology optimization and lattice optimization are employed on the links to replicate the radially graded structure characteristic of bone, this approach effectively reduces weight and inertia while simultaneously maximizing stiffness. A novel tensioning mechanism with adjustable tension is introduced to address cable relaxation, and a Tension-amplification tendon mechanism is implemented to increase the manipulator's overall stiffness and output. The paper presents a kinematic model based on virtual coupled joints, a comprehensive workspace analysis, and detailed calculations of output torques and stiffness for individual arm joints. The prototype arm has a total weight of 2.7 kg, with the end effector contributing only 0.818 kg. By positioning all actuators at the base, coupling disturbance are minimized. The paper includes a detailed mechanical design and validates the system's performance through semi-physical multi-body dynamics simulations, confirming the efficacy of the proposed design.

cs.RO↗
arXiv2024

NTIRE 2024 Quality Assessment of AI-Generated Content Challenge

This paper reports on the NTIRE 2024 Quality Assessment of AI-Generated Content Challenge, which will be held in conjunction with the New Trends in Image Restoration and Enhancement Workshop (NTIRE) at CVPR 2024. This challenge is to address a major challenge in the field of image and video processing, namely, Image Quality Assessment (IQA) and Video Quality Assessment (VQA) for AI-Generated Content (AIGC). The challenge is divided into the image track and the video track. The image track uses the AIGIQA-20K, which contains 20,000 AI-Generated Images (AIGIs) generated by 15 popular generative models. The image track has a total of 318 registered participants. A total of 1,646 submissions are received in the development phase, and 221 submissions are received in the test phase. Finally, 16 participating teams submitted their models and fact sheets. The video track uses the T2VQA-DB, which contains 10,000 AI-Generated Videos (AIGVs) generated by 9 popular Text-to-Video (T2V) models. A total of 196 participants have registered in the video track. A total of 991 submissions are received in the development phase, and 185 submissions are received in the test phase. Finally, 12 participating teams submitted their models and fact sheets. Some methods have achieved better results than baseline methods, and the winning methods in both tracks have demonstrated superior prediction performance on AIGC.

cs.CV↗
arXiv2024

An Invertible All-optical Logic Gate on Chip

We demonstrate an invertible all-optical gate on chip, with the roles of control and signal switchable by slightly adjusting their relative arrival time at the gate. It is based on quantum Zeno blockade driven by sum-frequency generation in a periodic-poled lithium niobate microring resonator. For two nearly-identical nanosecond pulses, the later arriving pulse is modulated by the earlier arriving one, resulting in 2.4 and 3.9 power extinction between the two, respectively, when their peak power is 1 mW and 2 mW. Our results, while to be improved and enriched, herald a new paradigm of logical gates and circuits for exotic applications.

physics.optics↗
arXiv2024

Spacetime Gaussian Feature Splatting for Real-Time Dynamic View Synthesis

Novel view synthesis of dynamic scenes has been an intriguing yet challenging problem. Despite recent advancements, simultaneously achieving high-resolution photorealistic results, real-time rendering, and compact storage remains a formidable task. To address these challenges, we propose Spacetime Gaussian Feature Splatting as a novel dynamic scene representation, composed of three pivotal components. First, we formulate expressive Spacetime Gaussians by enhancing 3D Gaussians with temporal opacity and parametric motion/rotation. This enables Spacetime Gaussians to capture static, dynamic, as well as transient content within a scene. Second, we introduce splatted feature rendering, which replaces spherical harmonics with neural features. These features facilitate the modeling of view- and time-dependent appearance while maintaining small size. Third, we leverage the guidance of training error and coarse depth to sample new Gaussians in areas that are challenging to converge with existing pipelines. Experiments on several established real-world datasets demonstrate that our method achieves state-of-the-art rendering quality and speed, while retaining compact storage. At 8K resolution, our lite-version model can render at 60 FPS on an Nvidia RTX 4090 GPU. Our code is available at https://github.com/oppo-us-research/SpacetimeGaussians.

cs.CV↗
arXiv2024

Parametric All-Optical Modulation on Chip

We demonstrate parametric all-optical modulation in a periodically-poled lithium niobate microring resonator on chip. It employs quantum Zeno blockade between two distinct waves, a signal and a pump, through their sum-frequency generation at a large per-photon efficiency of 8.2 MHz. With nanosecond pump pulses at 6 mW peak power, 85.7% modulation extinction is observed, marking over 30~times efficiency improvement across various previous implementations. With only 2 mW pump peak power, 43.0% modulation extinction is observed for a doubly-stronger signal at 4 mW. This demonstrates, for the first time, that optical transistors with cascadability and fan-out are possible with just parametric nonlinear optics. These results, together with inherent advantages in such photonic integrated circuits, open the door to scalable technology for all-optical and quantum information processing.

physics.optics↗
arXiv2024

High precision proton beam monitor system concept design on CSNS based on SiC

A high precision beam monitor system based on silicon carbide PIN sensor is designed for China Spallation Neutron Source 1.6 GeV proton beam to monitor the proton beam fluence.The concept design of the beam monitor system is finished together with front-end electronics with silicon carbide PIN sensors, readout system and mechanical system.Several tests are performed to study the performance of each component of the system.The charge collection of the SiC PIN sensors after proton radiation is studied with 80 MeV proton beam for continuous running. Research on the performance of the front-end electronics and readout system is finished for better data acquisition.The uncertainty of proton beam fluence is below 1% in the beam monitor system.

physics.acc-ph↗
arXiv2024

Application of Quantum Machine Learning in a Higgs Physics Study at the CEPC

Machine learning has blossomed in recent decades and has become essential in many fields. It significantly solved some problems in particle physics -- particle reconstruction, event classification, etc. However, it is now time to break the limitation of conventional machine learning with quantum computing. A support-vector machine algorithm with a quantum kernel estimator (QSVM-Kernel) leverages high-dimensional quantum state space to identify a signal from backgrounds. In this study, we have pioneered employing this quantum machine learning algorithm to study the $e^{+}e^{-} \rightarrow ZH$ process at the Circular Electron-Positron Collider (CEPC), a proposed Higgs factory to study electroweak symmetry breaking of particle physics. Using 6 qubits on quantum computer simulators, we optimised the QSVM-Kernel algorithm and obtained a classification performance similar to the classical support-vector machine algorithm. Furthermore, we have validated the QSVM-Kernel algorithm using 6-qubits on quantum computer hardware from both IBM and Origin Quantum: the classification performances of both are approaching noiseless quantum computer simulators. In addition, the Origin Quantum hardware results are similar to the IBM Quantum hardware within the uncertainties in our study. Our study shows that state-of-the-art quantum computing technologies could be utilised by particle physics, a branch of fundamental science that relies on big experimental data.

hep-ex↗
arXiv2024

GTAutoAct: An Automatic Datasets Generation Framework Based on Game Engine Redevelopment for Action Recognition

Current datasets for action recognition tasks face limitations stemming from traditional collection and generation methods, including the constrained range of action classes, absence of multi-viewpoint recordings, limited diversity, poor video quality, and labor-intensive manually collection. To address these challenges, we introduce GTAutoAct, a innovative dataset generation framework leveraging game engine technology to facilitate advancements in action recognition. GTAutoAct excels in automatically creating large-scale, well-annotated datasets with extensive action classes and superior video quality. Our framework's distinctive contributions encompass: (1) it innovatively transforms readily available coordinate-based 3D human motion into rotation-orientated representation with enhanced suitability in multiple viewpoints; (2) it employs dynamic segmentation and interpolation of rotation sequences to create smooth and realistic animations of action; (3) it offers extensively customizable animation scenes; (4) it implements an autonomous video capture and processing pipeline, featuring a randomly navigating camera, with auto-trimming and labeling functionalities. Experimental results underscore the framework's robustness and highlights its potential to significantly improve action recognition model training.

cs.CV↗
arXiv2023

A Highly Efficient and Pure Few-Photon Source on Chip

We report on multi-photon statistics of correlated twin beams produced in a periodic poled micro-ring resonator on thin-film lithium niobate. Owing to high cavity confinement and near perfect quasi-phase matching, the photons pairs are produced efficiently in single modes at rates reaching 27 MHz per $μ$W pump power. By using a pump laser whose pulse width impedance matches with the cavity, those photons are further created in single longitudinal modes with purity reaching 99\%, without relying on later-on filtering. With a dual-channel photon-number resolving detection system, we obtain directly the joint detection probabilities of multi-photon states up to three photons, with high coincidence to accidental contrast for each. Used as a single photon source, it gives heralded $g_H^{(2)}(0)$ around 0.04 at a single photon rate of 650 kHz on chip. The findings of our research highlight the potential of this nanophotonic platform as a promising platform for generating non-classical, few-photon states with ideal indistinguishability, for fundamental quantum optics studies and information applications.

quant-ph↗
arXiv2023

Characterising User Transfer Amid Industrial Resource Variation: A Bayesian Nonparametric Approach

In a multitude of industrial fields, a key objective entails optimising resource management whilst satisfying user requirements. Resource management by industrial practitioners can result in a passive transfer of user loads across resource providers, a phenomenon whose accurate characterisation is both challenging and crucial. This research reveals the existence of user clusters, which capture macro-level user transfer patterns amid resource variation. We then propose CLUSTER, an interpretable hierarchical Bayesian nonparametric model capable of automating cluster identification, and thereby predicting user transfer in response to resource variation. Furthermore, CLUSTER facilitates uncertainty quantification for further reliable decision-making. Our method enables privacy protection by functioning independently of personally identifiable information. Experiments with simulated and real-world data from the communications industry reveal a pronounced alignment between prediction results and empirical observations across a spectrum of resource management scenarios. This research establishes a solid groundwork for advancing resource management strategy development.

cs.LG↗
arXiv2023

Globally Composite-Learning-Based Intelligent Fast Finite-Time Control for Uncertain Strict-Feedback Systems with Nonlinearly Periodic Disturbances

This brief aims at the issue of globally composite-learning-based neural fast finite-time (F-FnT) tracking control for a class of uncertain systems in strict-feedback form subject to nonlinearly periodic disturbances. First, uncertain dynamics with periodic parameters are identified by incorporating Fourier series expansion (FSE) into an intelligent estimator, which leverages the feedback of newly designed prediction errors in updating weights to boost learning performance. Then, a novel switching mechanism is constructed to fulfill smooth switching from the composite FSE-based neural controller to robust control law when the inputs of the intelligent estimator transcend the valid approximation domain. By fusing the switching mechanism with an improved F-FnT backstepping algorithm, the globally F-FnT boundedness of all variables in the closed-loop system is guaranteed. Finally, a simulation study is conducted to evince the availability of the theoretical result.

eess.SY↗
arXiv2023

On the relative Morrison-Kawamata cone conjecture (II)

Assuming the Morrison-Kawamata cone conjecture for the generic fiber of a Calabi-Yau fibration and the abundance conjecture, we show (1) the finiteness of minimal models, (2) the existence of a weak rational polyhedral fundamental domain under the action of birational automorphism groups, and (3) the finiteness of varieties as targets of contractions. As an application, the finiteness of minimal models and the weak Morrison-Kawamata cone conjecture in relative dimensions $\leq 2$ are established.

math.AG↗
arXiv2023

Astrometric Reduction of Saturnian Satellites with Cassini-ISS Images Degraded by Trailed Stars

Imaging Science Subsystem (ISS) mounted on the Cassini spacecraft has taken a lot of images, which provides an important source of high-precision astrometry of some planets and satellites. However, some of these images are degraded by trailed stars. Previously, these degraded images cannot be used for astrometry. In this paper, a new method is proposed to detect and compute the centers of these trailed stars automatically. The method is then performed on the astrometry of ISS images with trailed stars. Finally, we provided 658 astrometric positions between 2004 and 2017 of several satellites that include Enceladus, Dione, Tethys, Mimas and Rhea. Compared with the JPL ephemeris SAT427, the mean residuals of these measurements are 0.11 km and 0.26 km in right ascension and declination, respectively. Their standard deviations are 1.08 km and 1.37 km, respectively. The results show that the proposed method performs astrometric measurements of Cassini ISS images with trailed stars effectively.

astro-ph.EP↗
arXiv2022

Adaptive Neural Network Backstepping Control Method for Aerial Manipulator Based on Variable Inertia Parameter Modeling

For the aerial manipulator that performs aerial work tasks, the actual operating environment it faces is very complex, and it is affected by internal and external multi-source disturbances. In this paper, to effectively improve the anti-disturbance control performance of the aerial manipulator, an adaptive neural network backstepping control method based on variable inertia parameter modeling is proposed. Firstly, for the intense internal coupling disturbance, we analyze and model it from the perspective of the generation mechanism of the coupling disturbance, and derive the dynamics model of the aerial manipulator system and the coupling disturbance model based on the variable inertia parameters. Through the proposed coupling disturbance model, we can compensate the strong coupling disturbance in a way of feedforward. Then, the adaptive neural network is proposed and applid to estimate and compensate the additional disturbances, and the closed-loop controller is designed based on the backstepping control method. Finally, we verify the correctness of the proposed coupling disturbance model through physical experiment under a large range motion of the manipulator. Two sets of comparative simulation results also prove the accurate estimation of the proposed adaptive neural network for additional disturbances and the effectiveness and superiority of the proposed control method.

cs.RO↗