arXiv subjects

Zihao Li

Publications and source records attributed to Zihao Li.

At least 199 records · Page 11Linked to original sources

arXiv2024

WorldGPT: A Sora-Inspired Video AI Agent as Rich World Models from Text and Image Inputs

Several text-to-video diffusion models have demonstrated commendable capabilities in synthesizing high-quality video content. However, it remains a formidable challenge pertaining to maintaining temporal consistency and ensuring action smoothness throughout the generated sequences. In this paper, we present an innovative video generation AI agent that harnesses the power of Sora-inspired multimodal learning to build skilled world models framework based on textual prompts and accompanying images. The framework includes two parts: prompt enhancer and full video translation. The first part employs the capabilities of ChatGPT to meticulously distill and proactively construct precise prompts for each subsequent step, thereby guaranteeing the utmost accuracy in prompt communication and accurate execution in following model operations. The second part employ compatible with existing advanced diffusion techniques to expansively generate and refine the key frame at the conclusion of a video. Then we can expertly harness the power of leading and trailing key frames to craft videos with enhanced temporal consistency and action smoothness. The experimental results confirm that our method has strong effectiveness and novelty in constructing world models from text and image inputs over the other methods.

cs.CV↗

arXiv2024

Regularized DeepIV with Model Selection

In this paper, we study nonparametric estimation of instrumental variable (IV) regressions. While recent advancements in machine learning have introduced flexible methods for IV estimation, they often encounter one or more of the following limitations: (1) restricting the IV regression to be uniquely identified; (2) requiring minimax computation oracle, which is highly unstable in practice; (3) absence of model selection procedure. In this paper, we present the first method and analysis that can avoid all three limitations, while still enabling general function approximation. Specifically, we propose a minimax-oracle-free method called Regularized DeepIV (RDIV) regression that can converge to the least-norm IV solution. Our method consists of two stages: first, we learn the conditional distribution of covariates, and by utilizing the learned distribution, we learn the estimator by minimizing a Tikhonov-regularized loss function. We further show that our method allows model selection procedures that can achieve the oracle rates in the misspecified regime. When extended to an iterative estimator, our method matches the current state-of-the-art convergence rate. Our method is a Tikhonov regularized variant of the popular DeepIV method with a non-parametric MLE first-stage estimator, and our results provide the first rigorous guarantees for this empirically used method, showcasing the importance of regularization which was absent from the original work.

cs.LG↗

arXiv2024

Multiview Subspace Clustering of Hyperspectral Images based on Graph Convolutional Networks

High-dimensional and complex spectral structures make clustering of hy-perspectral images (HSI) a challenging task. Subspace clustering has been shown to be an effective approach for addressing this problem. However, current subspace clustering algorithms are mainly designed for a single view and do not fully exploit spatial or texture feature information in HSI. This study proposed a multiview subspace clustering of HSI based on graph convolutional networks. (1) This paper uses the powerful classification ability of graph convolutional network and the learning ability of topologi-cal relationships between nodes to analyze and express the spatial relation-ship of HSI. (2) Pixel texture and pixel neighbor spatial-spectral infor-mation were sent to construct two graph convolutional subspaces. (3) An attention-based fusion module was used to adaptively construct a more discriminative feature map. The model was evaluated on three popular HSI datasets, including Indian Pines, Pavia University, and Houston. It achieved overall accuracies of 92.38%, 93.43%, and 83.82%, respectively and significantly outperformed the state-of-the-art clustering methods. In conclusion, the proposed model can effectively improve the clustering ac-curacy of HSI.

cs.CV↗

arXiv2024

Theoretical Insights for Diffusion Guidance: A Case Study for Gaussian Mixture Models

Diffusion models benefit from instillation of task-specific information into the score function to steer the sample generation towards desired properties. Such information is coined as guidance. For example, in text-to-image synthesis, text input is encoded as guidance to generate semantically aligned images. Proper guidance inputs are closely tied to the performance of diffusion models. A common observation is that strong guidance promotes a tight alignment to the task-specific information, while reducing the diversity of the generated samples. In this paper, we provide the first theoretical study towards understanding the influence of guidance on diffusion models in the context of Gaussian mixture models. Under mild conditions, we prove that incorporating diffusion guidance not only boosts classification confidence but also diminishes distribution diversity, leading to a reduction in the differential entropy of the output distribution. Our analysis covers the widely adopted sampling schemes including DDPM and DDIM, and leverages comparison inequalities for differential equations as well as the Fokker-Planck equation that characterizes the evolution of probability density function, which may be of independent theoretical interest.

cs.LG↗

arXiv2024

Sample Complexity of Preference-Based Nonparametric Off-Policy Evaluation with Deep Networks

A recently popular approach to solving reinforcement learning is with data from human preferences. In fact, human preference data are now used with classic reinforcement learning algorithms such as actor-critic methods, which involve evaluating an intermediate policy over a reward learned from human preference data with distribution shift, known as off-policy evaluation (OPE). Such algorithm includes (i) learning reward function from human preference dataset, and (ii) learning expected cumulative reward of a target policy. Despite the huge empirical success, existing OPE methods with preference data often lack theoretical understanding and rely heavily on heuristics. In this paper, we study the sample efficiency of OPE with human preference and establish a statistical guarantee for it. Specifically, we approach OPE by learning the value function by fitted-Q-evaluation with a deep neural network. By appropriately selecting the size of a ReLU network, we show that one can leverage any low-dimensional manifold structure in the Markov decision process and obtain a sample-efficient estimator without suffering from the curse of high data ambient dimensionality. Under the assumption of high reward smoothness, our results \textit{almost align with the classical OPE results with observable reward data}. To the best of our knowledge, this is the first result that establishes a \textit{provably efficient} guarantee for off-policy evaluation with RLHF.

cs.LG↗

arXiv2024

Double Duality: Variational Primal-Dual Policy Optimization for Constrained Reinforcement Learning

We study the Constrained Convex Markov Decision Process (MDP), where the goal is to minimize a convex functional of the visitation measure, subject to a convex constraint. Designing algorithms for a constrained convex MDP faces several challenges, including (1) handling the large state space, (2) managing the exploration/exploitation tradeoff, and (3) solving the constrained optimization where the objective and the constraint are both nonlinear functions of the visitation measure. In this work, we present a model-based algorithm, Variational Primal-Dual Policy Optimization (VPDPO), in which Lagrangian and Fenchel duality are implemented to reformulate the original constrained problem into an unconstrained primal-dual optimization. Moreover, the primal variables are updated by model-based value iteration following the principle of Optimism in the Face of Uncertainty (OFU), while the dual variables are updated by gradient ascent. Moreover, by embedding the visitation measure into a finite-dimensional space, we can handle large state spaces by incorporating function approximation. Two notable examples are (1) Kernelized Nonlinear Regulators and (2) Low-rank MDPs. We prove that with an optimistic planning oracle, our algorithm achieves sublinear regret and constraint violation in both cases and can attain the globally optimal policy of the original constrained problem.

cs.LG↗

arXiv2024

Are We There Yet? Unraveling the State-of-the-Art Smart Contract Fuzzers

Given the growing importance of smart contracts in various applications, ensuring their security and reliability is critical. Fuzzing, an effective vulnerability detection technique, has recently been widely applied to smart contracts. Despite numerous studies, a systematic investigation of smart contract fuzzing techniques remains lacking. In this paper, we fill this gap by: 1) providing a comprehensive review of current research in contract fuzzing, and 2) conducting an in-depth empirical study to evaluate state-of-the-art contract fuzzers' usability. To guarantee a fair evaluation, we employ a carefully-labeled benchmark and introduce a set of pragmatic performance metrics, evaluating fuzzers from five complementary perspectives. Based on our findings, we provide direction for the future research and development of contract fuzzers.

cs.SE↗

arXiv2024

A SPectroscopic survey of biased halos In the Reionization Era (ASPIRE): Impact of Galaxies on the CGM Metal Enrichment at z > 6 Using the JWST and VLT

We characterize the multiphase circumgalactic medium and galaxy properties at z = 6.0-6.5 in four quasar fields from the James Webb Space Telescope A SPectroscopic survey of biased halos In the Reionization Era (ASPIRE) program. We use the Very Large Telescope/X-shooter spectra of quasar J0305-3150 to identify one new metal absorber at z = 6.2713 with multiple transitions (OI, MgI, FeII and CII). They are combined with the published absorbing systems in Davies et al. (2023a) at the same redshift range to form of a sample of nine metal absorbers at z = 6.03 to 6.49. We identify eight galaxies within 1000 km s$^{-1}$ and 350 kpc around the absorbing gas from the ASPIRE spectroscopic data, with their redshifts secured by [OIII]($λλ$4959, 5007) doublets and H$β$ emission lines. Our spectral energy distribution fitting indicates that the absorbing galaxies have stellar mass ranging from 10$^{7.2}$ to 10$^{8.8}M_{\odot}$ and metallicity between 0.02 and 0.4 solar. Notably, the z = 6.2713 system in the J0305-3150 field resides in a galaxy overdensity region, which contains two (tentatively) merging galaxies within 350 kpc and seven galaxies within 1 Mpc. We measure the relative abundances of $α$ elements to iron ([$α$/Fe]) and find that the CGM gas in the most overdense region exhibits a lower [$α$/Fe] ratio. Our modeling of the galaxy's chemical abundance favors a top-heavy stellar initial mass function, and hints that we may be witnessing the contribution of the first generation Population III stars to the CGM at the end of reionization epoch.

astro-ph.GA↗

arXiv2024

Overview of Sensing Attacks on Autonomous Vehicle Technologies and Impact on Traffic Flow

While perception systems in Connected and Autonomous Vehicles (CAVs), which encompass both communication technologies and advanced sensors, promise to significantly reduce human driving errors, they also expose CAVs to various cyberattacks. These include both communication and sensing attacks, which potentially jeopardize not only individual vehicles but also overall traffic safety and efficiency. While much research has focused on communication attacks, sensing attacks, which are equally critical, have garnered less attention. To address this gap, this study offers a comprehensive review of potential sensing attacks and their impact on target vehicles, focusing on commonly deployed sensors in CAVs such as cameras, LiDAR, Radar, ultrasonic sensors, and GPS. Based on this review, we discuss the feasibility of integrating hardware-in-the-loop experiments with microscopic traffic simulations. We also design baseline scenarios to analyze the macro-level impact of sensing attacks on traffic flow. This study aims to bridge the research gap between individual vehicle sensing attacks and broader macroscopic impacts, thereby laying the foundation for future systemic understanding and mitigation.

eess.SY↗

arXiv2024

Lyapunov Function Consistent Adaptive Network Signal Control with Back Pressure and Reinforcement Learning

In traffic signal control, flow-based (optimizing the overall flow) and pressure-based methods (equalizing and alleviating congestion) are commonly used but often considered separately. This study introduces a unified framework using Lyapunov control theory, defining specific Lyapunov functions respectively for these methods. We have found interesting results. For example, the well-recognized back-pressure method is equal to differential queue lengths weighted by intersection lane saturation flows. We further improve it by adding basic traffic flow theory. Rather than ensuring that the control system be stable, the system should be also capable of adaptive to various performance metrics. Building on insights from Lyapunov theory, this study designs a reward function for the Reinforcement Learning (RL)-based network signal control, whose agent is trained with Double Deep Q-Network (DDQN) for effective control over complex traffic networks. The proposed algorithm is compared with several traditional and RL-based methods under pure passenger car flow and heterogenous traffic flow including freight, respectively. The numerical tests demonstrate that the proposed method outperforms the alternative control methods across different traffic scenarios, covering corridor and general network situations each with varying traffic demands, in terms of the average network vehicle waiting time per vehicle.

eess.SY↗

arXiv2024

A Comprehensive Review of Pre-Darcy Flows in Low-Permeability Porous Media

This paper reviews theories, experimental data, and modeling methods for pre-Darcy flow in low-permeability porous media, where Darcy velocity shows nonlinear dependence on pressure gradients at sufficiently low pressures, a deviation from Darcy's law. It begins by explaining the fundamental mechanisms of pre-Darcy flow, focusing on its unique characteristics like non-linear pressure gradients and fluid-rock interactions. Next, the paper compiles experimental studies on low-permeability geomaterials such as tight sandstones, shales, and clays, detailing methodologies employed, including core sample preparation, permeability measurement techniques, and threshold pressure gradient assessments. The experiments' findings, showing how pore geometry, fluid type, and pressure conditions affect pre-Darcy flow onset, are discussed. The review then covers empirical and theoretical models, plus simulation methods developed for interpreting data on pre-Darcy flow. It concludes by highlighting challenges in conducting and interpreting these experiments, suggesting directions for future research. This comprehensive analysis aims to assist those studying fluid dynamics in low-permeability geomaterials and has implications for applications like shale oil and gas recovery, contaminant transport in low-permeability aquifers, and geological nuclear waste disposal.

physics.geo-ph↗

arXiv2024

A Complete Landscape for the Price of Envy-Freeness

We study the efficiency of fair allocations using the well-studied price of fairness concept, which quantitatively measures the worst-case efficiency loss when imposing fairness constraints. Previous works provided partial results on the price of fairness with well-known fairness notions such as envy-freeness up to one good (EF1) and envy-freeness up to any good (EFX). In this paper, we give a complete characterization for the price of envy-freeness in various settings. In particular, we first consider the two-agent case under the indivisible-goods setting and present tight ratios for the price of EF1 (for scaled utility) and EFX (for unscaled utility), which resolve questions left open in the literature. Next, we consider the mixed goods setting which concerns a mixture of both divisible and indivisible goods. We focus on envy-freeness for mixed goods (EFM), which generalizes both envy-freeness and EF1, as well as its strengthening called envy-freeness up to any good for mixed goods (EFXM), which generalizes envy-freeness and EFX. To this end, we settle the price of EFM and EFXM by providing a complete picture of tight bounds for two agents and asymptotically tight bounds for $n$ agents, for both scaled and unscaled utilities.

cs.GT↗

arXiv2023

Pixel-Superpixel Contrastive Learning and Pseudo-Label Correction for Hyperspectral Image Clustering

Hyperspectral image (HSI) clustering is gaining considerable attention owing to recent methods that overcome the inefficiency and misleading results from the absence of supervised information. Contrastive learning methods excel at existing pixel level and super pixel level HSI clustering tasks. The pixel-level contrastive learning method can effectively improve the ability of the model to capture fine features of HSI but requires a large time overhead. The super pixel-level contrastive learning method utilizes the homogeneity of HSI and reduces computing resources; however, it yields rough classification results. To exploit the strengths of both methods, we present a pixel super pixel contrastive learning and pseudo-label correction (PSCPC) method for the HSI clustering. PSCPC can reasonably capture domain-specific and fine-grained features through super pixels and the comparative learning of a small number of pixels within the super pixels. To improve the clustering performance of super pixels, this paper proposes a pseudo-label correction module that aligns the clustering pseudo-labels of pixels and super-pixels. In addition, pixel-level clustering results are used to supervise super pixel-level clustering, improving the generalization ability of the model. Extensive experiments demonstrate the effectiveness and efficiency of PSCPC.

cs.CV↗

arXiv2023

Contrastive Multi-view Subspace Clustering of Hyperspectral Images based on Graph Convolutional Networks

High-dimensional and complex spectral structures make the clustering of hyperspectral images (HSI) a challenging task. Subspace clustering is an effective approach for addressing this problem. However, current subspace clustering algorithms are primarily designed for a single view and do not fully exploit the spatial or textural feature information in HSI. In this study, contrastive multi-view subspace clustering of HSI was proposed based on graph convolutional networks. Pixel neighbor textural and spatial-spectral information were sent to construct two graph convolutional subspaces to learn their affinity matrices. To maximize the interaction between different views, a contrastive learning algorithm was introduced to promote the consistency of positive samples and assist the model in extracting robust features. An attention-based fusion module was used to adaptively integrate these affinity matrices, constructing a more discriminative affinity matrix. The model was evaluated using four popular HSI datasets: Indian Pines, Pavia University, Houston, and Xu Zhou. It achieved overall accuracies of 97.61%, 96.69%, 87.21%, and 97.65%, respectively, and significantly outperformed state-of-the-art clustering methods. In conclusion, the proposed model effectively improves the clustering accuracy of HSI.

cs.CV↗

arXiv2023

Demystifying DeFi MEV Activities in Flashbots Bundle

Decentralized Finance, mushrooming in permissionless blockchains, has attracted a recent surge in popularity. Due to the transparency of permissionless blockchains, opportunistic traders can compete to earn revenue by extracting Miner Extractable Value (MEV), which undermines both the consensus security and efficiency of blockchain systems. The Flashbots bundle mechanism further aggravates the MEV competition because it empowers opportunistic traders with the capability of designing more sophisticated MEV extraction. In this paper, we conduct the first systematic study on DeFi MEV activities in Flashbots bundle by developing ActLifter, a novel automated tool for accurately identifying DeFi actions in transactions of each bundle, and ActCluster, a new approach that leverages iterative clustering to facilitate us to discover known/unknown DeFi MEV activities. Extensive experimental results show that ActLifter can achieve nearly 100% precision and recall in DeFi action identification, significantly outperforming state-of-the-art techniques. Moreover, with the help of ActCluster, we obtain many new observations and discover 17 new kinds of DeFi MEV activities, which occur in 53.12% of bundles but have not been reported in existing studies

cs.CR↗

arXiv2023

Robust and efficient verification of graph states in blind measurement-based quantum computation

Blind quantum computation (BQC) is a secure quantum computation method that protects the privacy of clients. Measurement-based quantum computation (MBQC) is a promising approach for realizing BQC. To obtain reliable results in blind MBQC, it is crucial to verify whether the resource graph states are accurately prepared in the adversarial scenario. However, previous verification protocols for this task are too resource consuming or noise susceptible to be applied in practice. Here, we propose a robust and efficient protocol for verifying arbitrary graph states with any prime local dimension in the adversarial scenario, which leads to a robust and efficient protocol for verifying the resource state in blind MBQC. Our protocol requires only local Pauli measurements and is thus easy to realize with current technologies. Nevertheless, it can achieve the optimal scaling behaviors with respect to the system size and the target precision as quantified by the infidelity and significance level, which has never been achieved before. Notably, our protocol can exponentially enhance the scaling behavior with the significance level.

quant-ph↗

arXiv2023

UnifiedVisionGPT: Streamlining Vision-Oriented AI through Generalized Multimodal Framework

In the current landscape of artificial intelligence, foundation models serve as the bedrock for advancements in both language and vision domains. OpenAI GPT-4 has emerged as the pinnacle in large language models (LLMs), while the computer vision (CV) domain boasts a plethora of state-of-the-art (SOTA) models such as Meta's SAM and DINO, and YOLOS. However, the financial and computational burdens of training new models from scratch remain a significant barrier to progress. In response to this challenge, we introduce UnifiedVisionGPT, a novel framework designed to consolidate and automate the integration of SOTA vision models, thereby facilitating the development of vision-oriented AI. UnifiedVisionGPT distinguishes itself through four key features: (1) provides a versatile multimodal framework adaptable to a wide range of applications, building upon the strengths of multimodal foundation models; (2) seamlessly integrates various SOTA vision models to create a comprehensive multimodal platform, capitalizing on the best components of each model; (3) prioritizes vision-oriented AI, ensuring a more rapid progression in the CV domain compared to the current trajectory of LLMs; and (4) introduces automation in the selection of SOTA vision models, generating optimal results based on diverse multimodal inputs such as text prompts and images. This paper outlines the architecture and capabilities of UnifiedVisionGPT, demonstrating its potential to revolutionize the field of computer vision through enhanced efficiency, versatility, generalization, and performance. Our implementation, along with the unified multimodal framework and comprehensive dataset, is made publicly available at https://github.com/LHBuilder/SA-Segment-Anything.

cs.CV↗

arXiv2023

A SPectroscopic survey of biased halos In the Reionization Era (ASPIRE): JWST Discovers an Overdensity around a Metal Absorption-selected Galaxy at $z\sim5.5$

The launch of ${\it JWST}$ opens a new window for studying the connection between metal-line absorbers and galaxies at the end of the Epoch of Reionization (EoR). Previous studies have detected absorber-galaxy pairs in limited quantities through ground-based observations. To enhance our understanding of the relationship between absorbers and their host galaxies at $z>5$, we utilized the NIRCam Wide Field Slitless Spectroscopy (WFSS) to search for absorber-associated galaxies by detecting their rest-frame optical emission lines (e.g., [OIII] + H$β$). We report the discovery of a MgII-associated galaxy at $z=5.428$ using data from the ${\it JWST}$ ASPIRE program. The MgII absorber is detected on the spectrum of quasar J0305--3150 with a rest-frame equivalent width of 0.74$\mathring{A}$. The associated galaxy has an [OIII] luminosity of $10^{42.5}\ {\rm erg\ s^{-1}}$ with an impact parameter of 24.9 proper kiloparsecs (pkpc). The joint ${\it HST}$-${\it JWST}$ spectral energy distribution (SED) implies a stellar mass and star-formation rate of ${\rm M_* \approx 10^{8.8}}$ ${\rm M_{\odot}}$, ${\rm SFR}\approx 10\ {\rm M_{\odot}\ yr^{-1}}$. Its [OIII] equivalent width and stellar mass are typical of [OIII] emitters at this redshift. Furthermore, connecting the outflow starting time to the SED-derived stellar age, the outflow velocity of this galaxy is $\sim300\ {\rm km\ s^{-1}}$, consistent with theoretical expectations. We identified six additional [OIII] emitters with impact parameters of up to $\sim300$ pkpc at similar redshifts ($|dv|<1000\ {\rm km\ s^{-1}}$). The observed number is consistent with that in cosmological simulations. This pilot study suggests that systematically investigating the absorber-galaxy connection within the ASPIRE program will provide insights into the metal-enrichment history in the early universe.

astro-ph.GA↗