arXiv subjects

Alex Zhang

Publications and source records attributed to Alex Zhang.

22 records · Page 2Linked to original sources

arXiv2023

Appearance Codes using Joint Embedding Learning of Multiple Modalities

The use of appearance codes in recent work on generative modeling has enabled novel view renders with variable appearance and illumination, such as day-time and night-time renders of a scene. A major limitation of this technique is the need to re-train new appearance codes for every scene on inference, so in this work we address this problem proposing a framework that learns a joint embedding space for the appearance and structure of the scene by enforcing a contrastive loss constraint between different modalities. We apply our framework to a simple Variational Auto-Encoder model on the RADIATE dataset \cite{sheeny2021radiate} and qualitatively demonstrate that we can generate new renders of night-time photos using day-time appearance codes without additional optimization iterations. Additionally, we compare our model to a baseline VAE that uses the standard per-image appearance code technique and show that our approach achieves generations of similar quality without learning appearance codes for any unseen images on inference.

cs.CV↗

arXiv2023

Building Scalable Video Understanding Benchmarks through Sports

Existing benchmarks for evaluating long video understanding falls short on two critical aspects, either lacking in scale or quality of annotations. These limitations arise from the difficulty in collecting dense annotations for long videos, which often require manually labeling each frame. In this work, we introduce an automated Annotation and Video Stream Alignment Pipeline (abbreviated ASAP). We demonstrate the generality of ASAP by aligning unlabeled videos of four different sports with corresponding freely available dense web annotations (i.e. commentary). We then leverage ASAP scalability to create LCric, a large-scale long video understanding benchmark, with over 1000 hours of densely annotated long Cricket videos (with an average sample length of ~50 mins) collected at virtually zero annotation cost. We benchmark and analyze state-of-the-art video understanding models on LCric through a large set of compositional multi-choice and regression queries. We establish a human baseline that indicates significant room for new research to explore. Our human studies indicate that ASAP can align videos and annotations with high fidelity, precision, and speed. The dataset along with the code for ASAP and baselines can be accessed here: https://asap-benchmark.github.io/.

cs.CV↗

arXiv2023

Transaction Fee Mining and Mechanism Design

Transaction fees represent a major incentive in many blockchain systems as a way to incentivize processing transactions. Unfortunately, they also introduce an enormous amount of incentive asymmetry compared to alternatives like fixed block rewards. We analyze some of the incentive compatibility issues that arise from transaction fees, which relate to the bids that users submit, the allocation rules that miners use to choose which transactions to include, and where they choose to mine in the context of longest-chain consensus. We start by surveying a variety of mining attacks including undercutting, fee sniping, and fee-optimized selfish mining. Then, we move to analyzing mechanistic notions of user incentive compatibility, myopic miner incentive compatibility, and off-chain-agreement-proofness, as well as why they are provably incompatible in their full form. Then, we discuss weaker notions of nearly and $γ$-weak incentive compatibility, and how all of these forms of incentive compatibility hold or fail in the trustless auctioneer setup of blockchains, examining classical mechanisms as well as more recent ones such as Ethereum's EIP-1559 mechanism and \cite{chung}'s burning second-price auction. Throughout, we generalize and interrelate existing notions, provide new unifying perspectives and intuitions on analysis, and discuss both specific and overarching open problems for future work.

cs.GT↗

arXiv2021

$FM^2$: Field-matrixed Factorization Machines for Recommender Systems

Click-through rate (CTR) prediction plays a critical role in recommender systems and online advertising. The data used in these applications are multi-field categorical data, where each feature belongs to one field. Field information is proved to be important and there are several works considering fields in their models. In this paper, we proposed a novel approach to model the field information effectively and efficiently. The proposed approach is a direct improvement of FwFM, and is named as Field-matrixed Factorization Machines (FmFM, or $FM^2$). We also proposed a new explanation of FM and FwFM within the FmFM framework, and compared it with the FFM. Besides pruning the cross terms, our model supports field-specific variable dimensions of embedding vectors, which acts as soft pruning. We also proposed an efficient way to minimize the dimension while keeping the model performance. The FmFM model can also be optimized further by caching the intermediate vectors, and it only takes thousands of floating-point operations (FLOPs) to make a prediction. Our experiment results show that it can out-perform the FFM, which is more complex. The FmFM model's performance is also comparable to DNN models which require much more FLOPs in runtime.

cs.IR↗