Searcharxiv⌕ Search

arXiv subjects

Ting Chen

Publications and source records attributed to Ting Chen.

At least 91 records · Page 5Linked to original sources
arXiv2023

Multi-dimension Queried and Interacting Network for Stereo Image Deraining

Eliminating the rain degradation in stereo images poses a formidable challenge, which necessitates the efficient exploitation of mutual information present between the dual views. To this end, we devise MQINet, which employs multi-dimension queries and interactions for stereo image deraining. More specifically, our approach incorporates a context-aware dimension-wise queried block (CDQB). This module leverages dimension-wise queries that are independent of the input features and employs global context-aware attention (GCA) to capture essential features while avoiding the entanglement of redundant or irrelevant information. Meanwhile, we introduce an intra-view physics-aware attention (IPA) based on the inverse physical model of rainy images. IPA extracts shallow features that are sensitive to the physics of rain degradation, facilitating the reduction of rain-related artifacts during the early learning period. Furthermore, we integrate a cross-view multi-dimension interacting attention mechanism (CMIA) to foster comprehensive feature interaction between the two views across multiple dimensions. Extensive experimental evaluations demonstrate the superiority of our model over EPRRNet and StereoIRR, achieving respective improvements of 4.18 dB and 0.45 dB in PSNR. Code and models are available at \url{https://github.com/chdwyb/MQINet}.

cs.CV↗
arXiv2023

Nilpotent center conditions in cubic switching polynomial Liénard systems by higher-order analysis

The aim of this paper is to investigate two classical problems related to nilpotent center conditions and bifurcation of limit cycles in switching polynomial systems. Due to the difficulty in calculating the Lyapunov constants of switching polynomial systems at non-elementary singular points, it is extremely difficult to use the existing Poincaré-Lyapunov method to study these two problems. In this paper, we develop a higher-order Poincaré-Lyapunov method to consider the nilpotent center problem in switching polynomial systems, with particular attention focused on cubic switching Liénard systems. With proper perturbations, explicit center conditions are derived for switching Liénard systems at a nilpotent center, which is characterized as global. Moreover, with Bogdanov-Takens bifurcation theory, the existence of five limit cycles around the nilpotent center is proved for a class of switching Liénard systems, which is a new lower bound of cyclicity for such polynomial systems around a nilpotent center.

math.DS↗
arXiv2023

Scalable Adaptive Computation for Iterative Generation

Natural data is redundant yet predominant architectures tile computation uniformly across their input and output space. We propose the Recurrent Interface Networks (RINs), an attention-based architecture that decouples its core computation from the dimensionality of the data, enabling adaptive computation for more scalable generation of high-dimensional data. RINs focus the bulk of computation (i.e. global self-attention) on a set of latent tokens, using cross-attention to read and write (i.e. route) information between latent and data tokens. Stacking RIN blocks allows bottom-up (data to latent) and top-down (latent to data) feedback, leading to deeper and more expressive routing. While this routing introduces challenges, this is less problematic in recurrent computation settings where the task (and routing problem) changes gradually, such as iterative generation with diffusion models. We show how to leverage recurrence by conditioning the latent tokens at each forward pass of the reverse diffusion process with those from prior computation, i.e. latent self-conditioning. RINs yield state-of-the-art pixel diffusion models for image and video generation, scaling to 1024X1024 images without cascades or guidance, while being domain-agnostic and up to 10X more efficient than 2D and 3D U-Nets.

cs.LG↗
arXiv2023

HyperFormer: Learning Expressive Sparse Feature Representations via Hypergraph Transformer

Learning expressive representations for high-dimensional yet sparse features has been a longstanding problem in information retrieval. Though recent deep learning methods can partially solve the problem, they often fail to handle the numerous sparse features, particularly those tail feature values with infrequent occurrences in the training data. Worse still, existing methods cannot explicitly leverage the correlations among different instances to help further improve the representation learning on sparse features since such relational prior knowledge is not provided. To address these challenges, in this paper, we tackle the problem of representation learning on feature-sparse data from a graph learning perspective. Specifically, we propose to model the sparse features of different instances using hypergraphs where each node represents a data instance and each hyperedge denotes a distinct feature value. By passing messages on the constructed hypergraphs based on our Hypergraph Transformer (HyperFormer), the learned feature representations capture not only the correlations among different instances but also the correlations among features. Our experiments demonstrate that the proposed approach can effectively improve feature representation learning on sparse features.

cs.IR↗
arXiv2023

FIT: Far-reaching Interleaved Transformers

We present FIT: a transformer-based architecture with efficient self-attention and adaptive computation. Unlike original transformers, which operate on a single sequence of data tokens, we divide the data tokens into groups, with each group being a shorter sequence of tokens. We employ two types of transformer layers: local layers operate on data tokens within each group, while global layers operate on a smaller set of introduced latent tokens. These layers, comprising the same set of self-attention and feed-forward layers as standard transformers, are interleaved, and cross-attention is used to facilitate information exchange between data and latent tokens within the same group. The attention complexity is $O(n^2)$ locally within each group of size $n$, but can reach $O(L^{{4}/{3}})$ globally for sequence length of $L$. The efficiency can be further enhanced by relying more on global layers that perform adaptive computation using a smaller set of latent tokens. FIT is a versatile architecture and can function as an encoder, diffusion decoder, or autoregressive decoder. We provide initial evidence demonstrating its effectiveness in high-resolution image understanding and generation tasks. Notably, FIT exhibits potential in performing end-to-end training on gigabit-scale data, such as 6400$\times$6400 images, or 160K tokens (after patch tokenization), within a memory capacity of 16GB, without requiring specific optimizations or model parallelism.

cs.LG↗
arXiv2023

On the Importance of Noise Scheduling for Diffusion Models

We empirically study the effect of noise scheduling strategies for denoising diffusion generative models. There are three findings: (1) the noise scheduling is crucial for the performance, and the optimal one depends on the task (e.g., image sizes), (2) when increasing the image size, the optimal noise scheduling shifts towards a noisier one (due to increased redundancy in pixels), and (3) simply scaling the input data by a factor of $b$ while keeping the noise schedule function fixed (equivalent to shifting the logSNR by $\log b$) is a good strategy across image sizes. This simple recipe, when combined with recently proposed Recurrent Interface Network (RIN), yields state-of-the-art pixel-based diffusion models for high-resolution images on ImageNet, enabling single-stage, end-to-end generation of diverse and high-fidelity images at 1024$\times$1024 resolution (without upsampling/cascades).

cs.CV↗
arXiv2023

SigRec: Automatic Recovery of Function Signatures in Smart Contracts

Millions of smart contracts have been deployed onto Ethereum for providing various services, whose functions can be invoked. For this purpose, the caller needs to know the function signature of a callee, which includes its function id and parameter types. Such signatures are critical to many applications focusing on smart contracts, e.g., reverse engineering, fuzzing, attack detection, and profiling. Unfortunately, it is challenging to recover the function signatures from contract bytecode, since neither debug information nor type information is present in the bytecode. To address this issue, prior approaches rely on source code, or a collection of known signatures from incomplete databases or incomplete heuristic rules, which, however, are far from adequate and cannot cope with the rapid growth of new contracts. In this paper, we propose a novel solution that leverages how functions are handled by Ethereum virtual machine (EVM) to automatically recover function signatures. In particular, we exploit how smart contracts determine the functions to be invoked to locate and extract function ids, and propose a new approach named type-aware symbolic execution (TASE) that utilizes the semantics of EVM operations on parameters to identify the number and the types of parameters. Moreover, we develop SigRec, a new tool for recovering function signatures from contract bytecode without the need of source code and function signature databases. The extensive experimental results show that SigRec outperforms all existing tools, achieving an unprecedented 98.7 percent accuracy within 0.074 seconds. We further demonstrate that the recovered function signatures are useful in attack detection, fuzzing and reverse engineering of EVM bytecode.

cs.CR↗
arXiv2023

DeSTSeg: Segmentation Guided Denoising Student-Teacher for Anomaly Detection

Visual anomaly detection, an important problem in computer vision, is usually formulated as a one-class classification and segmentation task. The student-teacher (S-T) framework has proved to be effective in solving this challenge. However, previous works based on S-T only empirically applied constraints on normal data and fused multi-level information. In this study, we propose an improved model called DeSTSeg, which integrates a pre-trained teacher network, a denoising student encoder-decoder, and a segmentation network into one framework. First, to strengthen the constraints on anomalous data, we introduce a denoising procedure that allows the student network to learn more robust representations. From synthetically corrupted normal images, we train the student network to match the teacher network feature of the same images without corruption. Second, to fuse the multi-level S-T features adaptively, we train a segmentation network with rich supervision from synthetic anomaly masks, achieving a substantial performance improvement. Experiments on the industrial inspection benchmark dataset demonstrate that our method achieves state-of-the-art performance, 98.6% on image-level AUC, 75.8% on pixel-level average precision, and 76.4% on instance-level average precision.

cs.CV↗
arXiv2023

Analog Bits: Generating Discrete Data using Diffusion Models with Self-Conditioning

We present Bit Diffusion: a simple and generic approach for generating discrete data with continuous state and continuous time diffusion models. The main idea behind our approach is to first represent the discrete data as binary bits, and then train a continuous diffusion model to model these bits as real numbers which we call analog bits. To generate samples, the model first generates the analog bits, which are then thresholded to obtain the bits that represent the discrete variables. We further propose two simple techniques, namely Self-Conditioning and Asymmetric Time Intervals, which lead to a significant improvement in sample quality. Despite its simplicity, the proposed approach can achieve strong performance in both discrete image generation and image captioning tasks. For discrete image generation, we significantly improve previous state-of-the-art on both CIFAR-10 (which has 3K discrete 8-bit tokens) and ImageNet-64x64 (which has 12K discrete 8-bit tokens), outperforming the best autoregressive model in both sample quality (measured by FID) and efficiency. For image captioning on MS-COCO dataset, our approach achieves competitive results compared to autoregressive models.

cs.CV↗
arXiv2023

Understanding and Improving Robustness of Vision Transformers through Patch-based Negative Augmentation

We investigate the robustness of vision transformers (ViTs) through the lens of their special patch-based architectural structure, i.e., they process an image as a sequence of image patches. We find that ViTs are surprisingly insensitive to patch-based transformations, even when the transformation largely destroys the original semantics and makes the image unrecognizable by humans. This indicates that ViTs heavily use features that survived such transformations but are generally not indicative of the semantic class to humans. Further investigations show that these features are useful but non-robust, as ViTs trained on them can achieve high in-distribution accuracy, but break down under distribution shifts. From this understanding, we ask: can training the model to rely less on these features improve ViT robustness and out-of-distribution performance? We use the images transformed with our patch-based operations as negatively augmented views and offer losses to regularize the training away from using non-robust features. This is a complementary view to existing research that mostly focuses on augmenting inputs with semantic-preserving transformations to enforce models' invariance. We show that patch-based negative augmentation consistently improves robustness of ViTs across a wide set of ImageNet based robustness benchmarks. Furthermore, we find our patch-based negative augmentation are complementary to traditional (positive) data augmentation, and together boost the performance further.

cs.LG↗
arXiv2023

Bubble or Not: Measurements, Analyses, and Findings on the Ethereum ERC721 and ERC1155 Non-fungible Token Ecosystem

The non-fungible token (NFT) is an emergent type of cryptocurrency that has garnered extensive attention since its inception. The uniqueness, indivisibility and humanistic value of NFTs are the key characteristics that distinguish them from traditional tokens. The market capitalization of NFT reached 21.5 billion USD in 2021, almost 200 times of all previous transactions. However, the subsequent rapid decline in NFT market fever in the second quarter of 2022 casts doubts on the ostensible boom in the NFT market. To date, there has been no comprehensive and systematic study of the NFT trade market or of the NFT bubble and hype phenomenon. To fill this gap, we conduct an in-depth investigation of the whole Ethereum ERC721 and ERC1155 NFT ecosystem via graph analysis and apply several metrics to measure the characteristics of NFTs. By collecting data from the whole blockchain, we construct three graphs, namely NFT create graph, NFT transfer graph, and NFT hold graph, to characterize the NFT traders, analyze the characteristics of NFTs, and discover many observations and insights. Moreover, we propose new indicators to quantify the activeness and value of NFT and propose an algorithm that combines indicators and graph analyses to find bubble NFTs. Real-world cases demonstrate that our indicators and approach can be used to discern bubble NFTs effectively.

cs.CY↗
arXiv2023

RiskProp: Account Risk Rating on Ethereum via De-anonymous Score and Network Propagation

As one of the most popular blockchain platforms supporting smart contracts, Ethereum has caught the interest of both investors and criminals. Differently from traditional financial scenarios, executing Know Your Customer verification on Ethereum is rather difficult due to the pseudonymous nature of the blockchain. Fortunately, as the transaction records stored in the Ethereum blockchain are publicly accessible, we can understand the behavior of accounts or detect illicit activities via transaction mining. Existing risk control techniques have primarily been developed from the perspectives of de-anonymizing address clustering and illicit account classification. However, these techniques cannot be used to ascertain the potential risks for all accounts and are limited by specific heuristic strategies or insufficient label information. These constraints motivate us to seek an effective rating method for quantifying the spread of risk in a transaction network. To the best of our knowledge, we are the first to address the problem of account risk rating on Ethereum by proposing a novel model called RiskProp, which includes a de-anonymous score to measure transaction anonymity and a network propagation mechanism to formulate the relationships between accounts and transactions. We demonstrate the effectiveness of RiskProp in overcoming the limitations of existing models by conducting experiments on real-world datasets from Ethereum. Through case studies on the detected high-risk accounts, we demonstrate that the risk assessment by RiskProp can be used to provide warnings for investors and protect them from possible financial losses, and the superior performance of risk score-based account classification experiments further verifies the effectiveness of our rating method.

cs.SI↗
arXiv2022

An automated approach to extracting positive and negative clinical research results

Failure is common in clinical trials since the successful failures presented in negative results always indicate the ways that should not be taken. In this paper, we proposed an automated approach to extracting positive and negative clinical research results by introducing a PICOE (Population, Intervention, Comparation, Outcome, and Effect) framework to represent randomized controlled trials (RCT) reports, where E indicates the effect between a specific I and O. We developed a pipeline to extract and assign the corresponding statistical effect to a specific I-O pair from natural language RCT reports. The extraction models achieved a high degree of accuracy for ICO and E descriptive words extraction through two rounds of training. By defining a threshold of p-value, we find in all Covid-19 related intervention-outcomes pairs with statistical tests, negative results account for nearly 40%. We believe that this observation is noteworthy since they are extracted from the published literature, in which there is an inherent risk of reporting bias, preferring to report positive results rather than negative results. We provided a tool to systematically understand the current level of clinical evidence by distinguishing negative results from the positive results.

cs.IR↗
arXiv2022

AntFuzzer: A Grey-Box Fuzzing Framework for EOSIO Smart Contracts

In the past few years, several attacks against the vulnerabilities of EOSIO smart contracts have caused severe financial losses to this prevalent blockchain platform. As a lightweight test-generation approach, grey-box fuzzing can open up the possibility of improving the security of EOSIO smart contracts. However, developing a practical grey-box fuzzer for EOSIO smart contracts from scratch is time-consuming and requires a deep understanding of EOSIO internals. In this work, we proposed AntFuzzer, the first highly extensible grey-box fuzzing framework for EOSIO smart contracts. AntFuzzer implements a novel approach that interfaces AFL to conduct AFL-style grey-box fuzzing on EOSIO smart contracts. Compared to black-box fuzzing tools, AntFuzzer can effectively trigger those hard-to-cover branches. It achieved an improvement in code coverage on 37.5% of smart contracts in our benchmark dataset. AntFuzzer provides unified interfaces for users to easily develop new detection plugins for continually emerging vulnerabilities. We have implemented 6 detection plugins on AntFuzzer to detect major vulnerabilities of EOSIO smart contracts. In our large-scale fuzzing experiments on 4,616 real-world smart contracts, AntFuzzer successfully detected 741 vulnerabilities. The results demonstrate the effectiveness and efficiency of AntFuzzer and our detection pl

cs.CR↗
arXiv2022

A Unified Sequence Interface for Vision Tasks

While language tasks are naturally expressed in a single, unified, modeling framework, i.e., generating sequences of tokens, this has not been the case in computer vision. As a result, there is a proliferation of distinct architectures and loss functions for different vision tasks. In this work we show that a diverse set of "core" computer vision tasks can also be unified if formulated in terms of a shared pixel-to-sequence interface. We focus on four tasks, namely, object detection, instance segmentation, keypoint detection, and image captioning, all with diverse types of outputs, e.g., bounding boxes or dense masks. Despite that, by formulating the output of each task as a sequence of discrete tokens with a unified interface, we show that one can train a neural network with a single model architecture and loss function on all these tasks, with no task-specific customization. To solve a specific task, we use a short prompt as task description, and the sequence output adapts to the prompt so it can produce task-specific output. We show that such a model can achieve competitive performance compared to well-established task-specific models.

cs.CV↗
arXiv2022

StackVAE-G: An efficient and interpretable model for time series anomaly detection

Recent studies have shown that autoencoder-based models can achieve superior performance on anomaly detection tasks due to their excellent ability to fit complex data in an unsupervised manner. In this work, we propose a novel autoencoder-based model, named StackVAE-G that can significantly bring the efficiency and interpretability to multivariate time series anomaly detection. Specifically, we utilize the similarities across the time series channels by the stacking block-wise reconstruction with a weight-sharing scheme to reduce the size of learned models and also relieve the overfitting to unknown noises in the training data. We also leverage a graph learning module to learn a sparse adjacency matrix to explicitly capture the stable interrelation structure among multiple time series channels for the interpretable pattern reconstruction of interrelated channels. Combining these two modules, we introduce the stacking block-wise VAE (variational autoencoder) with GNN (graph neural network) model for multivariate time series anomaly detection. We conduct extensive experiments on three commonly used public datasets, showing that our model achieves comparable (even better) performance with the state-of-the-art modelsand meanwhile requires much less computation and memory cost. Furthermore, we demonstrate that the adjacency matrix learned by our model accurately captures the interrelation among multiple channels, and can provide valuable information for failure diagnosis applications.

cs.LG↗
arXiv2022

Experimental violation of Leggett-Garg inequality in a three-level trapped-ion system

Leggett-Garg inequality (LGI) studies the temporal correlation in the evolution of physical systems. Classical systems obey the LGI but quantum systems may violate it. The extent of the violation depends on the dimension of the quantum system and the state update rule. In this work, we experimentally test the LGI in a three-level trapped-ion system under the model of a large spin precessing in a magnetic field. The Von Neumann and Lüders state update rules are employed in our system for direct comparative analysis. The maximum observed value of Leggett-Garg correlator under the Von Neumann state update rule is $K_3 = 1.739 \pm 0.014$, which demonstrates a violation of the Lüders bound by 17 standard deviations and is by far the most significant violation in natural three-level systems.

quant-ph↗
arXiv2022

Robust Learning of Deep Time Series Anomaly Detection Models with Contaminated Training Data

Time series anomaly detection (TSAD) is an important data mining task with numerous applications in the IoT era. In recent years, a large number of deep neural network-based methods have been proposed, demonstrating significantly better performance than conventional methods on addressing challenging TSAD problems in a variety of areas. Nevertheless, these deep TSAD methods typically rely on a clean training dataset that is not polluted by anomalies to learn the "normal profile" of the underlying dynamics. This requirement is nontrivial since a clean dataset can hardly be provided in practice. Moreover, without the awareness of their robustness, blindly applying deep TSAD methods with potentially contaminated training data can possibly incur significant performance degradation in the detection phase. In this work, to tackle this important challenge, we firstly investigate the robustness of commonly used deep TSAD methods with contaminated training data which provides a guideline for applying these methods when the provided training data are not guaranteed to be anomaly-free. Furthermore, we propose a model-agnostic method which can effectively improve the robustness of learning mainstream deep TSAD models with potentially contaminated data. Experiment results show that our method can consistently prevent or mitigate performance degradation of mainstream deep TSAD models on widely used benchmark datasets.

cs.LG↗