arXiv subjects

Jinyoung Park

Publications and source records attributed to Jinyoung Park.

At least 55 records · Page 3Linked to original sources

arXiv2023

Sketch-based Video Object Localization

We introduce Sketch-based Video Object Localization (SVOL), a new task aimed at localizing spatio-temporal object boxes in video queried by the input sketch. We first outline the challenges in the SVOL task and build the Sketch-Video Attention Network (SVANet) with the following design principles: (i) to consider temporal information of video and bridge the domain gap between sketch and video; (ii) to accurately identify and localize multiple objects simultaneously; (iii) to handle various styles of sketches; (iv) to be classification-free. In particular, SVANet is equipped with a Cross-modal Transformer that models the interaction between learnable object tokens, query sketch, and video through attention operations, and learns upon a per-frame set matching strategy that enables frame-wise prediction while utilizing global video context. We evaluate SVANet on a newly curated SVOL dataset. By design, SVANet successfully learns the mapping between the query sketches and video objects, achieving state-of-the-art results on the SVOL benchmark. We further confirm the effectiveness of SVANet via extensive ablation studies and visualizations. Lastly, we demonstrate its transfer capability on unseen datasets and novel categories, suggesting its high scalability in real-world applications.

cs.CV↗

arXiv2023

Threshold phenomena for random discrete structures

In this expository article, we give a gentle introduction to the Erdős-Rényi random graphs and threshold phenomena that they exhibit. We also mildly introduce the Kahn-Kalai Conjecture with several intuitive examples, mainly targeting the general audience.

math.HO↗

arXiv2023

The number of colorings of the middle layers of the Hamming cube

For an odd integer $n = 2d-1$, let $\mathcal B_d$ be the subgraph of the hypercube $Q_n$ induced by the two largest layers. In this paper, we describe the typical structure of proper $q$-colorings of $V(\mathcal B_d)$ and give asymptotics on the number of them. The proofs use various tools including information theory (entropy), Sapozhenko's graph container method and a recently developed method of M. Jenssen and W. Perkins that combines Sapozhenko's graph container lemma with the cluster expansion for polymer models from statistical physics.

math.CO↗

arXiv2023

Note on the number of antichains in generalizations of the Boolean lattice

We give a short and self-contained argument that shows that, for any positive integers $t$ and $n$ with $t =O\Bigl(\frac{n}{\log n}\Bigr)$, the number $α([t]^n)$ of antichains of the poset $[t]^n$ is at most \[\exp_2\Bigl(1+O\Bigl(\Bigl(\frac{t\log^3 n}{n}\Bigr)^{1/2}\Bigr)\Bigr)N(t,n)\,,\] where $N(t,n)$ is the size of a largest level of $[t]^n$. This, in particular, says that if $t \ll n/\log^3 n$ as $n \rightarrow \infty$, then $\logα([t]^n)=(1+o(1))N(t,n)$, giving a (partially) positive answer to a question of Moshkovitz and Shapira for $t, n$ in this range. Particularly for $t=3$, we prove a better upper bound: \[\logα([3]^n)\le(1+4\log 3/n)N(3,n),\] which is the best known upper bound on the number of antichains of $[3]^n$.

math.CO↗

arXiv2023

Relation-Aware Language-Graph Transformer for Question Answering

Question Answering (QA) is a task that entails reasoning over natural language contexts, and many relevant works augment language models (LMs) with graph neural networks (GNNs) to encode the Knowledge Graph (KG) information. However, most existing GNN-based modules for QA do not take advantage of rich relational information of KGs and depend on limited information interaction between the LM and the KG. To address these issues, we propose Question Answering Transformer (QAT), which is designed to jointly reason over language and graphs with respect to entity relations in a unified manner. Specifically, QAT constructs Meta-Path tokens, which learn relation-centric embeddings based on diverse structural and semantic relations. Then, our Relation-Aware Self-Attention module comprehensively integrates different modalities via the Cross-Modal Relative Position Bias, which guides information exchange between relevant entites of different modalities. We validate the effectiveness of QAT on commonsense question answering datasets like CommonsenseQA and OpenBookQA, and on a medical question answering dataset, MedQA-USMLE. On all the datasets, our method achieves state-of-the-art performance. Our code is available at http://github.com/mlvlab/QAT.

cs.CL↗

arXiv2023

A Proof of the Kahn-Kalai Conjecture

Proving the ``expectation-threshold'' conjecture of Kahn and Kalai, we show that for any increasing property $\mathcal{F}$ on a finite set $X$, $$p_c(\mathcal{F})=O(q(\mathcal{F})\log \ell(\mathcal{F})),$$ where $p_c(\mathcal{F})$ and $q(\mathcal{F})$ are the threshold and ``expectation threshold'' of $\mathcal{F}$, and $\ell(\mathcal{F})$ is the maximum of $2$ and the maximum size of a minimal member of $\mathcal{F}$.

math.CO↗

arXiv2023

Self-positioning Point-based Transformer for Point Cloud Understanding

Transformers have shown superior performance on various computer vision tasks with their capabilities to capture long-range dependencies. Despite the success, it is challenging to directly apply Transformers on point clouds due to their quadratic cost in the number of points. In this paper, we present a Self-Positioning point-based Transformer (SPoTr), which is designed to capture both local and global shape contexts with reduced complexity. Specifically, this architecture consists of local self-attention and self-positioning point-based global cross-attention. The self-positioning points, adaptively located based on the input shape, consider both spatial and semantic information with disentangled attention to improve expressive power. With the self-positioning points, we propose a novel global cross-attention mechanism for point clouds, which improves the scalability of global self-attention by allowing the attention module to compute attention weights with only a small set of self-positioning points. Experiments show the effectiveness of SPoTr on three point cloud tasks such as shape classification, part segmentation, and scene segmentation. In particular, our proposed model achieves an accuracy gain of 2.6% over the previous best models on shape classification with ScanObjectNN. We also provide qualitative analyses to demonstrate the interpretability of self-positioning points. The code of SPoTr is available at https://github.com/mlvlab/SPoTr.

cs.CV↗

arXiv2022

RainUNet for Super-Resolution Rain Movie Prediction under Spatio-temporal Shifts

This paper presents a solution to the Weather4cast 2022 Challenge Stage 2. The goal of the challenge is to forecast future high-resolution rainfall events obtained from ground radar using low-resolution multiband satellite images. We suggest a solution that performs data preprocessing appropriate to the challenge and then predicts rainfall movies using a novel RainUNet. RainUNet is a hierarchical U-shaped network with temporal-wise separable block (TS block) using a decoupled large kernel 3D convolution to improve the prediction performance. Various evaluation metrics show that our solution is effective compared to the baseline method. The source codes are available at https://github.com/jinyxp/Weather4cast-2022

cs.CV↗

arXiv2022

OCR-free Document Understanding Transformer

Understanding document images (e.g., invoices) is a core but challenging task since it requires complex functions such as reading text and a holistic understanding of the document. Current Visual Document Understanding (VDU) methods outsource the task of reading text to off-the-shelf Optical Character Recognition (OCR) engines and focus on the understanding task with the OCR outputs. Although such OCR-based approaches have shown promising performance, they suffer from 1) high computational costs for using OCR; 2) inflexibility of OCR models on languages or types of document; 3) OCR error propagation to the subsequent process. To address these issues, in this paper, we introduce a novel OCR-free VDU model named Donut, which stands for Document understanding transformer. As the first step in OCR-free VDU research, we propose a simple architecture (i.e., Transformer) with a pre-training objective (i.e., cross-entropy loss). Donut is conceptually simple yet effective. Through extensive experiments and analyses, we show a simple OCR-free VDU model, Donut, achieves state-of-the-art performances on various VDU tasks in terms of both speed and accuracy. In addition, we offer a synthetic data generator that helps the model pre-training to be flexible in various languages and domains. The code, trained model and synthetic data are available at https://github.com/clovaai/donut.

cs.LG↗

arXiv2022

Deformable Graph Transformer

Transformer-based models have recently shown success in representation learning on graph-structured data beyond natural language processing and computer vision. However, the success is limited to small-scale graphs due to the drawbacks of full dot-product attention on graphs such as the quadratic complexity with respect to the number of nodes and message aggregation from enormous irrelevant nodes. To address these issues, we propose Deformable Graph Transformer (DGT) that performs sparse attention via dynamically sampled relevant nodes for efficiently handling large-scale graphs with a linear complexity in the number of nodes. Specifically, our framework first constructs multiple node sequences with various criteria to consider both structural and semantic proximity. Then, combining with our learnable Katz Positional Encodings, the sparse attention is applied to the node sequences for learning node representations with a significantly reduced computational cost. Extensive experiments demonstrate that our DGT achieves state-of-the-art performance on 7 graph benchmark datasets with 2.5 - 449 times less computational cost compared to transformer-based graph models with full attention.

cs.LG↗

arXiv2022

Wide Effective Work-Function Tuning of Al/SiO$_2$/Si Junction Achieved with Graphene Interlayer at Al/SiO$_2$ Interface

The effective work-function of metal electrode is one of the major factors to determine the threshold voltage of metal/oxide/semiconductor junction. In this work, we demonstrate experimentally that the effective work-function of Aluminum (Al) electrode in Al/SiO$_2$/n-Si junction increases significantly by $\sim$1.04 eV with the graphene interlayer inserted at Al/SiO$_2$ interface. We also provide the device-physical analysis of solving Poisson equation when the flat-band voltage is applied to the junction, supporting that the wide tuning of Al effective work-function originates from the electrical dipole layer formed by the overlap of electron orbitals between Al and graphene layer. Our work suggests the feasibility of constructing the dual-metal gate CMOS circuitry just by using Al electrodes with area-specific underlying graphene interlayer.

cond-mat.mes-hall↗

arXiv2022

Explore-And-Match: Bridging Proposal-Based and Proposal-Free With Transformer for Sentence Grounding in Videos

Natural Language Video Grounding (NLVG) aims to localize time segments in an untrimmed video according to sentence queries. In this work, we present a new paradigm named Explore-And-Match for NLVG that seamlessly unifies the strengths of two streams of NLVG methods: proposal-free and proposal-based; the former explores the search space to find time segments directly, and the latter matches the predefined time segments with ground truths. To achieve this, we formulate NLVG as a set prediction problem and design an end-to-end trainable Language Video Transformer (LVTR) that can enjoy two favorable properties, which are rich contextualization power and parallel decoding. We train LVTR with two losses. First, temporal localization loss allows time segments of all queries to regress targets (explore). Second, set guidance loss couples every query with their respective target (match). To our surprise, we found that training schedule shows divide-and-conquer-like pattern: time segments are first diversified regardless of the target, then coupled with each target, and fine-tuned to the target again. Moreover, LVTR is highly efficient and effective: it infers faster than previous baselines (by 2X or more) and sets competitive results on two NLVG benchmarks (ActivityCaptions and Charades-STA). Codes are available at https://github.com/sangminwoo/Explore-And-Match.

cs.CV↗

arXiv2022

Metropolis-Hastings Data Augmentation for Graph Neural Networks

Graph Neural Networks (GNNs) often suffer from weak-generalization due to sparsely labeled data despite their promising results on various graph-based tasks. Data augmentation is a prevalent remedy to improve the generalization ability of models in many domains. However, due to the non-Euclidean nature of data space and the dependencies between samples, designing effective augmentation on graphs is challenging. In this paper, we propose a novel framework Metropolis-Hastings Data Augmentation (MH-Aug) that draws augmented graphs from an explicit target distribution for semi-supervised learning. MH-Aug produces a sequence of augmented graphs from the target distribution enables flexible control of the strength and diversity of augmentation. Since the direct sampling from the complex target distribution is challenging, we adopt the Metropolis-Hastings algorithm to obtain the augmented samples. We also propose a simple and effective semi-supervised learning strategy with generated samples from MH-Aug. Our extensive experiments demonstrate that MH-Aug can generate a sequence of samples according to the target distribution to significantly improve the performance of GNNs.

cs.LG↗

arXiv2021

Deformable Graph Convolutional Networks

Graph neural networks (GNNs) have significantly improved the representation power for graph-structured data. Despite of the recent success of GNNs, the graph convolution in most GNNs have two limitations. Since the graph convolution is performed in a small local neighborhood on the input graph, it is inherently incapable to capture long-range dependencies between distance nodes. In addition, when a node has neighbors that belong to different classes, i.e., heterophily, the aggregated messages from them often negatively affect representation learning. To address the two common problems of graph convolution, in this paper, we propose Deformable Graph Convolutional Networks (Deformable GCNs) that adaptively perform convolution in multiple latent spaces and capture short/long-range dependencies between nodes. Separated from node representations (features), our framework simultaneously learns the node positional embeddings (coordinates) to determine the relations between nodes in an end-to-end fashion. Depending on node position, the convolution kernels are deformed by deformation vectors and apply different transformations to its neighbor nodes. Our extensive experiments demonstrate that Deformable GCNs flexibly handles the heterophily and achieve the best performance in node classification tasks on six heterophilic graph datasets.

cs.LG↗

arXiv2021

On a problem of M. Talagrand

We address a special case of a conjecture of M. Talagrand relating two notions of "threshold" for an increasing family $\mathcal F$ of subsets of a finite set $V$. The full conjecture implies equivalence of the "Fractional Expectation-Threshold Conjecture," due to Talagrand and recently proved by the authors and B. Narayanan, and the (stronger) "Expectation-Threshold Conjecture" of the second author and G. Kalai. The conjecture under discussion here says there is a fixed $L$ such that if, for a given $\mathcal F$, $p\in [0,1]$ admits $λ:2^V \rightarrow \mathbb R^+$ with \[ \mbox{$\sum_{S\subseteq F}λ_S\ge 1 ~~\forall F\in \mathcal F$} \] and \[ \mbox{$\sum_Sλ_Sp^{|S|} \le 1/2$} \] (a.k.a. $\mathcal F$ is weakly $p$-small), then $p/L$ admits such a $λ$ taking values in $\{0,1\}$ ($\mathcal F$ is $(p/L)$-small). Talagrand showed this when $λ$ is supported on singletons and suggested, as a more challenging test case, proving it when $λ$ is supported on pairs. The present work provides such a proof.

math.CO↗

arXiv2021

Self-supervised Auxiliary Learning for Graph Neural Networks via Meta-Learning

In recent years, graph neural networks (GNNs) have been widely adopted in the representation learning of graph-structured data and provided state-of-the-art performance in various applications such as link prediction, node classification, and recommendation. Motivated by recent advances of self-supervision for representation learning in natural language processing and computer vision, self-supervised learning has been recently studied to leverage unlabeled graph-structured data. However, employing self-supervision tasks as auxiliary tasks to assist a primary task has been less explored in the literature on graphs. In this paper, we propose a novel self-supervised auxiliary learning framework to effectively learn graph neural networks. Moreover, this work is the first study showing that a meta-path prediction is beneficial as a self-supervised auxiliary task for heterogeneous graphs. Our method is learning to learn a primary task with various auxiliary tasks to improve generalization performance. The proposed method identifies an effective combination of auxiliary tasks and automatically balances them to improve the primary task. Our methods can be applied to any graph neural network in a plug-in manner without manual labeling or additional data. Also, it can be extended to any other auxiliary tasks. Our experiments demonstrate that the proposed method consistently improves the performance of node classification and link prediction.

cs.LG↗

arXiv2021

Note on the number of balanced independent sets in the Hamming cube

Let $Q_d$ be the $d$-dimensional Hamming cube and $N=|V(Q_d)|=2^d$. An independent set $I$ in $Q_d$ is called balanced if $I$ contains the same number of even and odd vertices. We show that the logarithm of the number of balanced independent sets in $Q_d$ is \[(1-Θ(1/\sqrt d))N/2.\] The key ingredient of the proof is an improved version of "Sapozhenko's graph container lemma."

math.CO↗

arXiv2021

Self-supervised Auxiliary Learning with Meta-paths for Heterogeneous Graphs

Graph neural networks have shown superior performance in a wide range of applications providing a powerful representation of graph-structured data. Recent works show that the representation can be further improved by auxiliary tasks. However, the auxiliary tasks for heterogeneous graphs, which contain rich semantic information with various types of nodes and edges, have less explored in the literature. In this paper, to learn graph neural networks on heterogeneous graphs we propose a novel self-supervised auxiliary learning method using meta-paths, which are composite relations of multiple edge types. Our proposed method is learning to learn a primary task by predicting meta-paths as auxiliary tasks. This can be viewed as a type of meta-learning. The proposed method can identify an effective combination of auxiliary tasks and automatically balance them to improve the primary task. Our methods can be applied to any graph neural networks in a plug-in manner without manual labeling or additional data. The experiments demonstrate that the proposed method consistently improves the performance of link prediction and node classification on heterogeneous graphs.

cs.LG↗