arXiv subjects

Jun Gao

Publications and source records attributed to Jun Gao.

At least 199 records · Page 11Linked to original sources

arXiv2023

Extracting Triangular 3D Models, Materials, and Lighting From Images

We present an efficient method for joint optimization of topology, materials and lighting from multi-view image observations. Unlike recent multi-view reconstruction approaches, which typically produce entangled 3D representations encoded in neural networks, we output triangle meshes with spatially-varying materials and environment lighting that can be deployed in any traditional graphics engine unmodified. We leverage recent work in differentiable rendering, coordinate-based networks to compactly represent volumetric texturing, alongside differentiable marching tetrahedrons to enable gradient-based optimization directly on the surface mesh. Finally, we introduce a differentiable formulation of the split sum approximation of environment lighting to efficiently recover all-frequency lighting. Experiments show our extracted models used in advanced scene editing, material decomposition, and high quality view interpolation, all running at interactive rates in triangle-based renderers (rasterizers and path tracers). Project website: https://nvlabs.github.io/nvdiffrec/ .

cs.CV↗

arXiv2023

Neural Fields meet Explicit Geometric Representation for Inverse Rendering of Urban Scenes

Reconstruction and intrinsic decomposition of scenes from captured imagery would enable many applications such as relighting and virtual object insertion. Recent NeRF based methods achieve impressive fidelity of 3D reconstruction, but bake the lighting and shadows into the radiance field, while mesh-based methods that facilitate intrinsic decomposition through differentiable rendering have not yet scaled to the complexity and scale of outdoor scenes. We present a novel inverse rendering framework for large urban scenes capable of jointly reconstructing the scene geometry, spatially-varying materials, and HDR lighting from a set of posed RGB images with optional depth. Specifically, we use a neural field to account for the primary rays, and use an explicit mesh (reconstructed from the underlying neural field) for modeling secondary rays that produce higher-order lighting effects such as cast shadows. By faithfully disentangling complex geometry and materials from lighting effects, our method enables photorealistic relighting with specular and shadow effects on several outdoor datasets. Moreover, it supports physics-based scene manipulations such as virtual object insertion with ray-traced shadow casting.

cs.CV↗

arXiv2023

Beyond Fixed Grid: Learning Geometric Image Representation with a Deformable Grid

In modern computer vision, images are typically represented as a fixed uniform grid with some stride and processed via a deep convolutional neural network. We argue that deforming the grid to better align with the high-frequency image content is a more effective strategy. We introduce \emph{Deformable Grid} DefGrid, a learnable neural network module that predicts location offsets of vertices of a 2-dimensional triangular grid, such that the edges of the deformed grid align with image boundaries. We showcase our DefGrid in a variety of use cases, i.e., by inserting it as a module at various levels of processing. We utilize DefGrid as an end-to-end \emph{learnable geometric downsampling} layer that replaces standard pooling methods for reducing feature resolution when feeding images into a deep CNN. We show significantly improved results at the same grid resolution compared to using CNNs on uniform grids for the task of semantic segmentation. We also utilize DefGrid at the output layers for the task of object mask annotation, and show that reasoning about object boundaries on our predicted polygonal grid leads to more accurate results over existing pixel-wise and curve-based approaches. We finally showcase DefGrid as a standalone module for unsupervised image partitioning, showing superior performance over existing approaches. Project website: http://www.cs.toronto.edu/~jungao/def-grid

cs.CV↗

arXiv2023

Magic3D: High-Resolution Text-to-3D Content Creation

DreamFusion has recently demonstrated the utility of a pre-trained text-to-image diffusion model to optimize Neural Radiance Fields (NeRF), achieving remarkable text-to-3D synthesis results. However, the method has two inherent limitations: (a) extremely slow optimization of NeRF and (b) low-resolution image space supervision on NeRF, leading to low-quality 3D models with a long processing time. In this paper, we address these limitations by utilizing a two-stage optimization framework. First, we obtain a coarse model using a low-resolution diffusion prior and accelerate with a sparse 3D hash grid structure. Using the coarse representation as the initialization, we further optimize a textured 3D mesh model with an efficient differentiable renderer interacting with a high-resolution latent diffusion model. Our method, dubbed Magic3D, can create high quality 3D mesh models in 40 minutes, which is 2x faster than DreamFusion (reportedly taking 1.5 hours on average), while also achieving higher resolution. User studies show 61.7% raters to prefer our approach over DreamFusion. Together with the image-conditioned generation capabilities, we provide users with new ways to control 3D synthesis, opening up new avenues to various creative applications.

cs.CV↗

arXiv2023

Exploring the Feasibility of ChatGPT for Event Extraction

Event extraction is a fundamental task in natural language processing that involves identifying and extracting information about events mentioned in text. However, it is a challenging task due to the lack of annotated data, which is expensive and time-consuming to obtain. The emergence of large language models (LLMs) such as ChatGPT provides an opportunity to solve language tasks with simple prompts without the need for task-specific datasets and fine-tuning. While ChatGPT has demonstrated impressive results in tasks like machine translation, text summarization, and question answering, it presents challenges when used for complex tasks like event extraction. Unlike other tasks, event extraction requires the model to be provided with a complex set of instructions defining all event types and their schemas. To explore the feasibility of ChatGPT for event extraction and the challenges it poses, we conducted a series of experiments. Our results show that ChatGPT has, on average, only 51.04% of the performance of a task-specific model such as EEQA in long-tail and complex scenarios. Our usability testing experiments indicate that ChatGPT is not robust enough, and continuous refinement of the prompt does not lead to stable performance improvements, which can result in a poor user experience. Besides, ChatGPT is highly sensitive to different prompt styles.

cs.CL↗

arXiv2023

Probing the Higgs trilinear self-coupling through Higgs+jet production

We present the calculation of the next-to-leading order (NLO) electroweak (EW) corrections proportional to the Higgs trilinear self-coupling ($λ_{HHH}$) for Higgs boson plus one jet production at the Large Hadron Collider (LHC). We use the method of large top quark mass expansion to tackle the most challenging two-loop virtual amplitude, and apply the Padé approximation to extend the region of convergence of the expansion. We find that the NLO EW corrections is $0.66\%$ for the total cross section. For the invariant mass distribution and Higgs boson transverse momentum distribution, the NLO corrections are almost flat with their values similar in size. Our results can be used to set extra constraints on $λ_{HHH}$ at the LHC.

hep-ph↗

arXiv2023

Many Hamiltonian subsets in large graphs with given density

A set of vertices in a graph is a Hamiltonian subset if it induces a subgraph containing a Hamiltonian cycle. Kim, Liu, Sharifzadeh and Staden proved that among all graphs with minimum degree $d$, $K_{d+1}$ minimises the number of Hamiltonian subsets. We prove a near optimal lower bound that takes also the order and the structure of a graph into account. For many natural graph classes, it provides a much better bound than the extremal one ($\approx 2^{d+1}$). Among others, our bound implies that an $n$-vertex $C_4$-free graphs with minimum degree $d$ contains at least $n2^{d^{2-o(1)}}$ Hamiltonian subsets.

math.CO↗

arXiv2023

Mask-then-Fill: A Flexible and Effective Data Augmentation Framework for Event Extraction

We present Mask-then-Fill, a flexible and effective data augmentation framework for event extraction. Our approach allows for more flexible manipulation of text and thus can generate more diverse data while keeping the original event structure unchanged as much as possible. Specifically, it first randomly masks out an adjunct sentence fragment and then infills a variable-length text span with a fine-tuned infilling model. The main advantage lies in that it can replace a fragment of arbitrary length in the text with another fragment of variable length, compared to the existing methods which can only replace a single word or a fixed-length fragment. On trigger and argument extraction tasks, the proposed framework is more effective than baseline methods and it demonstrates particularly strong results in the low-resource setting. Our further analysis shows that it achieves a good balance between diversity and distributional similarity.

cs.CL↗

arXiv2022

A Generative Approach for Script Event Prediction via Contrastive Fine-tuning

Script event prediction aims to predict the subsequent event given the context. This requires the capability to infer the correlations between events. Recent works have attempted to improve event correlation reasoning by using pretrained language models and incorporating external knowledge~(e.g., discourse relations). Though promising results have been achieved, some challenges still remain. First, the pretrained language models adopted by current works ignore event-level knowledge, resulting in an inability to capture the correlations between events well. Second, modeling correlations between events with discourse relations is limited because it can only capture explicit correlations between events with discourse markers, and cannot capture many implicit correlations. To this end, we propose a novel generative approach for this task, in which a pretrained language model is fine-tuned with an event-centric pretraining objective and predicts the next event within a generative paradigm. Specifically, we first introduce a novel event-level blank infilling strategy as the learning objective to inject event-level knowledge into the pretrained language model, and then design a likelihood-based contrastive loss for fine-tuning the generative model. Instead of using an additional prediction layer, we perform prediction by using sequence likelihoods generated by the generative model. Our approach models correlations between events in a soft way without any external knowledge. The likelihood-based prediction eliminates the need to use additional networks to make predictions and is somewhat interpretable since it scores each word in the event. Experimental results on the multi-choice narrative cloze~(MCNC) task demonstrate that our approach achieves better results than other state-of-the-art baselines. Our code will be available at https://github.com/zhufq00/mcnc.

cs.CL↗

arXiv2022

Search for Charm-quark Production via Dimuons in Neutrino Telescopes

Dimuon events induced by charm-quark productions from neutrino deep inelastic scattering (DIS) processes have been studied in traditional DIS experiments for decades. The recent progress in neutrino telescopes makes it possible to search such dimuon events at energies far beyond laboratory scale. In this paper, we construct a simulation framework to calculate yields and distributions of dimuon signals in an IceCube-like km3 scale neutrino telescope. Due to experimental limitation in the resolution of double-track lateral distance, only dimuon produced outside the detector volume are considered. Detailed information about simulation results for ten years exposure is demonstrated. Both an earlier work [1] and our work study a similar situation, we therefore use that paper as a baseline to conduct comparisons. We then estimate the impacts of different calculation methods of muon energy losses, and find that, the average energy loss formula used in that work results in more events by contrast with Monte Carlo simulation. Finally, we study the experimental potential of dimuon searches under the hypothesis of single-muon background-only. Our results based on a simplified double-track reconstruction indicate a moderate sensitivity especially with the ORCA configuration. Further developments on both the reconstruction algorithm and possible detector designs are thus required, and are under investigation.

hep-ph↗

arXiv2022

Title2Event: Benchmarking Open Event Extraction with a Large-scale Chinese Title Dataset

Event extraction (EE) is crucial to downstream tasks such as new aggregation and event knowledge graph construction. Most existing EE datasets manually define fixed event types and design specific schema for each of them, failing to cover diverse events emerging from the online text. Moreover, news titles, an important source of event mentions, have not gained enough attention in current EE research. In this paper, We present Title2Event, a large-scale sentence-level dataset benchmarking Open Event Extraction without restricting event types. Title2Event contains more than 42,000 news titles in 34 topics collected from Chinese web pages. To the best of our knowledge, it is currently the largest manually-annotated Chinese dataset for open event extraction. We further conduct experiments on Title2Event with different models and show that the characteristics of titles make it challenging for event extraction, addressing the significance of advanced study on this problem. The dataset and baseline codes are available at https://open-event-hub.github.io/title2event.

cs.CL↗

arXiv2022

TASA: Deceiving Question Answering Models by Twin Answer Sentences Attack

We present Twin Answer Sentences Attack (TASA), an adversarial attack method for question answering (QA) models that produces fluent and grammatical adversarial contexts while maintaining gold answers. Despite phenomenal progress on general adversarial attacks, few works have investigated the vulnerability and attack specifically for QA models. In this work, we first explore the biases in the existing models and discover that they mainly rely on keyword matching between the question and context, and ignore the relevant contextual relations for answer prediction. Based on two biases above, TASA attacks the target model in two folds: (1) lowering the model's confidence on the gold answer with a perturbed answer sentence; (2) misguiding the model towards a wrong answer with a distracting answer sentence. Equipped with designed beam search and filtering methods, TASA can generate more effective attacks than existing textual attack methods while sustaining the quality of contexts, in extensive experiments on five QA datasets and human evaluations.

cs.CL↗

arXiv2022

Li-rich and super Li-rich giants produced by element diffusion

Context. About 0.2-2% of giant stars are Li-rich, whose lithium abundance (A(Li)) is higher than 1.5 dex. Among them, near 6% are super Li-rich with A(Li) exceeding 3.2 dex. Meanwhile, the formation mechanism of these Li-rich and super Li-rich giants is still under debate. Aims. Considering the compact He core of red giants, attention is paid to the effect of element diffusion on A(Li). In particular, when the He core flash occurs, the element diffusion makes the thermohaline mixing zone extend inward and connect to the inner convection region of stars. Then, a large amount of 7Be produced by the He flash can be transferred to stellar surface, finally turning into 7Li. Thus, the goal of this work is to propose the mechanism of A(Li) enrichment and achieve the consistency between the theoretical and observation data. Methods. Using the Modules for Experiments in Stellar Astrophysics (MESA), we simulate the evolution of low-mass stars, with considering the effects of element diffusion on the Li abundances. The timescale ratio of Li-rich giants to normal giants is estimated by population synthesis method. Then we get the theoretical value of A(Li) and make a comparison with observations. Results. Considering the influence of element diffusion in the model results in the increase of lithium abundance up to about 1.8dex, which can reveal Li-rich giants. Simultaneously, introducing high constant diffusive mixing coefficients (Dmix) with the values from 10e11 to 10e15in the model allows A(Li) to increase from 2.4 to 4.5dex, which can explain the most of Li-rich and super Li-rich giant stars. The population synthesis method reveals that the amount of Li-rich giants among giants is about 0.2-2%, which is consistent with observation estimated levels.

astro-ph.SR↗

arXiv2022

Signature of a light charged Higgs boson from top quark pairs at the LHC

The charged Higgs boson is a smoking gun of extensions of the standard model with multiple Higgs-doublets, and has been searched for at various collider experiments. In this paper, we study signature of a light charged Higgs boson produced by top quark pairs at the LHC, with subsequent three-body decays into a W boson and a pair of bottom quarks. Cross sections on final states of two W bosons plus four bottom quarks have been measured by the ATLAS collaboration at the LHC 13 TeV. We reinterpret the experimental data under the scenario of a light charged Higgs boson, and find improved agreements. We obtain the first limit from LHC direct searches on the total branching ratio of the three-body decay, $Br(t\rightarrow H^+b)\times Br(H^+\rightarrow W^+b\bar b)$, and the strongest direct constraints on the parameter space of a class of type-I two-Higgs-Doublet models.

hep-ph↗

arXiv2022

Tight bounds towards a conjecture of Gallai

We prove that for $n>k\geq 3$, if $G$ is an $n$-vertex graph with chromatic number $k$ but any its proper subgraph has smaller chromatic number, then $G$ contains at most $n-k+3$ copies of cliques of size $k-1$. This answers a problem of Abbott and Zhou and provides a tight bound on a conjecture of Gallai.

math.CO↗

arXiv2022

On two cycles of consecutive even lengths

Bondy and Vince showed that every graph with minimum degree at least three contains two cycles of lengths differing by one or two.We prove the following average degree counterpart that every $n$-vertex graph $G$ with at least $\frac52(n-1)$ edges, unless $4|(n-1)$ and every block of $G$ is a clique $K_5$, contains two cycles of consecutive even lengths. Our proof is mainly based on structural analysis, and a crucial step which may be of independent interest shows that the same conclusion holds for every 3-connected graph with at least 6 vertices. This solves a special case of a conjecture of Verstraëte. The quantitative bound is tight and also provides the optimal extremal number for cycles of length two modulo four.

math.CO↗

arXiv2022

GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Images

As several industries are moving towards modeling massive 3D virtual worlds, the need for content creation tools that can scale in terms of the quantity, quality, and diversity of 3D content is becoming evident. In our work, we aim to train performant 3D generative models that synthesize textured meshes which can be directly consumed by 3D rendering engines, thus immediately usable in downstream applications. Prior works on 3D generative modeling either lack geometric details, are limited in the mesh topology they can produce, typically do not support textures, or utilize neural renderers in the synthesis process, which makes their use in common 3D software non-trivial. In this work, we introduce GET3D, a Generative model that directly generates Explicit Textured 3D meshes with complex topology, rich geometric details, and high-fidelity textures. We bridge recent success in the differentiable surface modeling, differentiable rendering as well as 2D Generative Adversarial Networks to train our model from 2D image collections. GET3D is able to generate high-quality 3D textured meshes, ranging from cars, chairs, animals, motorbikes and human characters to buildings, achieving significant improvements over previous methods.

cs.CV↗

arXiv2022

Empathetic Response Generation with State Management

A good empathetic dialogue system should first track and understand a user's emotion and then reply with an appropriate emotion. However, current approaches to this task either focus on improving the understanding of users' emotion or on proposing better responding strategies, and very few works consider both at the same time. Our work attempts to fill this vacancy. Inspired by task-oriented dialogue systems, we propose a novel empathetic response generation model with emotion-aware dialogue management. The emotion-aware dialogue management contains two parts: (1) Emotion state tracking maintains the current emotion state of the user and (2) Empathetic dialogue policy selection predicts a target emotion and a user's intent based on the results of the emotion state tracking. The predicted information is then used to guide the generation of responses. Experimental results show that dynamically managing different information can help the model generate more empathetic responses compared with several baselines under both automatic and human evaluations.

cs.CL↗