arXiv subjects

Yifan Song

Publications and source records attributed to Yifan Song.

70 records · Page 4Linked to original sources

arXiv2022

Calibrating Factual Knowledge in Pretrained Language Models

Previous literature has proved that Pretrained Language Models (PLMs) can store factual knowledge. However, we find that facts stored in the PLMs are not always correct. It motivates us to explore a fundamental question: How do we calibrate factual knowledge in PLMs without re-training from scratch? In this work, we propose a simple and lightweight method CaliNet to achieve this goal. To be specific, we first detect whether PLMs can learn the right facts via a contrastive score between right and fake facts. If not, we then use a lightweight method to add and adapt new parameters to specific factual texts. Experiments on the knowledge probing task show the calibration effectiveness and efficiency. In addition, through closed-book question answering, we find that the calibrated PLM possesses knowledge generalization ability after fine-tuning. Beyond the calibration performance, we further investigate and visualize the knowledge calibration mechanism.

cs.CL↗

arXiv2022

Learning Robust Representations for Continual Relation Extraction via Adversarial Class Augmentation

Continual relation extraction (CRE) aims to continually learn new relations from a class-incremental data stream. CRE model usually suffers from catastrophic forgetting problem, i.e., the performance of old relations seriously degrades when the model learns new relations. Most previous work attributes catastrophic forgetting to the corruption of the learned representations as new relations come, with an implicit assumption that the CRE models have adequately learned the old relations. In this paper, through empirical studies we argue that this assumption may not hold, and an important reason for catastrophic forgetting is that the learned representations do not have good robustness against the appearance of analogous relations in the subsequent learning process. To address this issue, we encourage the model to learn more precise and robust representations through a simple yet effective adversarial class augmentation mechanism (ACA), which is easy to implement and model-agnostic. Experimental results show that ACA can consistently improve the performance of state-of-the-art CRE models on two popular benchmarks.

cs.CL↗

arXiv2022

ConFiguRe: Exploring Discourse-level Chinese Figures of Speech

Figures of speech, such as metaphor and irony, are ubiquitous in literature works and colloquial conversations. This poses great challenge for natural language understanding since figures of speech usually deviate from their ostensible meanings to express deeper semantic implications. Previous research lays emphasis on the literary aspect of figures and seldom provide a comprehensive exploration from a view of computational linguistics. In this paper, we first propose the concept of figurative unit, which is the carrier of a figure. Then we select 12 types of figures commonly used in Chinese, and build a Chinese corpus for Contextualized Figure Recognition (ConFiguRe). Different from previous token-level or sentence-level counterparts, ConFiguRe aims at extracting a figurative unit from discourse-level context, and classifying the figurative unit into the right figure type. On ConFiguRe, three tasks, i.e., figure extraction, figure type classification and figure recognition, are designed and the state-of-the-art techniques are utilized to implement the benchmarks. We conduct thorough experiments and show that all three tasks are challenging for existing models, thus requiring further research. Our dataset and code are publicly available at https://github.com/pku-tangent/ConFiguRe.

cs.CL↗

arXiv2022

Less is More: Rethinking State-of-the-art Continual Relation Extraction Models with a Frustratingly Easy but Effective Approach

Continual relation extraction (CRE) requires the model to continually learn new relations from class-incremental data streams. In this paper, we propose a Frustratingly easy but Effective Approach (FEA) method with two learning stages for CRE: 1) Fast Adaption (FA) warms up the model with only new data. 2) Balanced Tuning (BT) finetunes the model on the balanced memory data. Despite its simplicity, FEA achieves comparable (on TACRED or superior (on FewRel) performance compared with the state-of-the-art baselines. With careful examinations, we find that the data imbalance between new and old relations leads to a skewed decision boundary in the head classifiers over the pretrained encoders, thus hurting the overall performance. In FEA, the FA stage unleashes the potential of memory data for the subsequent finetuning, while the BT stage helps establish a more balanced decision boundary. With a unified view, we find that two strong CRE baselines can be subsumed into the proposed training pipeline. The success of FEA also provides actionable insights and suggestions for future model designing in CRE.

cs.CL↗

arXiv2022

Skill requirements in job advertisements: A comparison of skill-categorization methods based on explanatory power in wage regressions

In this paper, we compare different methods to extract skill requirements from job advertisements. We consider three top-down methods that are based on expert-created dictionaries of keywords, and a bottom-up method of unsupervised topic modeling, the Latent Dirichlet Allocation (LDA) model. We measure the skill requirements based on these methods using a U.K. dataset of job advertisements that contains over 1 million entries. We estimate the returns of the identified skills using wage regressions. Finally, we compare the different methods by the wage variation they can explain, assuming that better-identified skills will explain a higher fraction of the wage variation in the labor market. We find that the top-down methods perform worse than the LDA model, as they can explain only about 20% of the wage variation, while the LDA model explains about 45% of it.

econ.GN↗

arXiv2022

Marine Bubble Flow Quantification Using Wide-Baseline Stereo Photogrammetry

Reliable quantification of natural and anthropogenic gas release (e.g.\ CO$_2$, methane) from the seafloor into the water column, and potentially to the atmosphere, is a challenging task. While ship-based echo sounders such as single beam and multibeam systems allow detection of free gas, bubbles, in the water even from a great distance, exact quantification utilizing the hydroacoustic data requires additional parameters such as rise speed and bubble size distribution. Optical methods are complementary in the sense that they can provide high temporal and spatial resolution of single bubbles or bubble streams from close distance. In this contribution we introduce a complete instrument and evaluation method for optical bubble stream characterization targeted at flows of up to 100ml/min and bubbles with a few millimeters radius. The dedicated instrument employs a high-speed deep sea capable stereo camera system that can record terabytes of bubble imagery when deployed at a seep site for later automated analysis. Bubble characteristics can be obtained for short sequences, then relocating the instrument to other locations, or in autonomous mode of definable intervals up to several days, in order to capture bubble flow variations due to e.g. tide dependent pressure changes or reservoir depletion. Beside reporting the steps to make bubble characterization robust and autonomous, we carefully evaluate the reachable accuracy to be in the range of 1-2\% of the bubble radius and propose a novel auto-calibration procedure that, due to the lack of point correspondences, uses only the silhouettes of bubbles. The system has been operated successfully in 1000m water depth at the Cascadia margin offshore Oregon to assess methane fluxes from various seep locations. Besides sample results we also report failure cases and lessons learnt during deployment and method development.

cs.CV↗

arXiv2021

Robustly Removing Deep Sea Lighting Effects for Visual Mapping of Abyssal Plains

The majority of Earth's surface lies deep in the oceans, where no surface light reaches. Robots diving down to great depths must bring light sources that create moving illumination patterns in the darkness, such that the same 3D point appears with different color in each image. On top, scattering and attenuation of light in the water makes images appear foggy and typically blueish, the degradation depending on each pixel's distance to its observed seafloor patch, on the local composition of the water and the relative poses and cones of the light sources. Consequently, visual mapping, including image matching and surface albedo estimation, severely suffers from the effects that co-moving light sources produce, and larger mosaic maps from photos are often dominated by lighting effects that obscure the actual seafloor structure. In this contribution a practical approach to estimating and compensating these lighting effects on predominantly homogeneous, flat seafloor regions, as can be found in the Abyssal plains of our oceans, is presented. The method is essentially parameter-free and intended as a preprocessing step to facilitate visual mapping, but already produces convincing lighting artefact compensation up to a global white balance factor. It does not require to be trained beforehand on huge sets of annotated images, which are not available for the deep sea. Rather, we motivate our work by physical models of light propagation, perform robust statistics-based estimates of additive and multiplicative nuisances that avoid explicit parameters for light, camera, water or scene, discuss the breakdown point of the algorithms and show results on imagery captured by robots in several kilometer water depth.

cs.CV↗

arXiv2021

Refractive Geometry for Underwater Domes

Underwater cameras are typically placed behind glass windows to protect them from the water. Spherical glass, a dome port, is well suited for high water pressures at great depth, allows for a large field of view, and avoids refraction if a pinhole camera is positioned exactly at the sphere's center. Adjusting a real lens perfectly to the dome center is a challenging task, both in terms of how to actually guide the centering process (e.g. visual servoing) and how to measure the alignment quality, but also, how to mechanically perform the alignment. Consequently, such systems are prone to being decentered by some offset, leading to challenging refraction patterns at the sphere that invalidate the pinhole camera model. We show that the overall camera system becomes an axial camera, even for thick domes as used for deep sea exploration and provide a non-iterative way to compute the center of refraction without requiring knowledge of exact air, glass or water properties. We also analyze the refractive geometry at the sphere, looking at effects such as forward- vs. backward decentering, iso-refraction curves and obtain a 6th-degree polynomial equation for forward projection of 3D points in thin domes. We then propose a pure underwater calibration procedure to estimate the decentering from multiple images. This estimate can either be used during adjustment to guide the mechanical position of the lens, or can be considered in photogrammetric underwater applications.

cs.CV↗

arXiv2021

Deep Sea Robotic Imaging Simulator

Nowadays underwater vision systems are being widely applied in ocean research. However, the largest portion of the ocean - the deep sea - still remains mostly unexplored. Only relatively few image sets have been taken from the deep sea due to the physical limitations caused by technical challenges and enormous costs. Deep sea images are very different from the images taken in shallow waters and this area did not get much attention from the community. The shortage of deep sea images and the corresponding ground truth data for evaluation and training is becoming a bottleneck for the development of underwater computer vision methods. Thus, this paper presents a physical model-based image simulation solution, which uses an in-air texture and depth information as inputs, to generate underwater image sequences taken by robots in deep ocean scenarios. Different from shallow water conditions, artificial illumination plays a vital role in deep sea image formation as it strongly affects the scene appearance. Our radiometric image formation model considers both attenuation and scattering effects with co-moving spotlights in the dark. By detailed analysis and evaluation of the underwater image formation model, we propose a 3D lookup table structure in combination with a novel rendering strategy to improve simulation performance. This enables us to integrate an interactive deep sea robotic vision simulation in the Unmanned Underwater Vehicles simulator. To inspire further deep sea vision research by the community, we will release the source code of our deep sea image converter to the public.

cs.CV↗

arXiv2020

Light Pose Calibration for Camera-light Vision Systems

Illuminating a scene with artificial light is a prerequisite for seeing in dark environments. However, nonuniform and dynamic illumination can deteriorate or even break computer vision approaches, for instance when operating a robot with headlights in the darkness. This paper presents a novel light calibration approach by taking multi-view and -distance images of a reference plane in order to provide pose information of the employed light sources to the computer vision system. By following a physical light propagation approach, under consideration of energy preservation, the estimation of light poses is solved by minimizing of the differences between real and rendered pixel intensities. During the evaluation we show the robustness and consistency of this method by statistically analyzing the light pose estimation results with different setups. Although the results are demonstrated using a rotationally-symmetric non-isotropic light, the method is suited also for non-symmetric lights.

cs.CV↗

arXiv2019

Effective repulsion in dense quark matter from non-perturbative gluon exchange

A moderately strong vector repulsion between quarks in dense quark matter is needed to explain how a quark core can support neutron stars heavier than two solar masses. We study this repulsion, parametrized by a four-fermion interaction with coupling g_V, in terms of non-perturbative gluon exchange in QCD in the Landau gauge. Matching the energy of quark matter, g_V n_q^2 (where n_q is the number density of quarks) with the quark exchange energy calculated in QCD with a gluon propagator parametrized by a finite gluon mass m_g and a frozen coupling alpha_s, at moderate quark densities, we find that gluon masses m_g in the range 200 - 600 MeV and alpha_s = 2 - 4 lead to a g_V consistent with neutron star phenomenology. Estimating the effects of quark masses and a color-flavor-locked (CFL) pairing gap, we find that g_V can be well approximated by a flavor-symmetric, decreasing function of density. We briefly discuss similar matchings for the isovector repulsion and for the pairing attraction.

astro-ph.HE↗

arXiv2018

Dynamics of magnetic skyrmion clusters driven by spin-polarized current with a spatially varied polarization

Magnetic skyrmions are promising candidates for future information technology. Here, we present a micromagnetic study of isolated skyrmions and skyrmion clusters in ferromagnetic nanodisks driven by the spin-polarized current with spatially varied polarization. The current-driven skyrmion clusters can be either dynamic steady or static, depending on the spatially varied polarization profile. For the dynamic steady state, the skyrmion cluster moves in a circle in the nanodisk, while for the static state, the skyrmion cluster is static. The frequency of the circular motion of skyrmion is also studied. Furthermore, the dependence of the skyrmion cluster dynamics on the magnetic anisotropy and Dzyaloshinskii-Moriya interaction is investigated. Our results may provide a pathway to realize magnetic skyrmion cluster based devices.

cond-mat.mes-hall↗

arXiv2017

From hadrons to quarks in neutron stars: a review

We review the equation of state of matter in neutron stars from the solid crust through the liquid nuclear matter interior to the quark regime at higher densities. We focus in detail on the question of how quark matter appears in neutron stars, and how it affects the equation of state. After discussing the crust and liquid nuclear matter in the core we briefly review aspects of microscopic quark physics relevant to neutron stars, and quark models of dense matter based on the Nambu--Jona-Lasinio framework, in which gluonic processes are replaced by effective quark interactions. We turn then to describing equations of state useful for interpretation of both electromagnetic and gravitational observations, reviewing the emerging picture of hadron-quark continuity in which hadronic matter turns relatively smoothly, with at most only a weak first order transition, into quark matter with increasing density. We review construction of unified equations of state that interpolate between the reasonably well understood nuclear matter regime at low densities and the quark matter regime at higher densities. The utility of such interpolations is driven by the present inability to calculate the dense matter equation of state in QCD from first principles. As we review, the parameters of effective quark models -- which have direct relevance to the more general structure of the QCD phase diagram of dense and hot matter -- are constrained by neutron star mass and radii measurements, in particular favoring large repulsive density-density and attractive diquark pairing interactions. We describe the structure of neutron stars constructed from the unified equations of states with crossover. Lastly we present the current equations of state -- called "QHC18" for quark-hadron crossover -- in a parametrized form practical for neutron star modeling.

astro-ph.HE↗

arXiv2017

Generalized Nambu-Goldstone pion in dense matter: a schematic NJL model

Chiral symmetry is always broken in cold, dense matter, by chiral condensation at low densities and by diquark condensation at high density. We construct here, within a schematic Nambu-Jona-Lasinio (NJL) model, the corresponding generalized Nambu-Goldstone pion, $π_G$. As we show, the $π_G$ mode naturally emerges as a linear combination of the $\langle \bar{q}q\rangle$ vacuum pion $π$ and the $\langle qq \rangle$ diquark condensate pion $\tildeπ$, with $q$ the quark field, and continuously evolves with increasing density from being $π$-like in the vacuum to $\tildeπ$-like in the high density diquark pairing phase. We calculate the density-dependent mass, decay constant, and coupling to quarks of the $π_G$, and derive a generalized Gell-Mann--Oakes--Renner (GMOR) relation in the presence of a finite bare quark mass $m_q$. We briefly discuss the implications of the results to possible Bose condensation of $π_G$ in more realistic models.

nucl-th↗

arXiv2015

Phenomenological QCD equations of state for neutron stars

We delineate the properties of QCD matter at baryon density n_B=1-10n_0 (n_0: nuclear saturation density), through the construction of neutron star equations of state that satisfy the neutron star mass-radius constraints as well as physical conditions on the speed of sound. The QCD matter is described in the 3-window modeling: at n_B < 2n_0 purely nuclear matter; at n_B > 5n_0 percolated quark matter; and at 2n_0 < n_B < 5n_0 matter intermediate between these two which are constructed by interpolation. Using a schematic quark model with effective interactions inspired from hadron and nuclear physics, we analyze the strength of interactions necessary to describe observed neutron star properties. Our finding is that the interactions should remain as strong as in the QCD vacuum, indicating that gluons at n_B =1-10 n_0 remain non-perturbative even after quark matter formation.

hep-ph↗

arXiv2015

Phenomenological QCD equation of state for massive neutron stars

We construct an equation of state for massive neutron stars based on quantum chromodynamics phenomenology. Our primary purpose is to delineate the relevant ingredients of equations of state that simultaneously have the required stiffness and satisfy constraints from thermodynamics and causality. These ingredients are: (i) a repulsive density-density interaction, universal for all flavors; (ii) the color-magnetic interaction active from low to high densities; (iii) confining effects, which become increasingly important as the baryon density decreases; (iv) non-perturbative gluons, which are not very sensitive to changes of the quark density. We use the following "3-window" description: At baryon densities below about twice normal nuclear density, 2n_0, we use the Akmal-Pandharipande-Ravenhall (APR) equation of state, and at high densities, > (4-7)n_0, we use the three-flavor Nambu-Jona-Lasinio (NJL) model supplemented by vector and diquark interactions. In the transition density region, we smoothly interpolate the hadronic and quark equations of state in the chemical potential-pressure plane. Requiring that the equation of state approach APR at low densities, we find that the quark pressure in non-confining models can be larger than the hadronic pressure, unlike in conventional equations of state. We show that consistent equations of state of stiffness sufficient to allow massive neutron stars are reasonably tightly constrained, suggesting that gluon dynamics remains non-perturbative even at baryon densities ~10n_0.

hep-ph↗