arXiv subjects

Dongyue Li

Publications and source records attributed to Dongyue Li.

46 records · Page 3Linked to original sources

arXiv2024

Boosting Multitask Learning on Graphs through Higher-Order Task Affinities

Predicting node labels on a given graph is a widely studied problem with many applications, including community detection and molecular graph prediction. This paper considers predicting multiple node labeling functions on graphs simultaneously and revisits this problem from a multitask learning perspective. For a concrete example, consider overlapping community detection: each community membership is a binary node classification task. Due to complex overlapping patterns, we find that negative transfer is prevalent when we apply naive multitask learning to multiple community detection, as task relationships are highly nonlinear across different node labeling. To address the challenge, we develop an algorithm to cluster tasks into groups based on a higher-order task affinity measure. We then fit a multitask model on each task group, resulting in a boosting procedure on top of the baseline model. We estimate the higher-order task affinity measure between two tasks as the prediction loss of one task in the presence of another task and a random subset of other tasks. Then, we use spectral clustering on the affinity score matrix to identify task grouping. We design several speedup techniques to compute the higher-order affinity scores efficiently and show that they can predict negative transfers more accurately than pairwise task affinities. We validate our procedure using various community detection and molecular graph prediction data sets, showing favorable results compared with existing methods. Lastly, we provide a theoretical analysis to show that under a planted block model of tasks on graphs, our affinity scores can provably separate tasks into groups.

cs.LG↗

arXiv2023

Identification of Negative Transfers in Multitask Learning Using Surrogate Models

Multitask learning is widely used in practice to train a low-resource target task by augmenting it with multiple related source tasks. Yet, naively combining all the source tasks with a target task does not always improve the prediction performance for the target task due to negative transfers. Thus, a critical problem in multitask learning is identifying subsets of source tasks that would benefit the target task. This problem is computationally challenging since the number of subsets grows exponentially with the number of source tasks; efficient heuristics for subset selection do not always capture the relationship between task subsets and multitask learning performances. In this paper, we introduce an efficient procedure to address this problem via surrogate modeling. In surrogate modeling, we sample (random) subsets of source tasks and precompute their multitask learning performances. Then, we approximate the precomputed performances with a linear regression model that can also predict the multitask performance of unseen task subsets. We show theoretically and empirically that fitting this model only requires sampling linearly many subsets in the number of source tasks. The fitted model provides a relevance score between each source and target task. We use the relevance scores to perform subset selection for multitask learning by thresholding. Through extensive experiments, we show that our approach predicts negative transfers from multiple source tasks to target tasks much more accurately than existing task affinity measures. Additionally, we demonstrate that for several weak supervision datasets, our approach consistently improves upon existing optimization methods for multitask learning.

cs.LG↗

arXiv2023

Robust Fine-Tuning of Deep Neural Networks with Hessian-based Generalization Guarantees

We consider fine-tuning a pretrained deep neural network on a target task. We study the generalization properties of fine-tuning to understand the problem of overfitting, which has often been observed (e.g., when the target dataset is small or when the training labels are noisy). Existing generalization measures for deep networks depend on notions such as distance from the initialization (i.e., the pretrained network) of the fine-tuned model and noise stability properties of deep networks. This paper identifies a Hessian-based distance measure through PAC-Bayesian analysis, which is shown to correlate well with observed generalization gaps of fine-tuned models. Theoretically, we prove Hessian distance-based generalization bounds for fine-tuned models. We also describe an extended study of fine-tuning against label noise, where overfitting remains a critical problem. We present an algorithm and a generalization error guarantee for this algorithm under a class conditional independent noise model. Empirically, we observe that the Hessian-based distance measure can match the scale of the observed generalization gap of fine-tuned models in practice. We also test our algorithm on several image classification tasks with noisy training labels, showing gains over prior methods and decreases in the Hessian distance measure of the fine-tuned model.

cs.LG↗

arXiv2023

Generalization in Graph Neural Networks: Improved PAC-Bayesian Bounds on Graph Diffusion

Graph neural networks are widely used tools for graph prediction tasks. Motivated by their empirical performance, prior works have developed generalization bounds for graph neural networks, which scale with graph structures in terms of the maximum degree. In this paper, we present generalization bounds that instead scale with the largest singular value of the graph neural network's feature diffusion matrix. These bounds are numerically much smaller than prior bounds for real-world graphs. We also construct a lower bound of the generalization gap that matches our upper bound asymptotically. To achieve these results, we analyze a unified model that includes prior works' settings (i.e., convolutional and message-passing networks) and new settings (i.e., graph isomorphism networks). Our key idea is to measure the stability of graph neural networks against noise perturbations using Hessians. Empirically, we find that Hessian-based measurements correlate with the observed generalization gaps of graph neural networks accurately. Optimizing noise stability properties for fine-tuning pretrained graph neural networks also improves test performance on several graph-level classification tasks.

cs.LG↗

arXiv2023

CylinderTag: An Accurate and Flexible Marker for Cylinder-Shape Objects Pose Estimation Based on Projective Invariants

High-precision pose estimation based on visual markers has been a thriving research topic in the field of computer vision. However, the suitability of traditional flat markers on curved objects is limited due to the diverse shapes of curved surfaces, which hinders the development of high-precision pose estimation for curved objects. Therefore, this paper proposes a novel visual marker called CylinderTag, which is designed for developable curved surfaces such as cylindrical surfaces. CylinderTag is a cyclic marker that can be firmly attached to objects with a cylindrical shape. Leveraging the manifold assumption, the cross-ratio in projective invariance is utilized for encoding in the direction of zero curvature on the surface. Additionally, to facilitate the usage of CylinderTag, we propose a heuristic search-based marker generator and a high-performance recognizer as well. Moreover, an all-encompassing evaluation of CylinderTag properties is conducted by means of extensive experimentation, covering detection rate, detection speed, dictionary size, localization jitter, and pose estimation accuracy. CylinderTag showcases superior detection performance from varying view angles in comparison to traditional visual markers, accompanied by higher localization accuracy. Furthermore, CylinderTag boasts real-time detection capability and an extensive marker dictionary, offering enhanced versatility and practicality in a wide range of applications. Experimental results demonstrate that the CylinderTag is a highly promising visual marker for use on cylindrical-like surfaces, thus offering important guidance for future research on high-precision visual localization of cylinder-shaped objects. The code is available at: https://github.com/wsakobe/CylinderTag.

cs.CV↗

arXiv2023

Optimal Intervention on Weighted Networks via Edge Centrality

Suppose there is a spreading process such as an infectious disease propagating on a graph. How would we reduce the number of affected nodes in the spreading process? This question appears in recent studies about implementing mobility interventions on mobility networks (Chang et al. (2021)). A practical algorithm to reduce infections on unweighted graphs is to remove edges with the highest edge centrality score (Tong et al. (2012)), which is the product of two adjacent nodes' eigenscores. However, mobility networks have weighted edges; Thus, an intervention measure would involve edge-weight reduction besides edge removal. Motivated by this example, we revisit the problem of minimizing top eigenvalue(s) on weighted graphs by decreasing edge weights up to a fixed budget. We observe that the edge centrality score of Tong et al. (2012) is equal to the gradient of the largest eigenvalue of $WW^{\top}$, where $W$ denotes the weight matrix of the graph. We then present generalized edge centrality scores as the gradient of the sum of the largest $r$ eigenvalues of $WW^{\top}$. With this generalization, we design an iterative algorithm to find the optimal edge-weight reduction to shrink the largest $r$ eigenvalues of $WW^{\top}$ under a given edge-weight reduction budget. We also extend our algorithm and its guarantee to time-varying graphs, whose weights evolve over time. We perform a detailed empirical study to validate our approach. Our algorithm significantly reduces the number of infections compared with existing methods on eleven weighted networks. Further, we illustrate several properties of our algorithm, including the benefit of choosing the rank $r$, fast convergence to global optimum, and an almost linear runtime per iteration.

cs.SI↗

arXiv2022

Populations of highly variable X-ray sources in the XMM$-$Newton slew survey

We present the identifications of a flux-limited sample of highly variable X-ray sources on long time-scales from the second catalogue of the XMM$-$Newton SLew survey (XMMSL2). The carefully constructed sample, comprising 265 sources (2.5 per cent) selected from the XMMSL2 clean catalogue, displayed X-ray variability of a factor of more than 10 in 0.2$-$2 keV compared to the ROSAT All Sky Survey. Of the sample sources, 94.3 per cent are identified. The identification procedure follows a series of cross-matches with astronomical data bases and multiwavelength catalogues to refine the source position and identify counterparts to the X-ray sources. Assignment of source type utilizes a combination of indicators including counterparts offset, parallax measurement, spectral colours, X-ray luminosity, and light-curve behaviour. We identified 40 per cent of the variables with stars, 10 per cent with accreting binaries, and at least 30.4 per cent with active galactic nuclei. The rest of the variables are identified as galaxies. It is found that the mean effective temperatures of the highly variable stars are lower than those of less variable stars. Our sample of highly variable AGN tend to have lower black hole masses, redshifts, and marginally lower soft X-ray luminosities compared to the less variable ones, while no difference was found in the Eddington ratio distributions. Five flaring events are tidal disruption events published previously. This study has significantly increased the number of variable sources in XMMSL2 with identifications and provides greater insight on the nature of many of the sources, enabling further studies of highly variable X-ray sources.

astro-ph.HE↗

arXiv2020

A Tidal Disruption Event Candidate Discovered in the Active Galactic Nucleus SDSS J022700.77-042020.6

We report the discovery of a Tidal Disruption Event (TDE) candidate occurring in the Active Galactic Nucleus SDSS J022700.77-042020.6. A sudden increase in flux of J0227-0420 during the second half of 2009 is clearly shown in the long-term optical, UV, and NIR light curves. A plateau phase, following an initial decline, is seen in the NUV and optical u, g, r, i light curves. Moreover, we find possible evidence that the plateau phase in the NUV band may lag behind the optical ones by approximately 70-80 days with also a much shorter duration, i.e. $\sim$7-15 days against $\sim$40-50 days. The long-term NUV/optical (after the plateau phase), NIR and MIR light curves can be well characterized with a form of $L(t)\propto t^{-β}$, consistent with the expectation of a TDE. The plateaus can be explained if the stellar streams collide with the pre-existing AGN disk at different radii. Though the overall fallback rate decreases, the material in the outer disk gradually drifts inward and increases the local accretion rate at the inner region, producing the optical and UV plateaus. The possible lag between the optical and NUV plateaus can then be attributed to viscosity delay. The index $β$ of the NIR $J, H, K_s$ bands ($\sim1.4-3.3$) is steeper than that of the UV/optical ($\sim0.7-1.3$) and MIR bands ($\sim0.9-1.8$), which may suggest that a certain fraction of the dust in the inner region of the dusty torus may be sublimated during the TDE phase. Our results indicate that, due to collisions between stellar debris and pre-existing disk, the light curves of TDEs occurring in AGN may show distinctive features, which may shed new light on the accretion process.

astro-ph.GA↗

arXiv2020

Multiwavelength Study of an X-ray Tidal Disruption Event Candidate in NGC 5092

We present multiwavelength studies of a transient X-ray source, XMMSL1 J131952.3+225958, associated with the galaxy NGC 5092 at $z=0.023$ detected in the XMM-Newton SLew survey (XMMSL). The source brightened in the 0.2--2 keV band by a factor of $>20$ in 2005 as compared with previous flux limits and then faded by a factor of $>200$ as observed with it XMM-Newton in 2013 and with it Swift in 2018. At the flaring state, the X-ray spectrum can be modeled with a blackbody at a temperature of $\sim$ 60 eV and an overall luminosity of $\sim$ $1.5 \times 10^{43}$ erg s$^{-1}$. A UV flare and optical flare were also detected with the Galaxy Evolution Explorer and the Sloan Digital Sky Survey, respectively, within several months of the X-ray flare, whose nonstellar UV--optical spectrum can be described with a blackbody at a temperature of $\sim$ $(1-2) \times 10^4$ K and a luminosity of $\sim$ $(2-6) \times 10^{43}$ erg s$^{-1}$. Interestingly, mid-infrared monitoring observations of NGC 5092 with the Wide-field Infrared Survey Explorer 5--13 yr later show a continuous flux decline. These dramatic variability properties, from the X-ray through UV and optical to infrared, appear to be orderly, suggestive of a stellar tidal disruption event (TDE) by a massive black hole, confirming the postulation by Kanner et al.(2013). This TDE candidate belongs to a rare sample with contemporaneous bright emission detected in the X-ray, UV, and optical, which are later echoed by dust-reprocessed light in the mid-infrared. The black hole has a mass of $\sim$ $5 \times 10^{7} \rm M_{\odot}$, residing in a galaxy that is dominated by a middle-aged stellar population of 2.5 Gyr.

astro-ph.HE↗

arXiv2019

Investigation of the force closure for Eulerian-Eulerian simulations: a validation study of nine gas-liquid flow cases

Gas-liquid flows can be simulated by the Eulerian-Eulerian (E-E) method. Whether to include a specific momentum interfacial exchange force model remains as a question with no answer. In this work, our aim is to seek a general numerical settings for the E-E method, which can provide competent results for industrial bubbly flows with different geometries under different operations. They were selected from different industries including chemical, nuclear, bio-processing and metallurgical engineering. Simulations were launched by the OpenFOAM solver reactingTwoPhaseEulerFoam, in which the E-E method was implemented with sophisticated numerical techniques to ensure stabilities. Predictions were compared against experimental data. It was found that the drag force and turbulent dispersion force play the most important role on the predictions and should be included for all simulations. The first one accounts for the two-way coupling while the second one accounts for the turbulence effect and ensures the E-E equations to be well-posed. The lift force and wall lubrication force should be included to address the phase fraction accumulation in the vicinity of the wall, especially for pipe flows with large aspect ratio. In other cases the lateral forces can be safely neglected. All the test case are open-sourced and are available as supplementary data for anyone to download as baseline test cases.

physics.flu-dyn↗