arXiv subjects

Feng Luo

Publications and source records attributed to Feng Luo.

At least 73 records · Page 4Linked to original sources

arXiv2024

Unsupervised Microscopy Video Denoising

In this paper, we introduce a novel unsupervised network to denoise microscopy videos featured by image sequences captured by a fixed location microscopy camera. Specifically, we propose a DeepTemporal Interpolation method, leveraging a temporal signal filter integrated into the bottom CNN layers, to restore microscopy videos corrupted by unknown noise types. Our unsupervised denoising architecture is distinguished by its ability to adapt to multiple noise conditions without the need for pre-existing noise distribution knowledge, addressing a significant challenge in real-world medical applications. Furthermore, we evaluate our denoising framework using both real microscopy recordings and simulated data, validating our outperforming video denoising performance across a broad spectrum of noise scenarios. Extensive experiments demonstrate that our unsupervised model consistently outperforms state-of-the-art supervised and unsupervised video denoising techniques, proving especially effective for microscopy videos.

eess.IV↗

arXiv2024

Effectively Prompting Small-sized Language Models for Cross-lingual Tasks via Winning Tickets

Current soft prompt methods yield limited performance when applied to small-sized models (fewer than a billion parameters). Deep prompt-tuning, which entails prepending parameters in each layer for enhanced efficacy, presents a solution for prompting small-sized models, albeit requiring carefully designed implementation. In this paper, we introduce the Lottery Ticket Prompt-learning (LTP) framework that integrates winning tickets with soft prompts. The LTP offers a simpler implementation and requires only a one-time execution. We demonstrate LTP on cross-lingual tasks, where prior works rely on external tools like human-designed multilingual templates and bilingual dictionaries, which may not be feasible in a low-resource regime. Specifically, we select a subset of parameters that have been changed the most during the fine-tuning with the Masked Language Modeling objective. Then, we prepend soft prompts to the original pre-trained language model and only update the selected parameters together with prompt-related parameters when adapting to the downstream tasks. We verify the effectiveness of our LTP framework on cross-lingual tasks, specifically targeting low-resource languages. Our approach outperforms the baselines by only updating 20\% of the original parameters.

cs.CL↗

arXiv2023

Image Super-resolution Via Latent Diffusion: A Sampling-space Mixture Of Experts And Frequency-augmented Decoder Approach

The recent use of diffusion prior, enhanced by pre-trained text-image models, has markedly elevated the performance of image super-resolution (SR). To alleviate the huge computational cost required by pixel-based diffusion SR, latent-based methods utilize a feature encoder to transform the image and then implement the SR image generation in a compact latent space. Nevertheless, there are two major issues that limit the performance of latent-based diffusion. First, the compression of latent space usually causes reconstruction distortion. Second, huge computational cost constrains the parameter scale of the diffusion model. To counteract these issues, we first propose a frequency compensation module that enhances the frequency components from latent space to pixel space. The reconstruction distortion (especially for high-frequency information) can be significantly decreased. Then, we propose to use Sample-Space Mixture of Experts (SS-MoE) to achieve more powerful latent-based SR, which steadily improves the capacity of the model without a significant increase in inference costs. These carefully crafted designs contribute to performance improvements in largely explored 4x blind super-resolution benchmarks and extend to large magnification factors, i.e., 8x image SR benchmarks. The code is available at https://github.com/amandaluof/moe_sr.

cs.CV↗

arXiv2023

Is GPT Powerful Enough to Analyze the Emotions of Memes?

Large Language Models (LLMs), representing a significant achievement in artificial intelligence (AI) research, have demonstrated their ability in a multitude of tasks. This project aims to explore the capabilities of GPT-3.5, a leading example of LLMs, in processing the sentiment analysis of Internet memes. Memes, which include both verbal and visual aspects, act as a powerful yet complex tool for expressing ideas and sentiments, demanding an understanding of societal norms and cultural contexts. Notably, the detection and moderation of hateful memes pose a significant challenge due to their implicit offensive nature. This project investigates GPT's proficiency in such subjective tasks, revealing its strengths and potential limitations. The tasks include the classification of meme sentiment, determination of humor type, and detection of implicit hate in memes. The performance evaluation, using datasets from SemEval-2020 Task 8 and Facebook hateful memes, offers a comparative understanding of GPT responses against human annotations. Despite GPT's remarkable progress, our findings underscore the challenges faced by these models in handling subjective tasks, which are rooted in their inherent limitations including contextual understanding, interpretation of implicit meanings, and data biases. This research contributes to the broader discourse on the applicability of AI in handling complex, context-dependent tasks, and offers valuable insights for future advancements.

cs.CL↗

arXiv2023

Contrastive Representation Disentanglement for Clustering

Clustering continues to be a significant and challenging task. Recent studies have demonstrated impressive results by applying clustering to feature representations acquired through self-supervised learning, particularly on small datasets. However, when dealing with datasets containing a large number of clusters, such as ImageNet, current methods struggle to achieve satisfactory clustering performance. In this paper, we introduce a novel method called Contrastive representation Disentanglement for Clustering (CDC) that leverages contrastive learning to directly disentangle the feature representation for clustering. In CDC, we decompose the representation into two distinct components: one component encodes categorical information under an equipartition constraint, and the other component captures instance-specific factors. To train our model, we propose a contrastive loss that effectively utilizes both components of the representation. We conduct a theoretical analysis of the proposed loss and highlight how it assigns different weights to negative samples during the process of disentangling the feature representation. Further analysis of the gradients reveals that larger weights emphasize a stronger focus on hard negative samples. As a result, the proposed loss exhibits strong expressiveness, enabling efficient disentanglement of categorical information. Through experimental evaluation on various benchmark datasets, our method demonstrates either state-of-the-art or highly competitive clustering performance. Notably, on the complete ImageNet dataset, we achieve an accuracy of 53.4%, surpassing existing methods by a substantial margin of +10.2%.

cs.LG↗

arXiv2023

LIVABLE: Exploring Long-Tailed Classification of Software Vulnerability Types

Prior studies generally focus on software vulnerability detection and have demonstrated the effectiveness of Graph Neural Network (GNN)-based approaches for the task. Considering the various types of software vulnerabilities and the associated different degrees of severity, it is also beneficial to determine the type of each vulnerable code for developers. In this paper, we observe that the distribution of vulnerability type is long-tailed in practice, where a small portion of classes have massive samples (i.e., head classes) but the others contain only a few samples (i.e., tail classes). Directly adopting previous vulnerability detection approaches tends to result in poor detection performance, mainly due to two reasons. First, it is difficult to effectively learn the vulnerability representation due to the over-smoothing issue of GNNs. Second, vulnerability types in tails are hard to be predicted due to the extremely few associated samples.To alleviate these issues, we propose a Long-taIled software VulnerABiLity typE classification approach, called LIVABLE. LIVABLE mainly consists of two modules, including (1) vulnerability representation learning module, which improves the propagation steps in GNN to distinguish node representations by a differentiated propagation method. A sequence-to-sequence model is also involved to enhance the vulnerability representations. (2) adaptive re-weighting module, which adjusts the learning weights for different types according to the training epochs and numbers of associated samples by a novel training loss.

cs.SE↗

arXiv2022

Multi-level Distillation of Semantic Knowledge for Pre-training Multilingual Language Model

Pre-trained multilingual language models play an important role in cross-lingual natural language understanding tasks. However, existing methods did not focus on learning the semantic structure of representation, and thus could not optimize their performance. In this paper, we propose Multi-level Multilingual Knowledge Distillation (MMKD), a novel method for improving multilingual language models. Specifically, we employ a teacher-student framework to adopt rich semantic representation knowledge in English BERT. We propose token-, word-, sentence-, and structure-level alignment objectives to encourage multiple levels of consistency between source-target pairs and correlation similarity between teacher and student models. We conduct experiments on cross-lingual evaluation benchmarks including XNLI, PAWS-X, and XQuAD. Experimental results show that MMKD outperforms other baseline models of similar size on XNLI and XQuAD and obtains comparable performance on PAWS-X. Especially, MMKD obtains significant performance gains on low-resource languages.

cs.CL↗

arXiv2022

Bias-reduced Multi-step Hindsight Experience Replay for Efficient Multi-goal Reinforcement Learning

Multi-goal reinforcement learning is widely applied in planning and robot manipulation. Two main challenges in multi-goal reinforcement learning are sparse rewards and sample inefficiency. Hindsight Experience Replay (HER) aims to tackle the two challenges via goal relabeling. However, HER-related works still need millions of samples and a huge computation. In this paper, we propose Multi-step Hindsight Experience Replay (MHER), incorporating multi-step relabeled returns based on $n$-step relabeling to improve sample efficiency. Despite the advantages of $n$-step relabeling, we theoretically and experimentally prove the off-policy $n$-step bias introduced by $n$-step relabeling may lead to poor performance in many environments. To address the above issue, two bias-reduced MHER algorithms, MHER($λ$) and Model-based MHER (MMHER) are presented. MHER($λ$) exploits the $λ$ return while MMHER benefits from model-based value expansions. Experimental results on numerous multi-goal robotic tasks show that our solutions can successfully alleviate off-policy $n$-step bias and achieve significantly higher sample efficiency than HER and Curriculum-guided HER with little additional computation beyond HER.

cs.LG↗

arXiv2022

Final bound-state formation effect on dark matter annihilation

If the annihilation products of dark matter (DM) are non-relativistic and couples directly to a light force mediator, the non-perturbation effect like final state bound state (FBS) formation and final state Sommerfeld (FSS) effect must be considered. Non-relativistic region of final particles will appear when there is small mass split between DM and products, so we study those effects in the degenerate region of mass (including kinematics forbidden case) using two specific models. We demonstrate that FBS effect will significantly modify the DM relic abundance comparing to the standard perturbation calculation in some mass split region. We emphasize that FBS effect is comparable to the FSS effect in those mass split. The conservation angular momentum are subtle considering FBS formation, in some cases there may be not $s$-wave, so we use two models exhibit the different partial wave FBS effect contribution. We also show that the FBS formation with vector boson emission process also contributes in DM relic abundance, and first calculate the $p$-wave FSS effect in the specific model.

hep-ph↗

arXiv2021

Understanding and Measuring Robustness of Multimodal Learning

The modern digital world is increasingly becoming multimodal. Although multimodal learning has recently revolutionized the state-of-the-art performance in multimodal tasks, relatively little is known about the robustness of multimodal learning in an adversarial setting. In this paper, we introduce a comprehensive measurement of the adversarial robustness of multimodal learning by focusing on the fusion of input modalities in multimodal models, via a framework called MUROAN (MUltimodal RObustness ANalyzer). We first present a unified view of multimodal models in MUROAN and identify the fusion mechanism of multimodal models as a key vulnerability. We then introduce a new type of multimodal adversarial attacks called decoupling attack in MUROAN that aims to compromise multimodal models by decoupling their fused modalities. We leverage the decoupling attack of MUROAN to measure several state-of-the-art multimodal models and find that the multimodal fusion mechanism in all these models is vulnerable to decoupling attacks. We especially demonstrate that, in the worst case, the decoupling attack of MUROAN achieves an attack success rate of 100% by decoupling just 1.16% of the input space. Finally, we show that traditional adversarial training is insufficient to improve the robustness of multimodal models with respect to decoupling attacks. We hope our findings encourage researchers to pursue improving the robustness of multimodal learning.

cs.LG↗

arXiv2021

The Spectroscopic Binaries from LAMOST Medium-Resolution Survey (MRS). I. Searching for Double-lined Spectroscopic Binaries (SB2s) with Convolutional Neural Network

We developed a convolutional neural network (CNN) model to distinguish the double-lined spectroscopic binaries (SB2s) from others based on single exposure medium-resolution spectra ($R\sim 7,500$). The training set consists of a large set of mock spectra of single stars and binaries synthesized based on the MIST stellar evolutionary model and ATLAS9 atmospheric model. Our model reaches a novel theoretic false positive rate by adding a proper penalty on the negative sample (e.g., 0.12\% and 0.16\% for the blue/red arm when the penalty parameter $Λ=16$). Tests show that the performance is as expected and favors FGK-type Main-sequence binaries with high mass ratio ($q \geq 0.7$) and large radial velocity separation ($Δv \geq 50\,\mathrm{km\,s^{-1}}$). Although the real false positive rate can not be estimated reliably, validating on eclipsing binaries identified from Kepler light curves indicates that our model predicts low binary probabilities at eclipsing phases (0, 0.5, and 1.0) as expected. The color-magnitude diagram also helps illustrate its feasibility and capability of identifying FGK MS binaries from spectra. We conclude that this model is reasonably reliable and can provide an automatic approach to identify SB2s with period $\lesssim 10$ days. This work yields a catalog of binary probabilities for over 5 million spectra of 1 million sources from the LAMOST medium-resolution survey (MRS), and a catalog of 2198 SB2 candidates whose physical properties will be analyzed in our following-up paper. Data products are made publicly available at the journal as well as our Github website.

astro-ph.SR↗

arXiv2021

MHER: Model-based Hindsight Experience Replay

Solving multi-goal reinforcement learning (RL) problems with sparse rewards is generally challenging. Existing approaches have utilized goal relabeling on collected experiences to alleviate issues raised from sparse rewards. However, these methods are still limited in efficiency and cannot make full use of experiences. In this paper, we propose Model-based Hindsight Experience Replay (MHER), which exploits experiences more efficiently by leveraging environmental dynamics to generate virtual achieved goals. Replacing original goals with virtual goals generated from interaction with a trained dynamics model leads to a novel relabeling method, model-based relabeling (MBR). Based on MBR, MHER performs both reinforcement learning and supervised learning for efficient policy improvement. Theoretically, we also prove the supervised part in MHER, i.e., goal-conditioned supervised learning with MBR data, optimizes a lower bound on the multi-goal RL objective. Experimental results in several point-based tasks and simulated robotics environments show that MHER achieves significantly higher sample efficiency than previous model-free and model-based multi-goal methods.

cs.LG↗

arXiv2021

The Binarity of Early-type Stars from LAMOST Medium-resolution Spectroscopic Survey

Massive binaries play significant roles in many fields. Identification of massive stars, particularly massive binaries, is of great importance. In this paper, by adopting the technique of measuring the equivalent widths of several spectral lines, we identified 9,382 early-type stars from LAMOST medium-resolution survey and divided the sample into four groups, T1 ($\sim$O-B4), T2 ($\sim$B5), T3 ($\sim$B7), and T4 ($\sim$B8-A). The relative radial velocities $RV_{\rm rel}$ were calculated using the Maximum Likelihood Estimation. The stars with significant changes of $RV_{\rm rel}$ and at least larger than 15.57km s$^{-1}$ were identified as spectroscopic binaries. We found that the observed spectroscopic binary fractions for the four groups are $24.6\%\pm0.5\%$, $20.8\%\pm0.6\%$, $13.7\%\pm0.3\%$, and $7.4\%\pm0.3\%$, respectively. Assuming that orbital period ($P$) and mass ratio ($q$) have intrinsic distributions as $f(P) \propto P^π$ (1\textless$P$\textless1000 days) and $f(q) \propto q^κ$ (0.1\textless$q$\textless1), respectively, we conducted a series of Monte-Carlo simulations to correct observational biases for estimating the intrinsic multiplicity properties. The results show that the intrinsic binary fractions for the four groups are 68$\%\pm8\%$, 52$\%\pm3\%$, 44$\%\pm6\%$, and 44$\%\pm6\%$, respectively. The best estimated values for $π$ are -1$\pm0.1$, -1.1$\pm0.05$, -1.1$\pm0.1$, and -0.6$\pm0.05$, respectively. The $κ$ cannot be constrained for groups T1 and T2 and is -2.4$\pm0.3$ for group T3 and -1.6$\pm0.3$ for group T4. We confirmed the relationship of a decreasing trend in binary fractions towards late-type stars. No correlation between the spectral type and the orbital period distribution has been found yet, possibly due to the limitation of observational cadence.

astro-ph.SR↗

arXiv2021

Co-evolution of Opinion and Social Tie Dynamics Towards Structural Balance

In this paper, we propose co-evolution models for both dynamics of opinions (people's view on a particular topic) and dynamics of social appraisals (the approval or disapproval towards each other). Opinion dynamics and dynamics of signed networks, respectively, have been extensively studied. We propose a co-evolution model, where each vertex $i$ in the network has a current opinion vector $v_i$ and each edge $(i,j)$ has a weight $w_{ij}$ that models the relationship between $i,j$. The system evolves as the opinions and edge weights are updated over time by the following rules, Opinion Dynamics and Appraisal Dynamics. We are interested in characterizing the long-time behavior of the dynamic model -- i.e., whether edge weights evolve to have stable signs (positive or negative) and structural balance (the multiplication of weights on any triangle is non-negative). Our main theoretical result solves the above dynamic system with time-evolving opinions $V(t)=[v1(t),\cdots,vn(t)]$ and social tie weights $W(t)=[w_{ij}(t)]_{n\times n}$. For a generic initial opinion vector $V(0)$ and weight matrix $W(0)$, one of the two phenomena must occur at the limit. The first one is that both sign stability and structural balance (for any triangle with individual $i,j,k$, $w_{ij}w_{jk}w_{ki}\geq0$) occur. In the special case that $V(0)$ is an eigenvector of $W(0)$, we are able to obtain the explicit solution to the co-evolution equation and give exact estimates on the blowup time and rate convergence. The second one is that all the opinions converge to $0$, i.e., $\lim_{t\rightarrow \infty}|V(t)|=0$.

cs.SI↗

arXiv2021

Sk-Unet Model with Fourier Domain for Mitosis Detection

Mitotic count is the most important morphological feature of breast cancer grading. Many deep learning-based methods have been proposed but suffer from domain shift. In this work, we construct a Fourier-based segmentation model for mitosis detection to address the problem. Swapping the low-frequency spectrum of source and target images is shown effective to alleviate the discrepancy between different scanners. Our Fourier-based segmentation method can achieve F1 with 0.7456 on the preliminary test set.

eess.IV↗

arXiv2021

Binary fraction of O and B-type stars from LAMOST data

Binary stars plays important role in the evolution of stellar populations . The intrinsic binary fraction ($f_{bin}$) of O and B-type (OB) stars in LAMOST DR5 was investigated in this work. We employed a cross-correlation approach to estimate relative radial velocities for each of the stellar spectra. The algorithm described by \cite{2013A&A...550A.107S} was implemented and several simulations were made to assess the performance of the approach. Binary fraction of the OB stars are estimated through comparing the uni-distribution between observations and simulations with the Kolmogorov-Smirnov tests. Simulations show that it is reliable for stars most of whom have $6,7$ and $8$ repeated observations. The uncertainty of orbital parameters of binarity become larger when observational frequencies decrease. By adopting the fixed power exponents of $π=-0.45$ and $κ=-1$ for period and mass ratio distributions, respectively, we obtain that $f_{bin}=0.4_{-0.06}^{+0.05}$ for the samples with more than 3 observations. When we consider the full samples with at least 2 observations, the binary fraction turns out to be $0.37_{-0.03}^{+0.03}$. These two results are consistent with each other in $1σ$.

astro-ph.SR↗

arXiv2021

A Coarse-to-Fine Instance Segmentation Network with Learning Boundary Representation

Boundary-based instance segmentation has drawn much attention since of its attractive efficiency. However, existing methods suffer from the difficulty in long-distance regression. In this paper, we propose a coarse-to-fine module to address the problem. Approximate boundary points are generated at the coarse stage and then features of these points are sampled and fed to a refined regressor for fine prediction. It is end-to-end trainable since differential sampling operation is well supported in the module. Furthermore, we design a holistic boundary-aware branch and introduce instance-agnostic supervision to assist regression. Equipped with ResNet-101, our approach achieves 31.7\% mask AP on COCO dataset with single-scale training and testing, outperforming the baseline 1.3\% mask AP with less than 1\% additional parameters and GFLOPs. Experiments also show that our proposed method achieves competitive performance compared to existing boundary-based methods with a lightweight design and a simple pipeline.

cs.CV↗

arXiv2021

Multi-level Knowledge Distillation via Knowledge Alignment and Correlation

Knowledge distillation (KD) has become an important technique for model compression and knowledge transfer. In this work, we first perform a comprehensive analysis of the knowledge transferred by different KD methods. We demonstrate that traditional KD methods, which minimize the KL divergence of softmax outputs between networks, are related to the knowledge alignment of an individual sample only. Meanwhile, recent contrastive learning-based KD methods mainly transfer relational knowledge between different samples, namely, knowledge correlation. While it is important to transfer the full knowledge from teacher to student, we introduce the Multi-level Knowledge Distillation (MLKD) by effectively considering both knowledge alignment and correlation. MLKD is task-agnostic and model-agnostic, and can easily transfer knowledge from supervised or self-supervised pretrained teachers. We show that MLKD can improve the reliability and transferability of learned representations. Experiments demonstrate that MLKD outperforms other state-of-the-art methods on a large number of experimental settings including different (a) pretraining strategies (b) network architectures (c) datasets (d) tasks.

cs.CV↗