Searcharxiv⌕ Search

arXiv subjects

Fangyuan Zhang

Publications and source records attributed to Fangyuan Zhang.

27 records · Page 2Linked to original sources

PILL: Plug Into LLM with Adapter Expert and Attention Gate

Due to the remarkable capabilities of powerful Large Language Models (LLMs) in effectively following instructions, there has been a growing number of assistants in the community to assist humans. Recently, significant progress has been made in the development of Vision Language Models (VLMs), expanding the capabilities of LLMs and enabling them to execute more diverse instructions. However, it is foreseeable that models will likely need to handle tasks involving additional modalities such as speech, video, and others. This poses a particularly prominent challenge of dealing with the complexity of mixed modalities. To address this, we introduce a novel architecture called PILL: Plug Into LLM with adapter expert and attention gate to better decouple these complex modalities and leverage efficient fine-tuning. We introduce two modules: Firstly, utilizing Mixture-of-Modality-Adapter-Expert to independently handle different modalities, enabling better adaptation to downstream tasks while preserving the expressive capability of the original model. Secondly, by introducing Modality-Attention-Gating, which enables adaptive control of the contribution of modality tokens to the overall representation. In addition, we have made improvements to the Adapter to enhance its learning and expressive capabilities. Experimental results demonstrate that our approach exhibits competitive performance compared to other mainstream methods for modality fusion. For researchers interested in our work, we provide free access to the code and models at https://github.com/DsaltYfish/PILL.

cs.CV↗

Blind Image Super-resolution with Rich Texture-Aware Codebooks

Blind super-resolution (BSR) methods based on high-resolution (HR) reconstruction codebooks have achieved promising results in recent years. However, we find that a codebook based on HR reconstruction may not effectively capture the complex correlations between low-resolution (LR) and HR images. In detail, multiple HR images may produce similar LR versions due to complex blind degradations, causing the HR-dependent only codebooks having limited texture diversity when faced with confusing LR inputs. To alleviate this problem, we propose the Rich Texture-aware Codebook-based Network (RTCNet), which consists of the Degradation-robust Texture Prior Module (DTPM) and the Patch-aware Texture Prior Module (PTPM). DTPM effectively mines the cross-resolution correlation of textures between LR and HR images by exploiting the cross-resolution correspondence of textures. PTPM uses patch-wise semantic pre-training to correct the misperception of texture similarity in the high-level semantic regularization. By taking advantage of this, RTCNet effectively gets rid of the misalignment of confusing textures between HR and LR in the BSR scenarios. Experiments show that RTCNet outperforms state-of-the-art methods on various benchmarks by up to 0.16 ~ 0.46dB.

cs.CV↗

Low-Confidence Samples Mining for Semi-supervised Object Detection

Reliable pseudo-labels from unlabeled data play a key role in semi-supervised object detection (SSOD). However, the state-of-the-art SSOD methods all rely on pseudo-labels with high confidence, which ignore valuable pseudo-labels with lower confidence. Additionally, the insufficient excavation for unlabeled data results in an excessively low recall rate thus hurting the network training. In this paper, we propose a novel Low-confidence Samples Mining (LSM) method to utilize low-confidence pseudo-labels efficiently. Specifically, we develop an additional pseudo information mining (PIM) branch on account of low-resolution feature maps to extract reliable large-area instances, the IoUs of which are higher than small-area ones. Owing to the complementary predictions between PIM and the main branch, we further design self-distillation (SD) to compensate for both in a mutually-learning manner. Meanwhile, the extensibility of the above approaches enables our LSM to apply to Faster-RCNN and Deformable-DETR respectively. On the MS-COCO benchmark, our method achieves 3.54% mAP improvement over state-of-the-art methods under 5% labeling ratios.

cs.CV↗

Intergenerational risk sharing in a Defined Contribution pension system: analysis with Bayesian optimization

We study a fully funded, collective defined-contribution (DC) pension system with multiple overlapping generations. We investigate whether the welfare of participants can be improved by intergenerational risk sharing (IRS) implemented with a realistic investment strategy (e.g., no borrowing) and without an outside entity (e.g., share holders) that helps finance the pension fund. To implement IRS, the pension system uses an automatic adjustment rule for the indexation of individual accounts, which adapts to the notional funding ratio of the pension system. The pension system has two parameters that determine the investment strategy and the strength of the adjustment rule, which are optimized by expected utility maximization using Bayesian optimization. The volatility of the retirement benefits and that of the funding ratio are analyzed, and it is shown that the trade-off between them can be controlled by the optimal adjustment parameter to attain IRS. Compared with the optimal individual DC benchmark using the life-cycle strategy, the studied pension system with IRS is shown to improve the welfare of risk-averse participants, when the financial market is volatile.

econ.GN↗

Personalized PageRank on Evolving Graphs with an Incremental Index-Update Scheme

{\em Personalized PageRank (PPR)} stands as a fundamental proximity measure in graph mining. Since computing an exact SSPPR query answer is prohibitive, most existing solutions turn to approximate queries with guarantees. The state-of-the-art solutions for approximate SSPPR queries are index-based and mainly focus on static graphs, while real-world graphs are usually dynamically changing. However, existing index-update schemes can not achieve a sub-linear update time. Motivated by this, we present an efficient indexing scheme to maintain indexed random walks in expected $O(1)$ time after each graph update. To reduce the space consumption, we further propose a new sampling scheme to remove the auxiliary data structure for vertices while still supporting $O(1)$ index update cost on evolving graphs. Extensive experiments show that our update scheme achieves orders of magnitude speed-up on update performance over existing index-based dynamic schemes without sacrificing the query efficiency.

cs.DB↗

On the equivalence between Value-at-Risk- and Expected Shortfall-based risk measures in non-concave optimization

We study a non-concave optimization problem in which a financial company maximizes the expected utility of the surplus under a risk-based regulatory constraint. For this problem, we consider four different prevalent risk constraints (Expected Shortfall, Expected Discounted Shortfall, Value-at-Risk, and Average Value-at-Risk), and investigate their effects on the optimal solution. Our main contributions are in obtaining an analytical solution under each of the four risk constraints, in the form of the optimal terminal wealth. We show that the four risk constraints lead to the same optimal solution, which differs from previous conclusions obtained from the corresponding concave optimization problem under a risk constraint. Compared with the benchmark (unconstrained) non-concave utility maximization problem, all four risk constraints effectively and equivalently reduce the set of zero terminal wealth, but do not fully eliminate this set, indicating the success and failure of the respective financial regulations.

math.OC↗

Semi-Supervised Object Detection with Adaptive Class-Rebalancing Self-Training

This study delves into semi-supervised object detection (SSOD) to improve detector performance with additional unlabeled data. State-of-the-art SSOD performance has been achieved recently by self-training, in which training supervision consists of ground truths and pseudo-labels. In current studies, we observe that class imbalance in SSOD severely impedes the effectiveness of self-training. To address the class imbalance, we propose adaptive class-rebalancing self-training (ACRST) with a novel memory module called CropBank. ACRST adaptively rebalances the training data with foreground instances extracted from the CropBank, thereby alleviating the class imbalance. Owing to the high complexity of detection tasks, we observe that both self-training and data-rebalancing suffer from noisy pseudo-labels in SSOD. Therefore, we propose a novel two-stage filtering algorithm to generate accurate pseudo-labels. Our method achieves satisfactory improvements on MS-COCO and VOC benchmarks. When using only 1\% labeled data in MS-COCO, our method achieves 17.02 mAP improvement over supervised baselines, and 5.32 mAP improvement compared with state-of-the-art methods.

cs.CV↗

Simultaneous Rank Tests in Analysis of Covariance Based on Pairwise Ranking

Nonparametric tests provide robust and powerful alternatives to the corresponding least squares methods. There are two approaches to nonparametric pairwise comparisons of treatment effects, the method based on pairwise rankings and the method based on overall ranking. The former is generally recommended in the literature because of its strong control of familywise error rate. However, this method is developed only for one-way layouts and randomized complete blocks. By combining the method of aligned ranks and pairwise ranking, we extend the Steel-Dwass pairwise comparisons to the analysis of covariance and factorial models for both one-sided and two-sided comparisons as well as testing for treatment versus control. Unlike the traditional two-sample standardization of test statistics, we propose a weighted estimate of the scale parameter for ranks and show through simulation that it has superior small sample performance by controlling the familywise error rate at nominal level. This method provides an improvement for large sample approximation of Steel-Dwass method for one-way layouts. The marginal and joint asymptotic distributions are derived and power comparisons are made with the method of aligned rank transformation and the least squares method.

stat.ME↗

Spacings Around An Order Statistic

We determine the joint limiting distribution of adjacent spacings around a central, intermediate, or an extreme order statistic $X_{k:n}$ of a random sample of size $n$ from a continuous distribution $F$. For central and intermediate cases, normalized spacings in the left and right neighborhoods are asymptotically i.i.d. exponential random variables. The associated independent Poisson arrival processes are independent of $X_{k:n}$. For an extreme $X_{k:n}$, the asymptotic independence property of spacings fails for $F$ in the domain of attraction of Fréchet and Weibull ($α\neq 1$) distributions. This work also provides additional insight into the limiting distribution for the number of observations around $X_{k:n}$ for all three cases.

math.ST↗