Searcharxiv⌕ Search

arXiv subjects

Yi Zhou

Publications and source records attributed to Yi Zhou.

At least 289 records · Page 16Linked to original sources
arXiv2024

Programming the scalable optical learning operator with spatial-spectral optimization

Electronic computers have evolved drastically over the past years with an ever-growing demand for improved performance. However, the transfer of information from memory and high energy consumption have emerged as issues that require solutions. Optical techniques are considered promising solutions to these problems with higher speed than their electronic counterparts and with reduced energy consumption. Here, we use the optical reservoir computing framework we have previously described (Scalable Optical Learning Operator or SOLO) to program the spatial-spectral output of the light after nonlinear propagation in a multimode fiber. The novelty in the current paper is that the system is programmed through an output sampling scheme, similar to that used in hyperspectral imaging in astronomy. Linear and nonlinear computations are performed by light in the multimode fiber and the high dimensional spatial-spectral information at the fiber output is optically programmed before it reaches the camera. We then used a digital computer to classify the programmed output of the multi-mode fiber using a simple, single layer network. When combining front-end programming and the proposed spatial-spectral programming, we were able to achieve 89.9% classification accuracy on the dataset consisting of chest X-ray images from COVID-19 patients. At the same time, we obtained a decrease of 99% in the number of tunable parameters compared to an equivalently performing digital neural network. These results show that the performance of programmed SOLO is comparable with cutting-edge electronic computing platforms, albeit with a much-reduced number of electronic operations.

physics.optics↗
arXiv2024

Assisted Learning for Organizations with Limited Imbalanced Data

In the era of big data, many big organizations are integrating machine learning into their work pipelines to facilitate data analysis. However, the performance of their trained models is often restricted by limited and imbalanced data available to them. In this work, we develop an assisted learning framework for assisting organizations to improve their learning performance. The organizations have sufficient computation resources but are subject to stringent data-sharing and collaboration policies. Their limited imbalanced data often cause biased inference and sub-optimal decision-making. In assisted learning, an organizational learner purchases assistance service from an external service provider and aims to enhance its model performance within only a few assistance rounds. We develop effective stochastic training algorithms for both assisted deep learning and assisted reinforcement learning. Different from existing distributed algorithms that need to frequently transmit gradients or models, our framework allows the learner to only occasionally share information with the service provider, but still obtain a model that achieves near-oracle performance as if all the data were centralized.

cs.LG↗
arXiv2024

Transfer Learning-Enhanced Instantaneous Multi-Person Indoor Localization by CSI

Passive indoor localization, integral to smart buildings, emergency response, and indoor navigation, has traditionally been limited by a focus on single-target localization and reliance on multi-packet CSI. We introduce a novel Multi-target loss, notably enhancing multi-person localization. Utilizing this loss function, our instantaneous CSI-ResNet achieves an impressive 99.21% accuracy at 0.6m precision with single-timestamp CSI. A preprocessing algorithm is implemented to counteract WiFi-induced variability, thereby augmenting robustness. Furthermore, we incorporate Nuclear Norm-Based Transfer Pre-Training, ensuring adaptability in diverse environments, which provides a new paradigm for indoor multi-person localization. Additionally, we have developed an extensive dataset, surpassing existing ones in scope and diversity, to underscore the efficacy of our method and facilitate future fingerprint-based localization research.

eess.SP↗
arXiv2024

EDTC: enhance depth of text comprehension in automated audio captioning

Modality discrepancies have perpetually posed significant challenges within the realm of Automated Audio Captioning (AAC) and across all multi-modal domains. Facilitating models in comprehending text information plays a pivotal role in establishing a seamless connection between the two modalities of text and audio. While recent research has focused on closing the gap between these two modalities through contrastive learning, it is challenging to bridge the difference between both modalities using only simple contrastive loss. This paper introduces Enhance Depth of Text Comprehension (EDTC), which enhances the model's understanding of text information from three different perspectives. First, we propose a novel fusion module, FUSER, which aims to extract shared semantic information from different audio features through feature fusion. We then introduced TRANSLATOR, a novel alignment module designed to align audio features and text features along the tensor level. Finally, the weights are updated by adding momentum to the twin structure so that the model can learn information about both modalities at the same time. The resulting method achieves state-of-the-art performance on AudioCaps datasets and demonstrates results comparable to the state-of-the-art on Clotho datasets.

cs.SD↗
arXiv2024

An Empirical Study of Data Ability Boundary in LLMs' Math Reasoning

Large language models (LLMs) are displaying emergent abilities for math reasoning tasks,and there is a growing attention on enhancing the ability of open-source LLMs through supervised fine-tuning (SFT).In this paper, we aim to explore a general data strategy for supervised data to help optimize and expand math reasoning ability.Firstly, we determine the ability boundary of reasoning paths augmentation by identifying these paths' minimal optimal set.Secondly, we validate that different abilities of the model can be cumulatively enhanced by Mix of Minimal Optimal Sets of corresponding types of data, while our models MMOS achieve SOTA performance on series base models under much lower construction costs.Besides, we point out GSM-HARD is not really hard and today's LLMs no longer lack numerical robustness.Also, we provide an Auto Problem Generator for robustness testing and educational applications.Our code and data are publicly available at https://github.com/cyzhh/MMOS.

cs.CL↗
arXiv2024

Brain-Inspired Two-Stage Approach: Enhancing Mathematical Reasoning by Imitating Human Thought Processes

Although large language models demonstrate emergent abilities in solving math word problems, there is a challenging task in complex multi-step mathematical reasoning tasks. To improve model performance on mathematical reasoning tasks, previous work has conducted supervised fine-tuning on open-source models by improving the quality and quantity of data. In this paper, we propose a novel approach, named Brain, to imitate human thought processes to enhance mathematical reasoning abilities, using the Frontal Lobe Model to generate plans, and then employing the Parietal Lobe Model to generate code and execute to obtain answers. First, we achieve SOTA performance in comparison with Code LLaMA 7B based models through this method. Secondly, we find that plans can be explicitly extracted from natural language, code, or formal language. Our code and data are publicly available at https://github.com/cyzhh/Brain.

cs.CL↗
arXiv2024

Periodic Schrödinger map flow on Kähler manifolds

Wei-Yue Ding \cite{Ding 2002} proposeed a proposition about Schrödinger map flow in 2002 International Congress of Mathematicians in Beijing, which is called Wei-Yue Ding conjecture by Rodnianski-Rubinstein-Staffilani \cite{Rodnianski 2009}. They proved \cite{Rodnianski 2009} that Schrödinger map flow for maps from the real line into Kähler manifolds and for maps from the circle into Riemann surfaces is globally well-posed which is the first significant advance in this conjecture by translating the Schrödinger map flow into nonlinear Schrödinger-type equations or (systems) and partially solved this conjecture. In this article, we will derive a new div-curl type lemma and combined it with energy and ``momentum" balance law to get some space-time estimates. Based on this, we prove the Schrödinger map flow for maps from the circle into Kähler manifolds is globally regular. So far, the Wei-Yue Ding's conjecture has been completely solved.

math.AP↗
arXiv2024

Memory-Assisted Sub-Prototype Mining for Universal Domain Adaptation

Universal domain adaptation aims to align the classes and reduce the feature gap between the same category of the source and target domains. The target private category is set as the unknown class during the adaptation process, as it is not included in the source domain. However, most existing methods overlook the intra-class structure within a category, especially in cases where there exists significant concept shift between the samples belonging to the same category. When samples with large concept shift are forced to be pushed together, it may negatively affect the adaptation performance. Moreover, from the interpretability aspect, it is unreasonable to align visual features with significant differences, such as fighter jets and civil aircraft, into the same category. Unfortunately, due to such semantic ambiguity and annotation cost, categories are not always classified in detail, making it difficult for the model to perform precise adaptation. To address these issues, we propose a novel Memory-Assisted Sub-Prototype Mining (MemSPM) method that can learn the differences between samples belonging to the same category and mine sub-classes when there exists significant concept shift between them. By doing so, our model learns a more reasonable feature space that enhances the transferability and reflects the inherent differences among samples annotated as the same category. We evaluate the effectiveness of our MemSPM method over multiple scenarios, including UniDA, OSDA, and PDA. Our method achieves state-of-the-art performance on four benchmarks in most cases.

cs.CV↗
arXiv2024

Publication bias adjustment in network meta-analysis: an inverse probability weighting approach using clinical trial registries

Network meta-analysis (NMA) is a useful tool to compare multiple interventions simultaneously in a single meta-analysis, it can be very helpful for medical decision making when the study aims to find the best therapy among several active candidates. However, the validity of its results is threatened by the publication bias issue. Existing methods to handle the publication bias issue in the standard pairwise meta-analysis are hard to extend to this area with the complicated data structure and the underlying assumptions for pooling the data. In this paper, we aimed to provide a flexible inverse probability weighting (IPW) framework along with several t-type selection functions to deal with the publication bias problem in the NMA context. To solve these proposed selection functions, we recommend making use of the additional information from the unpublished studies from multiple clinical trial registries. A comprehensive numerical study and a real example showed that our methodology can help obtain more accurate estimates and higher coverage probabilities, and improve other properties of an NMA (e.g., ranking the interventions).

stat.ME↗
arXiv2024

Enhancing In-context Learning via Linear Probe Calibration

In-context learning (ICL) is a new paradigm for natural language processing that utilizes Generative Pre-trained Transformer (GPT)-like models. This approach uses prompts that include in-context demonstrations to generate the corresponding output for a new query input. However, applying ICL in real cases does not scale with the number of samples, and lacks robustness to different prompt templates and demonstration permutations. In this paper, we first show that GPT-like models using ICL result in unreliable predictions based on a new metric based on Shannon entropy. Then, to solve this problem, we propose a new technique called the Linear Probe Calibration (LinC), a method that calibrates the model's output probabilities, resulting in reliable predictions and improved performance, while requiring only minimal additional samples (as few as five labeled data samples). LinC significantly enhances the ICL test performance of GPT models on various benchmark datasets, with an average improvement of up to 21%, and up to a 50% improvement in some cases, and significantly boosts the performance of PEFT methods, especially in the low resource regime. Moreover, LinC achieves lower expected calibration error, and is highly robust to varying label proportions, prompt templates, and demonstration permutations. Our code is available at \url{https://github.com/mominabbass/LinC}.

cs.CL↗
arXiv2024

Nonparametric worst-case bounds for publication bias on the summary receiver operating characteristic curve

The summary receiver operating characteristic (SROC) curve has been recommended as one important meta-analytical summary to represent the accuracy of a diagnostic test in the presence of heterogeneous cutoff values. However, selective publication of diagnostic studies for meta-analysis can induce publication bias (PB) on the estimate of the SROC curve. Several sensitivity analysis methods have been developed to quantify PB on the SROC curve, and all these methods utilize parametric selection functions to model the selective publication mechanism. The main contribution of this article is to propose a new sensitivity analysis approach that derives the worst-case bounds for the SROC curve by adopting nonparametric selection functions under minimal assumptions. The estimation procedures of the worst-case bounds use the Monte Carlo method to obtain the SROC curves along with the corresponding area under the curves in the worst case where the maximum possible PB under a range of marginal selection probabilities is considered. We apply the proposed method to a real-world meta-analysis to show that the worst-case bounds of the SROC curves can provide useful insights for discussing the robustness of meta-analytical findings on diagnostic test accuracy.

stat.ME↗
arXiv2024

On Unbalanced Optimal Transport: Gradient Methods, Sparsity and Approximation Error

We study the Unbalanced Optimal Transport (UOT) between two measures of possibly different masses with at most $n$ components, where the marginal constraints of standard Optimal Transport (OT) are relaxed via Kullback-Leibler divergence with regularization factor $τ$. Although only Sinkhorn-based UOT solvers have been analyzed in the literature with the iteration complexity of ${O}\big(\tfrac{τ\log(n)}{\varepsilon} \log\big(\tfrac{\log(n)}{\varepsilon}\big)\big)$ and per-iteration cost of $O(n^2)$ for achieving the desired error $\varepsilon$, their positively dense output transportation plans strongly hinder the practicality. On the other hand, while being vastly used as heuristics for computing UOT in modern deep learning applications and having shown success in sparse OT problem, gradient methods applied to UOT have not been formally studied. In this paper, we propose a novel algorithm based on Gradient Extrapolation Method (GEM-UOT) to find an $\varepsilon$-approximate solution to the UOT problem in $O\big( κ\log\big(\frac{τn}{\varepsilon}\big) \big)$ iterations with $\widetilde{O}(n^2)$ per-iteration cost, where $κ$ is the condition number depending on only the two input measures. Our proof technique is based on a novel dual formulation of the squared $\ell_2$-norm UOT objective, which fills the lack of sparse UOT literature and also leads to a new characterization of approximation error between UOT and OT. To this end, we further present a novel approach of OT retrieval from UOT, which is based on GEM-UOT with fine tuned $τ$ and a post-process projection step. Extensive experiments on synthetic and real datasets validate our theories and demonstrate the favorable performance of our methods in practice.

math.OC↗
arXiv2024

A Pure Integral-Type PLL with a Damping Branch to Enhance the Stability of Grid-Tied Inverter under Weak Grids

In a phase-locked loop (PLL) synchronized inverter, due to the strong nonlinear coupling between the PLL's parame-ters and the operation power angle, the equivalent damping coefficient will quickly deteriorate while the power angle is close to 90° under an ultra-weak grid, which causes the synchronous instability. To address this issue, in this letter, a pure integral-type phase-locked loop (IPLL) with a damping branch is proposed to replace the traditional PI-type PLL. The equivalent damping coefficient of an IPLL-synchronized inverter is decoupled with the steady-state power angle. As a result, the IPLL-synchronized inverter can stably operate under an ultra-weak grid when the equilibrium point exists. Finally, time-domain simulation results verify the effectiveness and correctness of the proposed IPLL.

eess.SY↗
arXiv2023

Balanced SNR-Aware Distillation for Guided Text-to-Audio Generation

Diffusion models have demonstrated promising results in text-to-audio generation tasks. However, their practical usability is hindered by slow sampling speeds, limiting their applicability in high-throughput scenarios. To address this challenge, progressive distillation methods have been effective in producing more compact and efficient models. Nevertheless, these methods encounter issues with unbalanced weights at both high and low noise levels, potentially impacting the quality of generated samples. In this paper, we propose the adaptation of the progressive distillation method to text-to-audio generation tasks and introduce the Balanced SNR-Aware~(BSA) method, an enhanced loss-weighting mechanism for diffusion distillation. The BSA method employs a balanced approach to weight the loss for both high and low noise levels. We evaluate our proposed method on the AudioCaps dataset and report experimental results showing superior performance during the reverse diffusion process compared to previous distillation methods with the same number of sampling steps. Furthermore, the BSA method allows for a significant reduction in sampling steps from 200 to 25, with minimal performance degradation when compared to the original teacher models.

cs.SD↗
arXiv2023

Physical space approach to wave equation bilinear estimates revisit

In the paper by Klainerman, Rodnianski and Tao \cite{Klainerman-Rodnianski-Tao}, they give a physical space proof to a classical result of Klainerman and Machedon \cite{Klainerman-Machedon} for the bilinear space-time estimates of null forms. In this paper, we shall give an alternative and very simple physical space proof of the same bilinear estimates by applying div-curl type lemma of Zhou \cite{Zhou} and Wang-Zhou \cite{Wang-Zhou-1}, \cite{Wang-Zhou-2}. As far as we known, the later development of wave maps \cite{Sterbenz-1}, \cite{Sterbenz-2}, \cite{Tao-1}, \cite{Tao-2}, \cite{Tataru-1}, \cite{Tataru-2}, and the proof of bounded curvature conjecture \cite{Klainerman-Rodnianski-Szeftel-1}, \cite{Klainerman-Rodnianski-Szeftel-2} rely on basic idea of Klainerman and Machedon \cite{Klainerman-Machedon} as well as Klainerman, Rodnianski and Tao \cite{Klainerman-Rodnianski-Tao}.

math.AP↗
arXiv2023

Possible chiral spin liquid state in the $S=1/2$ kagome Heisenberg model

The nature of the ground state for the $S = 1/2$ kagome Heisenberg antiferromagnet (KHAF) has been elusive. We revisit this challenging problem and provide numerical evidence that its ground state might be a chiral spin liquid. Combining the density matrix renormalization group method and analytical analyses, we demonstrate that the previously observed chiral spin liquid phase in the KHAF with longer-range couplings is stable in a broader region. We characterize the nature of the ground state by computing energy derivatives, revealing ground-state degeneracy arising from spontaneous breaking of time-reversal symmetry, and targeting the semion sector. We further investigate the phase diagram in the vicinity of the KHAF and observe a $\sqrt{3}\times\sqrt{3}$ magnetically ordered phase and two valence-bond crystal phases.

cond-mat.str-el↗
arXiv2023

A simple sensitivity analysis method for unmeasured confounders via linear programming with estimating equation constraints

In estimating the average treatment effect in observational studies, the influence of confounders should be appropriately addressed. To this end, the propensity score is widely used. If the propensity scores are known for all the subjects, bias due to confounders can be adjusted by using the inverse probability weighting (IPW) by the propensity score. Since the propensity score is unknown in general, it is usually estimated by the parametric logistic regression model with unknown parameters estimated by solving the score equation under the strongly ignorable treatment assignment (SITA) assumption. Violation of the SITA assumption and/or misspecification of the propensity score model can cause serious bias in estimating the average treatment effect. To relax the SITA assumption, the IPW estimator based on the outcome-dependent propensity score has been successfully introduced. However, it still depends on the correctly specified parametric model and its identification. In this paper, we propose a simple sensitivity analysis method for unmeasured confounders. In the standard practice, the estimating equation is used to estimate the unknown parameters in the parametric propensity score model. Our idea is to make inference on the average causal effect by removing restrictive parametric model assumptions while still utilizing the estimating equation. Using estimating equations as constraints, which the true propensity scores asymptotically satisfy, we construct the worst-case bounds for the average treatment effect with linear programming. Different from the existing sensitivity analysis methods, we construct the worst-case bounds with minimal assumptions. We illustrate our proposal by simulation studies and a real-world example.

stat.ME↗