arXiv subjects

Neha Verma

Publications and source records attributed to Neha Verma.

At least 19 recordsLinked to original sources

arXiv2026

There is No Theoretical Curse of Multilinguality For Embedding Space Structure

A central goal of multilingual NLP is to achieve high monolingual performance per language and cross-lingual alignment for large-scale language coverage with a multilingual model. The curse of multilinguality describes the phenomenon of degradation in multilingual model performance as we increase language coverage, posing a threat to the above goal. This paper asks whether multilingual embedding spaces are inherently incapable of achieving perfect multilinguality without a prohibitive increase in required capacity. We first formalize the goal of "perfect multilinguality", embodied in two multilinguality conditions. We then prove that the minimum dimensionality required for perfect multilinguality grows only logarithmically in the number of languages. That is, we show that there is no theoretical curse of multilinguality for embedding space structure. This suggests that the empirical curse of multilinguality is a result of real world data and training conditions. We back this understanding with a small-scale empirical study. Our paper provides the first theoretical and intrinsic perspective on the curse of multilinguality.

cs.CL↗

arXiv2026

ORBIT: Preserving Foundational Language Capabilities in GenRetrieval via Origin-Regulated Merging

Despite the rapid advancements in large language model (LLM) development, fine-tuning them for specific tasks often results in the catastrophic forgetting of their general, language-based reasoning abilities. This work investigates and addresses this challenge in the context of the Generative Retrieval (GenRetrieval) task. During GenRetrieval fine-tuning, we find this forgetting occurs rapidly and correlates with the distance between the fine-tuned and original model parameters. Given these observations, we propose ORBIT, a novel approach that actively tracks the distance between fine-tuned and initial model weights, and uses a weight averaging strategy to constrain model drift during GenRetrieval fine-tuning when this inter-model distance exceeds a maximum threshold. Our results show that ORBIT retains substantial text and retrieval performance by outperforming both common continual learning baselines and related regularization methods that also employ weight averaging.

cs.CL↗

arXiv2026

Evaluating Large Language Models' Responses to Sexual and Reproductive Health Queries in Nepali

As Large Language Models (LLMs) become integrated into daily life, they are increasingly used for personal queries, including Sexual and Reproductive Health (SRH), allowing users to chat anonymously without fear of judgment. However, current evaluation methods primarily focus on accuracy, often for objective queries in high-resource languages, and lack criteria to assess usability and safety, especially for low-resource languages and culturally sensitive domains like SRH. This paper introduces LLM Evaluation Framework (LEAF), that conducts assessments across multiple criteria: accuracy, language, usability gaps (including relevance, adequacy, and cultural appropriateness), and safety gaps (safety, sensitivity, and confidentiality). Using the LEAF framework, we assessed 14K SRH queries in Nepali from over 9K users. Responses were manually annotated by SRH experts according to the framework. Results revealed that only 35.1% of the responses were "proper", meaning they were accurate, adequate and had no major usability or safety related gaps. Insights include differences in performance between ChatGPT versions, such as similar accuracy but varying usability and safety aspects. This evaluation highlights significant limitations of current LLMs and underscores the need for improvement. The LEAF Framework is adaptable across domains and languages, particularly where usability and safety are critical, offering a pathway to better address sensitive topics.

cs.CL↗

arXiv2025

DOTResize: Reducing LLM Width via Discrete Optimal Transport-based Neuron Merging

Structured pruning methods designed for Large Language Models (LLMs) generally focus on identifying and removing the least important components to optimize model size. However, in this work, we question this prevalent approach by instead exploring how to recombine information from structures designated for pruning back into the reduced model. We specifically focus on neuron width reduction, and frame this problem as a Discrete Optimal Transport problem, and propose DOTResize, a novel Transformer compression method that uses optimal transport theory to transform and compress model width. To ensure applicability within the Transformer architecture, we motivate and incorporate necessary entropic regularization and matrix factorization techniques into the transportation maps produced by our method. Unlike pruning-based approaches which discard neurons based on importance measures, DOTResize re-projects the entire neuron width, allowing the retention and redistribution of useful signal across the reduced layer. Empirical results show that compared to simple or state-of-the-art neuron width-pruning techniques, DOTResize serves as a useful add-on to pruning, while achieving measurable reductions in real-world computational cost.

cs.LG↗

arXiv2025

Merging Feed-Forward Sublayers for Compressed Transformers

With the rise and ubiquity of larger deep learning models, the need for high-quality compression techniques is growing in order to deploy these models widely. The sheer parameter count of these models makes it difficult to fit them into the memory constraints of different hardware. In this work, we present a novel approach to model compression by merging similar parameter groups within a model, rather than pruning away less important parameters. Specifically, we select, align, and merge separate feed-forward sublayers in Transformer models, and test our method on language modeling, image classification, and machine translation. With our method, we demonstrate performance comparable to the original models while combining more than a third of model feed-forward sublayers, and demonstrate improved performance over a strong layer-pruning baseline. For instance, we can remove over 21% of total parameters from a Vision Transformer, while maintaining 99% of its original performance. Additionally, we observe that some groups of feed-forward sublayers exhibit high activation similarity, which may help explain their surprising mergeability.

cs.CL↗

arXiv2024

On estimation of Hankel determinants for certain class of starlike functions

In the present study, we consider two subclasses starlike and convex functions, denoted by $\mathcal{S}_{\mathcal{B}}^{*}$ and $\mathcal{C}_{\mathcal{B}}$ respectively, associated with a bean-shaped domain. Further, we estimate certain sharp initial coefficients, as well as second, third and fourth-order Hankel determinants for functions belonging to the class $\mathcal{S}_{\mathcal{B}}^{*}$. Additionally, we compute sharp second and third-order Hankel determinants for functions belonging to the $\mathcal{C}_{\mathcal{B}}$ class.

math.CV↗

arXiv2024

Higher order differential subordinations for certain starlike functions

In this paper, we employ a novel second and third-order differential subordination technique to establish the sufficient conditions for functions to belong to the classes $\mathcal{S}^*_s$ and $\mathcal{S}^*_{\rho}$, where $\mathcal{S}^*_s$ is the set of all normalized analytic functions $f$ satisfying $ zf'(z)/f(z)\prec 1+\sin z$ and $\mathcal{S}^*_{\rho}$ is the set of all normalized analytic functions $f$ satisfying $ zf'(z)/f(z)\prec 1+\sinh^{-1} z$.

math.CV↗

arXiv2024

Second and Third order differential subordination for exponential function

This article presents several findings regarding second and third-order differential subordination of the form: $$ p(z)+\gamma_1 zp'(z)+\gamma_2 z^2p''(z)\prec h(z)\implies p(z)\prec e^z $$ and $$ p(z)+\gamma_1 zp'(z)+\gamma_2 z^2p''(z)+\gamma_3 z^3p'''(z)\prec h(z)\implies p(z)\prec e^z. $$ Here, $\gamma_1$, $\gamma_2$, and $\gamma_3$ represent positive real numbers, and various selections of $h(z)$ are explored within the context of the class $\mathcal{S}^{*}_{e} := \{f \in \mathcal{A} : zf'(z)/f(z) \prec e^z\}$, which denotes the class of starlike functions associated with the exponential function.

math.CV↗

arXiv2024

Merging Text Transformer Models from Different Initializations

Recent work on permutation-based model merging has shown impressive low- or zero-barrier mode connectivity between models from completely different initializations. However, this line of work has not yet extended to the Transformer architecture, despite its dominant popularity in the language domain. Therefore, in this work, we investigate the extent to which separate Transformer minima learn similar features, and propose a model merging technique to investigate the relationship between these minima in the loss landscape. The specifics of the architecture, like its residual connections, multi-headed attention, and discrete, sequential input, require specific interventions in order to compute model permutations that remain within the same functional equivalence class. In merging these models with our method, we consistently find lower loss barriers between minima compared to model averaging, across models trained on a masked-language modeling task or fine-tuned on a language understanding benchmark. Our results show that the minima of these models are less sharp and isolated than previously understood, and provide a basis for future work on merging separately trained Transformer models.

cs.CL↗

arXiv2023

On a Subclass of Starlike Functions Associated with a Strip Domain

In the present investigation, we introduce a new subclass of starlike functions defined by $\mathcal{S}^{*}_{\tau}:=\{f\in \mathcal{A}:zf'(z)/f(z) \prec 1+\arctan z=:\tau(z)\}$, where $\tau(z)$ maps the unit disk $\mathbb {D}:= \{z\in \mathbb{C}:|z|<1\}$ onto a strip domain. We derive structural formulae, growth, and distortion theorems for $\mathcal{S}^{*}_{\tau}$. Also, inclusion relations with some well-known subclasses of $\mathcal{S}$ are established and obtain sharp radius estimates, as well as sharp coefficient bounds for the initial five coefficients and the second and third-order Hankel determinants of $\mathcal{S}^{*}_{\tau}$.

math.CV↗

arXiv2023

Differential Subordination of Certain Class of Starlike Functions

This paper presents several results concerning second and third-order differential subordination for the class $\mathcal{S}^{*}_{e}:=\{f\in \mathcal{A}:zf'(z)/f(z)\prec e^z\}$, which represents the class of starlike functions associated with exponential function.

math.CV↗

arXiv2023

Exploring Representational Disparities Between Multilingual and Bilingual Translation Models

Multilingual machine translation has proven immensely useful for both parameter efficiency and overall performance across many language pairs via complete multilingual parameter sharing. However, some language pairs in multilingual models can see worse performance than in bilingual models, especially in the one-to-many translation setting. Motivated by their empirical differences, we examine the geometric differences in representations from bilingual models versus those from one-to-many multilingual models. Specifically, we compute the isotropy of these representations using intrinsic dimensionality and IsoScore, in order to measure how the representations utilize the dimensions in their underlying vector space. Using the same evaluation data in both models, we find that for a given language pair, its multilingual model decoder representations are consistently less isotropic and occupy fewer dimensions than comparable bilingual model decoder representations. Additionally, we show that much of the anisotropy in multilingual decoder representations can be attributed to modeling language-specific information, therefore limiting remaining representational capacity.

cs.CL↗

arXiv2023

Multilingual Pixel Representations for Translation and Effective Cross-lingual Transfer

We introduce and demonstrate how to effectively train multilingual machine translation models with pixel representations. We experiment with two different data settings with a variety of language and script coverage, demonstrating improved performance compared to subword embeddings. We explore various properties of pixel representations such as parameter sharing within and across scripts to better understand where they lead to positive transfer. We observe that these properties not only enable seamless cross-lingual transfer to unseen scripts, but make pixel representations more data-efficient than alternatives such as vocabulary expansion. We hope this work contributes to more extensible multilingual models for all languages and scripts.

cs.CL↗

arXiv2022

On sharp third Hankel determinant for certain starlike functions

In this paper, we provide an estimation for the sharp bound of the third Hankel determinant of starlike functions of order $\alpha$, where $\alpha$ ranges in the interval $[0, 1/6]\cup \{1/2\}$ and thereby extending the result of Rath et al. (Complex Anal Oper Theory: No. 65, 16(5), 8 pp 2022).

math.CV↗

arXiv2022

IsoVec: Controlling the Relative Isomorphism of Word Embedding Spaces

The ability to extract high-quality translation dictionaries from monolingual word embedding spaces depends critically on the geometric similarity of the spaces -- their degree of "isomorphism." We address the root-cause of faulty cross-lingual mapping: that word embedding training resulted in the underlying spaces being non-isomorphic. We incorporate global measures of isomorphism directly into the Skip-gram loss function, successfully increasing the relative isomorphism of trained word embedding spaces and improving their ability to be mapped to a shared cross-lingual space. The result is improved bilingual lexicon induction in general data conditions, under domain mismatch, and with training algorithm dissimilarities. We release IsoVec at https://github.com/kellymarchisio/isovec.

cs.CL↗

arXiv2022

Certain Coefficient Problems of $\mathcal{S}_{e}^{*}$ and $\mathcal{C}_{e}$

In this current study, we consider the classes $\mathcal{S}^{*}_{e}$ and $\mathcal{C}_e$ to obtain sharp bounds for the third Hankel determinant for functions within these classes. Additionally, we provide estimates for the sixth and seventh coefficients while establishing the fourth-order Hankel determinant as well.

math.CV↗

arXiv2022

Coefficient problems for starlike functions associated with a petal shaped domain

In the present investigation, we consider a subclass of starlike functions associated with a petal shaped domain, recently introduced and defined by $$\mathcal{S}^{*}_{\rho}:=\{f\in \mathcal{A}:zf'(z)/f(z) \prec 1+\sinh^{-1} z\}.$$ We establish certain coefficient related problems such as sharp first five coefficient bounds along with sharp second and third order Hankel determinants for $\mathcal{S}^{*}_{\rho}$. Also, sixth and seventh coefficient bounds are estimated to obtain the fourth Hankel determinant bound for the same class.

math.CV↗

arXiv2022

A Conjecture on $H_3(1)$ For Certain Starlike Functions

We prove a conjecture concerning the third Hankel determinant, proposed in ``Anal. Math. Phys., https://doi.org/10.1007/s13324-021-00483-7", which states that $|H_3(1)|\leq 1/9$ is sharp for the class $\mathcal{S}_{\wp}^{*}=\{zf'(z)/f(z) \prec \varphi(z):=1+ze^z\}$. In addition, we also establish bounds for sixth and seventh coefficient, and $|H_4(1)|$ for functions in $\mathcal{S}_{\wp}^{*}$. The general bounds for two and three-fold symmetric functions related to the Ma-Minda classes $\mathcal{S}^*(\varphi)$ of starlike functions are also obtained.

math.CV↗