arXiv subjects

Hao-Yuan Chang

Publications and source records attributed to Hao-Yuan Chang.

6 recordsLinked to original sources

arXiv2024

Unitary Multi-Margin BERT for Robust Natural Language Processing

Recent developments in adversarial attacks on deep learning leave many mission-critical natural language processing (NLP) systems at risk of exploitation. To address the lack of computationally efficient adversarial defense methods, this paper reports a novel, universal technique that drastically improves the robustness of Bidirectional Encoder Representations from Transformers (BERT) by combining the unitary weights with the multi-margin loss. We discover that the marriage of these two simple ideas amplifies the protection against malicious interference. Our model, the unitary multi-margin BERT (UniBERT), boosts post-attack classification accuracies significantly by 5.3% to 73.8% while maintaining competitive pre-attack accuracies. Furthermore, the pre-attack and post-attack accuracy tradeoff can be adjusted via a single scalar parameter to best fit the design requirements for the target applications.

cs.CL↗

arXiv2023

Higher derivative couplings of hypermultiplets

We construct the four-derivative supersymmetric extension of $(1,0), 6D$ supergravity coupled to Yang-Mills and hypermultiplets. The hypermultiplet scalars are taken to parametrize the quaternionic projective space $Hp(n)=Sp(n,1)/Sp(n)\times Sp(1)_R$. The hyperscalar kinetic term is not deformed, and the quaternionic Kähler structure and symmetries of $Hp(n)$ are preserved. The result is a three parameter Lagrangian supersymmetric up to first order in these parameters. Considering the case of $Hp(1)$ we compare our result with that obtained from the compactification of $10D$ heterotic supergravity on four-torus, consistently truncated to $N=(1,0)$, in which the hyperscalars parametrize $SO(4,1)/SO(4)$. We find that depending on how $Sp(1) \subset Sp(1,1)$ is embedded in $SO(4)$, the results agree for a specific value of the parameter that governs the higher derivative hypermultiplet couplings.

hep-th↗

arXiv2022

Dualization of Higher Derivative Heterotic Supergravities in $6D$ and $10D$

There exist two four-derivative extensions of $N=(1,0)$ supergravity in six dimensions. A particular combination of them is known to dualize to the analog of the the Bergshoeff-de Roo (BdR) action in $10D$. Here we first show that the two extensions are not related to each other by any field redefinitions. Next, we dualize them separately thereby obtaining a two parameter dual theory. This is done directly at the level of the action, thus avoiding the laborious method of integrating equations of motion of the dualized theory into an action. To explore whether a similar phenomenon exists in $10D$, we study the dualization of the BdR action in $10D$ in detail. We find an obstacle in the separation of the result into a sum of two independent invariants because of the presence of terms which do not lift from $6D$ to $10D$. We also compare the dual of the BdR action with an existing result obtained in superspace. We find that the bosonic actions agree modulo field redefinitions.

hep-th↗

arXiv2021

Dimensional reduction of higher derivative heterotic supergravity

Higher derivative couplings of hypermultiplets to $6D, N=(1,0)$ supergravity are obtained from dimensional reduction of 10D heterotic supergravity that includes order $α'$ higher derivative corrections. Reduction on $T^4$ is followed by a consistent truncation. In the resulting action the hyperscalar fields parametrize the coset $SO(4,4)/(SO(4)\times SO(4))$. While the $SO(4,4)$ symmetry is ensured by Sen's construction based on string field theory, its emergence at the field theory level is a nontrivial phenomenon. A number of field redefinitions in the hypermultiplet sector are required to remove several terms that break the $SO(4)\times SO(4)$ down to its $SO(4)$ diagonal subgroup in the action and the supersymmetry transformation rules. Working with the Lorentz Chern-Simons term modified 3-form field strength, where the spin connection has the 3-form field strength as torsion, is shown to simplify considerably the dimensional reduction.

hep-th↗

arXiv2021

Deep Unitary Convolutional Neural Networks

Deep neural networks can suffer from the exploding and vanishing activation problem, in which the networks fail to train properly because the neural signals either amplify or attenuate across the layers and become saturated. While other normalization methods aim to fix the stated problem, most of them have inference speed penalties in those applications that require running averages of the neural activations. Here we extend the unitary framework based on Lie algebra to neural networks of any dimensionalities, overcoming the major constraints of the prior arts that limit synaptic weights to be square matrices. Our proposed unitary convolutional neural networks deliver up to 32% faster inference speeds and up to 50% reduction in permanent hard disk space while maintaining competitive prediction accuracy.

cs.LG↗

arXiv2021

A Projection Algorithm for the Unitary Weights

Unitary neural networks are promising alternatives for solving the exploding and vanishing activation/gradient problem without the need for explicit normalization that reduces the inference speed. However, they often require longer training time due to the additional unitary constraints on their weight matrices. Here we show a novel algorithm using a backpropagation technique with Lie algebra for computing approximated unitary weights from their pre-trained, non-unitary counterparts. The unitary networks initialized with these approximations can reach the desired accuracies much faster, mitigating their training time penalties while maintaining inference speedups. Our approach will be instrumental in the adaptation of unitary networks, especially for those neural architectures where pre-trained weights are freely available.

cs.LG↗