arXiv subjects

Yiran Liu

Publications and source records attributed to Yiran Liu.

26 records · Page 2Linked to original sources

arXiv2023

Does Debiasing Inevitably Degrade the Model Performance

Gender bias in language models has attracted sufficient attention because it threatens social justice. However, most of the current debiasing methods degraded the model's performance on other tasks while the degradation mechanism is still mysterious. We propose a theoretical framework explaining the three candidate mechanisms of the language model's gender bias. We use our theoretical framework to explain why the current debiasing methods cause performance degradation. We also discover a pathway through which debiasing will not degrade the model performance. We further develop a causality-detection fine-tuning approach to correct gender bias. The numerical experiment demonstrates that our method is able to lead to double dividends: partially mitigating gender bias while avoiding performance degradation.

cs.CL↗

arXiv2023

Swin Transformer coupling CNNs Makes Strong Contextual Encoders for VHR Image Road Extraction

Accurately segmenting roads is challenging due to substantial intra-class variations, indistinct inter-class distinctions, and occlusions caused by shadows, trees, and buildings. To address these challenges, attention to important texture details and perception of global geometric contextual information are essential. Recent research has shown that CNN-Transformer hybrid structures outperform using CNN or Transformer alone. While CNN excels at extracting local detail features, the Transformer naturally perceives global contextual information. In this paper, we propose a dual-branch network block named ConSwin that combines ResNet and SwinTransformers for road extraction tasks. This ConSwin block harnesses the strengths of both approaches to better extract detailed and global features. Based on ConSwin, we construct an hourglass-shaped road extraction network and introduce two novel connection structures to better transmit texture and structural detail information to the decoder. Our proposed method outperforms state-of-the-art methods on both the Massachusetts and CHN6-CUG datasets in terms of overall accuracy, IOU, and F1 indicators. Additional experiments validate the effectiveness of our proposed module, while visualization results demonstrate its ability to obtain better road representations.

cs.CV↗

arXiv2023

Knowledge is Power: Understanding Causality Makes Legal judgment Prediction Models More Generalizable and Robust

Legal Judgment Prediction (LJP), aiming to predict a judgment based on fact descriptions according to rule of law, serves as legal assistance to mitigate the great work burden of limited legal practitioners. Most existing methods apply various large-scale pre-trained language models (PLMs) finetuned in LJP tasks to obtain consistent improvements. However, we discover the fact that the state-of-the-art (SOTA) model makes judgment predictions according to irrelevant (or non-casual) information. The violation of rule of law not only weakens the robustness and generalization ability of models but also results in severe social problems like discrimination. In this paper, we use causal structural models (SCMs) to theoretically analyze how LJP models learn to make decisions and why they can succeed in passing the traditional testing paradigm without learning causality. According to our analysis, we provide two solutions intervening on data and model by causality, respectively. In detail, we first distinguish non-causal information by applying the open information extraction (OIE) technique. Then, we propose a method named the Causal Information Enhanced SAmpling Method (CIESAM) to eliminate the non-causal information from data. To validate our theoretical analysis, we further propose another method using our proposed Causality-Aware Self-Attention Mechanism (CASAM) to guide the model to learn the underlying causality knowledge in legal texts. The confidence of CASAM in learning causal information is higher than that of CIESAM. The extensive experimental results show that both our proposed methods achieve state-of-the-art (SOTA) performance on three commonly used legal-specific datasets. The stronger performance of CASAM further demonstrates that causality is the key to the robustness and generalization ability of models.

cs.CL↗

arXiv2022

Momentum Calibration for Text Generation

The input and output of most text generation tasks can be transformed to two sequences of tokens and they can be modeled using sequence-to-sequence learning modeling tools such as Transformers. These models are usually trained by maximizing the likelihood the output text sequence and assumes the input sequence and all gold preceding tokens are given during training, while during inference the model suffers from the exposure bias problem (i.e., it only has access to its previously predicted tokens rather gold tokens during beam search). In this paper, we propose MoCa ({\bf Mo}mentum {\bf Ca}libration) for text generation. MoCa is an online method that dynamically generates slowly evolving (but consistent) samples using a momentum moving average generator with beam search and MoCa learns to align its model scores of these samples with their actual qualities. Experiments on four text generation datasets (i.e., CNN/DailyMail, XSum, SAMSum and Gigaword) show MoCa consistently improves strong pre-trained transformers using vanilla fine-tuning and we achieve the state-of-the-art results on CNN/DailyMail and SAMSum datasets.

cs.CL↗

arXiv2022

Chern numbers of topological phonon band crossing determined with inelastic neutron scattering

Topological invariants in the band structure, such as Chern numbers, are crucial for the classification of topological matters and dictate the occurrence of exotic properties, yet their direct spectroscopic determination has been largely limited to electronic bands. Here, we use inelastic neutron scattering in conjunction with ab initio calculations to identify a variety of topological phonon band crossings in MnSi and CoSi single crystals. We find a distinct relation between the Chern numbers of a band-crossing node and the scattering intensity modulation in momentum space around the node. Given sufficiently high resolution, our method can be used to determine arbitrarily large Chern numbers of topological phonon band-crossing nodes.

cond-mat.mes-hall↗

arXiv2022

Magnetic molecular orbitals in MnSi

A large body of knowledge about magnetism is attained from models of interacting spins, which usually reside on magnetic ions. Proposals beyond the ionic picture are uncommon and seldom verified by direct observations in conjunction with microscopic theory. Here, using inelastic neutron scattering to study the itinerant near-ferromagnet MnSi, we find that the system's fundamental magnetic units are interconnected, extended molecular orbitals consisting of three Mn atoms each, rather than individual Mn atoms. This result is further corroborated by magnetic Wannier orbitals obtained by ab initio calculations. It contrasts the ionic picture with a concrete example, and presents a novel regime of the spin waves where the wavelength is comparable to the spatial extent of the molecular orbitals. Our discovery brings important insights into not only the magnetism of MnSi, but also a broad range of magnetic quantum materials where structural symmetry, electron itinerancy and correlations act in concert.

cond-mat.str-el↗

arXiv2021

Diffusion Approximations for $\text{Cox}/G_t/\infty$ Queues In A Fast Oscillatory Random Environment

We study infinite server queues driven by Cox processes in a fast oscillatory random environment. While exact performance analysis is difficult, we establish diffusion approximations to the (re-scaled) number-in-system process by proving functional central limit theorems (FCLTs) using a stochastic homogenization framework. This framework permits the establishment of quenched and annealed limits in a unified manner. At the quantitative level, we identity two parameter regimes, termed subcritical and supercritical indicating the relative dominance between the two underlying stochasticities driving our system: the randomness in the arrival intensity and that in the serivce times. We show that while quenched FCLTs can only be established in the subcritical regime, annealed FCLTs can be proved in both cases. Furthermore, the limiting diffusions in the annealed FCLTs display qualitatively different diffusivity properties in the two regimes, even though the stochastic primitives are identical. In particular, when the service time distribution is heavy-tailed, the diffusion is sub- and super-diffusive in the sub- and super-critical cases. The results illustrate intricate interactions between the underlying driving forces of our system.

math.PR↗

arXiv2020

Infinite server queues in a random fast oscillatory environment

In this paper, we consider a $G_t/G_t/\infty$ infinite server queueing model in a random environment. More specifically, the arrival rate in our server is modeled as a highly fluctuating stochastic process, which arguably takes into account some small time scale variations often observed in practice. We show a homogenization property for this system, which yields an approximation by a $M_t/G_t/\infty$ queue with modified parameters. Our limiting results include the description of the number of active servers, the total accumulated input and the solution of the storage equation. Hence in the fast oscillatory context under consideration, we show how the queuing system in a random environment can be approximated by a more classical Markovian system.

math.PR↗