arXiv subjects

Raymond Ng

Publications and source records attributed to Raymond Ng.

At least 19 recordsLinked to original sources

arXiv2026

GSpyNetTree-O4: an event validation tool used in the fourth LIGO-Virgo-KAGRA observing run

The frequent presence of non-Gaussian transient noise, or glitches, in gravitational-wave detector data can affect gravitational-wave searches, parameter estimation, and downstream analyses. To identify and mitigate transient noise near gravitational-wave candidates in a timely manner, the LIGO-Virgo-KAGRA Collaboration employs the Data Quality Report. In the fourth observing run, GSpyNetTree-O4 was deployed within this framework as a tool for glitch classification and event validation. We describe GSpyNetTree-O4 and the main developments relative to its predecessor, GSpyNetTree. The most important update was a new architecture that allowed the simultaneous identification of glitches and gravitational-wave signals when both were present in the same input. We also expanded and augmented the training set with examples in which simulated gravitational-wave signals overlapped with real glitches, and applied $60\,\mathrm{Hz}$ calibration corrections to better match the data expected during the fourth observing run. On test data, the low-mass, high-mass, and extremely high-mass classifiers identified $97.9\%$, $97.7\%$, and $95.4\%$ of glitches, respectively. Among samples without a glitch, including gravitational-wave-only and No Glitch samples, the classifiers correctly reported no data-quality issues in $97.1\%$, $96.6\%$, and $96.0\%$ of cases, respectively. We further assessed the robustness of GSpyNetTree-O4 on unseen glitch morphologies, a small set of Virgo glitches from the fourth observing run, and different choices of the $Q$-value used to construct the time-frequency inputs. GSpyNetTree-O4 was successfully deployed as a Data Quality Report tool and increased automation in gravitational-wave event validation workflows.

gr-qc↗

arXiv2026

Learning from Lost Provenance: Multiple Instance Learning for Cancer Registry Tumor Group Classification

Modernizing cancer registries with deep learning is opening new opportunities to automate labor-intensive tasks such as the coding of pathology reports. However, progress is constrained by the scarcity of report-level human-annotated training data. Cancer registries generate substantial volumes of expert-assigned labels as a routine product of their operations, but these exist at the patient level and are not linked to the individual pathology reports that informed them, limiting their direct use for training models. We develop an efficient framework for training deep learning classifiers by leveraging these operationally-generated labels without requiring per-report human annotation, demonstrated for tumor group classification at the BC Cancer Registry. We use Attention-Based Multiple Instance Learning (ABMIL) to recover the lost link between patient-level labels and the reports that informed them, leveraging the attention the model places on each report to distil a large, noisily-labeled corpus into a compact, high-quality per-report training dataset. A classifier fine-tuned on a distilled dataset achieved a macro F1 of 0.83, outperforming established baselines across most tumor groups. By turning routine operational labels into high-quality training data without additional annotation or large-scale computing infrastructure, ABMIL offers a practical and accessible route to automating cancer registry workflows.

cs.CL↗

arXiv2026

SEA-LION-Embedding: Open and Reproducible Text Embeddings for Southeast Asia

Text embeddings are fundamental to many downstream applications, making robustness important for real-world NLP. However, most recent state-of-the-art embedding models are not reproducible because they rely on closed or undisclosed training data, and they remain insufficiently robust for Southeast Asian languages. We present SEA-LION-Embedding, a fully open and reproducible text-embedding pipeline for Southeast Asian languages trained only on publicly available data, and use it to study three core factors of robust embedding design: data composition, training objective, and base encoder initialization. SEA-LION-Embedding achieves state-of-the-art results on SEA-BED while enabling systematic and reproducible analysis of robust text embeddings for the region.

cs.CL↗

arXiv2026

Rapid data quality investigations of gravitational-wave events with the Data Quality Report Builder toolkit

We present the Data Quality Report Builder toolkit, DQRbuild, a suite of data quality tools that have been developed to vet gravitational-wave events in preparation for the fourth LIGO-Virgo-KAGRA observing run. We explain the main functionality and the many scientific tests that we support. To validate the performance of the tools included in the toolkit, we run a series of tests on all significant candidates shared as public alerts in the third observing run to compare against what was manually reported using human intervention. We find that these automated tools can now identify 96% of the problems identified by humans during this previous observing run, with a 24% false alarm rate. We conclude with a commentary on the prospects and potential challenges for fully automating the process of vetting the data quality for gravitational-wave events identified in future observing runs.

astro-ph.IM↗

arXiv2026

Adapting Natural Language Processing Models Across Jurisdictions: A pilot Study in Canadian Cancer Registries

Population-based cancer registries depend on pathology reports as their primary diagnostic source, yet manual abstraction is resource-intensive and contributes to delays in cancer data. While transformer-based NLP systems have improved registry workflows, their ability to generalize across jurisdictions with differing reporting conventions remains poorly understood. We present the first cross-provincial evaluation of adapting BCCRTron, a domain-adapted transformer model developed at the British Columbia Cancer Registry, alongside GatorTron, a biomedical transformer model, for cancer surveillance in Canada. Our training dataset consisted of approximately 104,000 and 22,000 de-identified pathology reports from the Newfoundland & Labrador Cancer Registry (NLCR) for Tier 1 (cancer vs. non-cancer) and Tier 2 (reportable vs. non-reportable) tasks, respectively. Both models were fine-tuned using complementary synoptic and diagnosis focused report section input pipelines. Across NLCR test sets, the adapted models maintained high performance, demonstrating transformers pretrained in one jurisdiction can be localized to another with modest fine-tuning. To improve sensitivity, we combined the two models using a conservative OR-ensemble achieving a Tier 1 recall of 0.99 and reduced missed cancers to 24, compared with 48 and 54 for the standalone models. For Tier 2, the ensemble achieved 0.99 recall and reduced missed reportable cancers to 33, compared with 54 and 46 for the individual models. These findings demonstrate that an ensemble combining complementary text representations substantially reduce missed cancers and improve error coverage in cancer-registry NLP. We implement a privacy-preserving workflow in which only model weights are shared between provinces, supporting interoperable NLP infrastructure and a future pan-Canadian foundation model for cancer pathology and registry workflows.

cs.CL↗

arXiv2025

GSpyNetTreeS: a machine learning solution for glitch localization in time and frequency

Data from ground-based gravitational wave detectors are often contaminated by non-Gaussian instrumental artifacts or detector noise transients. Unbiased source property estimation relies on the ability to correctly identify and characterize these artifacts and remove them if necessary. To this end, the LIGO-Virgo-KAGRA Collaboration has implemented candidate vetting for all significant candidates to identify the presence of artifacts and assess the need for mitigation. The current candidate vetting process requires human experts to identify the frequency ranges and the time windows associated with any data quality issues present. Differences in judgment between human experts may cause inconsistency, making results difficult to reproduce across gravitational wave events. We present GSpyNetTreeS, an extension to GSpyNetTree based on the You Only Look Once algorithm, for the automatic detection, classification, and time-frequency localization of detector noise transients. As a proof of concept, we tested GSpyNetTreeS's performance on the data collected by the LIGO detectors during the third observing run for gravitational waves as well as common detector glitch classes included in GSpyNetTree: Blip, Low frequency blip, Low frequency line and Scratchy. We also demonstrated that GSpyNetTreeS is capable of accurately identifying common glitch classes and capturing the frequency and time information associated with detected detector noise transients, establishing its potential as an automatic event validation tool for LIGO-Virgo-KAGRA's observing runs.

astro-ph.IM↗

arXiv2025

SEA-BED: How Do Embedding Models Represent Southeast Asian Languages?

Multilingual text embeddings are often assumed to encode meaning in a perspective-independent semantic space, yielding stable similarity judgments across tasks and languages. Our results show that this assumption does not hold in practice. We introduce SEA-BED, a large-scale benchmark covering 10 Southeast Asian (SEA) languages and diverse embedding tasks, designed to systematically examine how embedding performance varies across tasks, languages, and language-task combinations. Across extensive evaluations, we observe that no single model performs uniformly well across SEA languages; task difficulty differs markedly within languages, and success on one task does not reliably generalize to others. Language-task analyses further reveal highly non-uniform performance landscapes, where performance varies across different language-task combinations. These findings call for closer attention to performance measurements that provide an expansive view across languages and tasks to uncover inconsistencies in semantic representation. Based on these observations, we provide insights for future model development, including data, algorithmic, and architectural considerations.

cs.CL↗

arXiv2025

Bridging AI Innovation and Healthcare Needs: Lessons Learned from Incorporating Modern NLP at The BC Cancer Registry

Automating data extraction from clinical documents offers significant potential to improve efficiency in healthcare settings, yet deploying Natural Language Processing (NLP) solutions presents practical challenges. Drawing upon our experience implementing various NLP models for information extraction and classification tasks at the British Columbia Cancer Registry (BCCR), this paper shares key lessons learned throughout the project lifecycle. We emphasize the critical importance of defining problems based on clear business objectives rather than solely technical accuracy, adopting an iterative approach to development, and fostering deep interdisciplinary collaboration and co-design involving domain experts, end-users, and ML specialists from inception. Further insights highlight the need for pragmatic model selection (including hybrid approaches and simpler methods where appropriate), rigorous attention to data quality (representativeness, drift, annotation), robust error mitigation strategies involving human-in-the-loop validation and ongoing audits, and building organizational AI literacy. These practical considerations, generalizable beyond cancer registries, provide guidance for healthcare organizations seeking to successfully implement AI/NLP solutions to enhance data management processes and ultimately improve patient care and public health outcomes.

cs.CL↗

arXiv2025

Small or Large? Zero-Shot or Finetuned? Guiding Language Model Choice for Specialized Applications in Healthcare

This study aims to guide language model selection by investigating: 1) the necessity of finetuning versus zero-shot usage, 2) the benefits of domain-adjacent versus generic pretrained models, 3) the value of further domain-specific pretraining, and 4) the continued relevance of Small Language Models (SLMs) compared to Large Language Models (LLMs) for specific tasks. Using electronic pathology reports from the British Columbia Cancer Registry (BCCR), three classification scenarios with varying difficulty and data size are evaluated. Models include various SLMs and an LLM. SLMs are evaluated both zero-shot and finetuned; the LLM is evaluated zero-shot only. Finetuning significantly improved SLM performance across all scenarios compared to their zero-shot results. The zero-shot LLM outperformed zero-shot SLMs but was consistently outperformed by finetuned SLMs. Domain-adjacent SLMs generally performed better than the generic SLM after finetuning, especially on harder tasks. Further domain-specific pretraining yielded modest gains on easier tasks but significant improvements on the complex, data-scarce task. The results highlight the critical role of finetuning for SLMs in specialized domains, enabling them to surpass zero-shot LLM performance on targeted classification tasks. Pretraining on domain-adjacent or domain-specific data provides further advantages, particularly for complex problems or limited finetuning data. While LLMs offer strong zero-shot capabilities, their performance on these specific tasks did not match that of appropriately finetuned SLMs. In the era of LLMs, SLMs remain relevant and effective, offering a potentially superior performance-resource trade-off compared to LLMs.

cs.CL↗

arXiv2025

SEA-LION: Southeast Asian Languages in One Network

Recently, Large Language Models (LLMs) have dominated much of the artificial intelligence scene with their ability to process and generate natural languages. However, the majority of LLM research and development remains English-centric, leaving low-resource languages such as those in the Southeast Asian (SEA) region under-represented. To address this representation gap, we introduce Llama-SEA-LION-v3-8B-IT and Gemma-SEA-LION-v3-9B-IT, two cutting-edge multilingual LLMs designed for SEA languages. The SEA-LION family of LLMs supports 11 SEA languages, namely English, Chinese, Indonesian, Vietnamese, Malay, Thai, Burmese, Lao, Filipino, Tamil, and Khmer. Our work leverages large-scale multilingual continued pre-training with a comprehensive post-training regime involving multiple stages of instruction fine-tuning, alignment, and model merging. Evaluation results on multilingual benchmarks indicate that our models achieve state-of-the-art performance across LLMs supporting SEA languages. We open-source the models to benefit the wider SEA community.

cs.CL↗

arXiv2025

ELM: A Hybrid Ensemble of Language Models for Automated Tumor Group Classification in Population-Based Cancer Registries

Background: Population-based cancer registries (PBCRs) manually extract data from unstructured pathology reports, a labor-intensive process where assigning reports to tumor groups can consume 900 person-hours annually for approximately 100,000 reports at a medium-sized registry. Current automated rule-based systems fail to handle the linguistic complexity of this classification task. Materials and Methods: We present ELM (Ensemble of Language Models), a novel hybrid approach combining small, encoder only language models and large language models (LLMs). ELM employs an ensemble of six fine-tuned encoder only models: three analyzing the top portion and three analyzing the bottom portion of each report to maximize text coverage given token limits. A tumor group is assigned when at least five of six models agree; otherwise, an LLM arbitrates using a carefully curated prompt constrained to likely tumor groups. Results: On a held-out test set of 2,058 pathology reports spanning 19 tumor groups, ELM achieves weighted precision and recall of 0.94, representing a statistically significant improvement (p<0.001) over encoder-only ensembles (0.91 F1-score) and substantially outperforming rule-based approaches. ELM demonstrates particular gains for challenging categories including leukemia (F1: 0.76 to 0.88), lymphoma (0.76 to 0.89), and skin cancer (0.44 to 0.58). Discussion: Deployed in production at British Columbia Cancer Registry, ELM has reduced manual review requirements by approximately 60-70%, saving an estimated 900 person-hours annually while maintaining data quality standards. Conclusion: ELM represents the first successful deployment of a hybrid small, encoder only models-LLM architecture for tumor group classification in a real-world PBCR setting, demonstrating how strategic combination of language models can achieve both high accuracy and operational efficiency.

cs.CL↗

arXiv2024

Black Swan: Abductive and Defeasible Video Reasoning in Unpredictable Events

The commonsense reasoning capabilities of vision-language models (VLMs), especially in abductive reasoning and defeasible reasoning, remain poorly understood. Most benchmarks focus on typical visual scenarios, making it difficult to discern whether model performance stems from keen perception and reasoning skills, or reliance on pure statistical recall. We argue that by focusing on atypical events in videos, clearer insights can be gained on the core capabilities of VLMs. Explaining and understanding such out-of-distribution events requires models to extend beyond basic pattern recognition and regurgitation of their prior knowledge. To this end, we introduce BlackSwanSuite, a benchmark for evaluating VLMs' ability to reason about unexpected events through abductive and defeasible tasks. Our tasks artificially limit the amount of visual information provided to models while questioning them about hidden unexpected events, or provide new visual information that could change an existing hypothesis about the event. We curate a comprehensive benchmark suite comprising over 3,800 MCQ, 4,900 generative and 6,700 yes/no questions, spanning 1,655 videos. After extensively evaluating various state-of-the-art VLMs, including GPT-4o and Gemini 1.5 Pro, as well as open-source VLMs such as LLaVA-Video, we find significant performance gaps of up to 32% from humans on these tasks. Our findings reveal key limitations in current VLMs, emphasizing the need for enhanced model architectures and training strategies. Our data and leaderboard is available at blackswan.cs.ubc.ca.

cs.CV↗

arXiv2024

Global MMLU: Understanding and Addressing Cultural and Linguistic Biases in Multilingual Evaluation

Cultural biases in multilingual datasets pose significant challenges for their effectiveness as global benchmarks. These biases stem not only from differences in language but also from the cultural knowledge required to interpret questions, reducing the practical utility of translated datasets like MMLU. Furthermore, translation often introduces artefacts that can distort the meaning or clarity of questions in the target language. A common practice in multilingual evaluation is to rely on machine-translated evaluation sets, but simply translating a dataset is insufficient to address these challenges. In this work, we trace the impact of both of these issues on multilingual evaluations and ensuing model performances. Our large-scale evaluation of state-of-the-art open and proprietary models illustrates that progress on MMLU depends heavily on learning Western-centric concepts, with 28% of all questions requiring culturally sensitive knowledge. Moreover, for questions requiring geographic knowledge, an astounding 84.9% focus on either North American or European regions. Rankings of model evaluations change depending on whether they are evaluated on the full portion or the subset of questions annotated as culturally sensitive, showing the distortion to model rankings when blindly relying on translated MMLU. We release Global MMLU, an improved MMLU with evaluation coverage across 42 languages -- with improved overall quality by engaging with compensated professional and community annotators to verify translation quality while also rigorously evaluating cultural biases present in the original dataset. This comprehensive Global MMLU set also includes designated subsets labeled as culturally sensitive and culturally agnostic to allow for more holistic, complete evaluation.

cs.CL↗

arXiv2024

GWSkyNet II : a refined machine learning pipeline for real-time classification of public gravitational wave alerts

Electromagnetic follow-up observations of gravitational wave events offer critical insights and provide significant scientific gain from this new class of astrophysical transients. Accurate identification of gravitational wave candidates and rapid release of sky localization information are crucial for the success of these electromagnetic follow-up observations. However, searches for gravitational wave candidates in real time suffer a non-negligible false alarm rate. By leveraging the sky localization information and other metadata associated with gravitational wave candidates, GWSkyNet, a machine learning classifier developed by Cabero et al. (2020), demonstrated promising accuracy for the identification of the origin of event candidates. We improve the performance of the classifier for LIGO-Virgo-KAGRA's fourth observing run by reviewing and updating the architecture and features used as inputs by the algorithm. We also retrain and fine-tune the classifier with data from the third observing run. To improve the prospect of electromagnetic follow-up observations, we incorporate GWSkyNet into LIGO-Virgo-KAGRA's low-latency infrastructure as an automatic pipeline for the evaluation of gravitational wave alerts in real time. We test the readiness of the algorithm on a LIGO-Virgo-KAGRA mock data challenge campaign. The results show that by thresholding on the GWSkyNet score, noise masquerading as astrophysical sources can be rejected efficiently and the majority of true astrophysical signals correctly identified.

astro-ph.IM↗

arXiv2023

A new method to distinguish gravitational-wave signals from detector noise transients with Gravity Spy

The Advanced LIGO and Advanced Virgo detectors have enabled the confident detection of dozens of mergers of black holes and neutron stars. However, the presence of detector noise transients (glitches) hinders the search for these gravitational wave (GW) signals. We prototyped a restructuring of Gravity Spy's classification model to distinguish between glitches and astrophysical signals. Our method is able to correctly classify three-quarters of retracted candidate events in O3b as non-astrophysical and 100\% of the confirmed astrophysical events as true signals. This approach will inform candidate event validation efforts in the latest observing run.

gr-qc↗

arXiv2023

COMET-M: Reasoning about Multiple Events in Complex Sentences

Understanding the speaker's intended meaning often involves drawing commonsense inferences to reason about what is not stated explicitly. In multi-event sentences, it requires understanding the relationships between events based on contextual knowledge. We propose COMET-M (Multi-Event), an event-centric commonsense model capable of generating commonsense inferences for a target event within a complex sentence. COMET-M builds upon COMET (Bosselut et al., 2019), which excels at generating event-centric inferences for simple sentences, but struggles with the complexity of multi-event sentences prevalent in natural text. To overcome this limitation, we curate a multi-event inference dataset of 35K human-written inferences. We trained COMET-M on the human-written inferences and also created baselines using automatically labeled examples. Experimental results demonstrate the significant performance improvement of COMET-M over COMET in generating multi-event inferences. Moreover, COMET-M successfully produces distinct inferences for each target event, taking the complete context into consideration. COMET-M holds promise for downstream tasks involving natural text such as coreference resolution, dialogue, and story understanding.

cs.CL↗

arXiv2023

GSpyNetTree: A signal-vs-glitch classifier for gravitational-wave event candidates

Despite achieving sensitivities capable of detecting the extremely small amplitude of gravitational waves (GWs), LIGO and Virgo detector data contain frequent bursts of non-Gaussian transient noise, commonly known as 'glitches'. Glitches come in various time-frequency morphologies, and they are particularly challenging when they mimic the form of real GWs. Given the higher expected event rate in the next observing run (O4), LIGO-Virgo GW event candidate validation will require increased levels of automation. Gravity Spy, a machine learning tool that successfully classified common types of LIGO and Virgo glitches in previous observing runs, has the potential to be restructured as a signal-vs-glitch classifier to accurately distinguish between glitches and GW signals. A signal-vs-glitch classifier used for automation must be robust and compatible with a broad array of background noise, new sources of glitches, and the likely occurrence of overlapping glitches and GWs. We present GSpyNetTree, the Gravity Spy Convolutional Neural Network Decision Tree: a multi-CNN classifier using CNNs in a decision tree sorted via total GW candidate mass tested under these realistic O4-era scenarios.

gr-qc↗

arXiv2023

What happens before and after: Multi-Event Commonsense in Event Coreference Resolution

Event coreference models cluster event mentions pertaining to the same real-world event. Recent models rely on contextualized representations to recognize coreference among lexically or contextually similar mentions. However, models typically fail to leverage commonsense inferences, which is particularly limiting for resolving lexically-divergent mentions. We propose a model that extends event mentions with temporal commonsense inferences. Given a complex sentence with multiple events, e.g., "The man killed his wife and got arrested", with the target event "arrested", our model generates plausible events that happen before the target event - such as "the police arrived", and after it, such as "he was sentenced". We show that incorporating such inferences into an existing event coreference model improves its performance, and we analyze the coreferences in which such temporal knowledge is required.

cs.CL↗