arXiv subjects

Hong Xu

Publications and source records attributed to Hong Xu.

At least 73 records · Page 4Linked to original sources

arXiv2023

Image2SSM: Reimagining Statistical Shape Models from Images with Radial Basis Functions

Statistical shape modeling (SSM) is an essential tool for analyzing variations in anatomical morphology. In a typical SSM pipeline, 3D anatomical images, gone through segmentation and rigid registration, are represented using lower-dimensional shape features, on which statistical analysis can be performed. Various methods for constructing compact shape representations have been proposed, but they involve laborious and costly steps. We propose Image2SSM, a novel deep-learning-based approach for SSM that leverages image-segmentation pairs to learn a radial-basis-function (RBF)-based representation of shapes directly from images. This RBF-based shape representation offers a rich self-supervised signal for the network to estimate a continuous, yet compact representation of the underlying surface that can adapt to complex geometries in a data-driven manner. Image2SSM can characterize populations of biological structures of interest by constructing statistical landmark-based shape models of ensembles of anatomical shapes while requiring minimal parameter tuning and no user assistance. Once trained, Image2SSM can be used to infer low-dimensional shape representations from new unsegmented images, paving the way toward scalable approaches for SSM, especially when dealing with large cohorts. Experiments on synthetic and real datasets show the efficacy of the proposed method compared to the state-of-art correspondence-based method for SSM.

cs.CV↗

arXiv2023

Particle-Based Shape Modeling for Arbitrary Regions-of-Interest

Statistical Shape Modeling (SSM) is a quantitative method for analyzing morphological variations in anatomical structures. These analyses often necessitate building models on targeted anatomical regions of interest to focus on specific morphological features. We propose an extension to \particle-based shape modeling (PSM), a widely used SSM framework, to allow shape modeling to arbitrary regions of interest. Existing methods to define regions of interest are computationally expensive and have topological limitations. To address these shortcomings, we use mesh fields to define free-form constraints, which allow for delimiting arbitrary regions of interest on shape surfaces. Furthermore, we add a quadratic penalty method to the model optimization to enable computationally efficient enforcement of any combination of cutting-plane and free-form constraints. We demonstrate the effectiveness of this method on a challenging synthetic dataset and two medical datasets.

cs.CV↗

arXiv2023

Adaptive Gating in Mixture-of-Experts based Language Models

Large language models, such as OpenAI's ChatGPT, have demonstrated exceptional language understanding capabilities in various NLP tasks. Sparsely activated mixture-of-experts (MoE) has emerged as a promising solution for scaling models while maintaining a constant number of computational operations. Existing MoE model adopts a fixed gating network where each token is computed by the same number of experts. However, this approach contradicts our intuition that the tokens in each sequence vary in terms of their linguistic complexity and, consequently, require different computational costs. Little is discussed in prior research on the trade-off between computation per token and model performance. This paper introduces adaptive gating in MoE, a flexible training strategy that allows tokens to be processed by a variable number of experts based on expert probability distribution. The proposed framework preserves sparsity while improving training efficiency. Additionally, curriculum learning is leveraged to further reduce training time. Extensive experiments on diverse NLP tasks show that adaptive gating reduces at most 22.5% training time while maintaining inference quality. Moreover, we conduct a comprehensive analysis of the routing decisions and present our insights when adaptive gating is used.

cs.CL↗

arXiv2023

Moby: Empowering 2D Models for Efficient Point Cloud Analytics on the Edge

3D object detection plays a pivotal role in many applications, most notably autonomous driving and robotics. These applications are commonly deployed on edge devices to promptly interact with the environment, and often require near real-time response. With limited computation power, it is challenging to execute 3D detection on the edge using highly complex neural networks. Common approaches such as offloading to the cloud induce significant latency overheads due to the large amount of point cloud data during transmission. To resolve the tension between wimpy edge devices and compute-intensive inference workloads, we explore the possibility of empowering fast 2D detection to extrapolate 3D bounding boxes. To this end, we present Moby, a novel system that demonstrates the feasibility and potential of our approach. We design a transformation pipeline for Moby that generates 3D bounding boxes efficiently and accurately based on 2D detection results without running 3D detectors. Further, we devise a frame offloading scheduler that decides when to launch the 3D detector judiciously in the cloud to avoid the errors from accumulating. Extensive evaluations on NVIDIA Jetson TX2 with real-world autonomous driving datasets demonstrate that Moby offers up to 91.9% latency improvement with modest accuracy loss over state of the art.

cs.NI↗

arXiv2022

Aryl: An Elastic Cluster Scheduler for Deep Learning

Companies build separate training and inference GPU clusters for deep learning, and use separate schedulers to manage them. This leads to problems for both training and inference: inference clusters have low GPU utilization when the traffic load is low; training jobs often experience long queueing time due to lack of resources. We introduce Aryl, a new cluster scheduler to address these problems. Aryl introduces capacity loaning to loan idle inference GPU servers for training jobs. It further exploits elastic scaling that scales a training job's GPU allocation to better utilize loaned resources. Capacity loaning and elastic scaling create new challenges to cluster management. When the loaned servers need to be returned, we need to minimize the number of job preemptions; when more GPUs become available, we need to allocate them to elastic jobs and minimize the job completion time (JCT). Aryl addresses these combinatorial problems using principled heuristics. It introduces the notion of server preemption cost which it greedily reduces during server reclaiming. It further relies on the JCT reduction value defined for each additional worker for an elastic job to solve the scheduling problem as a multiple-choice knapsack problem. Prototype implementation on a 64-GPU testbed and large-scale simulation with 15-day traces of over 50,000 production jobs show that Aryl brings 1.53x and 1.50x reductions in average queuing time and JCT, and improves cluster usage by up to 26.9% over the cluster scheduler without capacity loaning or elastic scaling.

cs.DC↗

arXiv2021

Neural sentence embedding models for semantic similarity estimation in the biomedical domain

BACKGROUND: In this study, we investigated the efficacy of current state-of-the-art neural sentence embedding models for semantic similarity estimation of sentences from biomedical literature. We trained different neural embedding models on 1.7 million articles from the PubMed Open Access dataset, and evaluated them based on a biomedical benchmark set containing 100 sentence pairs annotated by human experts and a smaller contradiction subset derived from the original benchmark set. RESULTS: With a Pearson correlation of 0.819, our best unsupervised model based on the Paragraph Vector Distributed Memory algorithm outperforms previous state-of-the-art results achieved on the BIOSSES biomedical benchmark set. Moreover, our proposed supervised model that combines different string-based similarity metrics with a neural embedding model surpasses previous ontology-dependent supervised state-of-the-art approaches in terms of Pearson's r (r=0.871) on the biomedical benchmark set. In contrast to the promising results for the original benchmark, we found our best models' performance on the smaller contradiction subset to be poor. CONCLUSIONS: In this study we highlighted the value of neural network-based models for semantic similarity estimation in the biomedical domain by showing that they can keep up with and even surpass previous state-of-the-art approaches for semantic similarity estimation that depend on the availability of laboriously curated ontologies when evaluated on a biomedical benchmark set. Capturing contradictions and negations in biomedical sentences, however, emerged as an essential area for further work.

cs.CL↗

arXiv2021

More Behind Your Electricity Bill: a Dual-DNN Approach to Non-Intrusive Load Monitoring

Non-intrusive load monitoring (NILM) is a well-known single-channel blind source separation problem that aims to decompose the household energy consumption into itemised energy usage of individual appliances. In this way, considerable energy savings could be achieved by enhancing household's awareness of energy usage. Recent investigations have shown that deep neural networks (DNNs) based approaches are promising for the NILM task. Nevertheless, they normally ignore the inherent properties of appliance operations in the network design, potentially leading to implausible results. We are thus motivated to develop the dual Deep Neural Networks (dual-DNN), which aims to i) take advantage of DNNs' learning capability of latent features and ii) empower the DNN architecture with identification ability of universal properties. Specifically in the design of dual-DNN, we adopt one subnetwork to measure power ratings of different appliances' operation states, and the other subnetwork to identify the running states of target appliances. The final result is then obtained by multiplying these two network outputs and meanwhile considering the multi-state property of household appliances. To enforce the sparsity property in appliance's state operating, we employ median filtering and hard gating mechanisms to the subnetwork for state identification. Compared with the state-of-the-art NILM methods, our dual-DNN approach demonstrates a 21.67% performance improvement in average on two public benchmark datasets.

cs.LG↗

arXiv2021

Modeling Discriminative Representations for Out-of-Domain Detection with Supervised Contrastive Learning

Detecting Out-of-Domain (OOD) or unknown intents from user queries is essential in a task-oriented dialog system. A key challenge of OOD detection is to learn discriminative semantic features. Traditional cross-entropy loss only focuses on whether a sample is correctly classified, and does not explicitly distinguish the margins between categories. In this paper, we propose a supervised contrastive learning objective to minimize intra-class variance by pulling together in-domain intents belonging to the same class and maximize inter-class variance by pushing apart samples from different classes. Besides, we employ an adversarial augmentation mechanism to obtain pseudo diverse views of a sample in the latent space. Experiments on two public datasets prove the effectiveness of our method capturing discriminative representations for OOD detection.

cs.CL↗

arXiv2021

Novel Slot Detection: A Benchmark for Discovering Unknown Slot Types in the Task-Oriented Dialogue System

Existing slot filling models can only recognize pre-defined in-domain slot types from a limited slot set. In the practical application, a reliable dialogue system should know what it does not know. In this paper, we introduce a new task, Novel Slot Detection (NSD), in the task-oriented dialogue system. NSD aims to discover unknown or out-of-domain slot types to strengthen the capability of a dialogue system based on in-domain training data. Besides, we construct two public NSD datasets, propose several strong NSD baselines, and establish a benchmark for future work. Finally, we conduct exhaustive experiments and qualitative analysis to comprehend key challenges and provide new guidance for future directions.

cs.CL↗

arXiv2020

Learning Embeddings of Directed Networks with Text-Associated Nodes---with Applications in Software Package Dependency Networks

A network embedding consists of a vector representation for each node in the network. Its usefulness has been shown in many real-world application domains, such as social networks and web networks. Directed networks with text associated with each node, such as software package dependency networks, are commonplace. However, to the best of our knowledge, their embeddings have hitherto not been specifically studied. In this paper, we propose PCTADW-1 and PCTADW-2, two algorithms based on neural networks that learn embeddings of directed networks with text associated with each node. We create two new node-labeled such networks: The package dependency networks in two popular GNU/Linux distributions, Debian and Fedora. We experimentally demonstrate that the embeddings produced by our algorithms resulted in node classification with better quality than those of various baselines on these two networks. We observe that there exist systematic presence of analogies (similar to those in word embeddings) in the network embeddings of software package dependency networks. To the best of our knowledge, this is the first time that such systematic presence of analogies is observed in network and document embeddings. We further demonstrate that these network embeddings can be novelly used for better understanding software attributes, such as the development process and user interface of software, etc.

cs.LG↗

arXiv2020

A sealed ceramic GEM-based neutron detector

The GEM-based neutron detector has flourished in the past decade. However almost all the GEM-based neutron detectors work in the flow-gas mode, and the long-term performances of the detectors may be unstable due to the dynamic changes of atmospheric pressure and ambient temperature. In this paper, a sealed ceramic GEM-based neutron detector was developed at China Spallation Neutron Source (CSNS) and its sensitive area was 100 mm * 100 mm. The characterizations of the detector were presented and discussed, which included the plateau curve, the neutron beam profile, the neutron wavelength spectrum, the spatial resolution (FWHM: 2.77 mm), the two-dimensional (2D) imaging ability, the neutron detection efficiency and the counting rate instability (Relative Standard Deviation (RSD): 0.7%). The results show that the detector has good performances in sealed mode, and it can be used for the measurement of the direct neutron beam at CSNS.

physics.ins-det↗

arXiv2020

Joint Switch-Controller Association and Control Devolution for SDN Systems: An Integration of Online Control and Online Learning

In software-defined networking (SDN) systems, it is a common practice to adopt a multi-controller design and control devolution techniques to improve the performance of the control plane. However, in such systems, the decision-making for joint switch-controller association and control devolution often involves various uncertainties, e.g., the temporal variations of controller accessibility, and computation and communication costs of switches. In practice, statistics of such uncertainties are unattainable and need to be learned in an online fashion, calling for an integrated design of learning and control. In this paper, we formulate a stochastic network optimization problem that aims to minimize time-average system costs and ensure queue stability. By transforming the problem into a combinatorial multi-armed bandit problem with long-term stability constraints, we adopt bandit learning methods and optimal control techniques to handle the exploration-exploitation tradeoff and long-term stability constraints, respectively. Through an integrated design of online learning and online control, we propose an effective Learning-Aided Switch-Controller Association and Control Devolution (LASAC) scheme. Our theoretical analysis and simulation results show that LASAC achieves a tunable tradeoff between queue stability and system cost reduction with a sublinear time-averaged regret bound over a finite time horizon.

cs.NI↗

arXiv2020

Predictive Switch-Controller Association and Control Devolution for SDN Systems

For software-defined networking (SDN) systems, to enhance the scalability and reliability of control plane, existing solutions adopt either multi-controller design with static switch-controller associations, or static control devolution by delegating certain request processing back to switches. Such solutions can fall short in face of temporal variations of request traffics, incurring considerable local computation costs on switches and their communication costs to controllers. So far, it still remains an open problem to develop a joint online scheme that conducts dynamic switch-controller association and dynamic control devolution. In addition, the fundamental benefits of predictive scheduling to SDN systems still remain unexplored. In this paper, we identify the non-trivial trade-off in such a joint design and formulate a stochastic network optimization problem that aims to minimize time-averaged total system costs and ensure long-term queue stability. By exploiting the unique problem structure, we devise a predictive online switch-controller association and control devolution (POSCAD) scheme, which solves the problem through a series of online distributed decision making. Theoretical analysis shows that without prediction, POSCAD can achieve near-optimal total system costs with a tunable trade-off for queue stability. With prediction, POSCAD can achieve even better performance with shorter latencies. We conduct extensive simulations to evaluate POSCAD. Notably, with mild-value of future information, POSCAD incurs a significant reduction in request latencies, even when faced with prediction errors.

cs.NI↗

arXiv2020

Attention-guided Quality Assessment for Automated Cryo-EM Grid Screening

Cryogenic electron microscopy (cryo-EM) has become an enabling technology in drug discovery and in understanding molecular bases of disease by producing near-atomic resolution (less than 0.4 nm) 3D reconstructions of biological macromolecules. The imaging process required for 3D reconstructions involves a highly iterative and empirical screening process, starting with the acquisition of low magnification images of the cryo-EM grids. These images are inspected for squares that are likely to contain useful molecular signals. Potentially useful squares within the grid are then imaged at progressively higher magnifications, with the goal of identifying sub-micron areas within circular holes (bounded by the squares) for imaging at high magnification. This arduous, multi-step data acquisition process represents a bottleneck for obtaining a high throughput data collection. Here, we focus on automating the early decision making for the microscope operator, scoring low magnification images of squares, and proposing the first deep learning framework, XCryoNet, for automated cryo-EM grid screening. XCryoNet is a semi-supervised, attention-guided deep learning approach that provides explainable scoring of automatically extracted square images using limited amounts of labeled data. Results show up to 8% and 37% improvements over a fully supervised and a no-attention solution, respectively, when labeled data is scarce.

cs.CV↗

arXiv2020

Structure and Properties of Thermoresponsive Diblock Copolymers Embedded with Metal Oxide Nanoparticles

Nanostructured polymer-metal oxide composites are a current research area of great importance due to its highlight applications in sensors, optics, catalysts and drug delivery. Particularly the use of thermoresponsive polymers gives more flexibilities and possibilities in the design and construction of polymer templates. In the present investigation, the structure and magnetic properties of hybrid metal oxide/DBC films composed of two kinds of polystyrene-block-poly (N-isopropylacrylamide)(PS-b-PNIPAM) diblock copolymers (DBCs) with PS and PNIPAM as the major polymer domains respectively, and iron oxide were investigated. The thermoresponsive PNIPAM has a lower critical solution temperature (LCST) in aqueous solution at 32°C, which enables the controllable volume ratio of PS and PNIPAM in the structure of PS-b-PNIPAM diblock copolymers (DBCs). Thus, a temperature and humidity controlling cell was designed and built for precisely tuning the block structure of PS-b-PNIPAM DBCs, which was investigated by in-situ small-angle X-ray scattering (SAXS) and grazing-incidence small-angle X-ray scattering (GISAXS) measurements. The superparamagnetic behavior of the heat-treated hybrid iron oxide/PS-b-PNIPAM DBC films was investigated using a superconducting quantum interference device (SQUID) magnetometer.

cond-mat.soft↗

arXiv2020

Simulating Performance of ML Systems with Offline Profiling

We advocate that simulation based on offline profiling is a promising approach to better understand and improve the complex ML systems. Our approach uses operation-level profiling and dataflow based simulation to ensure it offers a unified and automated solution for all frameworks and ML models, and is also accurate by considering the various parallelization strategies in a real system.

cs.DC↗

arXiv2019

MANELA: A Multi-Agent Algorithm for Learning Network Embeddings

Playing an essential role in data mining, machine learning has a long history of being applied to networks on multifarious tasks and has played an essential role in data mining. However, the discrete and sparse natures of networks often render it difficult to apply machine learning directly to networks. To circumvent this difficulty, one major school of thought to approach networks using machine learning is via network embeddings. On the one hand, this network embeddings have achieved huge success on aggregated network data in recent years. On the other hand, learning network embeddings on distributively stored networks still remained understudied: To the best of our knowledge, all existing algorithms for learning network embeddings have hitherto been exclusively centralized and thus cannot be applied to these networks. To accommodate distributively stored networks, in this paper, we proposed a multi-agent model. Under this model, we developed the multi-agent network embedding learning algorithm (MANELA) for learning network embeddings. We demonstrate MANELA's advantages over other existing centralized network embedding learning algorithms both theoretically and experimentally. Finally, we further our understanding in MANELA via visualization and exploration of its relationship to DeepWalk.

cs.MA↗

arXiv2019

Node-Constrained Traffic Engineering: Theory and Applications

Traffic engineering (TE) is a fundamental task in networking. Conventionally, traffic can take any path connecting the source and destination. Emerging technologies such as segment routing, however, use logical paths going through a predetermined set of middlepoints. Inspired by this, in this work we introduce the problem of node-constrained TE, where traffic must go through a set of middlepoints, and study its theoretical fundamentals. We show that the general node-constrained TE that constrains the traffic to take paths going through one or more middlepoints is NP-hard for directed graphs but strongly polynomial for undirected graphs, unveiling a profound dichotomy between the two cases. We additionally investigate the popular variant of node-constrained TE that uses shortest paths between middlepoints, and show that the problem can now be solved in weakly polynomial time for a fixed number of middlepoints. Yet if we constrain the end-to-end paths to be acyclic, the problem can become NP-hard. This explains why existing work focuses on the computationally tractable variant. An important application of our work concerns the computational complexity of flow centrality, first proposed in 1991 by Freeman et al. [21]: we show that it is NP-hard for directed but strongly polynomial for undirected graphs. Finally, we investigate the middlepoint selection problem in general node-constrained TE. We introduce group flow centrality as a solution concept for multi-commodity networks, study its complexity, and show that it is monotone but not submodular for both directed and undirected graphs. Our work provides a thorough theoretical treatment of node-constrained TE and its applications.

cs.NI↗