Searcharxiv⌕ Search

arXiv subjects

Hao Wang

Publications and source records attributed to Hao Wang.

At least 1,135 records · Page 63Linked to original sources
arXiv2023

UUKG: Unified Urban Knowledge Graph Dataset for Urban Spatiotemporal Prediction

Accurate Urban SpatioTemporal Prediction (USTP) is of great importance to the development and operation of the smart city. As an emerging building block, multi-sourced urban data are usually integrated as urban knowledge graphs (UrbanKGs) to provide critical knowledge for urban spatiotemporal prediction models. However, existing UrbanKGs are often tailored for specific downstream prediction tasks and are not publicly available, which limits the potential advancement. This paper presents UUKG, the unified urban knowledge graph dataset for knowledge-enhanced urban spatiotemporal predictions. Specifically, we first construct UrbanKGs consisting of millions of triplets for two metropolises by connecting heterogeneous urban entities such as administrative boroughs, POIs, and road segments. Moreover, we conduct qualitative and quantitative analysis on constructed UrbanKGs and uncover diverse high-order structural patterns, such as hierarchies and cycles, that can be leveraged to benefit downstream USTP tasks. To validate and facilitate the use of UrbanKGs, we implement and evaluate 15 KG embedding methods on the KG completion task and integrate the learned KG embeddings into 9 spatiotemporal models for five different USTP tasks. The extensive experimental results not only provide benchmarks of knowledge-enhanced USTP models under different task settings but also highlight the potential of state-of-the-art high-order structure-aware UrbanKG embedding methods. We hope the proposed UUKG fosters research on urban knowledge graphs and broad smart city applications. The dataset and source code are available at https://github.com/usail-hkust/UUKG/.

cs.AI↗
arXiv2023

A Data-Centric Multi-Objective Learning Framework for Responsible Recommendation Systems

Recommendation systems effectively guide users in locating their desired information within extensive content repositories. Generally, a recommendation model is optimized to enhance accuracy metrics from a user utility standpoint, such as click-through rate or matching relevance. However, a responsible industrial recommendation system must address not only user utility (responsibility to users) but also other objectives, including increasing platform revenue (responsibility to platforms), ensuring fairness (responsibility to content creators), and maintaining unbiasedness (responsibility to long-term healthy development). Multi-objective learning is a potent approach for achieving responsible recommendation systems. Nevertheless, current methods encounter two challenges: difficulty in scaling to heterogeneous objectives within a unified framework, and inadequate controllability over objective priority during optimization, leading to uncontrollable solutions. In this paper, we present a data-centric optimization framework, MoRec, which unifies the learning of diverse objectives. MoRec is a tri-level framework: the outer level manages the balance between different objectives, utilizing a proportional-integral-derivative (PID)-based controller to ensure a preset regularization on the primary objective. The middle level transforms objective-aware optimization into data sampling weights using sign gradients. The inner level employs a standard optimizer to update model parameters with the sampled data. Consequently, MoRec can flexibly support various objectives while maintaining the original model intact. Comprehensive experiments on two public datasets and one industrial dataset showcase the effectiveness, controllability, flexibility, and Pareto efficiency of MoRec, making it highly suitable for real-world implementation.

cs.IR↗
arXiv2023

Post-processing Private Synthetic Data for Improving Utility on Selected Measures

Existing private synthetic data generation algorithms are agnostic to downstream tasks. However, end users may have specific requirements that the synthetic data must satisfy. Failure to meet these requirements could significantly reduce the utility of the data for downstream use. We introduce a post-processing technique that improves the utility of the synthetic data with respect to measures selected by the end user, while preserving strong privacy guarantees and dataset quality. Our technique involves resampling from the synthetic data to filter out samples that do not meet the selected utility measures, using an efficient stochastic first-order algorithm to find optimal resampling weights. Through comprehensive numerical experiments, we demonstrate that our approach consistently improves the utility of synthetic data across multiple benchmark datasets and state-of-the-art synthetic data generation algorithms.

cs.LG↗
arXiv2023

A Unified Approach to Domain Incremental Learning with Memory: Theory and Algorithm

Domain incremental learning aims to adapt to a sequence of domains with access to only a small subset of data (i.e., memory) from previous domains. Various methods have been proposed for this problem, but it is still unclear how they are related and when practitioners should choose one method over another. In response, we propose a unified framework, dubbed Unified Domain Incremental Learning (UDIL), for domain incremental learning with memory. Our UDIL **unifies** various existing methods, and our theoretical analysis shows that UDIL always achieves a tighter generalization error bound compared to these methods. The key insight is that different existing methods correspond to our bound with different **fixed** coefficients; based on insights from this unification, our UDIL allows **adaptive** coefficients during training, thereby always achieving the tightest bound. Empirical results show that our UDIL outperforms the state-of-the-art domain incremental learning methods on both synthetic and real-world datasets. Code will be available at https://github.com/Wang-ML-Lab/unified-continual-learning.

cs.LG↗
arXiv2023

Fact-based Agent modeling for Multi-Agent Reinforcement Learning

In multi-agent systems, agents need to interact and collaborate with other agents in environments. Agent modeling is crucial to facilitate agent interactions and make adaptive cooperation strategies. However, it is challenging for agents to model the beliefs, behaviors, and intentions of other agents in non-stationary environment where all agent policies are learned simultaneously. In addition, the existing methods realize agent modeling through behavior cloning which assume that the local information of other agents can be accessed during execution or training. However, this assumption is infeasible in unknown scenarios characterized by unknown agents, such as competition teams, unreliable communication and federated learning due to privacy concerns. To eliminate this assumption and achieve agent modeling in unknown scenarios, Fact-based Agent modeling (FAM) method is proposed in which fact-based belief inference (FBI) network models other agents in partially observable environment only based on its local information. The reward and observation obtained by agents after taking actions are called facts, and FAM uses facts as reconstruction target to learn the policy representation of other agents through a variational autoencoder. We evaluate FAM on various Multiagent Particle Environment (MPE) and compare the results with several state-of-the-art MARL algorithms. Experimental results show that compared with baseline methods, FAM can effectively improve the efficiency of agent policy learning by making adaptive cooperation strategies in multi-agent reinforcement learning tasks, while achieving higher returns in complex competitive-cooperative mixed scenarios.

cs.AI↗
arXiv2023

Evolutionary tracks of massive stars with different rotation and metallicity in neutrino H-R diagram

Neutrino losses play a crucial role in the evolution of massive stars. We study the neutrino luminosity of stars ranging from 20 to 90 M_{\odot} from Zero Age Main Sequence (ZAMS) to Fe Core Collapse (FeCC) with different rotation and metallicity in a neutrino Hertzsprung-Russell diagram. In our simulations, we consider ω/ωcrit = 0 and 0.7 to represent non-rotation and high rotation, respectively, and set the metallicities to 0.014, 0.001, and 0.0001. During hydrogen burning stages, neutrino luminosity primarily originates from CNO cycle, and increases with higher stellar mass while decreasing with increasing metallicity. For the high metallicity models (Z = 0.014) during the helium burning stage, the reduction of the hydrogen envelope caused by a larger mass loss rate leads to a gradual decrease in neutrino luminosity. The rapid rotation results in extra mixing inside massive stars, which increases the neutrino luminosity during main sequence (MS), while decreases the neutrino luminosity during helium burning phase. Simultaneously, the rapid rotation also increases CO core mass, which enhances the neutrino luminosity during C and O burning phase. We also investigate the effect of neutrino magnetic moment (NMM) on the massive stars. We find that the energy loss caused by the NMM does not have effects on the evolutionary destiny of massive stars, and it does not significant change the compactness at the time of Fe core collapse.

astro-ph.SR↗
arXiv2023

Joint Optimization of Traffic Signal Control and Vehicle Routing in Signalized Road Networks using Multi-Agent Deep Reinforcement Learning

Urban traffic congestion is a critical predicament that plagues modern road networks. To alleviate this issue and enhance traffic efficiency, traffic signal control and vehicle routing have proven to be effective measures. In this paper, we propose a joint optimization approach for traffic signal control and vehicle routing in signalized road networks. The objective is to enhance network performance by simultaneously controlling signal timings and route choices using Multi-Agent Deep Reinforcement Learning (MADRL). Signal control agents (SAs) are employed to establish signal timings at intersections, whereas vehicle routing agents (RAs) are responsible for selecting vehicle routes. By establishing relevance between agents and enabling them to share observations and rewards, interaction and cooperation among agents are fostered, which enhances individual training. The Multi-Agent Advantage Actor-Critic algorithm is used to handle multi-agent environments, and Deep Neural Network (DNN) structures are designed to facilitate the algorithm's convergence. Notably, our work is the first to utilize MADRL in determining the optimal joint policy for signal control and vehicle routing. Numerical experiments conducted on the modified Sioux network demonstrate that our integration of signal control and vehicle routing outperforms controlling signal timings or vehicles' routes alone in enhancing traffic efficiency.

eess.SY↗
arXiv2023

Private Synthetic Data Meets Ensemble Learning

When machine learning models are trained on synthetic data and then deployed on real data, there is often a performance drop due to the distribution shift between synthetic and real data. In this paper, we introduce a new ensemble strategy for training downstream models, with the goal of enhancing their performance when used on real data. We generate multiple synthetic datasets by applying a differential privacy (DP) mechanism several times in parallel and then ensemble the downstream models trained on these datasets. While each synthetic dataset might deviate more from the real data distribution, they collectively increase sample diversity. This may enhance the robustness of downstream models against distribution shifts. Our extensive experiments reveal that while ensembling does not enhance downstream performance (compared with training a single model) for models trained on synthetic data generated by marginal-based or workload-based DP mechanisms, our proposed ensemble strategy does improve the performance for models trained using GAN-based DP mechanisms in terms of both accuracy and calibration of downstream models.

cs.CR↗
arXiv2023

GPFL: Simultaneously Learning Global and Personalized Feature Information for Personalized Federated Learning

Federated Learning (FL) is popular for its privacy-preserving and collaborative learning capabilities. Recently, personalized FL (pFL) has received attention for its ability to address statistical heterogeneity and achieve personalization in FL. However, from the perspective of feature extraction, most existing pFL methods only focus on extracting global or personalized feature information during local training, which fails to meet the collaborative learning and personalization goals of pFL. To address this, we propose a new pFL method, named GPFL, to simultaneously learn global and personalized feature information on each client. We conduct extensive experiments on six datasets in three statistically heterogeneous settings and show the superiority of GPFL over ten state-of-the-art methods regarding effectiveness, scalability, fairness, stability, and privacy. Besides, GPFL mitigates overfitting and outperforms the baselines by up to 8.99% in accuracy.

cs.LG↗
arXiv2023

B-Spine: Learning B-Spline Curve Representation for Robust and Interpretable Spinal Curvature Estimation

Spinal curvature estimation is important to the diagnosis and treatment of the scoliosis. Existing methods face several issues such as the need of expensive annotations on the vertebral landmarks and being sensitive to the image quality. It is challenging to achieve robust estimation and obtain interpretable results, especially for low-quality images which are blurry and hazy. In this paper, we propose B-Spine, a novel deep learning pipeline to learn B-spline curve representation of the spine and estimate the Cobb angles for spinal curvature estimation from low-quality X-ray images. Given a low-quality input, a novel SegRefine network which employs the unpaired image-to-image translation is proposed to generate a high quality spine mask from the initial segmentation result. Next, a novel mask-based B-spline prediction model is proposed to predict the B-spline curve for the spine centerline. Finally, the Cobb angles are estimated by a hybrid approach which combines the curve slope analysis and a curve-based regression model. We conduct quantitative and qualitative comparisons with the representative and SOTA learning-based methods on the public AASCE2019 dataset and our new proposed CJUH-JLU dataset which contains more challenging low-quality images. The superior performance on both datasets shows our method can achieve both robustness and interpretability for spinal curvature estimation.

eess.IV↗
arXiv2023

Large-scale photonic computing with nonlinear disordered media

Neural networks find widespread use in scientific and technological applications, yet their implementations in conventional computers have encountered bottlenecks due to ever-expanding computational needs. Photonic neuromorphic hardware, which manipulates information and represents data continuously in the optical domain, is one of the promising platforms with potential advantages of massive parallelism, ultralow latency, and reduced energy consumption. While linear photonic neural networks are within reach, photonic computing with large-scale optical nonlinear nodes remains largely unexplored. Here, we demonstrate a large-scale, high-performance nonlinear photonic neural system based on a disordered polycrystalline slab composed of lithium niobate nanocrystals. Mediated by random quasi-phase-matching and multiple scattering, linear and nonlinear optical speckle features are generated as the interplay between the simultaneous linear random scattering and the second-harmonic generation, defining a complex neural network in which the second-order nonlinearity acts as internal nonlinear activation functions. Benchmarked against linear random projection, such nonlinear mapping embedded with rich physical computational operations shows improved performance across a large collection of machine learning tasks in image classification, regression, and graph classification with varying complexity. Demonstrating up to 27,648 input and 3,500 nonlinear output nodes, the combination of optical nonlinearity and random scattering serves as a scalable computing engine for diverse applications.

physics.optics↗
arXiv2023

On Committor Functions in Milestoning

As an optimal one-dimensional reaction coordinate, the committor function not only describes the probability of a trajectory initiated at a phase space point first reaching the product state before reaching the reactant state, but also preserves the kinetics when utilized to run a reduced dynamics model. However, calculating the committor function in high-dimensional systems poses significant challenges. In this paper, within the framework of Milestoning, exact expressions for committor functions at two levels of coarse graining are given, including committor functions of phase space point to point (CFPP) and milestone to milestone (CFMM). When combined with transition kernels obtained from trajectory analysis, these expressions can be utilized to accurately and efficiently compute the committor functions. Furthermore, based on the calculated committor functions, an adaptive algorithm is developed to gradually refine the transition state region. Finally, two model examples are employed to assess the accuracy of these different formulations of committor functions.

physics.chem-ph↗
arXiv2023

ConvFormer: Revisiting Transformer for Sequential User Modeling

Sequential user modeling, a critical task in personalized recommender systems, focuses on predicting the next item a user would prefer, requiring a deep understanding of user behavior sequences. Despite the remarkable success of Transformer-based models across various domains, their full potential in comprehending user behavior remains untapped. In this paper, we re-examine Transformer-like architectures aiming to advance state-of-the-art performance. We start by revisiting the core building blocks of Transformer-based methods, analyzing the effectiveness of the item-to-item mechanism within the context of sequential user modeling. After conducting a thorough experimental analysis, we identify three essential criteria for devising efficient sequential user models, which we hope will serve as practical guidelines to inspire and shape future designs. Following this, we introduce ConvFormer, a simple but powerful modification to the Transformer architecture that meets these criteria, yielding state-of-the-art results. Additionally, we present an acceleration technique to minimize the complexity associated with processing extremely long sequences. Experiments on four public datasets showcase ConvFormer's superiority and confirm the validity of our proposed criteria.

cs.AI↗
arXiv2023

Biological Aggregations from Spatial Memory and Nonlocal Advection

We investigate a nonlocal single-species reaction-diffusion-advection model that integrates the spatial memory of previously visited locations and nonlocal detection in space, resulting in a coupled PDE-ODE system reflective of several existing models found in spatial ecology. We prove the existence and uniqueness of a Hölder continuous weak solution in one spatial dimension under some general conditions, allowing for discontinuous kernels such as the top-hat detection kernel. A robust spectral and bifurcation analysis is also performed, providing the rigorous analytical study not yet found in the existing literature. In particular, the essential spectrum is shown to be entirely negative, and we classify the nature of the bifurcation near the critical values obtained via a linear stability analysis. A pseudo-spectral method is used to solve and plot the steady states near and far away from these critical values, complementing the analytical insights.

math.AP↗
arXiv2023

Approximating First Hitting Point Distribution in Milestoning for Rare Event Kinetics

Milestoning is an efficient method for rare event kinetics calculation using short trajectory parallelization. Mean first passage time (MFPT) is the key kinetic output of Milestoning, whose accuracy crucially depends the initial distribution of the short trajectory ensemble. The true initial distribution, i.e., first hitting point distribution (FHPD), has no analytic expression in the general case. Here, we introduce two algorithms, local passage time weighted Milestoning (LPT-M) and Bayesian inference Milestoning (BI-M), to accurately and efficiently approximate FHPD for systems at equilibrium condition. Starting from sampling Boltzmann distribution on milestones, we calculate the proper weighting factor for the short trajectory ensemble. The methods are tested on two model examples for illustration purpose. Both methods improve significantly over the widely used classical Milestoning method in terms of the accuracy of MFPT. In particular, BI-M covers the directional Milestoning method as a special case in the deterministic Hamiltonian dynamics. LPT-M is especially advantageous in terms of computational costs and robustness with respect to the increasing number of intermediate milestones. Furthermore, a locally iterative correction algorithm for non-equilibrium stationary FHPD is developed for exact MFPT calculation, which can be combined with LPT-M/BI-M and is much cheaper than the exact Milestoning method.

physics.chem-ph↗
arXiv2023

Deep Learning-Based Connector Detection for Robotized Assembly of Automotive Wire Harnesses

The shift towards electrification and autonomous driving in the automotive industry results in more and more automotive wire harnesses being installed in modern automobiles, which stresses the great significance of guaranteeing the quality of automotive wire harness assembly. The mating of connectors is essential in the final assembly of automotive wire harnesses due to the importance of connectors on wire harness connection and signal transmission. However, the current manual operation of mating connectors leads to severe problems regarding assembly quality and ergonomics, where the robotized assembly has been considered, and different vision-based solutions have been proposed to facilitate a better perception of the robot control system on connectors. Nonetheless, there has been a lack of deep learning-based solutions for detecting automotive wire harness connectors in previous literature. This paper presents a deep learning-based connector detection for robotized automotive wire harness assembly. A dataset of twenty automotive wire harness connectors was created to train and evaluate a two-stage and a one-stage object detection model, respectively. The experiment results indicate the effectiveness of deep learning-based connector detection for automotive wire harness assembly but are limited by the design of the exteriors of connectors.

cs.RO↗
arXiv2023

Enhancing Text Generation with Cooperative Training

Recently, there has been a surge in the use of generated data to enhance the performance of downstream models, largely due to the advancements in pre-trained language models. However, most prevailing methods trained generative and discriminative models in isolation, which left them unable to adapt to changes in each other. These approaches lead to generative models that are prone to deviating from the true data distribution and providing limited benefits to discriminative models. While some works have proposed jointly training generative and discriminative language models, their methods remain challenging due to the non-differentiable nature of discrete data. To overcome these issues, we introduce a \textit{self-consistent learning} framework in the text field that involves training a discriminator and generator cooperatively in a closed-loop manner until a scoring consensus is reached. By learning directly from selected samples, our framework are able to mitigate training instabilities such as mode collapse and non-convergence. Extensive experiments on four downstream benchmarks, including AFQMC, CHIP-STS, QQP, and MRPC, demonstrate the efficacy of the proposed framework.

cs.CL↗
arXiv2023

Monotonic Neural Ordinary Differential Equation: Time-series Forecasting for Cumulative Data

Time-Series Forecasting based on Cumulative Data (TSFCD) is a crucial problem in decision-making across various industrial scenarios. However, existing time-series forecasting methods often overlook two important characteristics of cumulative data, namely monotonicity and irregularity, which limit their practical applicability. To address this limitation, we propose a principled approach called Monotonic neural Ordinary Differential Equation (MODE) within the framework of neural ordinary differential equations. By leveraging MODE, we are able to effectively capture and represent the monotonicity and irregularity in practical cumulative data. Through extensive experiments conducted in a bonus allocation scenario, we demonstrate that MODE outperforms state-of-the-art methods, showcasing its ability to handle both monotonicity and irregularity in cumulative data and delivering superior forecasting performance.

cs.LG↗