Searcharxiv⌕ Search

arXiv subjects

Hao Wang

Publications and source records attributed to Hao Wang.

At least 1,207 records · Page 67Linked to original sources
arXiv2023

Self-discipline on multiple channels

Self-distillation relies on its own information to improve the generalization ability of the model and has a bright future. Existing self-distillation methods either require additional models, model modification, or batch size expansion for training, which increases the difficulty of use, memory consumption, and computational cost. This paper developed Self-discipline on multiple channels(SMC), which combines consistency regularization with self-distillation using the concept of multiple channels. Conceptually, SMC consists of two steps: 1) each channel data is simultaneously passed through the model to obtain its corresponding soft label, and 2) the soft label saved in the previous step is read together with the soft label obtained from the current channel data through the model to calculate the loss function. SMC uses consistent regularization and self-distillation to improve the generalization ability of the model and the robustness of the model to noisy labels. We named the SMC containing only two channels as SMC-2. Comparative experimental results on both datasets show that SMC-2 outperforms Label Smoothing Regularizaion and Self-distillation From The Last Mini-batch on all models, and outperforms the state-of-the-art Sharpness-Aware Minimization method on 83% of the models.Compatibility of SMC-2 and data augmentation experimental results show that using both SMC-2 and data augmentation improves the generalization ability of the model between 0.28% and 1.80% compared to using only data augmentation. Ultimately, the results of the label noise interference experiments show that SMC-2 curbs the tendency that the model's generalization ability decreases in the late training period due to the interference of label noise. The code is available at https://github.com/JiuTiannn/SMC-Self-discipline-on-multiple-channels.

cs.LG↗
arXiv2023

Graph-Relational Domain Adaptation

Existing domain adaptation methods tend to treat every domain equally and align them all perfectly. Such uniform alignment ignores topological structures among different domains; therefore it may be beneficial for nearby domains, but not necessarily for distant domains. In this work, we relax such uniform alignment by using a domain graph to encode domain adjacency, e.g., a graph of states in the US with each state as a domain and each edge indicating adjacency, thereby allowing domains to align flexibly based on the graph structure. We generalize the existing adversarial learning framework with a novel graph discriminator using encoding-conditioned graph embeddings. Theoretical analysis shows that at equilibrium, our method recovers classic domain adaptation when the graph is a clique, and achieves non-trivial alignment for other types of graphs. Empirical results show that our approach successfully generalizes uniform alignment, naturally incorporates domain information represented by graphs, and improves upon existing domain adaptation methods on both synthetic and real-world datasets. Code will soon be available at https://github.com/Wang-ML-Lab/GRDA.

cs.LG↗
arXiv2023

A Reinforcement Learning-assisted Genetic Programming Algorithm for Team Formation Problem Considering Person-Job Matching

An efficient team is essential for the company to successfully complete new projects. To solve the team formation problem considering person-job matching (TFP-PJM), a 0-1 integer programming model is constructed, which considers both person-job matching and team members' willingness to communicate on team efficiency, with the person-job matching score calculated using intuitionistic fuzzy numbers. Then, a reinforcement learning-assisted genetic programming algorithm (RL-GP) is proposed to enhance the quality of solutions. The RL-GP adopts the ensemble population strategies. Before the population evolution at each generation, the agent selects one from four population search modes according to the information obtained, thus realizing a sound balance of exploration and exploitation. In addition, surrogate models are used in the algorithm to evaluate the formation plans generated by individuals, which speeds up the algorithm learning process. Afterward, a series of comparison experiments are conducted to verify the overall performance of RL-GP and the effectiveness of the improved strategies within the algorithm. The hyper-heuristic rules obtained through efficient learning can be utilized as decision-making aids when forming project teams. This study reveals the advantages of reinforcement learning methods, ensemble strategies, and the surrogate model applied to the GP framework. The diversity and intelligent selection of search patterns along with fast adaptation evaluation, are distinct features that enable RL-GP to be deployed in real-world enterprise environments.

cs.NE↗
arXiv2023

Fairness-Aware Optimization of Vehicle-to-Vehicle Interaction for Smart EV Charging Coordination

As the number of electric vehicles (EVs) continues to grow, there is an increasing need for smart charging strategies. This paper exploits the vehicle-to-vehicle (V2V) concept to leverage EVs' diverse charging patterns and unlock the value of flexibility by enabling energy transfer among EVs. We formulate a cost minimization problem for an EV charging station to optimize the V2V schedule together with vehicle-to-grid (V2G), grid-to-vehicle (G2V) charging, as well as the use of renewable energy. When EVs perform V2V to transfer energy to charge EVs, the traditional cost-minimizing approach may overuse some EVs with lower costs to perform V2V. We address the fairness issue by developing fair V2V energy transfer strategies to avoid the excess discharge from individual EVs. We introduced three kinds of fairness metrics in the V2V optimization problem to demonstrate the fair energy transfers. In addition, we formulate baseline optimization problems without V2G or V2V and compare the results to prove the potency of V2V concept along with V2G method. The simulation results demonstrate that V2V can significantly reduce EV charging costs and highlight the trade-off between fairness enforcement and charging cost minimization.

eess.SY↗
arXiv2023

Optimal Energy Storage Scheduling for Wind Curtailment Reduction and Energy Arbitrage: A Deep Reinforcement Learning Approach

Wind energy has been rapidly gaining popularity as a means for combating climate change. However, the variable nature of wind generation can undermine system reliability and lead to wind curtailment, causing substantial economic losses to wind power producers. Battery energy storage systems (BESS) that serve as onsite backup sources are among the solutions to mitigate wind curtailment. However, such an auxiliary role of the BESS might severely weaken its economic viability. This paper addresses the issue by proposing joint wind curtailment reduction and energy arbitrage for the BESS. We decouple the market participation of the co-located wind-battery system and develop a joint-bidding framework for the wind farm and BESS. It is challenging to optimize the joint-bidding because of the stochasticity of energy prices and wind generation. Therefore, we leverage deep reinforcement learning to maximize the overall revenue from the spot market while unlocking the BESS's potential in concurrently reducing wind curtailment and conducting energy arbitrage. We validate the proposed strategy using realistic wind farm data and demonstrate that our joint-bidding strategy responds better to wind curtailment and generates higher revenues than the optimization-based benchmark. Our simulations also reveal that the extra wind generation used to be curtailed can be an effective power source to charge the BESS, resulting in additional financial returns.

cs.LG↗
arXiv2023

Fengshenbang 1.0: Being the Foundation of Chinese Cognitive Intelligence

Nowadays, foundation models become one of fundamental infrastructures in artificial intelligence, paving ways to the general intelligence. However, the reality presents two urgent challenges: existing foundation models are dominated by the English-language community; users are often given limited resources and thus cannot always use foundation models. To support the development of the Chinese-language community, we introduce an open-source project, called Fengshenbang, which leads by the research center for Cognitive Computing and Natural Language (CCNL). Our project has comprehensive capabilities, including large pre-trained models, user-friendly APIs, benchmarks, datasets, and others. We wrap all these in three sub-projects: the Fengshenbang Model, the Fengshen Framework, and the Fengshen Benchmark. An open-source roadmap, Fengshenbang, aims to re-evaluate the open-source community of Chinese pre-trained large-scale models, prompting the development of the entire Chinese large-scale model community. We also want to build a user-centered open-source ecosystem to allow individuals to access the desired models to match their computing resources. Furthermore, we invite companies, colleges, and research institutions to collaborate with us to build the large-scale open-source model-based ecosystem. We hope that this project will be the foundation of Chinese cognitive intelligence.

cs.CL↗
arXiv2023

StarNet: Style-Aware 3D Point Cloud Generation

This paper investigates an open research task of reconstructing and generating 3D point clouds. Most existing works of 3D generative models directly take the Gaussian prior as input for the decoder to generate 3D point clouds, which fail to learn disentangled latent codes, leading noisy interpolated results. Most of the GAN-based models fail to discriminate the local geometries, resulting in the point clouds generated not evenly distributed at the object surface, hence degrading the point cloud generation quality. Moreover, prevailing methods adopt computation-intensive frameworks, such as flow-based models and Markov chains, which take plenty of time and resources in the training phase. To resolve these limitations, this paper proposes a unified style-aware network architecture combining both point-wise distance loss and adversarial loss, StarNet which is able to reconstruct and generate high-fidelity and even 3D point clouds using a mapping network that can effectively disentangle the Gaussian prior from input's high-level attributes in the mapped latent space to generate realistic interpolated objects. Experimental results demonstrate that our framework achieves comparable state-of-the-art performance on various metrics in the point cloud reconstruction and generation tasks, but is more lightweight in model size, requires much fewer parameters and less time for model training.

cs.CV↗
arXiv2023

ManiCLIP: Multi-Attribute Face Manipulation from Text

In this paper we present a novel multi-attribute face manipulation method based on textual descriptions. Previous text-based image editing methods either require test-time optimization for each individual image or are restricted to single attribute editing. Extending these methods to multi-attribute face image editing scenarios will introduce undesired excessive attribute change, e.g., text-relevant attributes are overly manipulated and text-irrelevant attributes are also changed. In order to address these challenges and achieve natural editing over multiple face attributes, we propose a new decoupling training scheme where we use group sampling to get text segments from same attribute categories, instead of whole complex sentences. Further, to preserve other existing face attributes, we encourage the model to edit the latent code of each attribute separately via an entropy constraint. During the inference phase, our model is able to edit new face images without any test-time optimization, even from complex textual prompts. We show extensive experiments and analysis to demonstrate the efficacy of our method, which generates natural manipulated faces with minimal text-irrelevant attribute editing. Code and pre-trained model are available at https://github.com/hwang1996/ManiCLIP.

cs.CV↗
arXiv2023

Analysis and Visualization of the Parameter Space of Matrix Factorization-based Recommender Systems

Recommender system is the most successful commercial technology in the past decade. Technical mammoth such as Temu, TikTok and Amazon utilize the technology to generate enormous revenues each year. Although there have been enough research literature on accuracy enhancement of the technology, explainable AI is still a new idea to the field. In 2022, the author of this paper provides a geometric interpretation of the matrix factorization-based methods and uses geometric approximation to solve the recommendation problem. We continue the research in this direction in this paper, and visualize the inner structure of the parameter space of matrix factorization technologies. We show that the parameters of matrix factorization methods are distributed within a hyper-ball. After further analysis, we prove that the distribution of the parameters is not multivariate normal.

cs.IR↗
arXiv2023

Evolution of the Online Rating Platform Data Structures and its Implications for Recommender Systems

Online rating platform represents the new trend of online cultural and commercial goods consumption. The user rating data on such platforms are foods for recommender system algorithms. Understanding the evolution pattern and its underlying mechanism is the key to understand the structures of input data for recommender systems. Prior research on input data analysis for recommender systems is quite limited, with a notable exception in 2018 [6]. In this paper, we take advantage of Poisson Process to analyze the evolution mechanism of the input data structures. We discover that homogeneous Poisson Process could not capture the mechanism of user rating behavior on online rating platforms, and inhomogeneous Poisson Process is compatible with the formation process.

cs.IR↗
arXiv2023

Prediction of novel final phases in aged uranium-niobium alloys

Ordered intermetallics are long believed to be the final products of the aging of U-Nb solid solutions at low temperatures, a crucial property for the practical applications of this alloy in engineering and industry. However, such conjectured ordered compounds have not been experimentally or theoretically established. Herein, numerical evidence for ordered intermetallic U-Nb compounds is presented using thorough first-principles structure predictions up to 500 GPa. Two stable U2Nb compounds and one metastable U2Nb and one metastable U3Nb were discovered. A unique hybridized transition driven by pressure was observed in U2Nb, which is a superposition of one first-order transition and another second-order transition, leading to striking features near the transition pressure of 21.6 GPa. The decomposition limit of these compounds at high temperature was also investigated. The strong stability of U2Nb in the region of low pressure and high temperature was revealed. This discovery of ordered U2Nb and its strong stability over a wide pressure range completely changed the phase diagram of U-Nb alloys and shed new light on the dynamic response and aging mechanism of U-Nb alloys.

cond-mat.mtrl-sci↗
arXiv2023

TAPS3D: Text-Guided 3D Textured Shape Generation from Pseudo Supervision

In this paper, we investigate an open research task of generating controllable 3D textured shapes from the given textual descriptions. Previous works either require ground truth caption labeling or extensive optimization time. To resolve these issues, we present a novel framework, TAPS3D, to train a text-guided 3D shape generator with pseudo captions. Specifically, based on rendered 2D images, we retrieve relevant words from the CLIP vocabulary and construct pseudo captions using templates. Our constructed captions provide high-level semantic supervision for generated 3D shapes. Further, in order to produce fine-grained textures and increase geometry diversity, we propose to adopt low-level image regularization to enable fake-rendered images to align with the real ones. During the inference phase, our proposed model can generate 3D textured shapes from the given text without any additional optimization. We conduct extensive experiments to analyze each of our proposed components and show the efficacy of our framework in generating high-fidelity 3D textured and text-relevant shapes.

cs.CV↗
arXiv2023

Distribution-restrained Softmax Loss for the Model Robustness

Recently, the robustness of deep learning models has received widespread attention, and various methods for improving model robustness have been proposed, including adversarial training, model architecture modification, design of loss functions, certified defenses, and so on. However, the principle of the robustness to attacks is still not fully understood, also the related research is still not sufficient. Here, we have identified a significant factor that affects the robustness of models: the distribution characteristics of softmax values for non-real label samples. We found that the results after an attack are highly correlated with the distribution characteristics, and thus we proposed a loss function to suppress the distribution diversity of softmax. A large number of experiments have shown that our method can improve robustness without significant time consumption.

cs.LG↗
arXiv2023

A Dual-branch Self-supervised Representation Learning Framework for Tumour Segmentation in Whole Slide Images

Supervised deep learning methods have achieved considerable success in medical image analysis, owing to the availability of large-scale and well-annotated datasets. However, creating such datasets for whole slide images (WSIs) in histopathology is a challenging task due to their gigapixel size. In recent years, self-supervised learning (SSL) has emerged as an alternative solution to reduce the annotation overheads in WSIs, as it does not require labels for training. These SSL approaches, however, are not designed for handling multi-resolution WSIs, which limits their performance in learning discriminative image features. In this paper, we propose a Dual-branch SSL Framework for WSI tumour segmentation (DSF-WSI) that can effectively learn image features from multi-resolution WSIs. Our DSF-WSI connected two branches and jointly learnt low and high resolution WSIs in a self-supervised manner. Moreover, we introduced a novel Context-Target Fusion Module (CTFM) and a masked jigsaw pretext task to align the learnt multi-resolution features. Furthermore, we designed a Dense SimSiam Learning (DSL) strategy to maximise the similarity of different views of WSIs, enabling the learnt representations to be more efficient and discriminative. We evaluated our method using two public datasets on breast and liver cancer segmentation tasks. The experiment results demonstrated that our DSF-WSI can effectively extract robust and efficient representations, which we validated through subsequent fine-tuning and semi-supervised settings. Our proposed method achieved better accuracy than other state-of-the-art approaches. Code is available at https://github.com/Dylan-H-Wang/dsf-wsi.

cs.CV↗
arXiv2023

Open problems in PDE models for knowledge-based animal movement via nonlocal perception and cognitive mapping

The inclusion of cognitive processes, such as perception, learning and memory, are inevitable in mechanistic animal movement modelling. Cognition is the unique feature that distinguishes animal movement from mere particle movement in chemistry or physics. Hence, it is essential to incorporate such knowledge-based processes into animal movement models. Here, we summarize popular deterministic mathematical models derived from first principles that begin to incorporate such influences on movement behaviour mechanisms. Most generally, these models take the form of nonlocal reaction-diffusion-advection equations, where the nonlocality may appear in the spatial domain, the temporal domain, or both. Mathematical rules of thumb are provided to judge the model rationality, to aid in model development or interpretation, and to streamline an understanding of the range of difficulty in possible model conceptions. To emphasize the importance of biological conclusions drawn from these models, we briefly present available mathematical techniques and introduce some existing "measures of success" to compare and contrast the possible predictions and outcomes. Throughout the review, we propose numerous open problems relevant to this relatively new area, ranging from precise technical mathematical challenges to broader conceptual challenges at the cross-section between mathematics and ecology. This review paper is expected to act as a synthesis of existing efforts while also pushing the boundaries of current modelling perspectives to better understand the influence of cognitive movement mechanisms on movement behaviours and space use outcomes.

math.AP↗
arXiv2023

Betti Number for Point Sets

Topology is the foundation for many industrial applications ranging from CAD to simulation analysis. Computational topology mostly focuses on structured data such as mesh, however unstructured dataset such as point set remains a virgin land for topology scientists. The significance of point-based topology can never be overemphasized, especially in the area of reverse engineering, geometric modeling and algorithmic analysis. In this paper, we propose a novel approach to compute the Betti number for point set data and illustrate its usefulness in real world examples. To the best of our knowledge, our work is pioneering and first of its kind in the fields of computational topology.

cs.CG↗
arXiv2023

PowerMat: context-aware recommender system without user item rating values that solves the cold-start problem

Recommender systems serves as an important technical asset in many modern companies. With the increasing demand for higher precision of the technology, more and more research and investment has been allocated to the field. One important sub-field of recommender systems that has been stagnating is context-aware recommender systems. Due to the difficulty of collecting input dataset, the amount of research on context-aware recommender systems is much less than other sub-fields of recommender systems. In this paper, we propose a new algorithm named PowerMat to tackle the context-aware recommendation problem. We build our theory on matrix factorization and Zipf's law, and also more recent research work such as DotMat. We prove by experiments that our method achieves superior results to the classic matrix factorization algorithm and other context-aware recommender systems such as MovieMat+. In addition, by theoretical analysis, we show that our algorithm solves the cold-start problem for context-aware recommendation.

cs.IR↗
arXiv2023

Kernel-CF: Collaborative filtering done right with social network analysis and kernel smoothing

Collaborative filtering is the simplest but oldest machine learning algorithm in the field of recommender systems. In spite of its long history, it remains a discussion topic in research venues. Usually people use users/items whose similarity scores with the target customer greater than 0 to compute the algorithms. However, this might not be the optimal solution after careful scrutiny. In this paper, we transform the recommender system input data into a 2-D social network, and apply kernel smoothing to compute preferences for unknown values in the user item rating matrix. We unifies the theoretical framework of recommender system and non-parametric statistics and provides an algorithmic procedure with optimal parameter selection method to achieve the goal.

cs.IR↗