arXiv subjects

Zheng Hu

Publications and source records attributed to Zheng Hu.

36 records · Page 2Linked to original sources

arXiv2024

WEPO: Web Element Preference Optimization for LLM-based Web Navigation

The rapid advancement of autonomous web navigation has significantly benefited from grounding pretrained Large Language Models (LLMs) as agents. However, current research has yet to fully leverage the redundancy of HTML elements for contrastive training. This paper introduces a novel approach to LLM-based web navigation tasks, called Web Element Preference Optimization (WEPO). WEPO utilizes unsupervised preference learning by sampling distance-based non-salient web elements as negative samples, optimizing maximum likelihood objective within Direct Preference Optimization (DPO). We evaluate WEPO on the Mind2Web benchmark and empirically demonstrate that WEPO aligns user high-level intent with output actions more effectively. The results show that our method achieved the state-of-the-art, with an improvement of 13.8% over WebAgent and 5.3% over the visual language model CogAgent baseline. Our findings underscore the potential of preference optimization to enhance web navigation and other web page based tasks, suggesting a promising direction for future research.

cs.CL↗

arXiv2024

Training Overhead Ratio: A Practical Reliability Metric for Large Language Model Training Systems

Large Language Models (LLMs) are revolutionizing the AI industry with their superior capabilities. Training these models requires large-scale GPU clusters and significant computing time, leading to frequent failures that significantly increase training costs. Despite its significance, this field lacks a metric for evaluating reliability. In this work, we introduce a novel reliability metric called \emph{Training Overhead Ratio} (TOR) to evaluate the reliability of fault-tolerant LLM training systems. TOR is defined as the ratio of optimal training time to the observed training time of a system, serving as a practical tool for users to estimate the actual time required to train an LLM on a given system. Furthermore, our investigation identifies the key factor for enhancing reliability and present TOR equations for various types of failures encountered in practice.

cs.DC↗

arXiv2024

Desigen: A Pipeline for Controllable Design Template Generation

Templates serve as a good starting point to implement a design (e.g., banner, slide) but it takes great effort from designers to manually create. In this paper, we present Desigen, an automatic template creation pipeline which generates background images as well as harmonious layout elements over the background. Different from natural images, a background image should preserve enough non-salient space for the overlaying layout elements. To equip existing advanced diffusion-based models with stronger spatial control, we propose two simple but effective techniques to constrain the saliency distribution and reduce the attention weight in desired regions during the background generation process. Then conditioned on the background, we synthesize the layout with a Transformer-based autoregressive generator. To achieve a more harmonious composition, we propose an iterative inference strategy to adjust the synthesized background and layout in multiple rounds. We constructed a design dataset with more than 40k advertisement banners to verify our approach. Extensive experiments demonstrate that the proposed pipeline generates high-quality templates comparable to human designers. More than a single-page design, we further show an application of presentation generation that outputs a set of theme-consistent slides. The data and code are available at https://whaohan.github.io/desigen.

cs.CV↗

arXiv2023

A Survey for Graphic Design Intelligence

Graphic design is an effective language for visual communication. Using complex composition of visual elements (e.g., shape, color, font) guided by design principles and aesthetics, design helps produce more visually-appealing content. The creation of a harmonious design requires carefully selecting and combining different visual elements, which can be challenging and time-consuming. To expedite the design process, emerging AI techniques have been proposed to automatize tedious tasks and facilitate human creativity. However, most current works only focus on specific tasks targeting at different scenarios without a high-level abstraction. This paper aims to provide a systematic overview of graphic design intelligence and summarize literature in the taxonomy of representation, understanding and generation. Specifically we consider related works for individual visual elements as well as the overall design composition. Furthermore, we highlight some of the potential directions for future explorations.

cs.HC↗

arXiv2023

Bert4XMR: Cross-Market Recommendation with Bidirectional Encoder Representations from Transformer

Real-world multinational e-commerce companies, such as Amazon and eBay, serve in multiple countries and regions. Some markets are data-scarce, while others are data-rich. In recent years, cross-market recommendation (XMR) has been proposed to bolster data-scarce markets by leveraging auxiliary information from data-rich markets. Previous XMR algorithms have employed techniques such as sharing bottom or incorporating inter-market similarity to optimize the performance of XMR. However, the existing approaches suffer from two crucial limitations: (1) They ignore the co-occurrences of items provided by data-rich markets. (2) They do not adequately tackle the issue of negative transfer stemming from disparities across diverse markets. In order to address these limitations, we propose a novel session-based model called Bert4XMR, which is able to model item co-occurrences across markets and mitigate negative transfer. Specifically, we employ the pre-training and fine-tuning paradigm to facilitate knowledge transfer across markets. Pre-training occurs on global markets to learn item co-occurrences, while fine-tuning happens in the target market for model customization. To mitigate potential negative transfer, we separate the item representations into market embeddings and item embeddings. Market embeddings model the bias associated with different markets, while item embeddings learn generic item representations. Extensive experiments conducted on seven real-world datasets illustrate our model's effectiveness. It outperforms the suboptimal model by an average of $4.82\%$, $4.73\%$, $7.66\%$, and $6.49\%$ across four metrics. Through the ablation study, we experimentally demonstrate that the market embedding approach helps prevent negative transfer, especially in data-scarce markets. Our implementations are available at https://github.com/laowangzi/Bert4XMR.

cs.IR↗

arXiv2023

DistSim: A performance model of large-scale hybrid distributed DNN training

With the ever-increasing computational demand of DNN training workloads, distributed training has been widely adopted. A combination of data, model and pipeline parallelism strategy, called hybrid parallelism distributed training, is imported to tackle the problem of deploying large-scale models. However, how to evaluate the hybrid strategy and the utilization of each device remains a challenge since existing works either profile on a real large-scale cluster with high time and money costs or only analyze a specific type of parallelism without considering the hybrid parallelism. In this work, we proposed DistSim, an event-based performance model to accurately analyze each device's computation and communication activities with low profiling costs. DistDim breaks down the model into events according to the given distributed strategy, which can be profiled on two nodes. Then DistSim leverages the hierarchy of different parallel strategies to generate the computation and communication event-flow from layer level to model level and finally the activity timeline of each device participating in training. Experiment shows that DistSim can reach \revise{<4\%} errors when predicting distributing training batch time and \revise{<5\%} errors when predicting a single device's activity time in various hybrid strategy settings. We also provide a use-case of DistSim, automatically evaluate and search the best distributed training strategy, and find a hybrid strategy with at most $7.37\times$ throughput improvement.

cs.DC↗

arXiv2023

Collaborative Recommendation Model Based on Multi-modal Multi-view Attention Network: Movie and literature cases

The existing collaborative recommendation models that use multi-modal information emphasize the representation of users' preferences but easily ignore the representation of users' dislikes. Nevertheless, modelling users' dislikes facilitates comprehensively characterizing user profiles. Thus, the representation of users' dislikes should be integrated into the user modelling when we construct a collaborative recommendation model. In this paper, we propose a novel Collaborative Recommendation Model based on Multi-modal multi-view Attention Network (CRMMAN), in which the users are represented from both preference and dislike views. Specifically, the users' historical interactions are divided into positive and negative interactions, used to model the user's preference and dislike views, respectively. Furthermore, the semantic and structural information extracted from the scene is employed to enrich the item representation. We validate CRMMAN by designing contrast experiments based on two benchmark MovieLens-1M and Book-Crossing datasets. Movielens-1m has about a million ratings, and Book-Crossing has about 300,000 ratings. Compared with the state-of-the-art knowledge-graph-based and multi-modal recommendation methods, the AUC, NDCG@5 and NDCG@10 are improved by 2.08%, 2.20% and 2.26% on average of two datasets. We also conduct controlled experiments to explore the effects of multi-modal information and multi-view mechanism. The experimental results show that both of them enhance the model's performance.

cs.IR↗

arXiv2022

Hard-sample Guided Hybrid Contrast Learning for Unsupervised Person Re-Identification

Unsupervised person re-identification (Re-ID) is a promising and very challenging research problem in computer vision. Learning robust and discriminative features with unlabeled data is of central importance to Re-ID. Recently, more attention has been paid to unsupervised Re-ID algorithms based on clustered pseudo-label. However, the previous approaches did not fully exploit information of hard samples, simply using cluster centroid or all instances for contrastive learning. In this paper, we propose a Hard-sample Guided Hybrid Contrast Learning (HHCL) approach combining cluster-level loss with instance-level loss for unsupervised person Re-ID. Our approach applies cluster centroid contrastive loss to ensure that the network is updated in a more stable way. Meanwhile, introduction of a hard instance contrastive loss further mines the discriminative information. Extensive experiments on two popular large-scale Re-ID benchmarks demonstrate that our HHCL outperforms previous state-of-the-art methods and significantly improves the performance of unsupervised person Re-ID. The code of our work is available soon at https://github.com/bupt-ai-cz/HHCL-ReID.

cs.CV↗

arXiv2022

Multi-sensor Suboptimal Fusion Student's $t$ Filter

A multi-sensor fusion Student's $t$ filter is proposed for time-series recursive estimation in the presence of heavy-tailed process and measurement noises. Driven from an information-theoretic optimization, the approach extends the single sensor Student's $t$ Kalman filter based on the suboptimal arithmetic average (AA) fusion approach. To ensure computationally efficient, closed-form $t$ density recursion, reasonable approximation has been used in both local-sensor filtering and inter-sensor fusion calculation. The overall framework accommodates any Gaussian-oriented fusion approach such as the covariance intersection (CI). Simulation demonstrates the effectiveness of the proposed multi-sensor AA fusion-based $t$ filter in dealing with outliers as compared with the classic Gaussian estimator, and the advantage of the AA fusion in comparison with the CI approach and the augmented measurement fusion.

eess.SY↗

arXiv2021

Construct Informative Triplet with Two-stage Hard-sample Generation

In this paper, we propose a robust sample generation scheme to construct informative triplets. The proposed hard sample generation is a two-stage synthesis framework that produces hard samples through effective positive and negative sample generators in two stages, respectively. The first stage stretches the anchor-positive pairs with piecewise linear manipulation and enhances the quality of generated samples by skillfully designing a conditional generative adversarial network to lower the risk of mode collapse. The second stage utilizes an adaptive reverse metric constraint to generate the final hard samples. Extensive experiments on several benchmark datasets verify that our method achieves superior performance than the existing hard-sample generation algorithms. Besides, we also find that our proposed hard sample generation method combining the existing triplet mining strategies can further boost the deep metric learning performance.

cs.CV↗

arXiv2021

Robust Dynamic Network Embedding via Ensembles

Dynamic Network Embedding (DNE) has recently attracted considerable attention due to the advantage of network embedding in various fields and the dynamic nature of many real-world networks. An input dynamic network to DNE is often assumed to have smooth changes over snapshots, which however would not hold for all real-world scenarios. It is natural to ask if existing DNE methods can perform well for an input dynamic network without smooth changes. To quantify it, an index called Degree of Changes (DoCs) is suggested so that the smaller DoCs indicates the smoother changes. Our comparative study shows several DNE methods are not robust enough to different DoCs even if the corresponding input dynamic networks come from the same dataset, which would make these methods unreliable and hard to use for unknown real-world applications. To propose an effective and more robust DNE method, we follow the notion of ensembles where each base learner adopts an incremental Skip-Gram embedding model. To further boost the performance, a simple yet effective strategy is designed to enhance the diversity among base learners at each timestep by capturing different levels of local-global topology. Extensive experiments demonstrate the superior effectiveness and robustness of the proposed method compared to state-of-the-art DNE methods, as well as the benefits of special designs in the proposed method and its scalability.

cs.SI↗

arXiv2021

Personal Fixations-Based Object Segmentation with Object Localization and Boundary Preservation

As a natural way for human-computer interaction, fixation provides a promising solution for interactive image segmentation. In this paper, we focus on Personal Fixations-based Object Segmentation (PFOS) to address issues in previous studies, such as the lack of appropriate dataset and the ambiguity in fixations-based interaction. In particular, we first construct a new PFOS dataset by carefully collecting pixel-level binary annotation data over an existing fixation prediction dataset, such dataset is expected to greatly facilitate the study along the line. Then, considering characteristics of personal fixations, we propose a novel network based on Object Localization and Boundary Preservation (OLBP) to segment the gazed objects. Specifically, the OLBP network utilizes an Object Localization Module (OLM) to analyze personal fixations and locates the gazed objects based on the interpretation. Then, a Boundary Preservation Module (BPM) is designed to introduce additional boundary information to guard the completeness of the gazed objects. Moreover, OLBP is organized in the mixed bottom-up and top-down manner with multiple types of deep supervision. Extensive experiments on the constructed PFOS dataset show the superiority of the proposed OLBP network over 17 state-of-the-art methods, and demonstrate the effectiveness of the proposed OLM and BPM components. The constructed PFOS dataset and the proposed OLBP network are available at https://github.com/MathLee/OLBPNet4PFOS.

cs.CV↗

arXiv2020

CRATOS: Cognition of Reliable Algorithm for Time-series Optimal Solution

Anomaly detection of time series plays an important role in reliability systems engineering. However, in practical application, there is no precisely defined boundary between normal and anomalous behaviors in different application scenarios. Therefore, different anomaly detection algorithms and processes ought to be adopted for time series in different situation. Although such strategy improve the accuracy of anomaly detection, it takes a lot of time for practitioners to configure various algorithms to millions of series, which greatly increases the development and maintenance cost of anomaly detection processes. In this paper, we propose CRATOS which is a self-adapt algorithms that extract features from time series, and then cluster series with similar features into one group. For each group we utilize evolutionary algorithm to search the best anomaly detection methods and processes. Our methods can significantly reduce the cost of development and maintenance of anomaly detection. According to experiments, our clustering methods achieves the state-of-art results. The accuracy of the anomaly detection algorithms in this paper is 85.1%.

cs.LG↗

arXiv2020

Superconductivity in Sm-doped 1,3,5-triphenylbenzene

We report the discovery of superconductivity at about 4.3 K in the samarium doped 1,3,5-triphenylbenzene. By using a solid state reaction method, the samarium doped 1,3,5-triphenylbenzene samples are successfully synthesized. These samples are characterized by magnetization, X-ray diffraction (XRD), scanning electron microscope(SEM) and energy dispersive spectroscopy (EDS) measurements. The X-ray diffractions reveal that the sample crystallizes in a space group of $P2/m$. The magnetization measurements reveal a superconducting transition at about 4.3 K. However, the superconducting shielding fraction is only about 1%, which is similar to the previous reports of the superconductivity in other aromatic hydrocarbons. Magnetization hysteresis loops (MHLs) of the sample show that it is a typical type-II superconductor.

cond-mat.supr-con↗

arXiv2018

Unprecedented High Irreversibility Line in Nontoxic Cuprate Superconductor (Cu,C)Ba2Ca3Cu4O11+delta

One of the key factors limiting the high power applications for a type-II superconductor is the irreversibility line Hirr(T) which reflects the very boundary of resistive dissipation in the phase diagram of magnetic field versus temperature. In cuprate family, the Y-, Bi-, Hg- and Tl-based systems have superconducting transition temperatures exceeding the liquid nitrogen boiling temperature (~77K). However, the toxic elements Hg and Tl in the latter two systems strongly constrain the possible applications. The Bi-based (2223) system is nontoxic, but the irreversibility magnetic field is strongly suppressed in the liquid nitrogen temperature region. For this purpose, the best perspective so far is relying on the YBa2Cu3O7 (Tc~90 K) system which is nontoxic and has a relatively high irreversibility magnetic field. Here we report the study of a nontoxic superconductor (Cu,C)Ba2Ca3Cu4O11+δ with Tc = 116K. It is found that the irreversibility magnetic field is unprecedentedly high among all superconductors and it thus provides a great potential of applications in the liquid nitrogen temperature region.

cond-mat.supr-con↗

arXiv2016

On Tie Strength Augmented Social Correlation for Inferring Preference of Mobile Telco Users

For mobile telecom operators, it is critical to build preference profiles of their customers and connected users, which can help operators make better marketing strategies, and provide more personalized services. With the deployment of deep packet inspection (DPI) in telecom networks, it is possible for the telco operators to obtain user online preference. However, DPI has its limitations and user preference derived only from DPI faces sparsity and cold start problems. To better infer the user preference, social correlation in telco users network derived from Call Detailed Records (CDRs) with regard to online preference is investigated. Though widely verified in several online social networks, social correlation between online preference of users in mobile telco networks, where the CDRs derived relationship are of less social properties and user mobile internet surfing activities are not visible to neighbourhood, has not been explored at a large scale. Based on a real world telecom dataset including CDRs and preference of more than $550K$ users for several months, we verified that correlation does exist between online preference in such \textit{ambiguous} social network. Furthermore, we found that the stronger ties that users build, the more similarity between their preference may have. After defining the preference inferring task as a Top-$K$ recommendation problem, we incorporated Matrix Factorization Collaborative Filtering model with social correlation and tie strength based on call patterns to generate Top-$K$ preferred categories for users. The proposed Tie Strength Augmented Social Recommendation (TSASoRec) model takes data sparsity and cold start user problems into account, considering both the recorded and missing recorded category entries. The experiment on real dataset shows the proposed model can better infer user preference, especially for cold start users.

cs.SI↗

arXiv2016

Critical size of ego communication networks

With the help of information and communication technologies, studies on the overall social networks have been extensively reported recently. However, investigations on the directed Ego Communication Networks (ECNs) remain insufficient, where an ECN stands for a sub network composed of a centralized individual and his/her direct contacts. In this paper, the directed ECNs are built on the Call Detail Records (CDRs), which cover more than 7 million people of a provincial capital city in China for half a year. Results show that there is a critical size for ECN at about 150, above which the average emotional closeness between ego and alters drops, the balanced relationship between ego and network collapses, and the proportion of strong ties decreases. This paper not only demonstrate the significance of ECN size in affecting its properties, but also shows accordance with the "Dunbar's Number". These results can be viewed as a cross-culture supportive evidence to the well-known Social Brain Hypothesis (SBH).

physics.soc-ph↗

arXiv2015

Consistence beats causality in recommender systems

The explosive growth of information challenges people's capability in finding out items fitting to their own interests. Recommender systems provide an efficient solution by automatically push possibly relevant items to users according to their past preferences. Recommendation algorithms usually embody the causality from what having been collected to what should be recommended. In this article, we argue that in many cases, a user's interests are stable, and thus the previous and future preferences are highly consistent. The temporal order of collections then does not necessarily imply a causality relationship. We further propose a consistence-based algorithm that outperforms the state-of-the-art recommendation algorithms in disparate real data sets, including \textit{Netflix}, \textit{MovieLens}, \textit{Amazon} and \textit{Rate Your Music}.

cs.IR↗