arXiv subjects

Haoyu Song

Publications and source records attributed to Haoyu Song.

32 records · Page 2Linked to original sources

arXiv2022

Towards End-to-End Quality-of-Service by Domain-Level Routing and Forwarding

We propose to raise network domain to the first-class citizen in packet routing and forwarding. By making domain explicit in packet header, it can be held responsible for committed services and quality of service. Since the service is easier to be monitored, verified, and compensated, a new business model catering for emerging cross-domain, high-value, and performance-sensitive applications can be fostered. The domain-level routing and forwarding essentially create a L3.5 in the network protocol stack. We extend IPv6 with new routing headers to support two types of forwarding approaches. One is based on the next domain lookup and the other realizes the domain-level source routing. We describe various design details covering header format, forwarding behavior, routing protocol, DNS, and OAM. Several use cases are presented to demonstrate the scheme potential. The design is compatible with today's Internet architecture and incrementally deployable on existing infrastructure with only moderate updates needed. We expect the preliminary work can trigger more discussion and detailed design in the research and Internet standard communities.

cs.NI↗

arXiv2022

Language Models are General-Purpose Interfaces

Foundation models have received much attention due to their effectiveness across a broad range of downstream applications. Though there is a big convergence in terms of architecture, most pretrained models are typically still developed for specific tasks or modalities. In this work, we propose to use language models as a general-purpose interface to various foundation models. A collection of pretrained encoders perceive diverse modalities (such as vision, and language), and they dock with a language model that plays the role of a universal task layer. We propose a semi-causal language modeling objective to jointly pretrain the interface and the modular encoders. We subsume the advantages and capabilities from both causal and non-causal modeling, thereby combining the best of two worlds. Specifically, the proposed method not only inherits the capabilities of in-context learning and open-ended generation from causal language modeling, but also is conducive to finetuning because of the bidirectional encoders. More importantly, our approach seamlessly unlocks the combinations of the above capabilities, e.g., enabling in-context learning or instruction following with finetuned encoders. Experimental results across various language-only and vision-language benchmarks show that our model outperforms or is competitive with specialized models on finetuning, zero-shot generalization, and few-shot learning.

cs.CL↗

arXiv2022

CLIP Models are Few-shot Learners: Empirical Studies on VQA and Visual Entailment

CLIP has shown a remarkable zero-shot capability on a wide range of vision tasks. Previously, CLIP is only regarded as a powerful visual encoder. However, after being pre-trained by language supervision from a large amount of image-caption pairs, CLIP itself should also have acquired some few-shot abilities for vision-language tasks. In this work, we empirically show that CLIP can be a strong vision-language few-shot learner by leveraging the power of language. We first evaluate CLIP's zero-shot performance on a typical visual question answering task and demonstrate a zero-shot cross-modality transfer capability of CLIP on the visual entailment task. Then we propose a parameter-efficient fine-tuning strategy to boost the few-shot performance on the vqa task. We achieve competitive zero/few-shot results on the visual question answering and visual entailment tasks without introducing any additional pre-training procedure.

cs.CV↗

arXiv2021

BoB: BERT Over BERT for Training Persona-based Dialogue Models from Limited Personalized Data

Maintaining consistent personas is essential for dialogue agents. Although tremendous advancements have been brought, the limited-scale of annotated persona-dense data are still barriers towards training robust and consistent persona-based dialogue models. In this work, we show how the challenges can be addressed by disentangling persona-based dialogue generation into two sub-tasks with a novel BERT-over-BERT (BoB) model. Specifically, the model consists of a BERT-based encoder and two BERT-based decoders, where one decoder is for response generation, and another is for consistency understanding. In particular, to learn the ability of consistency understanding from large-scale non-dialogue inference data, we train the second decoder in an unlikelihood manner. Under different limited data settings, both automatic and human evaluations demonstrate that the proposed model outperforms strong baselines in response quality and persona consistency.

cs.CL↗

arXiv2021

Profile Consistency Identification for Open-domain Dialogue Agents

Maintaining a consistent attribute profile is crucial for dialogue agents to naturally converse with humans. Existing studies on improving attribute consistency mainly explored how to incorporate attribute information in the responses, but few efforts have been made to identify the consistency relations between response and attribute profile. To facilitate the study of profile consistency identification, we create a large-scale human-annotated dataset with over 110K single-turn conversations and their key-value attribute profiles. Explicit relation between response and profile is manually labeled. We also propose a key-value structure information enriched BERT model to identify the profile consistency, and it gained improvements over strong baselines. Further evaluations on downstream tasks demonstrate that the profile consistency identification model is conducive for improving dialogue consistency.

cs.CL↗

arXiv2021

Generating Persona Consistent Dialogues by Exploiting Natural Language Inference

Consistency is one of the major challenges faced by dialogue agents. A human-like dialogue agent should not only respond naturally, but also maintain a consistent persona. In this paper, we exploit the advantages of natural language inference (NLI) technique to address the issue of generating persona consistent dialogues. Different from existing work that re-ranks the retrieved responses through an NLI model, we cast the task as a reinforcement learning problem and propose to exploit the NLI signals from response-persona pairs as rewards for the process of dialogue generation. Specifically, our generator employs an attention-based encoder-decoder to generate persona-based responses. Our evaluator consists of two components: an adversarially trained naturalness module and an NLI based consistency module. Moreover, we use another well-performed NLI model in the evaluation of persona-consistency. Experimental results on both human and automatic metrics, including the model-based consistency evaluation, demonstrate that the proposed approach outperforms strong generative baselines, especially in the persona-consistency of generated responses.

cs.AI↗

arXiv2021

Strongly Local Hypergraph Diffusions for Clustering and Semi-supervised Learning

Hypergraph-based machine learning methods are now widely recognized as important for modeling and using higher-order and multiway relationships between data objects. Local hypergraph clustering and semi-supervised learning specifically involve finding a well-connected set of nodes near a given set of labeled vertices. Although many methods for local clustering exist for graphs, there are relatively few for localized clustering in hypergraphs. Moreover, those that exist often lack flexibility to model a general class of hypergraph cut functions or cannot scale to large problems. To tackle these issues, this paper proposes a new diffusion-based hypergraph clustering algorithm that solves a quadratic hypergraph cut based objective akin to a hypergraph analog of Andersen-Chung-Lang personalized PageRank clustering for graphs. We prove that, for graphs with fixed maximum hyperedge size, this method is strongly local, meaning that its runtime only depends on the size of the output instead of the size of the hypergraph and is highly scalable. Moreover, our method enables us to compute with a wide variety of cardinality-based hypergraph cut functions. We also prove that the clusters found by solving the new objective function satisfy a Cheeger-like quality guarantee. We demonstrate that on large real-world hypergraphs our new method finds better clusters and runs much faster than existing approaches. Specifically, it runs in few seconds for hypergraphs with a few million hyperedges compared with minutes for flow-based technique. We furthermore show that our framework is general enough that can also be used to solve other p-norm based cut objectives on hypergraphs. Our code is available \url{github.com/MengLiuPurdue/LHQD}.

cs.SI↗

arXiv2020

Generate, Delete and Rewrite: A Three-Stage Framework for Improving Persona Consistency of Dialogue Generation

Maintaining a consistent personality in conversations is quite natural for human beings, but is still a non-trivial task for machines. The persona-based dialogue generation task is thus introduced to tackle the personality-inconsistent problem by incorporating explicit persona text into dialogue generation models. Despite the success of existing persona-based models on generating human-like responses, their one-stage decoding framework can hardly avoid the generation of inconsistent persona words. In this work, we introduce a three-stage framework that employs a generate-delete-rewrite mechanism to delete inconsistent words from a generated response prototype and further rewrite it to a personality-consistent one. We carry out evaluations by both human and automatic metrics. Experiments on the Persona-Chat dataset show that our approach achieves good performance.

cs.CL↗

arXiv2019

Combinatorial minimal surfaces in pseudomanifolds

We define combinatorial analogues of stable and unstable minimal surfaces in the setting of weighted pseudomanifolds. We prove that, under mild conditions, such combinatorial minimal surfaces always exist. We use a technique, adapted from work of Johnson and Thompson, called thin position. Thin position is defined using orderings of the cells of a pseudomanifold. In addition to defining and finding combinatorial minimal surfaces, from thin orderings, we derive invariants of even-dimensional closed simplicial pseudomanifolds called width and trunk. We study additivity properties of these invariants under connected sum and prove theorems analogous to those in knot theory and 3-manifold theory.

math.GT↗

arXiv2019

Tiny Buffer TCP for Data Center Networks

A low and stable buffer occupancy is critical to achieve high throughput, low packet drop rate, low latency, and low jitter for data center networks. It also allows switch chips to support higher port density, larger lookup tables, or richer functions. Tiny Buffer TCP creatively uses the common RED-based ECN with two novel congestion-window adjustment schemes to significantly reduce the required buffer size. Aiming to eliminate the residual packets in the bottleneck queue, Queue Canceling Decrease amortizes the ideal window reduction to the same number of flows so as to minimize the impact to active flows. In order to keep the buffer occupancy low and stable, Reduced Additive Increase recovers the flow window at a slower pace than normal. We implemented TBTCP in Linux kernel and conducted ns2-based simulations and real network-based tests. Our results show that compared to DCTCP, TBTCP reduces the switch buffer requirement by more than 80%, increases the bandwidth utilization by up to 15%, improves the FCT performance by up to 39%, and achieve a 71% better RTT fairness index.

cs.NI↗

arXiv2019

Exploiting Persona Information for Diverse Generation of Conversational Responses

In human conversations, due to their personalities in mind, people can easily carry out and maintain the conversations. Giving conversational context with persona information to a chatbot, how to exploit the information to generate diverse and sustainable conversations is still a non-trivial task. Previous work on persona-based conversational models successfully make use of predefined persona information and have shown great promise in delivering more realistic responses. And they all learn with the assumption that given a source input, there is only one target response. However, in human conversations, there are massive appropriate responses to a given input message. In this paper, we propose a memory-augmented architecture to exploit persona information from context and incorporate a conditional variational autoencoder model together to generate diverse and sustainable conversations. We evaluate the proposed model on a benchmark persona-chat dataset. Both automatic and human evaluations show that our model can deliver more diverse and more engaging persona-based responses than baseline approaches.

cs.AI↗

arXiv2016

Dynamic Network Probes: A Stepping Stone to Omni Network Visibility

Effective SDN control relies on the network data collecting capability as well as the quality and timeliness of the data. As open programmable data plane is becoming a reality, we further enhance it with the support of runtime interactive programming in order to cope with application dynamics, optimize data plane resource allocation, and reduce control-plane processing pressure. Based on the latest technologies, we propose the Dynamic Network Probes (DNP) as a means to support real-time and on-demand network visibility. DNPs serve as an important building block of an integrated networking data analytics platform which involves the network data plane as an active component for in-network computing. In this paper, we show the types of DNPs and their role in the big picture. We have implemented an NP-based hardware prototype to demonstrate the feasibility and efficiency of DNPs. We lay out the research challenges and our future work to realize the omni network visibility based on DNPs.

cs.NI↗

arXiv2016

Network Map Reduce

Networking data analytics is increasingly used for enhanced network visibility and controllability. We draw the similarities between the Software Defined Networking (SDN) architecture and the MapReduce programming model. Inspired by the similarity, we suggest the necessary data plane innovations to make network data plane devices function as distributed mappers and optionally, reducers. A streaming network data MapReduce architecture can therefore conveniently solve a series of network monitoring and management problems. Unlike the traditional networking data analytical system, our proposed system embeds the data analytics engine directly in the network infrastructure. The affinity leads to a concise system architecture and better cost performance ratio. On top of this architecture, we propose a general MapReduce-like programming model for real-time and one-pass networking data analytics, which involves joint in-network and out-of-network computing. We show this model can address a wide range of interactive queries from various network applications. This position paper strives to make a point that the white-box trend does not necessarily lead to simple and dumb networking devices. Rather, the defining characteristics of the next generation white-box are open and programmable, so that the network devices can be made smart and versatile to support new services and applications.

cs.NI↗

arXiv2014

Unified POF Programming for Diversified SDN Data Plane

In many real-world OpenFlow-based SDN deployments, the ability to program heterogeneous forwarding elements built with different forwarding architectures is a desirable capability. In this paper, we discuss a data plane programming framework suitable for a flexible and protocol-oblivious data plane and show how OpenFlow can evolve to provide a generic interface for platform-independent programming and platform-specific compiling. We also show how an abstract instruction set can play a pivotal role to support different programming styles mapping to different forwarding chip architectures. As an example, we compare the compiler-mode and interpreter-mode implementations for an NPU-based forwarding element and conclude that the compiler-mode implementation can achieve a performance similar to that of a conventional non-SDN implementation. Built upon our protocol-oblivious forwarding (POF) vision, this work presents our continuous efforts to complete the ecosystem and pave the SDN evolving path. The programming framework could be considered as a proposal for the OpenFlow 2.0 standard.

cs.NI↗