arXiv subjects

Luís Veiga

Publications and source records attributed to Luís Veiga.

18 recordsLinked to original sources

arXiv2026

Survey: On the Landscape of Graph Databases

Graph databases have become essential tools for managing complex and interconnected data, which is common in areas like social networks, bioinformatics, and recommendation systems. Unlike traditional relational databases, graph databases offer a more natural way to model and query intricate relationships, making them particularly effective for applications that demand flexibility and efficiency in handling interconnected data. Despite their increasing use, graph databases face notable challenges. One significant issue is the irregular nature of graph data, often marked by structural sparsity, such as in its adjacency matrix representation, which can lead to inefficiencies in data read and write operations. Other obstacles include the high computational demands of traversal-based queries, especially within large-scale networks, and complexities in managing transactions in distributed graph environments. Additionally, the reliance on traditional centralized architectures limits the scalability of Online Transaction Processing (OLTP), creating bottlenecks due to contention, CPU overhead, and network bandwidth constraints. This paper presents a thorough survey of graph databases. It begins by examining property models, query languages, and storage architectures, outlining the foundational aspects that users and developers typically engage with. Following this, it provides a detailed analysis of recent advancements in graph database technologies, evaluating these in the context of key aspects such as architecture, deployment, usage, and development, which collectively define the capabilities of graph database solutions.

cs.DB↗

arXiv2022

Adaptive Edge Content Delivery Networks for Web-Scale File Systems

The InterPlanetary File System (IPFS) is an hyper-media distribution protocol, addressed by content and identities. It aims to make the web faster, safer, and more open. The JavaScript implementation of IPFS runs on the browser, benefiting from the mass adoption potential that it yields. Startrail takes advantage of the IPFS ecosystem and strives to further evolve it, making it more scalable and performant through the implementation of an adaptive network caching mechanism. Our solution aims to add resilience to IPFS and improve its overall scalability, by avoiding overloading the nodes providing highly popular content, particularly during flash-crowd-like conditions where popularity and demand grow suddenly. We add a novel crucial key component to enable an IPFS-based decentralized Content Distribution Network (CDN). Following a peer-to-peer architecture, it runs on a scalable, highly available network of untrusted nodes that distribute immutable and authenticated objects which are cached progressively towards the sources of requests.

cs.NI↗

arXiv2022

SmartPubSub: Content-based Pub-Sub on IPFS

The InterPlanetary File System (IPFS) is a hypermedia distribution protocol enabling the creation of completely distributed applications. One of the most efficient and effective ways to distribute information is through notifications, with a producer of content (publisher) sharing content with other interested parts (subscribers). IPFS already implements topic-based publish-subscribe systems under an experimental flag. The goal of this work is to advance on that, by developing a content-based pub-sub system (with subscriptions as predicates about event content) to disseminate information on top of IPFS in an efficient and decentralized way, leveraging its infrastructure. We design two protocols: ScoutSubs that is completely decentralized; FastDelivery that is centered in the publisher. With these two approaches, we show the different advantages of having each of these protocols simultaneously by comparing ScoutSubs full decentralization, and FastDelivery centralization at data sources.

cs.NI↗

arXiv2021

An analysis of the graph processing landscape

The value of graph-based big data can be unlocked by exploring the topology and metrics of the networks they represent, and the computational approaches to this exploration take on many forms. The use-case of performing global computations over a graph, it is first ingested into a graph processing system from one of many digital representations. Extracting information from graphs involves processing all their elements globally, and can be done with single-machine systems (with varying approaches to hardware usage), distributed systems (either homogeneous or heterogeneous groups of machines) and systems dedicated to high-performance computing (HPC). We provide an overview of different aspects of the graph processing landscape and describe classes of systems based on a set of dimensions we detail. The dimensions we detail encompass paradigms to express graph processing, different types of systems to use, coordination and communication models in distributed graph processing, partitioning techniques and different definitions related to the potential for a graph to be updated. This survey is aimed at both the experienced software engineer or researcher as well as the newcomer looking for an understanding of the landscape of solutions (and their limitations) for graph processing.

cs.DC↗

arXiv2020

Latency-Sensitive Web Service Workflows: A Case for a Software-Defined Internet

The Internet, at large, remains under the control of service providers and autonomous systems. The Internet of Things (IoT) and edge computing provide an increasing demand and potential for more user control for their web service workflows. Network Softwarization revolutionizes the network landscape in various stages, from building, incrementally deploying, and maintaining the environment. Software-Defined Networking (SDN) and Network Functions Virtualization (NFV) are two core tenets of network softwarization. SDN offers a logically centralized control plane by abstracting away the control of the network devices in the data plane. NFV virtualizes dedicated hardware middleboxes and deploys them on top of servers and data centers as network functions. Thus, network softwarization enables efficient management of the system by enhancing its control and improving the reusability of the network services. In this work, we propose our vision for a Software-Defined Internet (SDI) for latency-sensitive web service workflows. SDI extends network softwarization to the Internet-scale, to enable a latency-aware user workflow execution on the Internet.

cs.DC↗

arXiv2020

Aion: Better Late than Never in Event-Time Streams

Processing data streams in near real-time is an increasingly important task. In the case of event-timestamped data, the stream processing system must promptly handle late events that arrive after the corresponding window has been processed. To enable this late processing, the window state must be maintained for a long period of time. However, current systems maintain this state in memory, which either imposes a maximum period of tolerated lateness, or causes the system to degrade performance or even crash when the system memory runs out. In this paper, we propose AION, a comprehensive solution for handling late events in an efficient manner, implemented on top of Flink. In designing AION, we go beyond a naive solution that transfers state between memory and persistent storage on demand. In particular, we introduce a proactive caching scheme, where we leverage the semantics of stream processing to anticipate the need for bringing data to memory. Furthermore, we propose a predictive cleanup scheme to permanently discard window state based on the likelihood of receiving more late events, to prevent storage consumption from growing without bounds. Our evaluation shows that AION is capable of maintaining sustainable levels of memory utilization while still preserving high throughput, low latency, and low staleness.

cs.DC↗

arXiv2019

VeilGraph: Streaming Graph Approximations

Graphs are found in a plethora of domains, including online social networks, the World Wide Web and the study of epidemics, to name a few. With the advent of greater volumes of information and the need for continuously updated results under temporal constraints, it is necessary to explore novel approaches that further enable performance improvements. In the scope of stream processing over graphs, we research the trade-offs between result accuracy and the speedup of approximate computation techniques. We see this as a natural path towards these performance improvements. Herein we present \name, through which we conducted our research. We showcase an innovative model for approximate graph processing, implemented in \texttt{Apache Flink}. We analyze our model and evaluate it with the case study of the PageRank algorithm \cite{pageRank}, perhaps the most famous measure of vertex centrality used to rank websites in search engine results. %In light of our model, we discuss the challenges driven by relations between result accuracy and potential performance gains. Our experiments, even when set up for favoring \texttt{Flink} for comparability, show that \name can improve performance up to 3X speedups, while achieving result quality above 95\% when compared to results of the traditional version of PageRank without any summarization or approximation techniques.

cs.DC↗

arXiv2018

ROLP: Runtime Object Lifetime Profiling for Big Data Memory Management

Low latency services such as credit-card fraud detection and website targeted advertisement rely on Big Data platforms (e.g., Lucene, Graphchi, Cassandra) which run on top of memory managed runtimes, such as the JVM. These platforms, however, suffer from unpredictable and unacceptably high pause times due to inadequate memory management decisions (e.g., allocating objects with very different lifetimes next to each other, resulting in memory fragmentation). This leads to long and frequent application pause times, breaking Service Level Agreements (SLAs). This problem has been previously identified and results show that current memory management techniques are ill-suited for applications that hold in memory massive amounts of middle to long-lived objects (which is the case for a wide spectrum of Big Data applications). Previous works try to reduce such application pauses by allocating objects off-heap or in special allocation regions/generations, thus alleviating the pressure on memory management. However, all these solutions require a combination of programmer effort and knowledge, source code access, or off-line profiling, with clear negative impact on programmer productivity and/or application performance. This paper presents ROLP, a runtime object lifetime profiling system. ROLP profiles application code at runtime in order to identify which allocation contexts create objects with middle to long lifetimes, given that such objects need to be handled differently (regarding short-lived ones). This profiling information greatly improves memory management decisions, leading to long tail latencies reduction of up to 51% for Lucene, 85% for GraphChi, and 60% for Cassandra, with negligible throughput and memory overhead. ROLP is implemented for the OpenJDK 8 HotSpot JVM and it does not require any programmer effort or source code access.

cs.DC↗

arXiv2018

Gelly-Scheduling: Distributed Graph Processing for Network Service Placement in Community Networks

Community networks (CNs) have seen an increase in the last fifteen years. Their members contact nodes which operate Internet proxies, web servers, user file storage and video streaming services, to name a few. Detecting communities of nodes with properties (such as co-location) and assessing node eligibility for service placement is thus a key-factor in optimizing the experience of users. We present a novel solution for the problem of service placement as a two-phase approach, based on: 1) community finding using a scalable graph label propagation technique and 2) a decentralized election procedure to address the multi-objective challenge of optimizing service placement in CNs. Herein we: i) highlight the applicability of leader election heuristics which are important for service placement in community networks and scheduler-dependent scenarios; ii) present a parallel and distributed solution designed as a scalable alternative for the problem of service placement, which has mostly seen computational approaches based on centralization and sequential execution.

cs.DC↗

arXiv2017

Client-Side Routing-Agnostic Gateway Selection for heterogeneous Wireless Mesh Networks

Citizens develop Wireless Mesh Networks (WMN) in many areas as an alternative or their only way for local interconnection and access to the Internet. This access is often achieved through the use of several shared web proxy gateways. These network infrastructures consist of heterogeneous technologies and combine diverse routing protocols. Network-aware state-of-art proxy selection schemes for WMNs do not work in this heterogeneous environment. We developed a client-side gateway selection mechanism that optimizes the client-gateway selection, agnostic to underlying infrastructure and protocols, requiring no modification of proxies nor the underlying network. The choice is sensitive to network congestion and proxy load, without requiring a minimum number of participating nodes. Extended Vivaldi network coordinates are used to estimate client-proxy network performance. The load of each proxy is estimated passively by collecting the Time-to-First-Byte of HTTP requests, and shared across clients. Our proposal was evaluated experimentally with clients and proxies deployed in guifi.net, the largest community wireless network in the world. Our selection mechanism avoids proxies with heavy load and slow internal network paths, with overhead linear to the number of clients and proxies.

cs.NI↗

arXiv2017

Gang-GC: Locality-aware Parallel Data Placement Optimizations for Key-Value Storages

Many cloud applications rely on fast and non-relational storage to aid in the processing of large amounts of data. Managed runtimes are now widely used to support the execution of several storage solutions of the NoSQL movement, particularly when dealing with big data key-value store-driven applications. The benefits of these runtimes can however be limited by modern parallel throughput-oriented GC algorithms, where related objects have the potential to be dispersed in memory, either in the same or different generations. In the long run this causes more page faults and degradation of locality on system-level memory caches. We propose, Gang-CG, an extension to modern heap layouts and to a parallel GC algorithm to promote locality between groups of related objects. This is done without extensive profiling of the applications and in a way that is transparent to the programmer, without the need to use specialized data structures. The heap layout and algorithmic extensions were implemented over the Parallel Scavenge garbage collector of the HotSpot JVM\@. Using microbenchmarks that capture the architecture of several key-value stores databases, we show negligible overhead in frequent operations such as the allocation of new objects and improvements to the access speed of data, supported by lower misses in system-level memory caches. Overall, we show a 6\% improvement in the average time of read and update operations and an average decrease of 12.4\% in page faults.

cs.PL↗

arXiv2017

SD-CPS: Taming the Challenges of Cyber-Physical Systems with a Software-Defined Approach

Cyber-Physical Systems (CPS) revolutionize various application domains with integration and interoperability of networking, computing systems, and mechanical devices. Due to its scale and variety, CPS faces a number of challenges and opens up a few research questions in terms of management, fault-tolerance, and scalability. We propose a software-defined approach inspired by Software-Defined Networking (SDN), to address the challenges for a wider CPS adoption. We thus design a middleware architecture for the correct and resilient operation of CPS, to manage and coordinate the interacting devices centrally in the cyberspace whilst not sacrificing the functionality and performance benefits inherent to a distributed execution.

cs.NI↗

arXiv2016

Smart Scheduling of Continuous Data-Intensive Workflows with Machine Learning Triggered Execution

To extract value from evergrowing volumes of data, coming from a number of different sources, and to drive decision making, organizations frequently resort to the composition of data processing workflows, since they are expressive, flexible, and scalable. The typical workflow model enforces strict temporal synchronization across processing steps without accounting the actual effect of intermediate computations on the final workflow output. However, this is not the most desirable behavior in a multitude of scenarios. We identify a class of applications for continuous data processing where workflow output changes slowly and without great significance in a short-to-medium time window, thus wasting compute resources and energy with current approaches. To overcome such inefficiency, we introduce a novel workflow model, for continuous and data-intensive processing, capable of relaxing triggering semantics according to the impact input data is assessed to have on changing the workflow output. To assess this impact, learn the correlation between input and output variation, and guarantee correctness within a given tolerated error constant, we rely on Machine Learning. The functionality of this workflow model is implemented in SmartFlux, a middleware framework which can be effortlessly integrated with existing workflow managers. Experimental results indicate we are able to save a significant amount of resources while not deviating the workflow output beyond a small error constant with high confidence level.

cs.DC↗

arXiv2016

JikesRVM: Internal Mechanisms Study and Garbage Collection with MMTk

High Level Language Virtual Machines is a core topic of interest for the researchers who are into virtual execution environments. As an open source virtual machine released to 16 universities, as early as 2001, Jikes RVM has been a major drive for many researches. While working on this project, we studied the JIT compilation of Jikes RVM as well as the Garbage Collection (GC) which is handled by the Memory Management Toolkit (MMTk), a part of the Jikes RVM. We also studied the Compressor Mark-Compact Collector algorithm and implemented it for MMTk. We have also implemented a micro-benchmark for the GC algorithms in Java, named "XPDBench", for benchmarking the implementations.

cs.PL↗

arXiv2016

SENDIM for Incremental Development of Cloud Networks

Due to the limited and varying availability of cheap infrastructure and resources, cloud network systems and applications are tested in simulation and emulation environments prior to physical deployments, at different stages of development. Configuration management tools manage deployments and migrations across different cloud platforms, mitigating tedious system administration efforts. However, currently a cloud networking simulation cannot be migrated as an emulation, or vice versa, without rewriting and manually re-deploying the simulated application. This paper presents SENDIM (Sendim is a northeastern Portuguese town close to the Spanish border, where the rare Mirandese language is spoken), a Simulation, Emulation, aNd Deployment Integration Middleware for cloud networks. As an orchestration platform for incrementally building Software-Defined Cloud Networks (SDCN), SENDIM manages the development and deployment of algorithms and architectures the entire length from visualization, simulation, emulation, to physical deployments. Hence, SENDIM optimizes the evaluation of cloud networks.

cs.NI↗

arXiv2016

A FIRM Approach to Software-Defined Service Composition

Service composition is an aggregate of services often leveraged to automate the enterprise business processes. While Service Oriented Architecture (SOA) has been a forefront of service composition, services can be realized as efficient distributed and parallel constructs such as MapReduce, which are not typically exploited in service composition. With the advent of Software\-Defined Networking (SDN), global view and control of the entire network is made available to the networking controller, which can further be leveraged in application level. This paper presents FIRM, an approach for Software-Defined Service Composition by leveraging SDN and MapReduce. FIRM comprises Find, Invoke, Return, and Manage, as the core procedures in achieving a QoS-Aware Service Composition.

cs.DC↗

arXiv2015

Not Every Flow is Equal: SMART Discrimination in Redundancy

Software-Defined Data Centers (SDDC) extend virtualization, software-defined networking and systems, and middleboxes to provide a better quality of service (QoS). While many network flow routing algorithms exist, most of them fail to adapt to the dynamic nature of the data center and cloud networks and their users' and enterprise requirements. This paper presents SMART, a Software-Defined Networking (SDN) middlebox architecture for reliable transfers. As an architectural enhancement for network flows allocation, routing, and control, SMART ensures timely delivery of flows by diverting them to a less congested path dynamically in the software-defined data center networks. SMART also clones packets of higher priority flows to route in an alternative path, along with the original flow. Hence SMART offers a differentiated QoS through varying levels of redundancy in the flows.

cs.NI↗

arXiv2014

Quality-of-Data for Consistency Levels in Geo-replicated Cloud Data Stores

Cloud computing has recently emerged as a key technology to provide individuals and companies with access to remote computing and storage infrastructures. In order to achieve highly-available yet high-performing services, cloud data stores rely on data replication. However, providing replication brings with it the issue of consistency. Given that data are replicated in multiple geographically distributed data centers, and to meet the increasing requirements of distributed applications, many cloud data stores adopt eventual consistency and therefore allow to run data intensive operations under low latency. This comes at the cost of data staleness. In this paper, we prioritize data replication based on a set of flexible data semantics that can best suit all types of Big Data applications, avoiding overloading both network and systems during large periods of disconnection or partitions in the network. Therefore we integrated these data semantics into the core architecture of a well-known NoSQL data store (e.g., HBase), which leverages a three-dimensional vector-field model (regarding timeliness, number of pending updates and divergence bounds) to provision data selectively in an on-demand fashion to applications. This enhances the former consistency model by providing a number of required levels of consistency to different applications such as, social networks or e-commerce sites, where priority of updates also differ. In addition, our implementation of the model into HBase allows updates to be tagged and grouped atomically in logical batches, akin to transactions, ensuring atomic changes and correctness of updates as they are propagated.

cs.DC↗