Aggregation of streamed data is key to the expansion of the Internet of Things. This paper addresses the problem of designing a topology for reliably aggregating data flows from many devices arriving at a datacenter. Reliability here means ensuring operation without data loss. We seek a frugal solution that prevents wasteful resource consumption (over-provisioning). This problem is salient when building an aggregation service out of components (here aggregation nodes) that exhibit hard constraints on the amount of information they can handle per unit of time. We first formalize the problem and provide an analysis of the relation between monitored devices (plus information they send), and the operations performed at aggregation nodes, in terms of data rates. Building on this rate analysis, we devise a novel algorithm, which we call CSA, that basically outputs an aggregation topology capable of handling those incoming data rates, preventing thereby empirical trial-and-error design. We analyze the algorithm, before validating it on the Amazon Kinesis platform, using a device dataset from a European telco operator.
Frugal Topology Construction for Stream Aggregation in the Cloud
Frugal Topology Construction for Stream Aggregation in the Cloud
Frugal Topology Construction for Stream Aggregation in the Cloud
Related Content
Accounts are often shared by multiple users, each of them having different item consumption and temporal habits. Identifying of the active user can lead to improvements in a variety of services by switching from account personalized services to user personalized services. To do so, we develop a topic model extending the Latent Dirichlet Allocation using a hidden…
Les liens DSL peuvent subir des pannes sporadiques entraînant des deconnexions ou un acces Internet dégradé. Ces pannes sont a l’origine d’une expérience utilisateur négative et générent des coûts pour les fournisseurs d’accès Internet (FAI) via des appels d’assistance technique. La prediction de pannes permet aux FAI de mettre en oeuvre des mesures proactives d…
The success of Google’s PageRank algorithm popularized graphs as a tool to model the web’s navigability. At that time, the web topology was resulting from human edition of hyper-links. Nowadays, that topology is mostly resulting from algorithms. In this paper, we propose to study the topology realized by a class of such algorithms: recommenders. By modeling the …
Webinar /Jun 2024
Blog Post /Jun 2025
Blog Post /Jun 2025