Data stream treatment using sliding windows with MapReduce

Knowledge Discovery in Databases (KDD) techniques present limitations when the volume of data to process is very large. Any KDD algorithm needs to do several iterations on the complete set of data in order to carry out its work. For continuous data stream processing it is necessary to store part of...

Descripción completa

Detalles Bibliográficos
Autores principales: Basgall, María José, Hasperué, Waldo, Naiouf, Marcelo
Formato: Articulo
Lenguaje:Inglés
Publicado: 2016
Materias:
Ciencias Informáticas
big data
mapreduce
stream processing
Acceso en línea:http://sedici.unlp.edu.ar/handle/10915/57265
http://journal.info.unlp.edu.ar/wp-content/uploads/2016/12/JCST-43-Paper-2.pdf
Aporte de:
SEDICI (UNLP) de Universidad Nacional de La Plata
id I19-R120-10915-57265
record_format dspace
institution Universidad Nacional de La Plata
institution_str I-19
repository_str R-120
collection SEDICI (UNLP)
language Inglés
topic Ciencias Informáticas
big data
mapreduce
stream processing
spellingShingle Ciencias Informáticas
big data
mapreduce
stream processing
Basgall, María José
Hasperué, Waldo
Naiouf, Marcelo
Data stream treatment using sliding windows with MapReduce
topic_facet Ciencias Informáticas
big data
mapreduce
stream processing
description Knowledge Discovery in Databases (KDD) techniques present limitations when the volume of data to process is very large. Any KDD algorithm needs to do several iterations on the complete set of data in order to carry out its work. For continuous data stream processing it is necessary to store part of it in a temporal window. In this paper, we present a technique that uses the size of the temporal window in a dynamic way, based on the frequency of the data arrival and the response time of the KDD task. The obtained results show that this technique reaches a great size window where each example of the stream is used in more than one iteration of the KDD task.
format Articulo
Articulo
author Basgall, María José
Hasperué, Waldo
Naiouf, Marcelo
author_facet Basgall, María José
Hasperué, Waldo
Naiouf, Marcelo
author_sort Basgall, María José
title Data stream treatment using sliding windows with MapReduce
title_short Data stream treatment using sliding windows with MapReduce
title_full Data stream treatment using sliding windows with MapReduce
title_fullStr Data stream treatment using sliding windows with MapReduce
title_full_unstemmed Data stream treatment using sliding windows with MapReduce
title_sort data stream treatment using sliding windows with mapreduce
publishDate 2016
url http://sedici.unlp.edu.ar/handle/10915/57265
http://journal.info.unlp.edu.ar/wp-content/uploads/2016/12/JCST-43-Paper-2.pdf
work_keys_str_mv AT basgallmariajose datastreamtreatmentusingslidingwindowswithmapreduce
AT hasperuewaldo datastreamtreatmentusingslidingwindowswithmapreduce
AT naioufmarcelo datastreamtreatmentusingslidingwindowswithmapreduce
bdutipo_str Repositorios
_version_ 1764820478071930884

Ejemplares similares