Static analysis for optimizing big data queries

Query languages for big data analysis provide user extensibility through a mechanism of user-deined operators (UDOs). These operators allow programmers to write proprietary functionalities on top of a relational query skeleton. However, achieving efective query optimization for such languages is ext...

Descripción completa

Guardado en:
Detalles Bibliográficos
Publicado: 2017
Materias:
Big Data
Query optimization
Static analysis
UDOs
Query languages
Software engineering
Data dependencies
Data query
Error prones
Optimizers
Real-world
Relational queries
Big data
Acceso en línea:https://bibliotecadigital.exactas.uba.ar/collection/paper/document/paper_97814503_vPartF130154_n_p932_Garbervetsky
http://hdl.handle.net/20.500.12110/paper_97814503_vPartF130154_n_p932_Garbervetsky
Aporte de:
Biblioteca Digital - Facultad de Ciencias Exactas y Naturales (UBA) de Universidad de Buenos Aires
id paper:paper_97814503_vPartF130154_n_p932_Garbervetsky
record_format dspace
spelling paper:paper_97814503_vPartF130154_n_p932_Garbervetsky2023-06-08T16:37:30Z Static analysis for optimizing big data queries Big Data Query optimization Static analysis UDOs Query languages Software engineering Static analysis Data dependencies Data query Error prones Optimizers Query optimization Real-world Relational queries UDOs Big data Query languages for big data analysis provide user extensibility through a mechanism of user-deined operators (UDOs). These operators allow programmers to write proprietary functionalities on top of a relational query skeleton. However, achieving efective query optimization for such languages is extremely challenging since the optimizer needs to understand data dependencies induced by UDOs. SCOPE, the query language from Microsoft, allows for hand coded declarations of UDO data dependencies. Unfortunately, most programmers avoid using this facility since writing and maintaining the declarations is tedious and error-prone. In this work, we designed and implemented two sound and robust static analyses for computing UDO data dependencies. The analyses can detect what columns of an input table are never used or pass-through a UDO unchanged. This information can be used to signiicantly improve execution of SCOPE scripts. We evaluate our analyses on thousands of real-world queries and show we can catch many unused and pass-through columns automatically without relying on any manually provided declarations. © 2017 Association for Computing Machinery. 2017 https://bibliotecadigital.exactas.uba.ar/collection/paper/document/paper_97814503_vPartF130154_n_p932_Garbervetsky http://hdl.handle.net/20.500.12110/paper_97814503_vPartF130154_n_p932_Garbervetsky
institution Universidad de Buenos Aires
institution_str I-28
repository_str R-134
collection Biblioteca Digital - Facultad de Ciencias Exactas y Naturales (UBA)
topic Big Data
Query optimization
Static analysis
UDOs
Query languages
Software engineering
Static analysis
Data dependencies
Data query
Error prones
Optimizers
Query optimization
Real-world
Relational queries
UDOs
Big data
spellingShingle Big Data
Query optimization
Static analysis
UDOs
Query languages
Software engineering
Static analysis
Data dependencies
Data query
Error prones
Optimizers
Query optimization
Real-world
Relational queries
UDOs
Big data
Static analysis for optimizing big data queries
topic_facet Big Data
Query optimization
Static analysis
UDOs
Query languages
Software engineering
Static analysis
Data dependencies
Data query
Error prones
Optimizers
Query optimization
Real-world
Relational queries
UDOs
Big data
description Query languages for big data analysis provide user extensibility through a mechanism of user-deined operators (UDOs). These operators allow programmers to write proprietary functionalities on top of a relational query skeleton. However, achieving efective query optimization for such languages is extremely challenging since the optimizer needs to understand data dependencies induced by UDOs. SCOPE, the query language from Microsoft, allows for hand coded declarations of UDO data dependencies. Unfortunately, most programmers avoid using this facility since writing and maintaining the declarations is tedious and error-prone. In this work, we designed and implemented two sound and robust static analyses for computing UDO data dependencies. The analyses can detect what columns of an input table are never used or pass-through a UDO unchanged. This information can be used to signiicantly improve execution of SCOPE scripts. We evaluate our analyses on thousands of real-world queries and show we can catch many unused and pass-through columns automatically without relying on any manually provided declarations. © 2017 Association for Computing Machinery.
title Static analysis for optimizing big data queries
title_short Static analysis for optimizing big data queries
title_full Static analysis for optimizing big data queries
title_fullStr Static analysis for optimizing big data queries
title_full_unstemmed Static analysis for optimizing big data queries
title_sort static analysis for optimizing big data queries
publishDate 2017
url https://bibliotecadigital.exactas.uba.ar/collection/paper/document/paper_97814503_vPartF130154_n_p932_Garbervetsky
http://hdl.handle.net/20.500.12110/paper_97814503_vPartF130154_n_p932_Garbervetsky
_version_ 1768543253700804608

Ejemplares similares