Skip to main navigation Skip to search Skip to main content

XG: A grid-enabled query processing engine

  • Radu Sion
  • , Ramesh Natarajan
  • , Inderpal Narang
  • , Thomas Phan
  • IBM

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

1 Scopus citations

Abstract

In [12] we introduce a novel architecture for data processing, based on a functional fusion between a data and a computation layer. In this demo we show how this architecture is leveraged to offer significant speedups for data processing jobs such as data analysis and mining over large data sets. One novel contribution of our solution is its data-driven approach. The computation infrastructure is controlled from within the data layer. Grid compute job submission events are based within the query processor on the DBMS side and in effect controlled by the data processing job to be performed. This allows the early deployment of on-the-fly data aggregation techniques, minimizing the amount of data to be transfered to/from compute nodes and is in stark contrast to existing Grid solutions that interact with data layers as external (mainly) "storage" components. By integrating scheduling intelligence in the data layer itself we show that it is possible to provide a close to optimal solution to the more general grid trade-off between required data replication costs and computation speed-up benefits. We validate this in a scenario derived from a real business deployment, involving financial customer profiling using common types of data analytics.

Original languageEnglish
Title of host publicationAdvances in Database Technology - EDBT 2006 - 10th International Conference on Extending Database Technology, Proceedings
Pages1115-1120
Number of pages6
DOIs
StatePublished - 2006
Event10th International Conference on Extending Database Technology, EDBT 2006 - Munich, Germany
Duration: Mar 26 2006Mar 31 2006

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume3896 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference10th International Conference on Extending Database Technology, EDBT 2006
Country/TerritoryGermany
CityMunich
Period03/26/0603/31/06

Fingerprint

Dive into the research topics of 'XG: A grid-enabled query processing engine'. Together they form a unique fingerprint.

Cite this