Provenance Framework for Multi-Depth Querying Using Zero-Information Loss Database

Abstract

Data provenance is a kind of metadata that describes the origin and derivation history of data. It provides the information about various direct and indirect sources of data and different transformations applied on it. Provenance information are beneficial in determining the quality, truthfulness, and authenticity of data. It also explains how, when, why, and by whom this data are created. In a relational database, fine-grained provenance captured at different stages (i.e., multi-layer provenance) is more significant and explanatory as it provides various remarkable information such as immediate and intermediate sources and origin of data. In this paper, we propose a novel multi-layer data provenance framework for Zero-Information Loss Relational Database (ZILRDB). The proposed framework is implemented on top of the relational database using the object relational database concepts to maintain all insert, delete, and update operations efficiently. It has the capability to capture multi-layer provenance for different query sets including historical queries. We also propose Provenance Relational Algebra (PRA) as an extension of traditional relational algebra to capture the provenance for ASPJU (Aggregate, Select, Project, Join, Union) queries in relational database. The framework provides a detailed provenance analysis through multi-depth provenance querying. We store the provenance data in both relational and graph database, and further evaluate the performance of the framework in terms of provenance storage overhead and average execution time for provenance querying. We observe that the graph database offers significant performance gains over relational database for executing multi-depth queries on provenance. We present two use case studies to explain the usefulness of proposed framework in various data-driven systems to increase the understandability of system’s behavior and functionalities.