From Provenance to Context: Systems for Post-Hoc Analysis of Data Workflows

Data science and machine learning workflows produce many artifacts whose value depends on context: cleaned datasets, feature matrices, embeddings, predictions, execution logs, prompts, generated code, and model outputs. These artifacts can support later auditing, reuse, comparison, and methodological analysis, but only when they remain connected to the transformations and decisions that produced them. Existing systems often preserve final outputs or manual documentation, while intermediate evidence for post-hoc analysis is lost or difficult to query. This dissertation argues that post-hoc analysis of data transformations can be automated by systems that capture scope-appropriate representations of workflow context and support universal queryability across stored workflows. It examines this claim across three levels of data science practice. At the level of array transformations, DSLog captures cell-level lineage between input and output arrays and compresses it using the ProvRC representation, enabling forward and backward lineage queries over compressed data. At the level of computational notebooks, DataInquirer records durable execution logs containing code, execution order, timing, outputs, errors, and line-level metadata, enabling systematic analysis of notebook behavior and methodological variation. At the level of AI-assisted machine learning workflows, TableVault stores generated artifacts with process records, descriptions, embeddings, structured properties, and lineage edges, enabling analysts and AI agents to search, traverse, and compare workflow context. Together, these systems show that different forms of post-hoc analysis require different representations, but that each representation becomes useful when it is queryable across workflows. DSLog automates provenance queries over transformed values. DataInquirer automates extraction of behavioral and methodological metrics from notebook histories. TableVault automates capture and retrieval of workflow context for AI-agent methodology analysis. Overall, the dissertation shows how data management can move from preserving isolated outputs toward preserving workflow context as reusable infrastructure for future analysis.