What Is Data Lineage? Tracing Data from Source to Decision

A number on an executive dashboard looks wrong. The CFO wants to know why — today. Now the real questions begin: which table feeds that chart? Which pipeline built that table? Which source system did the raw records come from, and what transformations touched them along the way? In most organisations, answering this means hours of detective work across pipeline code, SQL scripts, and the memory of whoever built the report two years ago.

Data lineage exists to make that detective work unnecessary. It is the documented, navigable path that data follows from its original source, through every transformation and intermediate store, to the report, model, or API where it finally drives a decision. Think of it as a family tree for data — hence the Arabic term, literally a "chain of descent" — or as a supply-chain map: for any data product, lineage tells you where every ingredient came from and what was done to it at each step.

This article explains what lineage is, the crucial difference between table-level and column-level lineage, the four use cases where it earns its keep, and — the part most introductions skip — how lineage actually gets captured in practice.

A working definition

Data lineage is metadata that records relationships of flow and transformation between data assets. A lineage graph has three building blocks:

• Nodes: the assets themselves — source tables, staging tables, warehouse models, dashboards, ML features, files, and APIs.
• Edges: the "feeds into" relationships between them — this table is read by that job, which writes that table, which powers that dashboard.
• Process context: the pipeline, query, or service that performs each hop, ideally with run times and status.

Two directions matter equally. Upstream lineage answers "where did this come from?" — trace a dashboard back to its sources. Downstream lineage answers "what depends on this?" — trace a source table forward to everything it ultimately feeds. Most of the value of lineage comes from being able to ask both questions in seconds rather than days.

Table-level vs column-level lineage

Not all lineage is equally precise, and the difference matters more than it first appears.

Table-level lineage

Table-level (or asset-level) lineage records that table A feeds table B, without saying which columns are involved. It is comparatively easy to capture, and it is enough for a surprising number of questions: which dashboards break if this source goes down? Which systems does this report depend on? For a first pass across the whole estate, table-level lineage is the right ambition.

Column-level lineage

Column-level lineage records that this specific column in the target was derived from those specific columns in the source — including the transformation logic between them: a rename, a join key, an aggregation, a masking function. This precision unlocks questions table-level lineage cannot answer:

• "The national_id column is classified as restricted — show me every downstream column derived from it, including copies that were renamed along the way."
• "We are changing the definition of net_revenue — exactly which reports and KPIs are affected?"
• "This column was supposed to be masked before reaching the analytics layer — prove that it was."

Column-level lineage is harder to capture and noisier to display, which is why mature teams pursue it selectively: broad table-level coverage everywhere, column-level depth for sensitive and business-critical data first. For personal data under the Personal Data Protection Law, column-level lineage is what turns data classification from a label on one table into something you can actually enforce across the estate.

Where lineage earns its keep: four use cases

1. Impact analysis — before you change something

Every schema change, column rename, or system migration carries the same anxious question: what will this break? With downstream lineage, the answer is a query, not a guess. Teams that check lineage before deploying changes stop discovering broken dashboards from angry business users on Sunday morning.

2. Root cause analysis — after something breaks

When a metric looks wrong, upstream lineage turns hours of archaeology into a guided walk: start at the dashboard, step backwards hop by hop, and find the failed job, late-arriving file, or changed source schema that caused it. Pairing lineage with data quality results makes this even faster — the broken hop is usually the one whose quality checks just turned red.

3. Compliance evidence — proving how data flows

Saudi regulatory frameworks increasingly expect organisations to demonstrate control over data flows, not just assert it. The NDMO framework — 15 domains, 77 controls, 191 specifications — expects documented data architecture and managed metadata; PDPL, enforced by SDAIA and fully in effect since September 2024, requires organisations to know where personal data lives and where it travels. When an auditor or a data subject asks "where is this personal data used?", a lineage graph is evidence; a verbal assurance is not.

4. Trust — knowing whether a number is safe to use

Analysts constantly judge whether a dataset is trustworthy. Lineage answers the trust questions directly: does this table come from the certified source or from someone's experimental copy? How many manual steps sit between the source and this number? Lineage shown alongside ownership and quality scores in a data catalog is what lets people decide to trust data instead of merely hoping.

How lineage is actually captured

No single technique covers everything. Real lineage programmes combine four:

• Pipeline and tool integration. Orchestration platforms, ETL/ELT tools, and BI tools already know what they read and write. Connectors harvest this directly, giving reliable lineage for every managed pipeline — usually the highest-fidelity source.
• SQL parsing. A large share of transformation logic lives in SQL — views, stored procedures, warehouse query logs. Parsing these statements reveals which tables and columns feed which, and is the workhorse for column-level lineage in the warehouse.
• Manual curation. Some hops are invisible to machines: a monthly extract emailed to a regulator, a file dropped by a partner, a legacy system with no logs. Stewards document these edges by hand so the graph reflects reality, not just what was automatable.
• Suggested lineage. Machine-assisted techniques propose probable edges from naming patterns, schema similarity, and usage behaviour — flagged as suggestions for a human to confirm or reject. Useful for bootstrapping coverage; never to be confused with verified lineage.

The practical lesson: treat automated capture as the foundation, manual curation as the patch for blind spots, and suggestions as an accelerator — and record how each edge was captured, because an auditor will eventually ask.

Don't forget the edges of the graph: APIs

Lineage discussions usually stop at the dashboard, but in modern architectures a great deal of data leaves the organisation through APIs — to mobile apps, partners, and government integration platforms. Each API endpoint is a lineage endpoint too: a terminal node that says "this data, from these tables, is exposed here, to these consumers." Including APIs in the lineage graph is what makes questions like "which external consumers receive personal data?" answerable — a question that sits at the heart of PDPL disclosure obligations, and one reason API governance and lineage belong in the same conversation.

Getting started

A pragmatic sequence for a team starting from zero:

1. Connect your warehouse and orchestration tool, and let automated capture build table-level lineage for the core estate.
2. Pick the five most critical dashboards and verify their lineage end to end — including any manual hops.
3. Enable column-level lineage for assets containing personal or restricted data first.
4. Add API endpoints that expose data externally as terminal nodes.
5. Make checking lineage a required step before schema changes — that habit alone repays the investment.

Lineage is most powerful when it lives where people already look for data. Goava builds data lineage directly into the catalog, so every asset shows where it came from and what depends on it — with the bilingual context and NDMO-aligned evidence trail Saudi data teams need.