How to run column level lineage (Beta)

Owned by Dean Nguyen
Oct 23, 2023
2 min read

This page explains how to process and use column level lineage

The function to run column level lineage is available to Admins.

About Column level lineage

K has the ability to processes queries its captures to provide column level lineage.

 

This process is currently a BETA feature and is run manually. The initial run will process all queries that are currently used in lineage and may take some time to complete (~50K queries per hour). Subsequent runs will be significantly shorter as only new queries are processed. Whilst this feature is in BETA we recommend running the initial run out of business hours.

 

Running Column level lineage

  • Select Platform Settings

  • Select Batch Manager

  • Run Query Lineage

     

Limitations with Column level lineage

Parsing code to produce column level lineage is not a perfect. Here are some examples of code patterns that cannot be currently processed.

#

Known limitations

Date raised

#

Known limitations

Date raised

1

Nested query where a select * is used

17 Oct 23

2

Create statement containing UNION ALL uses table aliases in the SELECT statements

17 Oct 23

Please raise any issues by raising a ticket via the KADA Support Page