What updates are real-time vs batch dependent?
- Chichi
Depending on the type of source connection and meta data some updates will update in real-time while others are dependent on when the next batch run is updated.
This page will articulate which updates are real-time vs batch dependent.
While some updates are real-time, for pages like Lineage maps, you will still be required to refresh the page for the update to be reflected
Real-Time Updates
Metadata provided via sources (e.g. usage data, column data) that are Directly Connected to K (and not via a Collector)
Changes to user role permissions
Contact your K Admin to understand if your Source is connected to K via Direct Connect or Collector method.
Batch Dependent Updates
Unless specified in the ‘Real-Time’ update section, it can be assumed that the update will not be immediate and you may need to wait for the next batch-run.
Most batches are run at a minimum daily. Below is a table listing default batch-run schedule that may impact how a data profile page is refreshed.
Metadata Type | Update Frequency | Comments |
|---|---|---|
? | Daily |
|
K Collections:
| Daily |
|
Search Results
Search indexing is performed in scheduled jobs. This means that while any updates made directly in the K platform (e.g. lineage updates, new collection/tag linkages) will immediately show on the data profile page, there will be a delay in when search results are refreshed to include the updates.
It is recommended that after a bulk update is performed, that you wait until the next business day to use the search function.
K Admin users have the ability to change batch timing and frequency. If you would like to confirm the frequency and timing of the above batch updates, contact your K Admin.