...
K is a Data Product Analytics platform for capturing, profiling and discovering how data products (data sets, analysis, reports etc) across an Enterprise is used. K focuses on identifying and storing how users work with data; leveraging this information to enable data producers to improve their products; data owners to take accountability for the proper use of their data; and to scale hidden knowledge to all data workers. The product vision is to become the central platform for all Enterprise data users to easily discover, understand and use data[BC1] data .
K Architecture
...
Services
Component | Description |
Ingestion | The service is used for loading metadata and logs from data sources and tools. |
Profiler | The service is used to identify and profile data assets and their usage. A set of proprietary algorithms are used to automatically match and analyse data assets over their lifecycle. |
Usage | The service is used to monitor and track data assets over time. |
Identity | The service is used to integrate with the Enterprise Identity Management service to provide single sign on. |
Search | The service provides fast, accurate and contextual search for all assets within K. |
Applications | The service is used to access dedicated applications built to solve specific data problems. E.g. migration assessment, impact assessment etc. |
Inventory | The service manages the hierarchical structure for all assets within. |
Scheduler | The service manages the integration and scheduling of ingestion of metadata and logs into K. |
...
Component | Description |
Data Sources | Data sources (e.g. Teradata, Hadoop, Snowflake, SQL Server etc.) where data is stored and used by the Enterprise data teams. K has integrators for many on-premise and cloud data sources and can also ingest custom data sources through the K ingestion framework. |
Data Tools | Reporting and Analytics applications (e.g. Tableau, Power BI etc.) used by the Enterprise data teams to create, manage and distribute content[BC1] . K has integrators for common data tools and can also ingest custom data tools through the K ingestion framework. |
Identity / SSO | Identity provider and user management sources (e.g. LDAP, SAML, OpenID Connect) that can provide single sign on and user and team data.[BC2] |
...
Deploying into the Enterprise
...
Considerations | Details |
Policies | Kubernetes service must have access to the Object Store. In the case where the Kubernetes service is using a Cloud Provider’s managed service (e.g. AWS, GCP, Azure) this may require cloud policies to be created to enable the service with the right read/write permissions. [BC1] Please consult your Cloud Provider’s documentation |
Internet Access | The K platform does NOT need internet access. The Kubernetes service needs to have internet access to download the K images from the KADA repository. |
...