About Collectors
Collectors are extractors that are developed and managed by you (A customer of K).
KADA provides python libraries that customers can use to quickly deploy a Collector.
Why you should use a Collector
There are several reasons why you may use a collector vs the direct connect extractor:
You are using the KADA SaaS offering and it cannot connect to your sources due to firewall restrictions
You want to push metadata to KADA rather than allow it pull data for Security reasons
You want to inspect the metadata before pushing it to K
Using a collector requires you to manage
Deploying and orchestrating the extract code
Managing a high water mark so the extract only pull the latest metadata
Storing and pushing the extracts to your K instance.
When using a Collector you will push metadata to a K landing directory.
To find your landing directory you will need to
Go to Platform Settings - Settings. Note down the value of this setting storage_azure_storage_account (if your platform is deployed on KADA SaaS)
Go to Sources - Edit the Source you have configured. Note down the landing directory in the About this Source section
You will also need a SAS token to push data to the landing directory. Request this from KADA Support (support@kada.ai)
…… airflow