| Scroll ignore | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||||||||
About Collectors
| Insert excerpt | ||||||
|---|---|---|---|---|---|---|
|
...
Python 3.6 - 3.10
Access to the KADA Collector repository that contains the DBT Cloud whl
The repository is currently hosted in KADA’s Azure Blob Storage. You will be given a SAS token to access the repository. Reach out to KADA Support (support@kada.ai) if you do not have access.
Download the DBT whl (e.g. kada_collectors_extractors_dbt-#.#.#-py3-none-any.whl)
Access to K landing directory
Access toDBT CloudK landing directory
Access toDBT Cloud
| Info |
|---|
Unlike the other collectors, the DBT extractor produces manifest, catalog and run_result json files instead of csv files. Do not be alarmed if you see these. This only works for DBT Cloud not DBT Core |
...
Step 1: Create the Source in K
...
Some python packages also have dependencies on the OS level packages, so you may be required to install additional OS packages if the below fails to install.
You can download the Latest Core Library and whl via Platform Settings → Sources → Download Collectors
...
Run the following command to install the collector
| Code Block |
|---|
pip install kada_collectors_extractors_dbt-3.0.0-py3<version>-none-any.whl |
You will also need to install the common library kada_collectors_lib -1.0.0 for this collector to function properly.
| Code Block |
|---|
pip install kada_collectors_lib-1.0.0-py3-<version>-none-any.whl |
| Info |
|---|
These are some known possible packages you may require depending on your OS, this is not exhaustive and only serves as a guide. |
...
Step 4: Configure the Collector
...
If you want prefer file managed hwm, you can edit the location of the hwn by following these instructions https://kadaai.atlassian.net/wiki/spaces/DATKSL/pages/18943181521902411777/Notes+v2.0.0#Storing-HWM-in-another-locationAdditional+Notes#Storing-High-Water-Marks-(HWM)
...
Step 7: Push the Extracts to K
...