...
| Scroll ignore | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||||||||
About Collectors
| Insert excerpt | ||||||
|---|---|---|---|---|---|---|
|
...
If you are handling external arguments of the runner yourself, you’ll need to consider the following for the run method https://kadaai.atlassian.net/wiki/spaces/DATKSL/pages/18943181521902411777/Notes+v2.0.0#TheAdditional+Notes#Extractor-run-method
| Code Block | ||
|---|---|---|
| ||
from kada_collectors.extractors.snowflake import Extractor
kwargs = {my args} # However you choose to construct your args
hwm_kwrgs = {"start_hwm": "end_hwm": } # The hwm values
ext = Extractor(**kwargs)
ext.run(**hwm_kwrgs) |
...
To edit the internal SQL being run refer to https://kadaai.atlassian.net/wiki/spaces/DATKSL/pages/18943181521902411777/Notes+v2.0.0#AddingAdditional+Notes#Adding-Custom-SQL
...
Step 7: Check the Collector Outputs
...
If you want prefer file managed hwm, you can edit the location of the hwn by following these instructions https://kadaai.atlassian.net/wiki/spaces/DATKSL/pages/18943181521902411777/Notes+v2.0.0#Storing-HWM-in-another-locationAdditional+Notes#Storing-High-Water-Marks-(HWM)
...
Step 8: Push the Extracts to K
...