Page Comparison

This page will walkthrough the setup of dbt core (self hosted dbt) in K.

...

• Generate the Manifest using the dbt compile command for each project. This will ensure the manifest file contain compiled SQL. The following docs (and filenames are expected)

  • https://docs.getdbt.com/reference/artifacts/manifest-json/

  • Must contain compiled SQL which is generated following DBT run / compile

  • filename: <project_id>_manifest_YYYYMMDDhhmmss.json

• (Optional) Generate the Catalog file using the dbt docs generate command

  • https://docs.getdbt.com/reference/artifacts/catalog-json

  • file name: <project_id>_catalog.json

• Run results are generated after each dbt run.

  • https://docs.getdbt.com/reference/artifacts/run-results-json

  • file name: <project_id>_run_results_YYYYMMDDHHmmss.json

• Use an orchestration tool like Airflow to align the filenames and push the docs (manifest, catalog, run_results) to the landing folder created in Step 3

...