Loading a full manifest.json for dbt (Cloud & Core)

Owned by Dean Nguyen
Last updated: Sept 18, 2022
1 min read

Open in new tab

Currently a full manifest.json (generated after a dbt compile) is required to ensure all dbt models are loaded into K. We recommend pushing a new full manifest.json as part of your dbt CI/CD process so that every time you add/update/delete a dbt model, you update K as well.

We are working to remove this step for dbt cloud users using the new dbt APIs that have been recently been made available.

Step 1) Generate the manifest.json

  • If you are using dbt Cloud, log into your dbt cloud account and go a project (you will need to repeat this for every project)

  • Create a new job to run dbt compile.

    • You do not need to run any other dbt command or schedule this job.

  • Run the dbt compile job

  • After the job has been completed, download the manifest.json from the artifacts tab

  • Open the manifest.json file and find the "DBT_CLOUD_PROJECT_ID" for dbt Cloud

    • Use "project_id" for dbt Core

  • Append the ID to the start of the manifest.json file name

    • e.g. 11231_manifest.json

Step 2) Upload the manifest.json to the dbt landing folder