Document toolboxDocument toolbox

dbt Cloud

This page will walkthrough the setup of dbt Cloud in K using the direct connect method

Each dbt Cloud account will need to be added as a separate dbt source.

Integration details

Scope

Included

Comments

Scope

Included

Comments

Metadata

YES

 

Tests

YES

 

Lineage

YES

 

Usage

YES

 

Sensitive Data Scanner

N/A

 


Step 1) Create a Service Token

  • Log into your dbt cloud account and go to Account Settings > Service Tokens

  • Click New Token.

  • Give the Token a name

  • Add Read Only Access to All Projects

  • Copy the Token for use in the next step


Step 2) Add dbt Cloud as a New Source

  • Select Platform Settings in the side bar

  • In the pop-out side panel, under Integrations click on Sources

  • Click Add Source and select dbt CLOUD

 


Step 3) Schedule dbt Cloud source load

  • Select Platform Settings in the side bar

  • In the pop-out side panel, under Integrations click on Sources

  • Locate your new dbt Cloud Source and click on the Schedule Settings (clock) icon to set the schedule

Note that scheduling a source can take up to 15 minutes to propagate the change.


Step 4) Manually run an ad hoc load to test dbt Cloud setup

  • Next to your new Source, click on the Run manual load icon

     

  • Confirm how your want the source to be loaded

  • After the source load is triggered, a pop up bar will appear taking you to the Monitor tab in the Batch Manager page. This is the usual page you visit to view the progress of source loads

     

A manual source load will also require a manual run of

  • DAILY

  • GATHER_METRICS_AND_STATS

To load all metrics and indexes with the manually loaded metadata. These can be found in the Batch Manager page

 

Troubleshooting failed loads

  • If the job failed at the extraction step

    • Check the error. Contact KADA Support if required.

    • Rerun the source job

  • If the job failed at the load step, the landing folder failed directory will contain the file with issues.

    • Find the bad record and fix the file

    • Rerun the source job