/
dbt Cloud

dbt Cloud

This page will walkthrough the setup of dbt Cloud in K using the direct connect method

Each dbt Cloud account will need to be added as a separate dbt source.

Integration details

Scope

Included

Comments

Scope

Included

Comments

Metadata

YES

 

Tests

YES

 

Lineage

YES

 

Usage

YES

 

Sensitive Data Scanner

N/A

 


Step 1) Create a Service Token

  • Log into your dbt cloud account and go to Account Settings > Service Tokens

  • Click New Token.

  • Give the Token a name

  • Add Read Only Access to All Projects

  • Copy the Token for use in the next step


Step 2) Add dbt Cloud as a New Source

  • Select Platform Settings in the side bar

  • In the pop-out side panel, under Integrations click on Sources

  • Click Add Source and select dbt CLOUD

 

  • Select Direct Connect and add your dbt Cloud details and click Next

  • Fill in the Source Settings and click Save & Next

    • Name: Give the dbt source a name in K

    • Host: add your cloud.getdbt.com.[account id]

    • Account ID: You can find your Account ID in the URL when you log into your dbt cloud console. e.g. dbt Cloud [account ID]/projects/[project ID]/dashboard/

    • Environment ID: Add the ID for each Environment you want to include. Add the Environment ID’s as a comma separated list

      • Each Project may have more than one environments. You can find the ID by going to the Environments page for a project and looking at the URL.

      • The value must be the ID, not the name of the environment.

    • Project mapping: Each DBT project is associated with a source. Add a mapping between the DBT project and the source by following the instructions here Host / Database Mapping. You will need to add a mapping for each project.

      • NOTE: You can find the ID by going to the Project page in your DBT cloud account and looking at the URL

  • Add your Token copied from Step 1 and click Save

  • Test your connection and click Save

  • Click Finish Setup


Step 3) Schedule dbt Cloud source load

  • Select Platform Settings in the side bar

  • In the pop-out side panel, under Integrations click on Sources

  • Locate your new dbt Cloud Source and click on the Schedule Settings (clock) icon to set the schedule

Note that scheduling a source can take up to 15 minutes to propagate the change.


Step 4) Manually run an ad hoc load to test dbt Cloud setup

  • Next to your new Source, click on the Run manual load icon

     

  • Confirm how your want the source to be loaded

  • After the source load is triggered, a pop up bar will appear taking you to the Monitor tab in the Batch Manager page. This is the usual page you visit to view the progress of source loads

     

A manual source load will also require a manual run of

  • DAILY

  • GATHER_METRICS_AND_STATS

To load all metrics and indexes with the manually loaded metadata. These can be found in the Batch Manager page

 

Troubleshooting failed loads

  • If the job failed at the extraction step

    • Check the error. Contact KADA Support if required.

    • Rerun the source job

  • If the job failed at the load step, the landing folder failed directory will contain the file with issues.

    • Find the bad record and fix the file

    • Rerun the source job

 

Related content

Host / Database Mapping
Host / Database Mapping
More like this
Integration Guides
Integration Guides
Read with this
dbt core (self hosted)
dbt core (self hosted)
More like this
PowerBI
Read with this
DBT Cloud (via Collector method) - v3.1.0
DBT Cloud (via Collector method) - v3.1.0
More like this
Snowflake
Read with this