Page Comparison

Scroll ignore

scroll-viewport	true
scroll-pdf	true
scroll-office	true
scroll-chm	true
scroll-docbook	true
scroll-eclipsehelp	true
scroll-html	true
scroll-epub	true

Open in new tab

About Collectors

Insert excerpt

	Collector Method
	Collector Method
name	about

...

Some python packages also have dependencies on the OS level packages, so you may be required to install additional OS packages if the below fails to install.

You can download the Latest latest Core Library and ThoughtSpot whl via Platform Settings → Sources → Download Collectors

...

You will also need to install the common library kada_collectors_lib -1.0.0 for this collector to function properly.

...

FIELD

FIELD TYPE

DESCRIPTION

EXAMPLE

username

string

Username to log into the Thoughtspot account

“myuser”

password

string

Password to log into the Thoughtspot account

org_ids

list<integer>

A list of organisation ids you wish to extract from

If you are unable to locate the Organisation ID through the Admin console, you may use this python script instead to list the IDs

Code Block

import requests
import json
BASE_URL = 'REPLACE BASE URL' # e.g. https://kada.thoughtspot.cloud
LOGIN_URL = BASE_URL + '/callosum/v1/tspublic/v1/session/login'  
ORG_URL = BASE_URL + '/callosum/v1/tspublic/v1/session/orgs'  
data = {
    'username': "REPLACE USERNAME",
    'password': "REPLACE PASSWORD"
}
with requests.Session() as session:
    session.post(LOGIN_URL, verify=True, data=data)

    res = session.get(ORG_URL, verify=True)
    res_json = json.loads(res.text)
    for org in res_json.get('orgs', []):
        print("Org Name: {}, Org Id: {}".format(org['orgName'], org['orgId']))

[123, 34]

output_path

string

Absolute path to the output location where files are to be written

“/tmp/output”

mask

boolean

To enable masking or not

true

timeout

integer

Timeout in seconds allowed against the powerbi APIs, for slower connections we recommend 30, default is 20

20

mapping

JSON

Mapping file of data source names against the onboarded host and database name in K

Assuming I have a “myDSN” data source name in powerbi, I’ll map it to host “myhost” and database “mydatabase” onboarded in K, snowflake type references are handled automatically

Code Block
{ "myDSN": { "host": "myhost", "database": "mydatabase" } }

compress

boolean

To gzip the output or not

true

meta_only

trueboolean

To only extract metadata or not. Only true is supported for this source

true

base_api_url

string

Should be in the form https://x.thoughtspot.cloud depending on your account

https://kada.thoughtspot.cloud

These parameters can be added directly into the run or you can use pass the parameters in via a JSON file. The following is an example you can use that is included in the example run code below.

...

Code Block
{ "username": "", "password": "", "org_ids": [], "output_path": "/tmp/output", "timeout": 20, "mapping": {}, "compress": true, "meta_only": true, "base_api_url": "https://sample.thoughtspot.cloud" }

...

Step 6: Run the Collector

...

Code Block

import os
import argparse
from kada_collectors.extractors.utils import load_config, get_hwm, publish_hwm, get_generic_logger
from kada_collectors.extractors.thoughtspot import Extractor

get_generic_logger('root') # Set to use the root logger, you can change the context accordingly or define your own logger

_type = 'teradatathoughtspot'
dirname = os.path.dirname(__file__)
filename = os.path.join(dirname, 'kada_{}_extractor_config.json'.format(_type))

parser = argparse.ArgumentParser(description='KADA TeradataThoughtspot Extractor.')
parser.add_argument('--config', '-c', dest='config', default=filename, help='Location of the configuration json, default is the config json in the same directory as the script.')
args = parser.parse_args()

start_hwm, end_hwm = get_hwm(_type)

ext = Extractor(**load_config(args.config))
ext.test_connection()
ext.run(**{"start_hwm": start_hwm, "end_hwm": end_hwm})

publish_hwm(_type, end_hwm)

...

Versions Compared

Old Version 3

New Version Current

Key

About Collectors

Step 6: Run the Collector