| Scroll ignore | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||||||||
About Collectors
| Insert excerpt | ||||||
|---|---|---|---|---|---|---|
|
...
The collector requires a set of parameters to connect to and extract metadata from Power BI
FIELD | FIELD TYPE | SUPPORTED VERSION | DESCRIPTION | EXAMPLE | |||||
|---|---|---|---|---|---|---|---|---|---|
client | string | 2.0.0+ | Onboarded client in Azure to access powerbi | ||||||
secret | string | 2.0.0+ | Onboarded client secret in Azure to access powerbi | ||||||
tenant | string | 2.0.0+ | Tenant ID of where powerbi exists | ||||||
output_path | string | 2.0.0+ | Absolute path to the output location where files are to be written | “/tmp/output” | |||||
mask | boolean | 2.0.0+ | To enable masking or not | true | |||||
timeout | integer | 2.0.0+ | Timeout in seconds allowed against the powerbi APIs, for slower connections we recommend 30, default is 20 | 20 | |||||
filter_flag | boolean | 2.1.0+ | Enable or disable filtering workspaces based on filter_workspaces | false | |||||
filter_workspaces | list<string> | 2.1.0+ | List of workspace names that should be processed, this is case insensitive. Note that personal workspaces are excluded globally and will never be included even if you include it here. | [“data lab”, “analysis models”] | |||||
mapping | JSON | 2.0.0+ | Add the mapping for each data source in Power BI to a data source that is loaded into K. You will need to map the data source name in Power BI to the data source host name in K (these can be found on the Sources page) Skip any data sources are not onboarded in K (these will be loaded in as references until the source is added to K). See Host / Database Mapping for more details | Where somehost is the alternate name created in Power BI for the Analytics database that has been onboarded to K
|
These parameters can be added directly into the run or you can use pass the parameters in via a JSON file. The following is an example you can use that is included in the example run code below.
...
| Code Block | ||
|---|---|---|
| ||
import os
import argparse
from kada_collectors.extractors.utils import load_config, get_hwm, publish_hwm, get_generic_logger
from kada_collectors.extractors.powerbi import Extractor
get_generic_logger('root') # Set to use the root logger, you can change the context accordingly or define your own logger
_type = 'powerbi'
dirname = os.path.dirname(__file__)
filename = os.path.join(dirname, 'kada_{}_extractor_config.json'.format(_type))
parser = argparse.ArgumentParser(description='KADA PowerBI Extractor.')
parser.add_argument('--config', '-c', dest='config', default=filename, help='Location of the configuration json, default is the config json in the same directory as the script.')
args = parser.parse_args()
start_hwm, end_hwm = get_hwm(_type)
ext = Extractor(**load_config(args.config))
ext.test_connection()
ext.run(**{"start_hwm": start_hwm, "end_hwm": end_hwm})
publish_hwm(_typeargs.name, end_hwm) |
Advance options:
...