...
| Scroll ignore | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||||||||
About Collectors
| Insert excerpt | ||||||
|---|---|---|---|---|---|---|
|
...
Informatica 9.1+ with repository hosted in Oracle.
Python 3.6 8 - 3.1011
Access to K landing directory
Access to Informatica Repository (see section below)
...
The collector requires a set of parameters to connect to and extract metadata from Informatica
FIELD | FIELD TYPE | DESCRIPTION | EXAMPLE |
|---|---|---|---|
username | string | Username to log into Oracle | “myuser” |
password | string | Password to log into Oracle | |
dsn | string | Datasource Name for Oracle, this can be one of the following forms <tnsname> | “preprod” |
repo_owner | string | This is the owner of all the tables required by the extractor | “inf” |
oracle_client_path | string | Full path to the location of the Oracle Client libraries | “/tmp/drivers/lib/oracleinstantclient_11_9” |
cached | boolean | If set to true if will prevent re-extracting data | false |
input_path | string | Absolute path to the input location where | “/tmp/input” |
output_path | string | Absolute path to the output location where files are to be written | “/tmp/output” |
mask | boolean | To enable masking or not | true |
compress | boolean | To gzip the output or not | true |
KADA provides an out of the box script that reads a configuration JSON file and runs the extractor. Below is the configuration file.
...
| Code Block | ||
|---|---|---|
| ||
import os
import argparse
from kada_collectors.extractors.utils import load_config, get_hwm, publish_hwm, get_generic_logger
from kada_collectors.extractors.informatica import Extractor
get_generic_logger('root') # Set to use the root logger, you can change the context accordingly or define your own logger
_type = 'informatica'
dirname = os.path.dirname(__file__)
filename = os.path.join(dirname, 'kada_{}_extractor_config.json'.format(_type))
parser = argparse.ArgumentParser(description='KADA Informatica Extractor.')
parser.add_argument('--config', '-c', dest='config', default=filename, help='Location of the configuration json, default is the config json in the same directory as the script.')
parser.add_argument('--name', '-n', dest='name', default=_type, help='Name of the collector instance.')
args = parser.parse_args()
start_hwm, end_hwm = get_hwm(_typeargs.name)
ext = Extractor(**load_config(args.config))
ext.test_connection()
ext.run(**{"start_hwm": start_hwm, "end_hwm": end_hwm})
publish_hwm(_type, end_hwm) |
...