| Info |
|---|
This collector is for Informatica Intelligent Cloud Services (IICS) |
| Scroll ignore | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||||||||
About Collectors
About Collectors
| Insert excerpt | ||||||
|---|---|---|---|---|---|---|
|
...
Pre-requisites
Collector Server Minimum Requirements
...
| Insert excerpt | |||||
|---|---|---|---|---|---|
|
...
|
IICS Requirements
IICS with access to both V2 and V3 APIs
Python 3.6 - 3.10
Access to K landing directory
Access to Informatica Repository (see section below)
Establish IICS Access
Dean Nguyen need to populate this part with screenshots of the check boxes
| Note |
|---|
Collector Limitations
|
...
Step 1: Create the Source in K
...
Go to Settings, Select Sources and click Add Source
Select “Load from File” option
Give the source a Name - e.g. Informatica IICS Production
Add the Host name for the Informatica ServerIICS Production
Click Finish Setup
...
Step 2: Getting Access to the Source Landing Directory
...
FIELD | FIELD TYPE | DESCRIPTION | EXAMPLE |
|---|---|---|---|
username | string | Username to log into IICS | “myuser” |
password | string | Password to log into IICS | |
login_url | string | This is the base url for your IICS login service, see https://docs.informatica.com/integration-cloud/b2b-gateway/current-version/rest-api-reference/platform-rest-api-version-2-resources/login.html for more details | |
days_active | integer | Number of days which a task must have run to be considered active for lineage resolution | 60 |
timeout | integer | Timeout in seconds for IICS API responses, sometimes IICS server can be slow so tune this accordingly if needed | 20 |
output_path | string | Absolute path to the output location where files are to be written | “/tmp/output” |
mask | boolean | To enable masking or not | true |
compress | boolean | To gzip the output or not | true |
...
| Code Block | ||
|---|---|---|
| ||
import os
import argparse
from kada_collectors.extractors.utils import load_config, get_hwm, publish_hwm, get_generic_logger
from kada_collectors.extractors.iics import Extractor
get_generic_logger('root') # Set to use the root logger, you can change the context accordingly or define your own logger
_type = 'iics'
dirname = os.path.dirname(__file__)
filename = os.path.join(dirname, 'kada_{}_extractor_config.json'.format(_type))
parser = argparse.ArgumentParser(description='KADA IICS Extractor.')
parser.add_argument('--config', '-c', dest='config', default=filename, help='Location of the configuration json, default is the config json in the same directory as the script.')
parser.add_argument('--name', '-n', dest='name', default=_type, help='Name of the collector instance.')
args = parser.parse_args()
start_hwm, end_hwm = get_hwm(_typeargs.name)
ext = Extractor(**load_config(args.config))
ext.test_connection()
ext.run(**{"start_hwm": start_hwm, "end_hwm": end_hwm})
publish_hwm(_type, end_hwm) |
...
If you are handling external arguments of the runner yourself, you’ll need to consider the following for the run method https://kadaai.atlassian.net/wiki/spaces/KSL/pages/1902411777/Additional+Notes#Extractor-run-method
| Code Block | ||
|---|---|---|
| ||
from kada_collectors.extractors.snowflake import Extractor
kwargs = {my args} # However you choose to construct your args
hwm_kwrgs = {"start_hwm": "end_hwm": } # The hwm values
ext = Extractor(**kwargs)
ext.run(**hwm_kwrgs) |
...
username: username to sign into server
password: password to sign into server
login_url: IICS login address
active_days: assessment window in days to consider tasks to be active for lineage resolution
timeout: timeout in seconds for API responses
output_path: full or relative path to where the outputs should go
mask: to mask files or not
compress: To gzip output files or not
...
If you want prefer file managed hwm, you can edit the location of the hwn by following these instructions Additional Notes
...
Step 8: Push the Extracts to K
...