Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

Version 1 Next »

You can deploy a KADA collector in your environment using AWS. This requires deploying a AWS EC2 instance, installing python & libraries, deploying, scheduling and executing the packages. Additional steps may be needed if integrating with AWS Secret Manager is required.

Deploying AWS EC2

The following minimum specifications are required for the collector

  • 2 VCPU

  • 4GB Memory

  • 50GB Storage

  • O/S (suggested is CENTOS or UBUNTU or any UNIX based O/S available that can execute bash shell)

In AWS you can use a T2.medium instance.

Installing Python & Libraries

Once the instance is created the following is required

  • Python 3.7+ Environment (Recommended 3.8/3.9)

  • Python libraries

    • Oracle

      • cx-Oracle = "==8.1.0"

      • pytz = "==2020.1"

    • Snowflake 

      python-dateutil = "==2.8.1"  # Snowflake Connector Requirement
requests = "==2.23.0" # Snowflake Connector Requirement
pytz = "==2020.1" # Snowflake Connector Requirement
cryptography = "==2.9.2" # Snowflake Connector Requirement
cx-Oracle = "==8.1.0" # https://cx-oracle.readthedocs.io/en/latest/user_guide/installation.html
asn1crypto = "==1.4.0" # Snowflake Connector Requirement
azure-common = "==1.1.25" # Snowflake Connector Requirement
azure-core = "==1.8.2" # Snowflake Connector Requirement
azure-storage-blob = "==12.5.0" # Snowflake Connector Requirement
boto3 = "==1.15.18" # Snowflake Connector Requirement
botocore = "==1.18.18" # Snowflake Connector Requirement
certifi = "==2020.6.20" # Snowflake Connector Requirement
cffi = "==1.14.3" # Snowflake Connector Requirement
chardet = "==3.0.4" # Snowflake Connector Requirement
idna = "==2.10" # Snowflake Connector Requirement
isodate = "==0.6.0" # Snowflake Connector Requirement
jmespath = "==0.10.0" # Snowflake Connector Requirement
msrest = "==0.6.19" # Snowflake Connector Requirement
oauthlib = "==3.1.0" # Snowflake Connector Requirement
oscrypto = "==1.2.1" # Snowflake Connector Requirement
pycparser = "==2.20" # Snowflake Connector Requirement
pycryptodomex = "==3.9.8" # Snowflake Connector Requirement
PyJWT = "==1.7.1" # Snowflake Connector Requirement
pyOpenSSL = "==19.1.0" # Snowflake Connector Requirement
requests-oauthlib = "==1.3.0" # Snowflake Connector Requirement
s3transfer = "==0.3.3" # Snowflake Connector Requirement
six = "==1.15.0" # Snowflake Connector Requirement
urllib3 = "==1.25.11" # Snowflake Connector Requirement
snowflake-connector-python = "==2.4.4" # https://docs.snowflake.com/en/user-guide/python-connector-install.html

Deploying, scheduling & executing the packages

See the extractor package for details on deploying the extractors. The KADA team will support you for this step.

  • No labels