Document toolboxDocument toolbox

Setup: Deploy a KADA collector (AWS)

Owned by Dean Nguyen
Last updated: Sept 09, 2021
2 min read

You can deploy a KADA collector in your environment using AWS. This requires deploying a AWS EC2 instance, installing python & libraries, deploying, scheduling and executing the packages. Additional steps may be needed if integrating with AWS Secret Manager is required.

 

Deploying AWS EC2

The following minimum specifications are required for the collector

  • 2 VCPU

  • 4GB Memory

  • 50GB Storage

  • O/S (suggested is CENTOS or UBUNTU or any UNIX based O/S available that can execute bash shell)

In AWS you can use a T2.medium instance.

 

Installing Python & Libraries

Once the instance is created the following is required

  • Python 3.7+ Environment (Recommended 3.8/3.9)

  • Python libraries

    • Oracle

      cx-Oracle = "==8.1.0" pytz = "==2020.1"

       

    • Snowflake

      python-dateutil = "==2.8.1" # Snowflake Connector Requirement requests = "==2.23.0" # Snowflake Connector Requirement pytz = "==2020.1" # Snowflake Connector Requirement cryptography = "==2.9.2" # Snowflake Connector Requirement cx-Oracle = "==8.1.0" # https://cx-oracle.readthedocs.io/en/latest/user_guide/installation.html asn1crypto = "==1.4.0" # Snowflake Connector Requirement azure-common = "==1.1.25" # Snowflake Connector Requirement azure-core = "==1.8.2" # Snowflake Connector Requirement azure-storage-blob = "==12.5.0" # Snowflake Connector Requirement boto3 = "==1.15.18" # Snowflake Connector Requirement botocore = "==1.18.18" # Snowflake Connector Requirement certifi = "==2020.6.20" # Snowflake Connector Requirement cffi = "==1.14.3" # Snowflake Connector Requirement chardet = "==3.0.4" # Snowflake Connector Requirement idna = "==2.10" # Snowflake Connector Requirement isodate = "==0.6.0" # Snowflake Connector Requirement jmespath = "==0.10.0" # Snowflake Connector Requirement msrest = "==0.6.19" # Snowflake Connector Requirement oauthlib = "==3.1.0" # Snowflake Connector Requirement oscrypto = "==1.2.1" # Snowflake Connector Requirement pycparser = "==2.20" # Snowflake Connector Requirement pycryptodomex = "==3.9.8" # Snowflake Connector Requirement PyJWT = "==1.7.1" # Snowflake Connector Requirement pyOpenSSL = "==19.1.0" # Snowflake Connector Requirement requests-oauthlib = "==1.3.0" # Snowflake Connector Requirement s3transfer = "==0.3.3" # Snowflake Connector Requirement six = "==1.15.0" # Snowflake Connector Requirement urllib3 = "==1.25.11" # Snowflake Connector Requirement snowflake-connector-python = "==2.4.4" # https://docs.snowflake.com/en/user-guide/python-connector-install.html

 

Deploying, scheduling & executing the packages

See the extractor package for details on deploying the extractors. The KADA team will support you for this step.