| Scroll ignore | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||||||||
About Collectors
| Insert excerpt | ||||||
|---|---|---|---|---|---|---|
|
...
Pre-requisites
Python 3.6 8 - 3.1011
Access to K landing directory
Access to MySQL. Supported MySQL versions include:
5.7x
8.0x
Access to the following Information Schema tables
INFORMATION_SCHEMA.VIEWSINFORMATION_SCHEMA.TABLESINFORMATION_SCHEMA.COLUMNSINFORMATION_SCHEMA.KEY_COLUMN_USAGE
...
The MySQL collector only extracts metadata and does not extract or process query usage on the database.
FIELD | FIELD TYPE | DESCRIPTION | EXAMPLE |
|---|---|---|---|
username | string | Username to log into MySQL | “myuser” |
password | string | Password to log into MySQL | “password” |
server | string | The server to connect to the MySQL instance, if the MySQL instance uses a port other than the default port, use a comma to indicate the port e.g. local.example.com,3309 | “local.example.com,3309” |
host_name | string | The onboarded host in K for the MySQL Source | “local.example.com” |
database_name | string | The onboarded database name in K for the MySQL Source, this will be the same as the source name for MySQL | “mysqldb” |
meta_only | boolean | If logging is not enabled, set this to true | true |
output_path | string | Absolute path to the output location where files are to be written | “/tmp/output” |
mask | boolean | To enable masking or not | true |
compress | boolean | To enable compression or not to .csv.gz | true |
use_ssl | boolean | If the MySQL instances uses SSL | true |
These parameters can be added directly into the run or you can use pass the parameters in via a JSON file. The following is an example you can use that is included in the example run code below.
...
| Code Block |
|---|
import os
import argparse
from kada_collectors.extractors.utils import load_config, get_hwm, publish_hwm, get_generic_logger
from kada_collectors.extractors.oracle_analytics import Extractor
get_generic_logger('root') # Set to use the root logger, you can change the context accordingly or define your own logger
_type = 'mysql'
dirname = os.path.dirname(__file__)
filename = os.path.join(dirname, 'kada_{}_extractor_config.json'.format(_type))
parser = argparse.ArgumentParser(description='KADA MySQL Extractor.')
parser.add_argument('--config', '-c', dest='config', default=filename, help='Location of the configuration json, default is the config json in the same directory as the script.')
parser.add_argument('--name', '-n', dest='name', default=_type, help='Name of the collector instance.')
args = parser.parse_args()
start_hwm, end_hwm = get_hwm(_typeargs.name)
ext = Extractor(**load_config(args.config))
ext.test_connection()
ext.run(**{"start_hwm": start_hwm, "end_hwm": end_hwm})
publish_hwm(_type, end_hwm) |
...