Nornir Compliance Check

 Credits to IPvZero for this who posted the original solution which was modified to suit my needs : https://github.com/IPvZero/nornir-test-compliance/blob/master/comply.py

colorama==0.4.4
genie==21.4
netmiko==3.4.0
nornir==3.1.1
nornir-netmiko==0.1.1
nornir-utils==0.1.2
pyats==21.4

from nornir import InitNornir

from nornir_netmiko.tasks import netmiko_send_command, netmiko_send_config

from nornir_utils.plugins.functions import print_result, print_title

from nornir.core.exceptions import NornirExecutionError

from nornir.core.task import Task, Result

import getpass

import sys

import os

from colorama import Fore, Style

import threading

import ntp_check

nr = InitNornir(config_file="config.yml")

# to run this script use: "env $(gpg --decrypt encrypted.env.gpg) python3 compliance-nornir3-encrypted.py"

nr.inventory.defaults.username = os.getenv("DEFAULT_USERNAME")

nr.inventory.defaults.password = os.getenv("DEFAULT_PASSWORD")

LOCK = threading.Lock()

with open('compliance-cmds.txt', 'r') as f:

    filelines = f.readlines()

clear_command = "clear"

#os.system(clear_command)

def ntp_check(task):

    mylist=[]

    output = task.run(task=netmiko_send_command, command_string="show run", use_genie=False)

    for cmd in filelines:

        if not cmd in output.result:

            mylist.append(cmd)

        else:

            print(Fore.GREEN + Style.BRIGHT+ "*" * 80)

            print(Fore.GREEN + f"ALERT: {task.host} is compliant!")

            print(Fore.GREEN + "Well Done!")

    if not mylist:

        john = "blah-de-blah"

    else:

        LOCK.acquire()

        print(Fore.GREEN + Style.BRIGHT+ "*" * 80)

        print(Fore.RED + f"ALERT: {task.host} is not NTP compliant!")

        print(Fore.YELLOW + "The following commands are missing:")

        try:

            for items in mylist:

                print(items)

        finally:

            LOCK.release()

            

results = nr.run(task=ntp_check)

print_title("COMPLETED COMPLIANCE TEST")

#print_result(results)

Radius authicatication using py-radius

  pip install py-radius

$ python -m radius

Host [default: 'radius']: 1.1.1.1

Port [default: 1812]: 1812

Enter RADIUS Secret: s3cr3t

Enter your username: foobar

Enter your password: qux

...

Authentication Successful

For Python3 an error message may be triggered:

  File "/home/user/venv-folder/lib/python3.8/site-packages/radius.py", line 583, in main

    host = raw_input("Host [default: 'radius']: ")

    NameError: name 'raw_input' is not defined

This is because Python3 uses "input" instead of "raw_input"

Editing the file "radius.py" and replacing "raw_input" with "input" fixes the issue. This is a quick fix but a long term solution should be used. This would be using a library written for Python3.

Mosquitto MQTT Bridge

A bridge allows 2 mosquitto mqtt brokers to connect to each other. Only one broker needs to be configured.

Add the following config to mosquitto.conf file:

# =================================================================
# Bridges
# =================================================================

connection homeassistant
address 1.1.1.1:1883
topic # both 0
cleansession false
notifications false
remote_clientid broker0
remote_password mypass
remote_username myuser
start_type automatic

# =================================================================
# Listeners
# =================================================================
listener 1883 0.0.0.0

The above config will start a Listener on port 1883 and will connect to another broker 1.1.1.1 also on port 1883. The ports can be different but for the purpose of this example the default was used.

Test (1.1.1.1 is the standalone broker and 2.2.2.2 is the bridge broker):

 # mosquitto_sub -h 1.1.1.1 -p 1883 -t '#' -u myuser -P mypass

0

1

/ # mosquitto_sub -h 2.2.2.2 -p 1883 -t '#' -u myuser -P mypass

0

1

The same messages are being received. 2.2.2.2 gets the messages from 1.1.1.1 because it acts as a bridge.

Syslog-NG to MySql

Install prerequisites:

 apt install syslog-ng

apt install syslog-ng-libdbi

apt install libdbi-drivers

apt install libdbi-devel

apt install libdbi-dbd-mysql

apt install syslog-ng-libdbi

Create "syslog" database and a MySQL user which has DELETE, INSERT, SELECT, UPDATE on the syslog database / schema.:

Create Table: CREATE TABLE `logs` (

  `host` varchar(32) DEFAULT NULL,

  `facility` varchar(10) DEFAULT NULL,

  `priority` varchar(10) DEFAULT NULL,

  `level` varchar(10) DEFAULT NULL,

  `tag` varchar(10) DEFAULT NULL,

  `datetime` datetime DEFAULT NULL,

  `program` varchar(15) DEFAULT NULL,

  `msg` text,

  `seq` bigint(20) unsigned NOT NULL AUTO_INCREMENT,

  PRIMARY KEY (`seq`),

  KEY `date_host` (`datetime`,`host`)

) ENGINE=InnoDB AUTO_INCREMENT=14120 DEFAULT CHARSET=utf8




Edit syslog.conf:

nano /etc/syslog-ng/syslog-ng.conf

@version: 3.13

#options {

#flush_lines (0);

#time_reopen (10);

#log_fifo_size (1000);

#long_hostnames (off);

#use_dns (yes);

#use_fqdn (yes);

#create_dirs (yes);

#keep_hostname (yes);

#};

 

# syslog-ng log source

source s_net { udp(ip("10.0.0.115") port(514)); };

#perm(0644) dir_perm(0700) create_dirs(yes)); };

 

 

source s_sys {

file ("/proc/kmsg" program_override("kernel: "));

unix-stream ("/dev/log");

internal();

};

 

destination d_file { file("/var/log/messagestest"); };

#perm(0644) dir_perm(0700) create_dirs(yes)); };

destination d_cons { file("/dev/console"); };

destination d_mesg { file("/var/log/messages"); };

destination d_auth { file("/var/log/secure"); };

destination d_mail { file("/var/log/maillog" flush_lines(10)); };

destination d_spol { file("/var/log/spooler"); };

destination d_boot { file("/var/log/boot.log"); };

destination d_cron { file("/var/log/cron"); };

destination d_kern { file("/var/log/kern"); };

destination d_mlal { usertty("*"); };

 

# MySQL define destination

destination d_mysql {

sql(

type(mysql)

username("root")

password("password")

database("syslog")

host("10.0.0.65")

table("logs")

columns("host", "facility", "priority", "level", "tag", "datetime", "program", "msg")

values("$HOST", "$FACILITY", "$PRIORITY", "$LEVEL", "$TAG","$YEAR-$MONTH-$DAY $HOUR:$MIN:$SEC","$PROGRAM", "$MSG")

indexes("datetime", "host")

);

};

 

 

filter f_kernel { facility(kern); };

filter f_default { level(info..emerg) and

not (facility(mail)

or facility(authpriv)

or facility(cron)); };

filter f_auth { facility(authpriv); };

filter f_mail { facility(mail); };

filter f_emergency { level(emerg); };

filter f_news { facility(uucp) or

(facility(news)

and level(crit..emerg)); };

filter f_boot { facility(local7); };

filter f_cron { facility(cron); };

 

# MySQL log to destination

log {source(s_net); destination(d_file); destination(d_mysql);};

 

# map source to destination fields.

log { source(s_sys); filter(f_kernel); destination(d_kern); };

log { source(s_sys); filter(f_default); destination(d_mesg); };

log { source(s_sys); filter(f_auth); destination(d_auth); };

log { source(s_sys); filter(f_mail); destination(d_mail); };

log { source(s_sys); filter(f_emergency); destination(d_mlal); };

log { source(s_sys); filter(f_news); destination(d_spol); };

log { source(s_sys); filter(f_boot); destination(d_boot); };

log { source(s_sys); filter(f_cron); destination(d_cron); };

Create a Docker container using Dockerfile

Find the project you want to create a container for and copy the entire folder and an empty file "Dockerfile" into the same directory. In this example we will copy "aci_app_ro" and "Dockerfile" inside the newly created directory "docker".

$pwd

/root/home

$ls

aci_app_ro Dockerfile

$mkdir docker

$cp aci_app_ro/ docker

$cp Dockerfile docker

Once you have all the files in the "docker" directory edit the "Dockerfile" with the following:

$cd docker

~/docker$ nano Dockerfile

FROM ubuntu:18.04

RUN apt-get update -y

RUN apt-get install -y apt-utils

RUN apt-get install -y python-pip

RUN apt-get install -y python-dev build-essential

COPY aci_app_ro /aci_app_ro

WORKDIR /aci_app_ro

RUN pip install -r requirements.txt

ENTRYPOINT ["python"]

CMD ["oneaciapp.py"]

Save the Dockerfile and build the container:

~/docker$ docker build . --tag aci_app_ro

Sending build context to Docker daemon  2.814MB

Step 1/10 : FROM ubuntu:18.04

 ---> 2c047404e52d

Step 2/10 : RUN apt-get update -y

 ---> Using cache

 ---> 1b7c84406e57

Step 3/10 : RUN apt-get install -y apt-utils

 ---> Using cache

 ---> 3297130974f4

Step 4/10 : RUN apt-get install -y python-pip

 ---> Using cache

 ---> e718357e701c

Step 5/10 : RUN apt-get install -y python-dev build-essential

 ---> Using cache

 ---> 96e1cd0f87a0

Step 6/10 : COPY aci_app_ro /aci_app_ro

 ---> Using cache

 ---> 5cfada21415a

Step 7/10 : WORKDIR /aci_app_ro

 ---> Using cache

 ---> 1fd61751593f

Step 8/10 : RUN pip install -r requirements.txt

 ---> Using cache

 ---> 1423f72bfbb7

Step 9/10 : ENTRYPOINT ["python"]

 ---> Using cache

 ---> 8970d0c9c7a8

Step 10/10 : CMD ["oneaciapp.py"]

 ---> Using cache

 ---> f5df0cc62b98

Successfully built f5df0cc62b98

Check the you have the docker image in the repository:

 ~/docker$ docker image ls | grep f5df0cc62b98

aci_app_ro_container         latest                f5df0cc62b98        2 minutes ago        468MB

Create new project and push to Github

work on a project locally inside a folder. Before you execute any git commands make sure you exclude any sensitive files where you may have credentials. For example you may have a creds.py file to avoid saving usernames, passwords or API keys directly into your development code. 

$ echo creds.py >> .gitignore 

## Initialize a new repository from command line

echo "# Flask-Portal" >> README.md
git init
git add README.md
git commit -m "first commit"
git branch -M master
git remote add origin https://github.com/aresik/Flask-Portal.git
git push -u origin master