Permanently add a variable to $PATH in Linux

Typically when we add a new path variable in Linux we will run something like:

export PATH=$PATH:/path/to/bin

If you wish to make this permanent we simply need to add the above line to:

vi ~/.profile

Read more

Disabling IPv6 in CentOS 7

If you ever find yourself in the situation where it is absolutely necessary to disable IPv6 - this can be performed as below:

vi /etc/sysctl.conf

and add:

net.ipv6.conf.all.disable_ipv6 = 1

and then ensure changes are persistent:

sysctl -p

Finally reboot the system:

shutdown -r now

Read more

Securing IPTables in CentOS 7 / attack mitigation

Below I have compiled a list of simple (but effective) ways of helping protect yourself from denial of service style attacks with layer 3.

Smurf attacks: This happens when an attacker spoofs your address and sends a ICMP packet to a broadcast address on a network - all hosts on this subnet then will potentially send an echo reply to the victim which could render the victims machine unusable due to the sheer amount of traffic. The following rules will help mitigate smurf attacks my limiting the amount of inbound ICMP traffic and restricting specific ICMP methods that are used to help the attacks:

iptables -A INPUT -p icmp -m icmp -m limit --limit 1/second -j ACCEPT
iptables -A INPUT -p icmp -m icmp --icmp-type address-mask-request -j DROP
iptables -A INPUT -p icmp -m icmp --icmp-type timestamp-request -j DROP

or we can simply drop all inbound ICMP packets (recommended):

iptables -A INPUT -p icmp -j DROP

Invalid packets: We don't want to process them or log them during an attack - so simply drop them:

iptables -A INPUT -m state --state INVALID -j DROP
iptables -A FORWARD -m state --state INVALID -j DROP
iptables -A OUTPUT -m state --state INVALID -j DROP

Port scans: We want to prevent any port scans being run on our server - the following will block anyone who attempts to run a portscan on the server for 24 hours:

iptables -A INPUT -m recent --name portscan --rcheck --seconds 86400 -j DROP
iptables -A FORWARD -m recent --name portscan --rcheck --seconds 86400 -j DROP

and then the following removes the 'ban' after 24 hours:

iptables -A INPUT   -m recent --name portscan --remove
iptables -A FORWARD -m recent --name portscan --remove

We can also log the hosts initiating the port scan with:

iptables -A INPUT   -p tcp -m tcp --dport 139 -m recent --name portscan --set -j LOG --log-prefix "Portscan:"
iptables -A INPUT   -p tcp -m tcp --dport 139 -m recent --name portscan --set -j DROP

iptables -A FORWARD -p tcp -m tcp --dport 139 -m recent --name portscan --set -j LOG --log-prefix "Portscan:"
iptables -A FORWARD -p tcp -m tcp --dport 139 -m recent --name portscan --set -j DROP

Address spoofing: We also want to ensure that any WAN only connected machines do block any LAN addresses:

iptables -A INPUT -s 10.0.0.0/8 -j DROP
iptables -A INPUT -s 169.254.0.0/16 -j DROP
iptables -A INPUT -s 172.16.0.0/12 -j DROP
iptables -A INPUT -s 127.0.0.0/8 -j DROP
iptables -A INPUT -s 224.0.0.0/4 -j DROP
iptables -A INPUT -d 224.0.0.0/4 -j DROP
iptables -A INPUT -s 240.0.0.0/5 -j DROP
iptables -A INPUT -d 240.0.0.0/5 -j DROP
iptables -A INPUT -s 0.0.0.0/8 -j DROP
iptables -A INPUT -d 0.0.0.0/8 -j DROP
iptables -A INPUT -d 239.255.255.0/24 -j DROP
iptables -A INPUT -d 255.255.255.255 -j DROP

XMAS attacks: These attacks simply have every option set for whichever protocol is used and should almost always be treated as suspect - we can drop these with:

iptables -A INPUT -p tcp --tcp-flags ALL ALL -j DROP

Read more

Diagnosing Cisco switch performance issues

Cisco switches make use of ASIC hardware to quickly switches both packets and frames - for example ACL and QoS tables are cached into the hardware in the form of tables - the switch will then query the tables when it needs to make decisions - such as evaluate whether a source IP matches a specific ACL in a table.

In contrast to RAM - CAM (Content Addressable Memory) uses data to retrieve where the data is stored - while RAM does the opposite - retrieving data from a memory address.

The CAM table is used primarily for layer 2 switching and returns either 'true' (0) or 'false' (1) - e.g. if data is being sent to another host on the switch the MAC address will be looked up in the CAM table and the result is either true (returns the switch port) or false (flood the switch ports.)

TCAM (Ternary Content Addressable Memory) is like CAM - but it can store a third state which can be any value - hence is useful for storing layer 3 operations like switching and qos.

When the TCAM cable is full it will 'punt' further entries to the CPU.

Check overall process CPU usage and sort by CPU utilization:
show proc cpu sorted

Example output:

CPU utilization for five seconds: 44%/30%; one minute: 42%; five minutes: 32%
 PID Runtime(ms)     Invoked      uSecs   5Sec   1Min   5Min TTY Process
  96          88      147299          0  1.11%  1.04%  0.92%   0 Ethernet Msec Ti
 117          40       36582          1  0.15%  0.19%  0.17%   0 IPAM Manager  
 240          28       36535          0  0.15%  0.14%  0.12%   0 MMON MENG

** Note: The first percentage figure (44%) show the overall CPU utilization and the second figure shows the percentage of the prior figure of which is caused by traffic (30%) **

The following processes are used for handling punted packets to the CPU and hence can be used as an indication of whether punted packets are causing CPU perofrmance issues:

- HLFM address lea
- Check heaps
- Virtual exec
- RedEarth Tx Mana
- HPM counter proc

Some common causes for traffic being punted to the CPU are:

- ACL logging
- Broadcast storms
- TCAM table space used up

To generate a smaller report of the overall CPU usage I like to use:

show proc cpu extended

Check CPU usage on IOS threads (does not work on 2960 series)
show proc cpu detailed process iosd sorted

We can show packet counts for all CPU receive queues with:

show controllers cpu-interface

We can issue the following to get an overview of the TCAM utilization:

show platform tcam utilization

Checking the device memory:

show processes memory sorted

show processes memory detailed process iosd sorted

Sources: http://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst3750/software/troubleshooting/cpu_util.html#pgfId-1028395
https://supportforums.cisco.com/document/60831/cam-content-addressable-memory-vs-tcam-ternary-content-addressable-memory

Read more

How to reset vCenter 6.0 (and above) SSO password

In order to reset a lost vCenter 6+ SSO password we will firstly need to jump onto the server hosting the vCenter instance and then launch the VDC admin tool from the command prompt:

"%VMWARE_CIS_HOME%\vmdird\vdcadmintool.exe"

Hit '3' to perform a password reset and enter the SSO login (by default this is: [email protected])

It will then generate a new password you can use to login with.

Finally ensure you change password once logged in.

Read more

Setting up IPTables from scratch on CentOS 7

This tutorial will provide you with a basic IPTables configuration to help you get up and running initially.

We will firstly start by flushing all of our IPTables rules.

** Warning: This could potentially lock you out of SSH if performed incorrectly - ensure you have console access to the server if something goes wrong! **

Ensure that a default-accept rule is in place on all of the default chains:

sudo iptables -P INPUT ACCEPT
sudo iptables -P FORWARD ACCEPT
sudo iptables -P OUTPUT ACCEPT

and then flush the chains as well as any non-standard chains:

sudo iptables -t nat -F
sudo iptables -t mangle -F
sudo iptables -F
sudo iptables -X

Now we will start by allowing traffic to freely flow out and in from our loopback interface:

sudo iptables -A INPUT -i lo -j ACCEPT
sudo iptables -A OUTPUT -o lo -j ACCEPT

We will also want to ensure that already established connections can get back to the server - i.e. allow stateful connections.

sudo iptables -A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT

We will likely also want to allow all outbound traffic from connections that are currently established:

sudo iptables -A OUTPUT -m conntrack --ctstate ESTABLISHED -j ACCEPT

We will also likely wish to allow SSH access from a specific host network:

sudo iptables -A INPUT -p tcp -s 10.11.12.13/32 --dport 22 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT

and allow the incoming SSH connection outbound back to the SSH initiator:

sudo iptables -A OUTPUT -p tcp --sport 22 -m conntrack --ctstate ESTABLISHED -j ACCEPT

We might also wish to accept ICMP echo requests:

sudo iptables -A INPUT -p icmp -j ACCEPT

And also log and then drop any other packets:

sudo iptables -N LOGGING
sudo iptables -A INPUT -j LOGGING
sudo iptables -A FORWARD -j LOGGING
sudo iptables -A LOGGING -m limit --limit 2/min -j LOG --log-prefix "IPTables-Dropped: " --log-level 4
sudo iptables -A LOGGING -j DROP

By leaving the default chain policies as 'ACCEPT' we ensure that if someone accidentally flushes a chain they are not going to lock themselves out.

Read more

Setup AAA and SSH on a Cisco 2960

Firstly create a local user for use with AAA:

conf t
username test privilage 15 secret $tr0ngPa$$w0rd!

Proceed by enabling AAA by issuing the following:

aaa new-model

If you wish to use remote TACACS+ servers - we can define them as follows:

tacacs-server host 10.99.99.253 key YourSecureKey
tacacs-server host 10.99.99.252 key YourSecureKey

and then creating a login authentication method list:

aaa authentication login default group tacacs+ local

or if you do have have any remote tacacs+ servers:

aaa authentication login default local

and then apply the login list to the relevent methods, for example:

line console 0
login authentication default

line vty 0
login authentication default

Define a domain for SSH:

ip domain-name yourdomain.local

and then generate the RSA key:

crypto key generate rsa modulus 2048

We can now proceed to setup SSH by enabling SSH v2:

ip ssh version 2

enable the line:

line vty 0
transport input ssh

We should also lock down the SSH access by creating an appropraite ACL:

ip access-list standard mgmt-ssh
10 permit <management-subnet> <management-wildcardmask>
20 deny any log

and then apply it to a vty line:

line vty 0
access-class mgmt-ssh in

Read more

Setup a transparent proxy with Squid and CentOS 7

Forenote: For this tutorial I will be using the 'core' edition of CentOS 7.

We should firstly ensure our system is up to date:

yum update

and proceed by disabling firewalld

systemctl stop firewalld
systemctl mask firewalld

and also disabling selinux:

vi /etc/selinux/config

and add / modify the line:

SELINUX=disabled

and setting permisive mode (so we don't need to do a reboot):

setenforce 0

and then download and compile squid from source:

yum install perl perl-Crypt-OpenSSL-X509 && yum groupinstall "Development Tools"

cd /tmp
curl -O http://www.squid-cache.org/Versions/v3/3.5/squid-3.5.19.tar.gz
tar zxvf squid-3.5.19.tar.gz

cd squid-3.5.19

./configure --prefix=/usr/local/squid --enable-delay-pools
make all
make install

create our squid user:

useradd squid -s /sbin/nologin
chown -R squid:squid /usr/local/squid/var/logs
chown -R squid:squid /usr/local/squid/var/cache

and then lets create our squid config:

vi /etc/squid/squid.conf

and add:

cache_dir ufs /usr/local/squid/var/cache/squid 15000 16 256

cache_effective_user squid
cache_effective_group squid

# tweaks
dns_v4_first on

visible_hostname myproxy.local
http_port <your-interface-address>:3128 transparent

## Define our network ##
acl our_network src <your-subnet/24>

## make sure that our network is allowed ##
http_access allow our_network

## finally deny everything else ##
http_access deny all

and initialize the cache:

/usr/local/squid/sbin/squid -f /etc/squid/squid.conf -z

/usr/local/squid/sbin/squid -z

Now lets setup the firewall configuration...

Firstly ensure ip forwarding is enabled:

vi /etc/sysctl.conf

and add:

net.ipv4.ip_forward = 1

and ensure it persists reboot:

/sbin/sysctl -p /etc/sysctl.conf

and then install / enable iptables:

yum install iptables-services
systemctl enable iptables
systemctl start iptables

and create our iptables ruleset:

iptables -t filter -I INPUT 1 -i <source-interface> -p tcp --dport 80 -m state --state NEW,ESTABLISHED -j ACCEPT
iptables -t filter -I INPUT 1 -i <source-interface> -p tcp --dport 3128 -m state --state NEW,ESTABLISHED -j ACCEPT

iptables -t nat -A POSTROUTING -o <source-interface> -j MASQUERADE
iptables -t nat -A PREROUTING -i <source-interface> -p tcp --dport 80 -j REDIRECT --to-port 3128

Read more

Logging dropped iptables traffic

Generally speaking it is good practice to log dropped traffic on your firewall - this can be achieved pretty easily with iptables.

Firstly create a new chain called 'logging':

iptables -N LOGGING

and then instruct the input chain to send any unmatched packets to the 'logging' chain:

iptables -A INPUT -j LOGGING

and we can also do the same for FORWARD traffic:

iptables -A FORWARD -j LOGGING

** Make sure that there are no catch-all drop rules in the input chain as this will prevent the unmatched packets from being dropped (they will be dropped when they are evaluated in the logging chain.) **

We then define rate-limiting to prevent a build-up of logs and define a prefix / log level for the logs:

iptables -A LOGGING -m limit --limit 2/min -j LOG --log-prefix "IPTables-Dropped: " --log-level 4

Finally we apply the drop-all rule to our 'logging' chain:

iptables -A LOGGING -j DROP

Read more

Understanding superblocks in Linux file systems

A disk consists of sectors - a block is made up of these sectors.

Available block sizes differ dependent on the filesystem - for example:

ext2: 1024 (1Kb), 2048 (2Kb) or 4096 (4Kb) bytes

ext3: 1024 (1Kb), 2048 (2Kb), 4096 (4Kb) or 8096 (8kB) bytes

ext4: 1024 (1Kb) to 65536 (64Kb) bytes

xfs: 512 (0.5kB) to 65536 (64kB) bytes

The block size is defined when initially creating the file system e.g.:

mkfs -t ext3 -b 4096 /dev/sda1

Performance Considerations

Typically if you are going to be storing large files on the disk you will want to ensure you have a large[er] block size - look at it this way: if you have a large file with many blocks (the blocks combine together to create the file) it will take longer to read all of those individual blocks than if you had larger blocks - meaning less blocks to read.

And in reverse - if you have a large amount of small files you will want a small block size as there will be a greater amount of blocks available to accommodate the files and because the blocks are smaller the system is having to read less sectors and hence improves the performance.

Hard disks

A hard disk will contain a (constant) sector size - usually 512. It is important to note that a block size can't be lower than that of the hard disks sector size  - it must be a multiple of it's sector size e.g. 1024,2048 and so on.

Block Groups 

(Specific to EXT file systems - see 'Allocation Groups' for XFS) Block groups are made of individual blocks - which makes reading and writing large amounts of data easier.

Each block group contains a redundant copy of the super block (mentioned below) and file system descriptors.

The physical structure of the file system is made up as follows:

Superblocks

A superblock contains important filesystem meta data like:

- Blocks per block group
- Number of free blocks in the filesystem
- Mount and write times
- File system state (e.g. clean or dirty mount)

Hence protecting superblocks is extremely important - and loss of a superblock can render a filesystem unmountable!

Read more

Working with views and templates with Django

This tutorial serves as a quick guide on how to utilize templates with your views in Django.

Lets first deploy a new instance of Django:

pip install Django==1.9.7

cmd
mkdir C:%HOMEPATH%\Desktop\django
cd C:%HOMEPATH%\Desktop\django

django-admin startproject myapp

and then create our new web app:

cd myapp
python manage.py startapp nagios

python manage.py migrate
python manage.py runserver

http://127.0.0.1:8000/

And ensure the website comes up OK.

Now lets make a start on creating out template object - this will be done interactively via the shell initially:

python manage.py shell

from django import template
import datetime
myTemplateObject = template.Template('<html><body>The current time is {{ thetime }}.</body></html>')
myContextObject = template.Context({'thetime': datetime.datetime.now()})
print(myTemplateObject.render(myContextObject))

The above will simply output the date and time. In other languages this is very similar to achors which allow you insert specific data into a specific point in a document.

Now - a slightly more complex example might be where we wish to insert a number of rows into an HTML table - we can utilize python types like lists as well - for example:

from django import template

resultString = ''

tableelements = template.Template('<td> {{ element }} </td>')
for element in ('Harry', 'John', 'Ben'):
    resultString += tableelements.render(template.Context({'element': element}))

myTemplateObject = template.Template('<html><body><table style="width:100%"><tr> {{ resultString }} </tr></table></body></html>')
myContextObject = template.Context({'resultString': resultString})
print(myTemplateObject.render(myContextObject))

Now we will apply this to a view, for example in our views.py file:

Create a the folder 'mymonitor' and the file (template) template.html and ensure that you add the anchor in: {{ element }}

and then add the following to your views.py file:

def table1(request):
    return render(request, 'mymonitor/template.html', {'element': resultString})

Read more

lsof cand fuser command reference

lsof is a useful tool that allows you to view files that currently have a handle on them by a process (i.e. open) - below is a few common commands that will help when debugging.

View files open by a specific user

lsof -u limited

View processes that are listening on a tcp port:

sudo lsof -i TCP:80

View handles on a specific file:

sudo lsof /var/www/html/vhost/123.zip

View open files for a specific process id:

sudo lsof -p 5123

Where similarly fuser allows you to view the processes that have handles on them - but more specifically it allows you to view open processes on a mount point:

fuser -m /mount/remote-smb-share

and also automatically kill any processes that have a handle open e.g.:

fuser -k -m /mount/remote-smb-share/file.locked

Read more

Throttling internal internet access with the ASA

To perform this we will utilize a service policy - the example below only limits download speed (not upload speed) - since the latter is not usually contested.

Define an access list to match the traffic we wish to throttle (including subnets we do not wish to e.g. internal ones):

access-list rate-limit-acl extended deny ip 10.50.0.0 255.255.0.0 10.100.0.0 255.255.0.0
access-list rate-limit-acl extended permit ip any 10.100.0.0 255.255.0.0

Create a class-map to classify the traffic:

class-map throttle_classmap
match access-list rate-limit-acl
exit

Define a policy map to setup qos (this will limit bandwidth to 2 megabits and allow a burst speed of 16000 bits) :

policy-map throttle_policymap
class throttle_classmap
police output 2000000 16000
police input 2000000 16000
exit
exit

and finally create a service policy apply it to your inside interface:

service-policy throttle_policymap interface inside

Read more

Working with views and templates with Django

This tutorial serves as a quick guide (my notes) on how to utilize templates with your views in Django.

Lets first deploy a new instance of Django:

pip install Django==1.9.7

cmd
mkdir C:%HOMEPATH%\Desktop\django
cd C:%HOMEPATH%\Desktop\django

django-admin startproject myapp

and then create our new web app:

cd myapp
python manage.py startapp nagios

python manage.py migrate
python manage.py runserver

http://127.0.0.1:8000/

And ensure the website comes up OK.

Now lets make a start on creating out template object - this will be done interactively via the shell initially:

python manage.py shell

from django import template
import datetime
myTemplateObject = template.Template('<html><body>The current time is {{ thetime }}.</body></html>')
myContextObject = template.Context({'thetime': datetime.datetime.now()})
print(myTemplateObject.render(myContextObject))

The above will simply output the date and time. In other languages this is very similar to anchors which allow you insert specific data into a specific point in a document.

Now - a slightly more complex example might be where we wish to insert a number of rows into an HTML table - we can utilize other python types like lists as well - for example:

from django import template

resultString = ''

tableelements = template.Template('<td> {{ element }} </td>')
for element in ('Harry', 'John', 'Ben'):
    resultString += tableelements.render(template.Context({'element': element}))

myTemplateObject = template.Template('<html><body><table style="width:100%"><tr> {{ resultString }} </tr></table></body></html>')
myContextObject = template.Context({'resultString': resultString})
print(myTemplateObject.render(myContextObject))

Now we will apply this to a view, for example in our views.py file:

Create a the folder 'mymonitor' and the file (template) template.html and ensure that you add the anchor in: {{ element }}

and then add the following to your views.py file:

def table1(request):
    return render(request, 'mymonitor/template.html', {'element': resultString})

Read more

Identifying account lockouts with Windows Event Log

GPMC >> Default Domain Controllers Policy >> Computer Configuration >> Windows Settings >> Security Settings >> Local Policies >> Audit Policy and ensure the following are set:

Account Logon Events – Failure
Account Management – Success
Logon Events – Failure

You should look out for event ID 644 (which will appear as a Success event) on the DC with the PDC emulator.

Read more

Logging to file with haproxy

By default haproxy logs information to syslog - in order to get the output piped out to a text file we should perform the following:

sudo vi /etc/rsyslog.d/49-haproxy.conf

and ensure the following line is uncommented / present:

if $programname startswith 'haproxy' then /var/log/haproxy.log

This simply instructs the syslog daemon to pipe out any data submitted by the process 'haproxy' to /var/log/haproxy.log

You should also ensure that the syslog daemon is running:

sudo vi /etc/rsyslog.conf

and ensuring the following is uncommented:

$ModLoad imudp
$UDPServerRun 514
$UDPServerAddress 127.0.0.1

sudo service rsyslogd restart

tail -f  /var/log/haproxy.log

and try generating some traffic...

Read more