Hiera and how it works within Puppet

Hiera and how it works within Puppet Hiera allows us to modify settings from modules within Puppet - for this example I will be tweaking some of the default settings from the saz/ssh module. Let's start by firstly install hiera: puppet module install puppet/hiera Hiera makes use of hierarchies - for example servers in a specific location might need a paticular DNS server, however all servers might require a specific SSH configuration. These settings are defined within the hiera.yaml file: cat /etc/puppetlabs/code/environments/production/hiera.yaml --- version:...

Read more

Creating files from templates with Puppet

To utilise a templates when creating new files we can issue something like: /etc/puppetlabs/code/environments/production/manifests/site.pp  file { '/etc/issue':     ensure  => present,     owner   => 'root',     group   => 'root',     mode    => 0644,     content => template($module_name/issue.erb),   } The source / content must be stored within a puppet module - so in the case we were using Saz's SSH module - we would place the template...

Read more

Changing regional settings (locate, time zone, keyboard mappings) in CentOS 7 / RHEL

Quite often when deploying new instances of CentOS the process of setting regional settings like time is often hidden from the user behind the OS installer and more often than not it is not necessary to change these. However this post will outline the steps that need to be taken if a server has been moved geographically or has simply not been configured correctly in the first place! We'll start my changing the time zone - this is pretty straight forward and you can find timezone settings available to the system in: ls -l /usr/share/zoneinfo/ In...

Read more

vi(m) Cheat Sheet

The following is a list of common commands that I will gradually compile for working with vi / vim. Find and Replace (Current Line) :/s/csharp/java Find and Replace (All Line) :$s/csharp/jav...

Read more

Adding a new disk with LVM

Identify the new disk with: lsblk and add the disk as a physical volume: pvcreate /dev/sdb Verify it with: pvdisplay Now create a new virtual group with: vgcreate myvg /dev/sdb and finally a new logical volume (ensuring all space is allocated to it): lvcreate -n mylg -l 100%FREE myvg and verify with: lvdisplay and finally create a new filesystem on the volume: mkfs.xfs /dev/myvg/m...

Read more

Using pip with a Python virtual environment (venv)

A venv is a way of isolating the host environment from that of a python project. I only came across these because Pycharm decided to adopt them by default now when creating new projects. To install additional modules using pip we must firstly enter the virtual environment - your project should look something like: ├── bin │   ├── activate │   ├── activate.csh │   ├── activate.fish │   ├── activate_this.py │   ├── easy_install │   ├── easy_install-3.6 │   ├── pip │   ├──...

Read more

vSphere Replication 6.5 Bug: 'Not Active' Status

This happened to myself when setting up a brand new vSphere lab with vSphere 6.5 and the vSphere Replication Appliance 6.5.1. After setting up a new replicated VM I was presented with the 'Not Active' status - although there was no information presented in the tool tip. So to dig a little deeper we can use the CLI to query the replicated VM status - but firstly we'll need to obtain the VM id number: vim-cmd vmsvc/getallvms and then query the...

Read more

Change default calendar permissions of a mailbox in Exchange 2010 / 2013 / 2016

Firstly obtain the current permissions with: Get-MailboxFolderPermission -identity "<useremail>:\Calendar" and to change them issue something like: Set-MailboxFolderPermission -identity "<useremail>@healix.com:\Calendar" -user Default -AccessRights Revie...

Read more

Using USB storage with ESXI / vSphere 6.0 / 6.5

In order to get USB drives working with ESXI (which is not officially supported) we'll need to ensure the USB arbitrator service has been stopped (this will unfortunately prevent you from using USB pass through devices in your VM's - however in a development environment I can afford to for go this.): /etc/init.d/usbarbitrator stop and ensure it is also disabled upon reboot: chkconfig usbarbitrator off We'll now plug the device in and identify the disk with dmesg or: ls /dev/disks Create a new GPT table on the disk: partedUtil mklabel /dev/disks/mpx.vmhba37\:C0\:T0\:L0...

Read more

Quickstart: Accessing an SQLite database from the command line

We'll firstly obtain the relevant packages: sudo dnf install sqlite or on Debian based distro's: sudo apt-get install sqlite3 Then open the database with: sqlite3 /path/to/database.sqlite To view the tables we should issue: .tables and to review the rows within them: select * from <table-name>; and to describe the table schema issue: .schema <table-name> to insert issue: insert into <table-name> values('testing',123); and to delete issue: delete from <table-name> where <column-name>...

Read more

Configuring Dog Tag (PKI) Certificate Authority on Fedora

After trialling several web based CA's Dog Tag was one of the few CA's I found a reasonable amount of documentation for and has readily available packages for CentOS / Fedora. Firstly let's install the package (it's not currently available in the stable repo yet): sudo yum --enablerepo=updates-testing install dogtag-pki 389-ds-base We will use 389 Directory Server to create a new LDAP server instance that Dogtag can use: sudo setup-ds.pl...

Read more

Implementing 802.1X with a Cisco 2960, FreeRADIUS and Windows 7 / 10

802.1X: 802.1X allows you to securely authenticate devices connecting to a network - while often employed in wireless networks it is also often used along side wired ones as well. A typical transaction will involve the authenticator (the switch in this case) sending a EAP message to the supplicant (the client workstation in this case) and will then send back an EAP response. For this lab we will be focusing on wired networks and will be attempting...

Read more

SELinux: Adding a trusted directory into the httpd policy

By default on CentOS 7 / RHEL the '/var/www' directory is not permitted as part of the httpd policy - so instead we need to use semanage command in order to add this directory: semanage fcontext -a -t httpd_sys_rw_content_t '/var/www/' and then apply the context changes with: restorecon -v /var/www/ you will also need to apply the context changes to any files within the directory as well e.g.: restorecon -v /var/www/index.html...

Read more

Using Arachni Scanner with cookies / restricted areas

Below is a command line example I like to use with the Arachni Scanner - it allows you to use a session cookie (you can obtain from something like tamperdata) and ensures that specific URL's are not caled - for example logoff - which would (obviously) kill our session: ./arachni --http-cookie-string "cookie123" --scope-exclude-pattern logoff --scope-exclude-pattern login https://yourdomain.com/auth/restrictedar...

Read more

Python Example: Viewing members of a group with ldap3

Although the ldap3 module for python is well documented I didn't find many good examples - so I decided to publish this one for others: from ldap3 import Server, Connection, ALL, NTLM, SUBTREE import re # Global varsBindUser = 'domain\\username'BindPassword = '<yourpassword>'SearchGroup = 'Domain Admins'ADServer = 'dc01.domain.tld'SearchBase = 'DC=domain,DC=tld' def getUsersInGroup(username, password, group): server = Server(ADServer) conn = Connection(server, user=username, password=password, authentication=NTLM, auto_bind=True) ...

Read more

Python ldap: AttributeError: module 'ldap' has no attribute 'open'

Uninstall the older ldap module: sudo pip uninstall ldap and install the newer 'python-ldap' module: sudo pip install python-l...

Read more

Changing a puppet master certificate

In the event you want to change a puppet server's hostname you will need to also generate a new certificate and re-issue a certificate to each of it's agents. Firstly delete the existing certificate on the puppet master: rm -Rf /etc/puppetlabs/puppet/ssl/ and on the puppetserver / CA issue: sudo puppet cert destroy <puppetserver.tld> and then on the puppetserver generate a new CA with: puppet cert generate puppetserver.int --dns_alt_names=puppetserver,puppetdb start the server: puppet master --no-daemonize --debug and on...

Read more

Forwarding mail for the root user to an external address

Quite often I find mail such as those generated by cron jobs are sent to the user they are executed under - for example root. Using the 'aliases' file we can instruct any mail destined for a specific user to be forwarded to another (internal or external) address - for example by adding the following to /etc/aliases: sudo vi /etc/aliases root: yourname@externalemail.com and ensure those changes take effect by issuing: sudo newaliases and finally reloading the mail server: sudo service postfix relo...

Read more

Installing / setting up Samba on CentOS 7

Firslty install the required packages: sudo dnf install samba samba-client samba-common We'll use /mnt/backup for the directory we wish to share: mkdir -p /mnt/backup Make a backup copy of the existing samba configuration: sudo cp /etc/samba/smb.conf cp /etc/samba/smb.conf.orig and adding the following into /etc/samba/smb.conf: [global] workgroup = WORKGROUP netbios name = centos security = user [ARCHIVE] comment = archive share path = /mnt/backup public = no valid users = samba1, @sambausers writable = yes browseable = yes create mask =...

Read more

Erasing an MBR (or GPT) and / or partition table and data of a disk

This can performed with dd - in order to wipe the MBR (the first sector this is executed after the BIOS / hardware initialisation) we should issue: sudo dd if=/dev/zero of=/dev/sdx bs=446 count=1 This wipes the first 446 bytes of the disk - while if we want to erase the MBR and the partition table we need to zero the first 512 bytes: sudo dd if=/dev/zero of=/dev/sdx bs=512 count=1 And then to erase the data on the disk we can issue: sudo dd if=/dev/zero of=/dev/sdx bs=4M count=1 Note: While strictly speaking the vast majority of modern drives...

Read more

Setting up DKIM for your domain / MTA

What is DKIM and how is it different to SPF? Both DKIM and SPF provide protection for your email infrastructure. SPF is used to prevent disallowed IP addresses from spoofing emails originating from your domain. DKIM validates that the message was initially sent by a specific domain and ensures its integrity. The two can (and should) be used together - since using DKIM might ensure the integrity of the email - but they can be re-sent (providing the message isn't modified) and potentially used for spam or phishing - hence employing SPF in addition...

Read more

Creating an internal / NAT'd network using a vSwitch on Server 2012 / 2016

We'll firstly need to install the Hyper V role - since we'll require the management tools in order to create our interface: Install-WindowsFeature Hyper-V –IncludeManagementTools Install-WindowsFeature Routing -IncludeManagementTools However I had the following message returned when attempting installation: Hyper-V cannot be installed: A hypervisor is already running. As I was running under VMWare I had to install the feature using a slightly different method (bare in mind we have no intention of using the Hyper V hypervisor - however we do...

Read more

Useful find command examples in Linux

The below is a compilation of 'find' commands that I often use myself. Finding files greater (or small) than 50mb find /path/to/directory -size +50m find /path/to/directory -size -50m Finding files with a specific file extension find /path/to/directory -name "prefix_*.php" Finding files (or folders) with specific permissions find /home -type f -perm 777 Finding files that have been changed  in the last hour find / -cmin -60 Performing an action with matched files (-exec switch) find / -cmin -60 -exec rm {}...

Read more

Adding a custom / unlisted resolution in Fedora / CentOS / RHEL

Sometimes I find that xrandr doesn't always advertise all of the supported resolutions for graphic cards - this can sometimes be down to using an unofficial driver or an older one. However in Fedora the latest drivers are usually bundled in for Intel graphics cards - unfortunately xrandr is only reporting that one resolution is available: xrandr -q Screen 0: minimum 320 x 200, current 1440 x 900, maximum 8192 x 8192 XWAYLAND0 connected (normal left inverted right x axis y axis)    1440x900      59.75 + In order to...

Read more

Wine: Could Not Initialize Graphics System. Make sure that your video card and driver are compatible with Direct Draw

For anyone else getting this problem when attempting to run older games on Wine - in my case this due to the graphics card not supporting the native resolution of the game (800x600) - you can check supported resolution types with: xrandr -q However you might be able to add custom resolutions as well. Otherwise within the Wine configuration you will need to ensure 'Emulate a virtual desktop' is ticked and the appropriate resolution for the game is s...

Read more

Mac Book Air: Installing the Broadcom BCM4360 - 14E4:43A0 module on Fedora

Firstly confirm you have the appropriate hardware version (there are two for the BCM4360!) lspci -vnn | grep Net The 'wl' module only supports the '14e4:43a0' version. The RPM fusion repository have kindly already packaged it up for us - so let's firstly add the repo: sudo dnf install -y https://download1.rpmfusion.org/nonfree/fedora/rpmfusion-nonfree-release-26.noarch.rpm https://download1.rpmfusion.org/free/fedora/rpmfusion-free-release-26.noarch.rpm sudo dnf install -y broadcom-wl kernel-develsudo akmods --force --kernel `uname -r` --akmod...

Read more

curl: 8 Command Line Examples

curl is a great addition to any scripter's arsenal and I tend to use it quite a lot - so I thought I would demonstrate some of its features in this post. Post data (as parameters) to url curl -d _username="admin" -d password="<password>" https://127.0.0.1/login.php Ensure curl follows redirects curl -L google.com Limit download bandwidth (2 MB per second) curl --limit-rate 2M -O http://speedtest.newark.linode.com/100MB-newark.bin Perform basic authentication curl -u username:password https://localhost/restrictedarea Enabling...

Read more

Windows Containers / Docker Networking: Inbound Communication

When working with Windows Containers I got a really bad headache trying to work out how to setup inbound communication to the container from external hosts. To summerize my findings: In order to allow inbound communication you will either need to use the '--expose'  or  '--expose' a long with  '--ports' switch - each of them do slightly different things. '--expose': When specifying this Docker will expose (make accessible) a port that is available to other containers only.  '--ports':  When used in conjunction...

Read more

git: Removing sensitive information from a repository

While you can use the 'filter-branch' switch to effectively erase all trace of a file from a repository - there is a much quicker way to do this using BFG Repo-Cleaner. Firstly grab an up to date copy of the repo with: git pull https://github.com/user123/project.git master Remove the file from the current branch: git rm 'dirtyfile.txt' Commit the changes to the local repo: git commit -m "removal" Push changes to the remote repo: git push origin master Download and execute BFG Repo-Cleaner: cd /tmp yum install jre-headless wget http://repo1.maven.org/maven2/com/madgag/bfg/1.12.15/bfg-1.12.15.jar cd...

Read more

Querying a PostgreSQL database

Firstly ensure your user has the adequate permissions to connect to the postgres server in pg_hba.conf. For the purposes of this tutorial I will be using the postgres user: sudo su - postgres psql \list \connect snorby or psql snorby For help we can issue: \? to list the databases: \l and to view the tables: \dt to get a description of the table we issue: \d+ <table-name> we can then query the table e.g.: select * from <table> where <column-name> between '2017-07-19 15:31:09.444' and '2017-07-21 15:31:09.444'; and...

Read more

Resolved: wkhtmltopdf: cannot connect to X server

Unfortunately the latest versions of wkhtmltopdf are not headless and as a result you will need to download wkhtmltopdf version 0.12.2 in order to get it running in a CLI environment. I haven't had any luck with any other versions - but please let me know if there are any other versions confirmed working. The other alternative is to fake an X server - however (personally) I prefer to avoid this approach. You can download version 0.12.2 from here: cd /tmp wget https://github.com/wkhtmltopdf/wkhtmltopdf/releases/download/0.12.2/wkhtmltox-0.12.2_linux-centos7-amd64.rpm rpm...

Read more

Exporting MSSQL Databases (schema and data) from Azure

Because Microsoft have disabled the ability to perform backups / exports of MSSQL databases from Azure directly from the SQL Management Studio (why?!) we now have to perform this from the Azure Portal. A new format introduced as a 'bacpac' file allows you to store both the database schema and data within a single (compressed) file. Open up the Resource Group in the Azure Portal, select the relevant database >> Overview >> and then select...

Read more

A crash course on Bash / Shell scripting concepts

if statement if [[ $1 == "123" ]] then echo "Argument 1 equals to 123" else echo "Argument 1 does not equal to 123" fi inverted if statement if ! [[ $1 == "123" ]] then echo "Argument 1 does not equal to 123" fi regular expression (checking for number) regex='^[0-9]+$' if [[ $num =~ regex ]] then echo "This is a valid number!" fi while loop NUMBER=1 while [[ $NUMBER -le "20" ]] do echo "The number ($NUMBER) is less than 20" NUMBER=$((NUMBER + 1)) done awk (separate by char) LINE=this,is,a,test echo "$LINE" | awk -F ',' '{print...

Read more

Setting up Octopus Tentacle on Windows Server 2012/2016 Core

For this tutorial I will be setting up Octopus Tentacle in a container running Server 2016 Core. Let's firstly create our container with: docker run -it --cpus 2 --memory 4G --network=<network-id> --name windowscore -h <your-hostname> microsoft/windowsservercore cmd.exe Ensure that your computer name is correct and setup in DNS - so both Octopus and the server running the Tentacle can communicate with each other. and for the purposes of this tutorial we will use a static IP and also join to to our domain with Powershell: Get-NetIPInterface...

Read more

Script to remove bad characters from a set of files

The need for this script was prompted by a series of files being uploaded to Sharepoint which had special characters within their filenames such as an astrix or tilde. Although there are many ways to achieve this I chose for a simplistic approach using cp and sed. We can use the sed substitute function to replace any bad characters - we have the following directory we wish to 'cleanse': ls /tmp/test drwxrwxr-x.  2 limited limited  120 Jul 13 13:45 . drwxrwxrwt. 40 root    root    1280 Jul 13 13:43 .. -rw-rw-r--....

Read more

Configuring NICs on Windows Server 2016 from the command line / Powershell

It seems that in Server 2016 they have removed some of the functionality in some older utilities such as netsh, netdom etc. So in order to configure IP addresses from the command line it looks like we should put our trust solely in Powershell (*cringes*.) In order to set a static IP address there are a few commands we need to run - firstly disabling the DHCP on the relevant NIC: Get-NetIPInterface | FL # grab the relevant interface ID from here Set-NetIPInterface -InterfaceIndex 22 -DHCP Disabled *Note: We can also use the 'InterfaceAlias'...

Read more

Windows Containers / Docker - Creating a 'bridged' or 'transparent' network

By default Windows Containers (or Docker on Server 2016) uses WinNAT to provide NAT functionality to containers - however in some cases you will likely want to run a container in bridged mode - i.e. provide direct network access to the container. We can do this fairly easily with: docker network create -d transparent -o com.docker.network.windowsshim.interface="Ethernet0" TransparentNet Confirm with: docker network ls We can also check the details of the network with the 'inspect' switch for example: docker network inspect <network-id> Static...

Read more

Snippet: Copying only modified files within X days

The following spinet allows you to copy only modified within the last 7 days (from the current date) to a predefined destination. find /var/log -d -mtime -7 -exec cp {} /home/user/modified_logs \; The -mtime command specifies the time (in days) of how far you wish to span back. You can also check for files that have been accessed with the '-atime' switch and similarly the creation time with 'ctime'. The '-exec' switch allows us to execute a custom command (in this case cp) and transplant the output of the find command into the custom cp command...

Read more

CentOS / RHEL: Enabling automatic updates of critical security patches

While I certainty wouldn't recommend enabling automatic updates (even general security updates) on a production server I would however (in most cases) recommend enabling automatic updates for critical security patches. We can do this with the yum-cron tool - which (as suggested) creates a cronjob to perform the updates: sudo -y install yum-cron We can then configure yum-cron - ensuring it only applies critical security updates: vi /etc/yum/yum-cron.conf and setup a mail host and destination - while ensuring that the update_cmd is set accordingly: update_cmd minimal-security-severity:Critical start...

Read more

Linux: Benchmarking disk I/O and determining block size

A quick command that can be used to benchmark the disk write speed: time sh -c "dd if=/dev/zero of=ddfile bs=8k count=250000 && sync"; rm ddfile and with a slightly larger block size: time sh -c "dd if=/dev/zero of=ddfile bs=16M count=10 && sync"; rm ddfile Note: Finding the appropriate block size when working with dd will be dependant on the hardware - the block size determines how how data is kept in memory during the copy - so you can imagine that if the block size is 2G in size you would need to ensure that you have at least...

Read more

RAM, CPU and I/O Throttling with Windows Server Containers / Docker

CPU Docker provides several options for limiting CPU with containers - one of the more common switches is '--cpus' - which allows you to limit the number of CPU's a container can use - for example if I wanted to ensure a container only used 1 and a half processors I could issue: docker run -it --name windowscorecapped  --cpus 1.5 microsoft/windowsservercore cmd.exe There is also a switch called '--cpu-shares' which allows you to delegate which containers would get priority (weighted) access to CPU cycles during (and only during) contention...

Read more