Wednesday, 13 December 2017

Using pip with a Python virtual environment (venv)

A venv is a way of isolating the host environment from that of a python project. I only came across these because Pycharm decided to adopt them by default now when creating new projects.

To install additional modules using pip we must firstly enter the virtual environment - your project should look something like:

├── bin
│   ├── activate
│   ├── activate.csh
│   ├── activate.fish
│   ├── activate_this.py
│   ├── easy_install
│   ├── easy_install-3.6
│   ├── pip
│   ├── pip3
│   ├── pip3.6
│   ├── python -> python3.6
│   ├── python3 -> python3.6
│   ├── python3.6
│   ├── python-config
│   ├── watchmedo
│   └── wheel
├── include
│   └── python3.6m -> /usr/include/python3.6m
├── lib
│   └── python3.6
├── lib64 -> lib
└── pip-selfcheck.json

In order to enter the virtual environment cd to the project directory e.g.:

cd /path/to/your/project

and then issue the following:

source venv/bin/activate

we can then use pip as usual to install modules:

pip install watchdog

vSphere Replication 6.5 Bug: 'Not Active' Status

This happened to myself when setting up a brand new vSphere lab with vSphere 6.5 and the vSphere Replication Appliance 6.5.1.

After setting up a new replicated VM I was presented with the 'Not Active' status - although there was no information presented in the tool tip.

So to dig a little deeper we can use the CLI to query the replicated VM status - but firstly we'll need to obtain the VM id number:

vim-cmd vmsvc/getallvms

and then query the state with:

vim-cmd hbrsvc/vmreplica.getState <id>

Retrieve VM running replication state:
        The VM is configured for replication. Current replication state: Group: CGID-1234567-9f6e-4f09-8487-1234567890 (generation=1234567890)
        Group State: full sync (0% done: checksummed 0 bytes of 1.0 TB, transferred 0 bytes of 0 bytes)

So it looks like it's at least attempting to perform the replication - however is stuck at 0% - so now devling into the logs:

cat /var/log/vmkernel.log | grep Hbr

2017-12-13T10:12:18.983Z cpu21:17841592)WARNING: Hbr: 4573: Failed to establish connection to [10.11.12.13]:10000(groupID=CGID-123456-9f6e-4f09-
8487-123456): Timeout
2017-12-13T10:12:45.102Z cpu18:17806591)WARNING: Hbr: 549: Connection failed to 10.11.12.13 (groupID=CGID-123456-9f6e-4f09-8487-123456): Timeout

It looks like the ESXI host is failing to connect to 10.11.12.13 (the Virtual Replication Appliance in my case) - so we can double check this
with:

cat /dev/zero | nc -v 10.11.12.13 10000

(Fails)

However if we attempt to ping it:

ping 10.11.12.13

we get a responce - so it looks like it's a firewall issue.

I attempt to connect to the replication appliance from another server:

cat /dev/zero | nc -v 10.11.12.13 10000

Ncat: Version 7.60 ( https://nmap.org/ncat )
Ncat: Connected to 10.11.12.13:10000.

So it looks like the firewall on this specific host is blocking outbound connections on port 10000.

My suspisions were confirmed when I reviewed the firewall rules from within vCenter on the Security Profile tab of the ESXI host:



Usually the relevent firewall rules are created automatically - however this time for whatever reason they have not been - so we'll need to
proceed by creating a custom firewall rule (which unfortuantely is quite cumbersome...):

SSH into the problematic ESXI host and create a new firewall config with:

touch /etc/vmware/firewall/replication.xml

and set the relevent write permissions:

chmod 644 /etc/vmware/firewall/replication.xml
chmod +t /etc/vmware/firewall/replication.xml

vi /etc/vmware/firewall/replication.xml

<!-- Firewall configuration information for vSphere Replication -->
<ConfigRoot>
<service>
<id>vrepl</id>
<rule id='0000'>
<direction>outbound</direction>
<protocol>tcp</protocol>
<porttype>dst</porttype>
<port>
<begin>10000</begin>
<end>10010</end>
</port>
</rule>
<enabled>true</enabled>
<required>false</required>
</service>
</ConfigRoot>

Revert the permsissions with:

chmod 444 /etc/vmware/firewall/replication.xml

and restart the firewall service:

esxcli network firewall refresh

and check its there with:

esxcli network firewall ruleset list

(Make sure it's set to 'enabled' - if not you can enable it via the vSphere GUI: ESXI Host >> Configuration >> Security Profile >> Edit Firewall
Settings.)

and the rules with:

esxcli network firewall ruleset rule list | grep vrepl

Then re-check connectivity with:

cat /dev/zero | nc -v 10.11.12.13 10000
Connection to 10.0.15.151 10000 port [tcp/*] succeeded!

Looks good!

After reviewing the vSphere Replication monitor everything had started syncing again.


Sources:

https://kb.vmware.com/s/article/2008226
https://kb.vmware.com/s/article/2059893




Friday, 8 December 2017

Thursday, 7 December 2017

Using USB storage with ESXI / vSphere 6.0 / 6.5

In order to get USB drives working with ESXI (which is not officially supported) we'll need to ensure the USB arbitrator service has been stopped (this will unfortunately prevent you from using USB pass through devices in your VM's - however in a development environment I can afford to for go this.):

/etc/init.d/usbarbitrator stop

and ensure it is also disabled upon reboot:

chkconfig usbarbitrator off

We'll now plug the device in and identify the disk with dmesg or:

ls /dev/disks

Create a new GPT table on the disk:

partedUtil mklabel /dev/disks/mpx.vmhba37\:C0\:T0\:L0 gpt

* Note: Some disks will come up as 'naa...' as well *

We now need to identify the start / end sectors:

partedUtil getptbl /dev/disks/mpx.vmhba37\:C0\:T0\:L0

>> gpt
>> 38913 255 63 625142448

To work out the end sector we do:

[quantity-of-cylinders] * [quantity-of-heads] * [quantity-of-sectors-per-track] - 1

so: 38913 * 255 * 63 - 1

which equals:

625137344

The start sector is always 2048 - however in earlier versions of VMFS (e.g. VMFS3 - this was 128)

partedUtil setptbl /dev/disks/mpx.vmhba37\:C0\:T0\:L0 gpt "1 2048 625137344 AA31E02A400F11DB9590000C2911D1B8 0"

and finally create the filesystem:

vmkfstools -C vmfs6 -S USB-Stick /dev/disks/mpx.vmhba37\:C0\:T0\:L0:1

Wednesday, 6 December 2017

Quickstart: Accessing an SQLite database from the command line

We'll firstly obtain the relevant packages:

sudo dnf install sqlite

or on Debian based distro's:

sudo apt-get install sqlite3

Then open the database with:

sqlite3 /path/to/database.sqlite

To view the tables we should issue:

.tables

and to review the rows within them:

select * from <table-name>;

and to describe the table schema issue:

.schema <table-name>

to insert issue:

insert into <table-name> values('testing',123);

and to delete issue:

delete from <table-name> where <column-name> = 1;

Wednesday, 8 November 2017

Configuring Dog Tag (PKI) Certificate Authority on Fedora

After trialling several web based CA's Dog Tag was one of the few CA's I found a reasonable amount of documentation for and has readily available packages for CentOS / Fedora.

Firstly let's install the package (it's not currently available in the stable repo yet):

sudo yum --enablerepo=updates-testing install dogtag-pki 389-ds-base

We will use 389 Directory Server to create a new LDAP server instance that Dogtag can use:

sudo setup-ds.pl --silent\
 General.FullMachineName=`hostname`\
 General.SuiteSpotUserID=nobody\
 General.SuiteSpotGroup=nobody\
 slapd.ServerPort=389\
 slapd.ServerIdentifier=pki-tomcat\
 slapd.Suffix=dc=example,dc=com\
 slapd.RootDN="cn=Directory Manager"\
 slapd.RootDNPwd=yourpassword

and then create our CA subsystem with:

sudo su -
pkispawn

Subsystem (CA/KRA/OCSP/TKS/TPS) [CA]: 

Tomcat:
  Instance [pki-tomcat]: 
  HTTP port [8080]: 
  Secure HTTP port [8443]: 
  AJP port [8009]: 
  Management port [8005]: 

Administrator:
  Username [caadmin]: 
  Password: 
  Verify password: 
  Import certificate (Yes/No) [N]? 
  Export certificate to [/root/.dogtag/pki-tomcat/ca_admin.cert]: 

Directory Server:
  Hostname [YOURVM.LOCAL]: 
  Use a secure LDAPS connection (Yes/No/Quit) [N]? 
  LDAP Port [389]: 
  Bind DN [cn=Directory Manager]: 
  Password: 
  Base DN [o=pki-tomcat-CA]: 

Security Domain:
  Name [LOCAL Security Domain]: 

Begin installation (Yes/No/Quit)? Y

Log file: /var/log/pki/pki-ca-spawn.20171108144208.log
Installing CA into /var/lib/pki/pki-tomcat.
Storing deployment configuration into /etc/sysconfig/pki/tomcat/pki-tomcat/ca/deployment.cfg.
Notice: Trust flag u is set automatically if the private key is present.
Created symlink /etc/systemd/system/multi-user.target.wants/pki-tomcatd.target → /usr/lib/systemd/system/pki-tomcatd.target.

    ==========================================================================
                                INSTALLATION SUMMARY
    ==========================================================================

      Administrator's username:             caadmin
      Administrator's PKCS #12 file:
            /root/.dogtag/pki-tomcat/ca_admin_cert.p12

      To check the status of the subsystem:
            systemctl status pki-tomcatd@pki-tomcat.service

      To restart the subsystem:
            systemctl restart pki-tomcatd@pki-tomcat.service

      The URL for the subsystem is:
            https://YOURVM.LOCAL:8443/ca

      PKI instances will be enabled upon system boot

    ==========================================================================

We can now either use the CLI or web-based interface to manage the server - however we will firstly need to ensure that the certificate we generated prior (/root/.dogtag/pki-tomcat/ca_admin.cert) is included in the Firefox certificate store:

Firefox >> Edit >> Preferences >> Advanced >> Certificates >> View Certificates >> 'Your Certificates' >> 'Import...'

Browse to the web ui: https://YOURVM.LOCAL:8443/ca

You should be presented with a certificate dialog like below:


End users can access the following URL in order to request certificates:

https://yourvm.local:8443/ca/ee/ca/

For this purposes of this tutorial we will keep things simple and generate a certificate for use on a Windows machine - so let's firstly select 'Manual Server Certificate Enrolment':

openssl genrsa -out computer.key 2048
openssl req -new -sha256 -key computer.key -out computer.csr

cat computer.csr

and paste the certificate in as below:


Hopefully then (after hitting submit) we'll see:

Now - let's head to the admin section and approve the request:

https://yourvm.local:8443/ca/agent/ca/

List Requests >> Find


Click on the request, check the details etc. and finally hit 'Approve' at the bottom.

Once it has been approved you should see a copy of the (BASE64 encoded) certificate at the bottom of the confirmation page.

Finally we'll want to package this up a long with the corresponding private key:

openssl pkcs12 -inkey computer.key -in computer.pem -export -out computer.pfx

(where 'computer.pem' is the public portion that has just been generated.)

Proceed by importing the PFX file into the Windows Computer certificate store under 'Personal'.

Sources

Fedora Project :: PKI :: Quick Start

Wednesday, 1 November 2017

Implementing 802.1X with a Cisco 2960, FreeRADIUS and Windows 7 / 10

802.1X: 802.1X allows you to securely authenticate devices connecting to a network - while often employed in wireless networks it is also often used along side wired ones as well. A typical transaction will involve the authenticator (the switch in this case) sending a EAP message to the supplicant (the client workstation in this case) and will then send back an EAP response.

For this lab we will be focusing on wired networks and will be attempting to address the problem of visiting employees from company A from plugging in their equipment into Company B's infrastructure.

To start we will need the following components:

- A client machine running Windows 7 / 10 (192.168.20.2/24)
- A Cisco 2960 switch with IOS > 15 (192.168.20.1/24)
- A linux box running FreeRadius (192.168.20.254/24)

Switch

So let's firstly look at the switch portion - we'll configure dot1x and radius on the switch:

conf t
aaa new-model

radius server dot1x-auth-serv
 address ipv4 192.168.20.2 auth-port 1812 acct-port 1813
 timeout 3
 key (7) <sharedkey>

aaa group server radius dot1x-auth
server name dot1x-auth-serv

aaa authentication dot1x default group dot1x-auth
aaa accounting dot1x default start-stop group dot1x-auth
aaa authorization network default group dot1x-auth

and proceed by enabling dot1x:

dot1x system-auth-control

and we'll then enable it on the relevant ports:

int range gi0/1-5
switchport mode access
authentication port-control auto
dot1x pae authenticator

FreeRADIUS Server

We'll continue on the server by installing radiusd:

sudo yum install freeradius

and then use samba to communicate with the domain:

sudo yum install samba samba-common samba-common-tools krb5-workstation openldap-clients policycoreutils-python

(We are not actually setting up a samba server - instead just using some of the tools that are provided with it!)

vi /etc/samba/smb.conf


[global]
        workgroup = <domain-name>
        security = user
        winbind use default domain = no
        password server = <ad-server>
        realm = <domain-name>
        #passdb backend = tdbsam


[homes]
        comment = Home Directories
        valid users = %S, %D%w%S
        browseable = No
        read only = No
        inherit acls = Yes

and then edit the kerberos configuration:

vi /etc/krb5.conf

 Configuration snippets may be placed in this directory as well
includedir /etc/krb5.conf.d/

[logging]
 default = FILE:/var/log/krb5libs.log
 kdc = FILE:/var/log/krb5kdc.log
 admin_server = FILE:/var/log/kadmind.log

[libdefaults]
 dns_lookup_realm = false
 ticket_lifetime = 24h
 renew_lifetime = 7d
 forwardable = true
 rdns = false
 # default_realm = EXAMPLE.COM
 default_ccache_name = KEYRING:persistent:%{uid}

[realms]
 domain.com = {
  kdc = pdc.domain.com
 }

[domain_realm]
 .domain.com = DOMAIN.COM
 domain.com = DOMAIN.COM

and then modify NSS to ensure that it performs lookups using windbind:

vi /etc/nsswitch.conf

passwd:     files winbind
shadow:     files winbind
group:      files winbind
protocols:  files winbind
services:   files winbind
netgroup:   files winbind
automount:  files winbind

ensure samba starts on boot and restart the system:

sudo systemctl enable smb
sudo systemctl enable winbind
sudo shutdown -r now

Now join the domain with:

net join -U <admin-user>

and attempt to authenticate against a user with:

wbinfo -a <username>%<password>

should return something like:

Could not authenticate user user%pass with plaintext password
challenge/response password authentication succeeded

We also need to ensure NTLM authentication works (as this is what is used with FreeRadius):

ntlm_auth --request-nt-key --domain=<domain-name> --username=<username>

should return:

NT_STATUS_OK: Success (0x0)

(Providing the account is in good order e.g. not locked etc.)

The 'ntlm_auth' program needs access to the 'winbind_privileged' directory - so we should ensure that the user running the radius server is within the 'wbpriv' group:

usermod -a -G wbpriv radiusd

and then proceed to install and setup freeradius:

sudo yum install freeradius
sudo systemctl enable radiusd

mv /etc/raddb/clients.conf /etc/raddb/clients.conf.orig
vi /etc/raddb/clients.conf

and add the following:

client <switch-ip> {
        secret                = <secret-key>
        shortname             = <switch-ip>
        nastype               = cisco
}

We'll now configure mschap:

vi /etc/raddb/mods-enabled/mschap

and ensure the following is set:

with_ntdomain_hack = yes

ntlm_auth = "/usr/bin/ntlm_auth --request-nt-key --username=%{%{Stripped-User-Name}:-%{%{User-Name}:-None}} --challenge=%{%{mschap:Challenge}:-00} --nt-response=%{%{mschap:NT-Response}:-00}"

and the EAP configuration:

sudo vi /etc/raddb/mods-available/eap

Search for the following line:

tls-config tls-common

and uncomment:

random_file = /dev/urandom

Make sure your firewall is setup correctly:

sudo iptables -A INPUT -p udp -m state --state NEW -m udp --dport 1812 -j ACCEPT
sudo iptables -A INPUT -p udp -m state --state NEW -m udp --dport 1813 -j ACCEPT

and then test the configuration by running FreeRadius in test mode:

sudo radiusd -XXX

Then on the Windows 7 / 10 client workstation ensure that the 'Wired AutoConfig' service has been started (and has also been set to 'Automatic'.)

On the relevant NIC properties ensure that 'Enable 802.1X authentication' has been enabled:


Within Windows 10 you should not need to perform this - however it's always best to check the defaults just in case!

If you are not using a certificate ensure that the 'Verify the server's identity by validating the certificate' is unchecked as below (within the PEAP Settings):



Click 'Configure...' next to 'Secured password (EAP-MSCHAP v2) and ensure that 'Automatically use my Windows logon name and password' is ticked.

Finally hit 'OK' and back on the 'Authentication' tab - click on 'Additional settings...' and ensure 'Specify authentication mode' is ticked and is set to 'User authentication'.

We can now attempt to plug the client into the switch and with any luck we will obtain network access!


Sources

https://documentation.meraki.com/MS/Access_Control/Configuring_802.1X_Wired_Authentication_on_a_Windows_7_Client

http://wiki.freeradius.org/guide/freeradius-active-directory-integration-howto