Thursday, 15 October 2015

System Deployment (5 of x)

Fixing the dodgy network settings. The debian-installer finds a hostname from DHCP of another site running on my providers service. Manually editing /etc/hostname and then rebooting to solve this.

Seems like the static configuration worked out well. Seeing this output gives me a nice warm feeling:

main@face:~$ netstat -l
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 *:ssh *:* LISTEN
tcp6 0 0 [::]:ssh [::]:* LISTEN

I like to know exactly what is running on a server, and how many access points it adds for an attacker. In this state the box should be safe until another openssh exploit is discovered, and they do not happen every year. Good enough.

Next I need to bring up bind9 for the domain (ha, there goes any security hopes :). Every step now will be done twice:

  • Live steps on the server (configuration drift)
  • Replicating the changes in the file-sytem overlay inside the remastering .iso

At the end I'll blast away the live system to check the installer can rebuild it properly. This won't happen very often, because the KVM simulated access to the raw disk is as slow as it is possible to be. The automated install (minus the manual modprobe for the driver, which is still bugging the crap out of me) takes about 40 minutes. Not so nice.

Bind9

The best way to configure bind9 is using an old configuration that works :) Otherwise proceed very carefully, and read as much as you can about security errors in bind9 configurations. For recovery of old files I took a not so elegant approach to backing up the live server: dd the disk and copy it to a workstation. Mounting this loopback (before killing the old server) gives me a simple way to recover things.

Disaster: apparently I did not dd the disk offsite. Instead I took the shortcut of tar'ing the entire file-system as it was running. This obviously made some sort of sense at the time. For inexplicable reasons it means that the /root directory is missing, along with the .bash_history inside it that was a record of the steps in building the server directly. Lesson: next time be ye not a dick and dd the disk offsite.

Not to worry, I deliberately cultivate an anally-rententive OCD-level journal of work. The steps to build the server will be in there... Disaster the 2nd: as I was running on 16-hour workdays with little sleep when the server was first installed there is a blank spot of a month in my notes. Oh bugger. Lesson: kind of unclear...

Well it can't be that hard to lookup again, I've already written the zone files once... Oh look, there's the zone files and everything is sealed inside a chroot jail so that I can run bind9 as a non-privleged user. That's a really cool idea, how the hell did I do that then?

Hmm, so it all changed with jessie then? I don't think that systemd and I will become friends after I prize its charred tentacles off of my drive.

The late_command in the preseed.cfg is getting a bit busy so we'll add a new script to the iso called chroot_bind9.bash:

#!/bin/bash
mkdir -p /var/bind9/chroot/{etc,dev,var/cache/bind,var/run/named}
mknod /var/bind9/chroot/dev/null c 1 3
mknod /var/bind9/chroot/dev/random c 1 8
chmod 660 /var/bind9/chroot/dev/{null,random}
mv /etc/bind /var/bind9/chroot/etc   # Installed default from package
ln -s /var/bind9/chroot/etc/bind /etc/bind
cp /etc/localtime /var/bind9/chroot/etc/
chown -R bind:bind /etc/bind/*
chmod 775 /var/bind9/chroot/var/{cache/bind,run/named}
chmod 775 /var/bind9/chroot/var/{cache/bind,run/named}
chgrp bind /var/bind9/chroot/var/{cache/bind,run/named}
cp /cdrom/initd_bind9 /etc/init.d/bind9
# Next line is deliberately fragile - should be checked / rewritten if there is a major update to bind9
# Also - this is gnu sed style argument, not bsd.
sed -i 's:PIDFILE=/var/run/named/named.pid:PIDFILE=/var/bind9/chroot/var/run/named/named.pid:' /etc/init.d/bind9
echo "\$AddUnixListenSocket /var/bind9/chroot/dev/log" > /etc/rsyslog.d/bind-chroot.conf
# Skip service restarts as we will reboot soon

The makefile needs to be updated to get the new info into the .iso:

remaster.iso: copy
cp preseed.cfg copy/
cp isolinux.cfg copy/isolinux/
cp /home/amoss/.ssh/rsa_face.pub copy/
cp chroot_bind9.bash copy/
chmod +x copy/chroot_bind9.bash
cp mechani.db copy/
tar czf copy/overlay.tgz -C config etc home/main
genisoimage -b isolinux/isolinux.bin -c isolinux/boot.cat -o remaster.iso -J -R -no-emul-boot -boot-load-size 4 -boot-info-table copy/

And lastly the preseed is updated to execute the new script:

d-i preseed/late_command string \
tar xzf /cdrom/overlay.tgz -C /target ; \
in-target chown -R main:main /home/main ; \
in-target chown root:root /etc/hosts ; \
in-target chown root:root /etc/ssh/sshd_config ; \
chmod 700 /target/home/main/.ssh ; \
in-target chown main:main /home/main/.ssh/authorized_keys ; \
chmod 600 /target/home/main/.ssh/authorized_keys ; \
/cdrom/chroot_bind9.bash

Again, for emphasis: none of this has been tested yet - but what fun is life if we do not live dangerously, eh? After a robust exchange of views with my registrar about the quality of service they have rebuild the entente cordial by manually flicking some switches somewhere, and lo and behold:

dig face.mechani.se +trace
; <<>> DiG 9.8.3-P1 <<>> face.mechani.se +trace
;; global options: +cmd
. 14196 IN NS j.root-servers.net.
. 14196 IN NS d.root-servers.net.
. 14196 IN NS c.root-servers.net.
. 14196 IN NS i.root-servers.net.
. 14196 IN NS h.root-servers.net.
. 14196 IN NS l.root-servers.net.
. 14196 IN NS k.root-servers.net.
. 14196 IN NS f.root-servers.net.
. 14196 IN NS g.root-servers.net.
. 14196 IN NS e.root-servers.net.
. 14196 IN NS m.root-servers.net.
. 14196 IN NS b.root-servers.net.
. 14196 IN NS a.root-servers.net.
;; Received 228 bytes from 8.8.8.8#53(8.8.8.8) in 89 ms

se. 172800 IN NS a.ns.se.
se. 172800 IN NS b.ns.se.
se. 172800 IN NS c.ns.se.
se. 172800 IN NS d.ns.se.
se. 172800 IN NS e.ns.se.
se. 172800 IN NS f.ns.se.
se. 172800 IN NS g.ns.se.
se. 172800 IN NS i.ns.se.
se. 172800 IN NS j.ns.se.
;; Received 492 bytes from 192.203.230.10#53(192.203.230.10) in 123 ms

mechani.se. 86400 IN NS face.mechani.se.
;; Received 63 bytes from 130.239.5.114#53(130.239.5.114) in 175 ms

face.mechani.se. 1800 IN A 46.246.89.132
mechani.se. 1800 IN NS face.mechani.se.
;; Received 63 bytes from 46.246.89.132#53(46.246.89.132) in 49 ms


Is good, no? The domain has only been offline for a month due to a "routine upgrade" :) Next up I will restore the gitolite configuration and mirror my lonely git server...

No comments:

Post a Comment