Millstream Software

Notes on how I set up my ADSL connected web server using OpenBSD.

This web site is being served to you from a computer in the Millstream Software office. This computer was installed and configured in 2004 and uses an ADSL connection to the local telephone exchange. This server also handles email and web browsing. Below are a few miscellaneous rough notes for the inquisitive and anyone considering a similar installation.

Configuring a server in this fashion is amusing for technically minded people but not something others should attempt. The shape of the configuration has been influenced by my interest in learning various internet technologies. Most ADSL users should delegate as much as possible to their internet service provider including DNS and web serving. A larger business would need a more complex configuration.


Asymmetric Digital Subscriber Line (ADSL)

ADSL is a permanently on high-speed connection to the internet and is available to the parts of the UK close to an upgraded local telephone exchange. It is asymmetric because you can get data from the internet faster than you can send it to the internet. Visit my internet service provider Andrews and Arnold for details.

Domain Name System (DNS)

This is the internet equivalent of the telephone book. It allows your computer to look up a domain name such as millstream.com and find the corresponding internet telephone number (called an IP address) which in this case is 81.2.66.10. It is also sometimes useful to look up an IP address and get the domain name. This is called reverse DNS and is answered by a separate telephone book arranged by number rather than name. I run some free software BIND on my server to contribute my entries to both these internet telephone books. The official documents - RFCs describing DNS can be somewhat opaque but need to be understood if you are going to run your own DNS. (DNS related RFCs include 1032, 1033, 1034, 1035, 1535, 1536, 1912, 2181, 2317.) You can study your DNS and various other aspects of your configuration as seen from the internet by using DNS Stuff

These grey boxes contain technical details and configuration files from 
my server. For an overview just ignore this information. The files may be 
specific to OpenBSD 3.5 and specific to my precise configuration and even 
then may not be the best solution so use them for learning not as a template.
/var/named/etc/named.conf

options {
        version "";
        pid-file "named.pid";
        recursion no;
        additional-from-cache no;
};

zone "."              { type hint;   file "standard/root.hint"; };
zone "millstream.com" { type master; file "master/millstream.com"; };

zone "55.65.2.81.in-addr.arpa"  { type master; file "master/violet" ; };
zone  "8.66.2.81.in-addr.arpa"  { type master; file "master/black"  ; };
zone  "9.66.2.81.in-addr.arpa"  { type master; file "master/white"  ; };
zone "10.66.2.81.in-addr.arpa"  { type master; file "master/red"    ; };
zone "11.66.2.81.in-addr.arpa"  { type master; file "master/yellow" ; };
zone "12.66.2.81.in-addr.arpa"  { type master; file "master/blue"   ; };
zone "13.66.2.81.in-addr.arpa"  { type master; file "master/green"  ; };
zone "14.66.2.81.in-addr.arpa"  { type master; file "master/orange" ; };
zone "15.66.2.81.in-addr.arpa"  { type master; file "master/pink"   ; };


/var/named/master/millstream.com

$TTL 1D
@ IN  SOA  @  r\.kelsall (
        45   ; Zone Serial
        1D   ; Secondary Refresh
        6H   ; Secondary Retry 
       40D   ; Secondary Expire
        1D ) ; Resource Record Cached For

  NS  @
  NS  secondary-dns.co.uk.

  MX  10  @
  MX  20  secondary-mx.co.uk.

  A  81.2.66.10

violet  A  81.2.65.55
black   A  81.2.66.8
white   A  81.2.66.9
yellow  A  81.2.66.11
blue    A  81.2.66.12
green   A  81.2.66.13
orange  A  81.2.66.14
pink    A  81.2.66.15

www  CNAME  millstream.com.


/var/named/master/red

$TTL 1D
@  IN  SOA  millstream.com.  r\.kelsall.millstream.com.  (45  1D  6H  40D  1D)
  NS   millstream.com.
  NS   secondary-dns.co.uk.
  PTR  millstream.com.


/var/named/master/blue

$TTL 1D
@  IN  SOA  millstream.com.  r\.kelsall.millstream.com.  (45  1D  6H  40D  1D)
  NS   millstream.com.
  NS   secondary-dns.co.uk.
  PTR  blue.millstream.com.

Static vs Dynamic IP Addresses

Any computer using the internet has to have a unique IP address - the internet equivalent of a telephone number. For an ordinary dial-up connection, for example when you browse the web using a modem, your ISP lends your computer an IP address for the duration of the call. For the next dial-up connection you are likely to get a different IP address. This IP address which is temporarily given to your computer is called a Dynamic IP address. The purpose of this is to reduce the number of IP addresses required by the ISP. IP addresses are a finite resource and are allocated carefully. In contrast a Static IP address is permanently assigned to you. Your name goes against it in a central database and you have to have a reason for possessing it. To run a web server or an email server you will need a static IP address. Multiple domain names can be served from a single IP address but some security mechanisms including Secure Sockets Layer (SSL) require the corresponding domain name to be returned from a reverse DNS query so you will need a separate IP address for each domain name using SSL. Some ADSL providers do not give you any static IP addresses or only a restricted number and charge heavily for extra IP addresses. Calculate your requirements before choosing an ADSL provider. A mechanism is available to perform dynamic DNS based on your dynamic IP address but this looked restrictive and more complicated to administer so I obtained some static IP addresses for my server.

/etc/hosts

::1       localhost.millstream.com localhost
127.0.0.1 localhost.millstream.com localhost
81.2.66.9     white.millstream.com white
81.2.66.10          millstream.com red
81.2.66.11   yellow.millstream.com yellow
81.2.66.12     blue.millstream.com blue
81.2.66.13    green.millstream.com green
81.2.66.14   orange.millstream.com orange


/etc/networks

BASE-ADDRESS.MCAST.NET	224
loopback		127	loop

 white.millstream.com 81.2.66.9
       millstream.com 81.2.66.10
yellow.millstream.com 81.2.66.11
  blue.millstream.com 81.2.66.12
 green.millstream.com 81.2.66.13
orange.millstream.com 81.2.66.14

OpenBSD

This is the open source operating system I use to run my server. I installed OpenBSD 3.5 onto a clean computer from the CD. I chose OpenBSD because it is secure and free. More secure means less work. OpenBSD is similar to Linux and other flavours of Unix. I found the notes from a short Unix course I attended more than ten years ago were relevant and with a little research, experimentation and following of instructions had no great problems using it.

As with any software it will be more secure if you only install the parts you need and after having installed it switch off any features you aren't going to use. So I didn't install the games nor any of the X graphical user interface. I use the text based interface which is fast and practical once you have become familiar with it and the knowledge you gain from this can be immediately applied to automating tasks. I find graphical user interfaces distance you from automating and combining the commands you want to perform.

If it is practical the security can be improved and the configuration work reduced by only allowing local terminal access to the server. This is what I have done and so I did not need to install ssh. I run all my server functions on one computer but it would be more secure to run each function on a different machine. Each extra function on a machine contributes an exponential increase in risk as a breach in one function may provide access to the others on the same machine. I have reduced the size of the exponent by 'sand-boxing' my functions using systrace.

Here are a few commands to get you started. Read about them in 
the manual. q or Control+c are the usual ways out of a command. 

man man
who -Tu
df -hi
ps -ax
find / -name pf.conf
cd /etc
ls -Flo
grep block *
cp pf.conf testfile
ls -Flo|more
vi testfile
chmod -w testfile
rm testfile
cd ..
alias
l
history
!!
cd /var/log
head -2 messages > testmessages
cat testmessages
tail -5 messages >> testmessages
mail -s "Test email to postmaster" postmaster < testmessages
arp -a
ifconfig -Am
netstat -rn -f inet
tcpdump -i xl0
route show
mount -t cd9660 -r /dev/cd0a /mnt
umount /mnt
logout
halt

The server is primarily set up by editing plain text files with names ending ".conf". To edit these files I use the vi editor.

A few commands in the vi editor you will find useful.

:q Enter       Quit (exit) vi.
:wq Enter      Write changes to the file and quit.
:q! Enter      Quit discarding changes.
/string Enter  Search forward for string.
/ Enter        Search again.
a              Append after cursor. Type text then Esc to exit append mode.
i              Insert. As Append but before cursor. 
Y              Yank (copy) a line. (2Y to yank 2 lines etc) 
p              Put (insert) yanked lines below cursor. (ditto)
dd             Delete a line. (ditto)
J              Join line at cursor with next line. (ditto)
nG             Go to line n.
0              (Zero.) Go to beginning of line.
$              Go to end of line.

The Hardware

To be economical and green my server is a Pentium III that has been pensioned off from Windows service. This is more than adequate for the task. An older PC would do just as well though it is useful to have a PC with a CD-ROM boot option in the BIOS which more modern PCs provide. My ISP sold me an ADSL router ready configured for my static IP addresses and a splitter for separating the ADSL and ordinary phone signals. An internal ADSL card for the PC might be a cheaper solution. I use two 100 Mbps ethernet cards in the server, which is excessive even for a fully loaded ADSL connection. (If a card doesn't work try swopping it for a different one.) One ethernet card connects to the ADSL router and the other to my Windows network. For historic reasons the server is configured as a bridge. It would be better to reconfigure it as a router with NAT. Starting from scratch NAT would be a simpler configuration.

/etc/hostname.xl0

inet 81.2.66.10 255.255.255.248 NONE 


/etc/hostname.xl1

inet 81.2.66.10 255.255.255.248 NONE 


/etc/mygate

81.2.66.9


/etc/myname

red

Firewalls

On my server I run the firewall that comes with OpenBSD called Packet Filter (PF). This just needs to be switched on and some rules defined in the pf.conf file. In addition to a rule set concocted from various documentation I have added rules to ensure the dreaded Windows File Sharing and similar Windows mechanisms can not get across the bridge. I have done this by completely blocking ports 135, 137, 138, 139 and 445. It is vital to block these ports if you have any Windows machines.

Note that although the bridge has its own set of rules the normal pf.conf rules apply to traffic passing across the bridge.

Even with a well configured firewall running on the server it is still worth having ZoneAlarm installed on any Windows PCs behind the firewall because it gives the ability to grant internet access rights to individual programs and can stop unauthorised programs escaping. It will also provide a second line of defense and some protection if an infected laptop is connected to your LAN.

/etc/pf.conf

ExtIf="xl0"
ServerIP="81.2.66.10"
BlockPorts="{135, 137, 138, 139, 445}"
    
# These addresses get blocked.
table <DeadIP> const {10/8, 172.16/12, \
                         192.168/16, 255.255.255.255/32}

# These addresses are behind the firewall.
table <LocalIP> const {$ServerIP, 127.0.0.1, \
                          81.2.66.11, 81.2.66.12, 81.2.66.13, 81.2.66.14}

set loginterface $ExtIf
scrub in all fragment reassemble
block log all

# Kill any incorrectly addressed packets.
block log quick on $ExtIf from 127/8 to any
block log quick on $ExtIf from any to 127/8
block log quick from <DeadIP> to any
block log quick from any to <DeadIP>
block in  log quick on $ExtIf from <LocalIP> to any
block out log quick on $ExtIf from any to <LocalIP>
block in  log quick on $ExtIf from any to ! <LocalIP>
block out log quick on $ExtIf from ! <LocalIP> to any

# Kill any Windows file sharing.
block log quick proto {tcp, udp} from any port $BlockPorts to any
block log quick proto {tcp, udp} from any to any port $BlockPorts

# Let machines contact the server.
pass in proto icmp       from any to $ServerIP keep state
pass in proto {tcp, udp} from any to $ServerIP port {25, 53, 80, 123} \
                                               flags S/SA keep state
pass in proto {tcp, udp} from <LocalIP> to $ServerIP port 110

# Let local machines contact the internet.
pass proto {icmp, udp} from <LocalIP> to any keep state
pass proto tcp         from <LocalIP> to any flags S/SA modulate state

Web Browsing

With the bridge and firewall established web browsing from my Windows PC just required the settings under Control Panel : Network : TCP/IP Properties to be adjusted to spring into life. Some of these settings will be given to you by your ISP. The 'DNS Server' is a machine provided by your ISP that will resolve your DNS queries. The 'Gateway' is the machine that lets you out onto the internet - in my case my ADSL router. The 'IP Address and Subnet Mask' can be set automatically using a thing called DHCP for a big local network but can be done just as easily by hand for a few PCs. A 'subnet' is a group of machines which can talk to each other without having to go through a gateway, in my case all the PCs on my office ethernet network. All these PCs need to be given IP addresses in the same block of IP addresses - the size of which is defined by the Subnet Mask. The Subnet Mask for all your machines will be identical as will the top part of the IP address down to the bits of the IP address which are not masked.

Email

I wanted to continue to use the same email program on my Windows PC for reading and sending emails but have my server perform the email functions previously provided by my ISP. To do this I run two pieces of software on the server : Sendmail and Popa3d. Sendmail is the postman of the email world and despite its name it both sends and receives email. There are other similar programs which can be used for the same purpose and they are generically known as Mail Transfer Agents (MTAs). Given an email to deliver they lookup the address of the recipient in the DNS and send the email on to a willing MTA closer to the destination. You can see all the MTA hops your emails have taken by looking at the detailed headers of your emails. The headers also usefully give an exact address for the origin of any spam if want to get them terminated. The email program on my Windows PC is generically termed a Mail User Agent (MUA). MUAs are for reading, composing and storing emails but need to access an MTA to send any emails. They often access the MTA using a protocol called SMTP so there is likely to be a setting in your MUA called something like 'Outgoing SMTP Server' to point it at your MTA. Conversely Sendmail needs to recognise the MUA machine as a legitimate sender of emails. This is achieved by an entry in the file local-host-names. The failure to constrain which machines can send emails via your server makes it an 'open mail rely'. This will result in your server being used to generate spam and consequently blacklisted. To check you haven't done this accidentally you can test your server using ORDB.

To get Sendmail to work you have to switch it on, give it some email 
addresses, create any necessary usernames (with adduser) and provide 
it with a configuration file. 

All the email addresses you need are put into the file called aliases 
which has to be converted to the more efficient aliases.db format before 
use by Sendmail. Aliases connects email addresses with usernames and can 
allow several email addresses to go to a single username. If Sendmail 
receives any emails for addresses not in this file they will bounce. 

The Sendmail configuration file can be composed in two formats - the 
complex old CF format or the easier new MC format. An MC file is compiled 
down into the old CF format using the m4 macro compiler. The resultant CF 
file is then placed where Sendmail will look for its configuration file. 
For my simple setup I created a file called millstream.com.mc from 
openbsd-proto.mc in the cf directory and added DOMAIN('millstream.com'). 
I then copied generic.m4 to millstream.com.m4 in the domain directory 
and compiled millstream.com.mc using m4 and moved the resultant cf file
to the right place. 

Another file called virtusertable can be used for cleverer email address 
redirection such as 

@millstream.com  other 

which will send emails addressed to anything@millstream.com to 'other'. 
But don't do 

@millstream.com  r.kelsall@millstream.com 

because it will get into an endless recursive loop.

The incoming emails are stored on the server until my MUA requests them. Just as with a normal ISP I download my emails using a protocol called POP3. The program Popa3d runs on the server to provide POP3 access to the stored emails. For this to work a username is created on the server with no login privileges and Popa3d checks the username-password given against this before allowing the emails to be downloaded. This username is the one you directed your emails to using the aliases file in Sendmail.

Precision Timing

Computer clocks drift away from the correct time and can be accidentally changed. My previous server had somehow got about two hours out. I have synchronised the time on the new server (and from it the date and time on my Windows PCs) with the atomic clocks on the internet. It may seem unimportant to have sub-second accuracy for all your computer clocks but if you ever want to unravel a sequence of events that occurred across several computers it makes life much easier. It's also nice to be certain exactly when you did send or receive that email and some security mechanisms require accurate clocks. I am using the ntp package (ntp-4.1.1c) which isn't on the OpenBSD 3.5 CD but is easy to download and install from the OpenBSD site. This runs continuously checking the server clock against accurate clocks on the internet and nudges the server clock gradually towards the correct time. Set the server clock precisely before running ntp or could take a day or two to converge.

Web Server

I am using the free web server software Apache. As with all software it is more secure and therefore less work if you only enable the functions you need. Experimenting and reading about the modules in Apache I got mine down to just five modules : dir, mime, log-config, setenvif and expires. I recompiled it with these modules and switched off the version display in the HTTP headers using 'ServerTokens ProductOnly'.

Taming my Daemons

To reduce the possibility of my server being misused by someone on the internet I have 'sand-boxed' most of my daemons such as Apache and Sendmail by using Systrace. Systrace sits between the program and the rest of the computer vetting all access to the machine. Given a policy (set of rules) defining what the program is allowed to do it can prevent any other access to the machine and log attempts to misbehave. Systrace can almost write your policy for you. It can do this because it can watch the program you want to sand-box during normal operation and so learn what the program should be allowed to do.

To create your policy you need to run Systrace in 'learn' mode to make it write a policy that lists exactly which system calls the program uses. Exercise the program with Systrace watching in learn mode and do all the things you will need to do with the program otherwise the learned policy may be too restrictive. Next switch off Systrace and edit the policy it has learned to make it more general purpose. For example it will learn specific IP addresses and temporary filenames when you want it to work with any IP address and any file in the temporary directory. (The policy it learns is the most restrictive possible which is not what you will normally want.) This is easily done by replacing any part of a string that is likely to change with an asterisk and altering the corresponding condition from 'eq' to 'match'. Next switch Systrace back into learn mode and run the program again to see whether it adds any new rules to the policy. Study and generalise these new rules as before. When learn mode ceases to add new rules the policy is ready for switching into 'enforce' mode where Systrace will disallow anything not in the policy rather than adding new rules. Any attempts now by the program to make system calls not in the policy will be logged in /var/log/messages and Systrace will return an error to the program rather than executing the system call. So in enforce mode you will need to look at the messages occasionally and decide whether any new ones are legitimate or aberrant behaviour.

N.B. Changing the /etc/rc and /etc/rc.conf files can leave your 
machine in a serious mess so make sure you take a copy of the 
originals. For upgrade reasons it's better to make changes to 
rc.conf.local rather than rc.conf as I did.

To /etc/rc.conf I added these lines

systrace_flags="-A -U"          # Learn = "-A -U"
                                # Enforce = "-a -U"
                                # Off = NO

and to /etc/rc I added these lines below the reference to /etc/rc.conf

# set systrace options for daemons
daemon_systrace=""
if [ X"${systrace_flags}" != X"NO" ]; then
        daemon_systrace="systrace ${systrace_flags}"
        echo 'daemons systraced'
else
        echo 'daemons not systraced'
fi

and prefixed the daemons I wanted to Systrace like this

${daemon_systrace} syslogd ${syslogd_flags}

The policy files are all created in /etc/systrace. For some reason 
this didn't seem to work for pflogd and named so I didn't Systrace 
these daemons. (pflogd didn't create the expected sbin_pflogd file 
in learn mode and named appeared to find extra rules when switched 
to enforce mode. I didn't have time to investigate this properly.) 
Using this method a reboot is obviously necessary to switch Systrace 
modes.