About a year ago, I started playing around with the PacketProtector (www.packetprotector.org/) load on a Linksys WRTSL54GS. I had intended it to be a replacement for my existing firewall, but it never got past acceptance testing as it crashed out after a while. After a lull and a VLAN misconfiguration that locked me out from access to the LAN ports, I decided to look into the error.
To recover the unit, I needed to see what at what point it was failing and try to regain control of it. Since PacketProtector is derived from OpenWRT, I perused their forums for answers. I purchased a USB TTL controller (http://www.amazon.com/gp/product/B006JKNWLE/ref=oh_details_o01_s00_i00), as recommended by the OpenWRT recovery article (http://wiki.openwrt.org/doc/hardware/port.serial) and the hardware specific article (http://wiki.openwrt.org/toh/linksys/wrtsl54gs?s[]=wrtsl54gs). When allowed to run for a while, this error shows up on the console, ad nauseum:
__alloc_pages: 0-order allocation failed (gfp=0x20/0)
It seemed to be indicative of either a USB OHCI error, or a memory error from Google searches. The following article had the most relevant data (https://packetprotector.org/forum/viewtopic.php?id=4362). Other Openwrt forum posts indicate that usb ohci failures are possible. However, most of the posts pointed to a memory leak that consumes everything. To test this, I set the unit to run for a while with periodic process and 'free' assessments to watch how the memory flows out. If it was leaking, I wanted to determine which service it is. The analysis script follows:
# cat /packetprotector/memanal.sh
#!/bin/sh
while [[ 1 == 1 ]]; do
echo;
echo "####################";
ps | sort | uniq;
date;
free;
sleep 60;
done
When run from the TTL console at /dev/ttyUSB0 under minicom with console capture engaged, it provided a usable log from which to determine how long expected runtime was, as well as what the process is hogging up the memory. Here was the last log before failure:
Sat Jan 1 00:54:36 UTC 2000
total used free shared buffers
Mem: 30512 28284 2228 0 316
Swap: 65524 38804 26720
Total: 96036 67088 28948
####################
1 root 136 S init
2 root SW [keventd]
3 root RWN [ksoftirqd_CPU0]
4 root DW [kswapd]
5 root SW [bdflush]
6 root SW [kupdated]
9 root SW [mtdblockd]
54 root SWN [jffs2_gcd_mtd4]
67 root 136 S logger -s -p 6 -t
69 root 152 S /bin/ash --login
78 root 124 S /sbin/syslogd -C16 -m 0
81 root 112 S /sbin/klogd
118 root 92 S /sbin/hotplug2 --override --persistent --max-children
222 root 168 S udhcpc -t 0 -i eth1 -b -p /var/run/eth1.pid -R
255 root SW [khubd]
376 root SW [usb-storage-0]
377 root SW [scsi_eh_0]
641 root 244 S crond -c /etc/crontabs
646 root 96 S /usr/sbin/dropbear -p 22
663 nobody 108 S /usr/sbin/dnsmasq --dhcp-range=lan,192.168.1.100,192.
713 root 100 S mini_httpd -C /etc/mini_httpd/mini_httpd.conf
731 root 160 S /packetprotector/usr/sbin/snort-inline -Q -D -c /pack
786 root 208 S /packetprotector/usr/sbin/openvpn /packetprotector/et
799 nobody 124 S /packetprotector/usr/sbin/tinyproxy -c /packetprotect
800 nobody 100 S /packetprotector/usr/sbin/tinyproxy -c /packetprotect
801 nobody 100 S /packetprotector/usr/sbin/tinyproxy -c /packetprotect
802 nobody 100 S /packetprotector/usr/sbin/tinyproxy -c /packetprotect
803 nobody 100 S /packetprotector/usr/sbin/tinyproxy -c /packetprotect
804 nobody 100 S /packetprotector/usr/sbin/tinyproxy -c /packetprotect
805 nobody 100 S /packetprotector/usr/sbin/tinyproxy -c /packetprotect
806 nobody 100 S /packetprotector/usr/sbin/tinyproxy -c /packetprotect
807 nobody 100 S /packetprotector/usr/sbin/tinyproxy -c /packetprotect
808 root 120 S /packetprotector/usr/sbin/dansguardian -c /packetprot
809 root 116 S /packetprotector/usr/sbin/dansguardian -c /packetprot
810 root 116 S /packetprotector/usr/sbin/dansguardian -c /packetprot
811 root 116 S /packetprotector/usr/sbin/dansguardian -c /packetprot
812 root 120 S /packetprotector/usr/sbin/dansguardian -c /packetprot
813 root 120 S /packetprotector/usr/sbin/dansguardian -c /packetprot
814 root 120 S /packetprotector/usr/sbin/dansguardian -c /packetprot
816 root 120 S /packetprotector/usr/sbin/dansguardian -c /packetprot
817 root 120 S /packetprotector/usr/sbin/dansguardian -c /packetprot
818 root 120 S /packetprotector/usr/sbin/dansguardian -c /packetprot
819 root 120 S /packetprotector/usr/sbin/dansguardian -c /packetprot
836 root 120 S /bin/sh /packetprotector/usr/sbin/snort_check
839 root 120 S /bin/sh /packetprotector/usr/sbin/snort_check
840 root 248 R top -n1
841 root 112 S grep snort
842 root 112 S grep 731
843 root 136 S /bin/sh /packetprotector/usr/sbin/snort_check
844 root 124 S /bin/sh /packetprotector/usr/sbin/snort_check
848 root 120 S /bin/sh /packetprotector/usr/sbin/snort_check
851 root 120 S /bin/sh /packetprotector/usr/sbin/snort_check
852 root 260 R top -n1
853 root 120 S grep snort
854 root 112 S grep 731
855 root 136 S /bin/sh /packetprotector/usr/sbin/snort_check
856 root 124 S /bin/sh /packetprotector/usr/sbin/snort_check
860 root 120 S /bin/sh /packetprotector/usr/sbin/snort_check
863 root 120 S /bin/sh /packetprotector/usr/sbin/snort_check
[...] x 1798 times
1852 root 196 D top -n1
1853 root 92 S grep snort
1854 root 80 S grep 731
1855 root 132 S /bin/sh /packetprotector/usr/sbin/snort_check
1856 root 120 S /bin/sh /packetprotector/usr/sbin/snort_check
1861 root 116 S /bin/sh /packetprotector/usr/sbin/snort_check
1863 root 116 S /bin/sh /packetprotector/usr/sbin/snort_check
1864 root 164 D top -n1
1865 root 104 S grep snort
1866 root 80 S grep 731
1867 root 132 S /bin/sh /packetprotector/usr/sbin/snort_check
1868 root 120 S /bin/sh /packetprotector/usr/sbin/snort_check
1870 root 104 S /bin/sh -c /packetprotector/usr/sbin/proxylogrotate.s
1873 root 116 S /bin/sh /packetprotector/usr/sbin/snort_check
1875 root 124 D /bin/sh /packetprotector/usr/sbin/proxylogrotate.sh
1877 root 104 S /bin/sh -c /packetprotector/usr/sbin/watchdog > /dev/
1878 root 112 D /bin/sh /packetprotector/usr/sbin/watchdog
1879 root /bin/sh /packetprotector/usr/sbin/watchdog
1880 root 116 D /bin/sh /packetprotector/usr/sbin/snort_check
The failure happened at 55 minutes after boot. This was interesting, as the swap was being consumed by prior runs at about one megabyte per minute. Something caused it to die before it hit the end of swap, possibly due to overallocation of processing resources. However, the memory leak could clearly be traced back to the many 'snort_check' scripts that are not dying as expected. A copy of that script follows:
$ cat /packetprotector/usr/sbin/snort_check
#!/bin/sh
#
# Changed snort_check 12/22/09 - CMH714
#
#
# Set High that snort shouldnt go over
# Default should be something very high like 140
# The memory snort eats is relative to the rules enabled.
HI_PCT="130"
echo "hi pct - " $HI_PCT
pid=`pidof snort-inline`
echo "pid - " "$pid"
if [ -n "$pid" ] ; then
SNORTPCT=`top -n1 | grep snort | grep $pid | cut -d% -f1 | awk '{print $NF}'`
fi
echo "snortpct - " "$SNORTPCT"
SNORT_INLINE=`grep "snort-inline=" /etc/packetprotector.conf | cut -d "=" -f 2`
if [ -n "$SNORTPCT" ] && [ -n "$pid" ] ; then
echo "Snorts running - Checking its Memory Usage"
if [ "$SNORTPCT" -gt "$HI_PCT" ] && [ -n "$pid" ] ; then
EPOCHTIME=`date +%s`
LOGDIR="/packetprotector/logs"
mv $LOGDIR/snort-inline/alert $LOGDIR/snort-inline/alert.$EPOCHTIME && touch $LOGDIR/snort-inline/alert
# Create a log file, assumes directory exists already
echo "snort needs to restart - killing snort" > /packetprotector/logs/snort-restart.log
echo `date` >> /packetprotector/logs/snort-restart.log
# killall snort-inline 2> /dev/null
killall -1 snort-inline 2> /dev/null
# restart snort - dont need to do this as watchdog will get it next time around
# /packetprotector/etc/init.d/S11snort-inline
fi
elif ([ -n "$SNORTPCT" ] && [ -z "$pid" ]) || ([ -z "$SNORTPCT" ] && [ -n "$pid" ]) ; then
echo "Snort may be going up or down. Will check again next minute..."
elif [ -z "$SNORTPCT" ] && [ -z "$pid" ] && [ "X$SNORT_INLINE" = "X1" ] ; then
echo "restarting snort"
echo "restarting snort" >> /packetprotector/logs/snort-restart.log
echo `date` >> /packetprotector/logs/snort-restart.log
/packetprotector/etc/init.d/S11snort-inline
else
return
fi
After review, it became clear that the script wouldn't reach conclusion and close on its own. Consider the following line:
SNORTPCT=`top -n1 | grep snort | grep $pid | cut -d% -f1 | awk '{print $NF}'`
The TOP command was called without the '-b' option to put it into batch mode. This kept the TOP process in memory, constantly updating and consuming memory as a running process, keeping the script and its following pipes alive until memory is exhausted.
Patching the line to add the batch flag seemed to have slowed the stacking up of the scripts, but did not eliminate the problem. The modification merely dragged the failure out for a few hours more:
# diff snort_check snort_check.mod
16c16
< SNORTPCT=`top -n1 | grep snort | grep $pid | cut -d% -f1 | awk '{print $NF}'`
---
> SNORTPCT=`top -b -n1 | grep snort | grep $pid | cut -d% -f1 | awk '{print $NF}'`
Further investigation running the command manually revealed that TOP did not exit as it should from the count mode (-n 1), holding and updating the process as it ran forward. The following memory status at failure mirrored the one before.
Sat Jan 1 01:03:37 UTC 2000
total used free shared buffers
Mem: 30512 29868 644 0 288
Swap: 65524 33356 32168
Total: 96036 63224 32812
top proc: 49
snort procs: 252
TOP not exiting caused over-consumption and, ultimately, a process contention problem. TOP was called within the /packetprotector/usr/sbin/snort_check script, which itself was executed every minute from the /packetprotector/usr/sbin/watchdog script as scheduled by CRON. To alleviate the problem immediately, the watchdog was modified to remove the snort_check script, as below.
# cat /packetprotector/usr/sbin/watchdog
#!/bin/sh
/packetprotector/usr/sbin/nas_check &
#/packetprotector/usr/sbin/snort_check &
After seven hours of running with no traffic, the memory utilization stabilized and held constant at 32MB. This indicated a need to modify the snort_check script to improve the memory monitoring process. Since TOP was not performing as documented per the options, another method was needed to get the memory information. /proc/[pid]/status has the VmRSS information that is reported to TOP, so this was used to draw the information directly for the script.
/proc/[pid]/status - look for VmRSS for the in memory value
/proc/[pid]/cmdline - look for the command line executed for a grep
The following modifications were made:
# diff snort_check snort_check.mod
10c10,12
< HI_PCT="130"
---
> # 20120805 - Changing PCT to an absolute value for a limit. Adjust as needed. Removes the need to use TOP.
> #HI_PCT="130"
> HI_PCT="20000"
16c18
< SNORTPCT=`top -n1 | grep snort | grep $pid | cut -d% -f1 | awk '{print $NF}'`
---
> SNORTPCT=`cat /proc/$pid/status | grep VmRSS | awk '{print $2}'`
/packetprotector/usr/sbin/watchdog was modified to call the new script.
# cat usr/sbin/watchdog
#!/bin/sh
/packetprotector/usr/sbin/nas_check &
#/packetprotector/usr/sbin/snort_check &
/packetprotector/usr/sbin/snort_check.mod &
The new script was run under monitoring for a while to ensure the new watchdog process was working. Consumed memory was now unaffected and the script is confirmed to be working properly. The change was made permanent to my unit and has eliminated crashing for more than a week.
Saturday, August 25, 2012
Saturday, January 7, 2012
Analysis of a Freshly Installed Windows 7's Teredo Tunneling Traffic
While reviewing packet drops, I noticed a system that looked like a low, slow series of repeated failures.
Digging into it, I identified the system as our newly installed Windows 7 host. This seemed a bit odd. Most of the dropped packets were to a single destination, 65.55.158.118.
Searching through the DNS logs, I found that the IP was only involved in a request for teredo.ipv6.microsoft.com. The picture below shows the result of the query, however the googleusercontent.com return is an error in the way DNSMASQ mixes the logs for simultaneous return records. So, there is only one domain to IP relationship between teredo.ipv6.microsoft.com and 65.55.158.118.
To see the activity of just the traffic to this host, I decided to graph the activity to IP address 65.55.158.118 across all of the datasets. This would show the interrelationship of DNS, firewall, and proxy resources over the last seven days. I used this query in my Splunk search:
The columns only indicated DNS and firewall would be factors since no proxy logs matched. I clicked 'Show Report' at the upper-right and selected a line graph with a logrithmic aspect. Here are my results:
So, my new Windows 7 host commits nearly constant DNS resolutions to teredo.ipv6.microsoft.com, resolves to 65.55.158.118, then sends roughly nine times as much direct traffic to that IP to die at the firewall as it is apparently not proxy aware.
My understanding has been that Microsoft disabled Teredo Tunneling by default after Windows XP Service Pack 1 due to industry concerns about data control. This is confirmed by a TechNet article released in 2007. A separate post found at the Microsoft Answers site indicates for Windows 7, "Teredo is installed by default and enabled so that it remains in a standby mode and comes into action only when required." Apparently not since it makes queries ALL THE TIME.
Teredo-by-default remains a potential security and data-loss issue for companies, many of whom lack tools to analyze the 6to4 wrapped protocol for network intrusion detection, enforcing proxy use for regulatory compliance, or other purposes. Some analysis of the protocol has been undertaken, as indicated by this Internet Storm Center article about a year ago, and followed by an execellent analysis by Johannes Ullrich in May. However, this level of depth in monitoring this protocol is not packaged or attainable to most businesses today.
To address the issue, administrators not utilizing Teredo for their domain should disable it via Global Policy. For home users of Windows 7 and Vista, this article at Windows Seven Forums provides instructions to disable the service.
In the Windows menu search bar, type 'cmd.exe', but don't open it. Right-click cmd.exe and select 'Run as Administrator'. Click Yes if prompted to confirm running the program with elevated privileges. Now follow the commands indicated in the output below:
Reboot the system to have the setting take effect.
Digging into it, I identified the system as our newly installed Windows 7 host. This seemed a bit odd. Most of the dropped packets were to a single destination, 65.55.158.118.
Searching through the DNS logs, I found that the IP was only involved in a request for teredo.ipv6.microsoft.com. The picture below shows the result of the query, however the googleusercontent.com return is an error in the way DNSMASQ mixes the logs for simultaneous return records. So, there is only one domain to IP relationship between teredo.ipv6.microsoft.com and 65.55.158.118.
To see the activity of just the traffic to this host, I decided to graph the activity to IP address 65.55.158.118 across all of the datasets. This would show the interrelationship of DNS, firewall, and proxy resources over the last seven days. I used this query in my Splunk search:
65.55.158.118 earliest=-7d | fields sourcetype | timechart limit=0 span=1h count(sourcetype) by sourcetype
The columns only indicated DNS and firewall would be factors since no proxy logs matched. I clicked 'Show Report' at the upper-right and selected a line graph with a logrithmic aspect. Here are my results:
So, my new Windows 7 host commits nearly constant DNS resolutions to teredo.ipv6.microsoft.com, resolves to 65.55.158.118, then sends roughly nine times as much direct traffic to that IP to die at the firewall as it is apparently not proxy aware.
My understanding has been that Microsoft disabled Teredo Tunneling by default after Windows XP Service Pack 1 due to industry concerns about data control. This is confirmed by a TechNet article released in 2007. A separate post found at the Microsoft Answers site indicates for Windows 7, "Teredo is installed by default and enabled so that it remains in a standby mode and comes into action only when required." Apparently not since it makes queries ALL THE TIME.
Teredo-by-default remains a potential security and data-loss issue for companies, many of whom lack tools to analyze the 6to4 wrapped protocol for network intrusion detection, enforcing proxy use for regulatory compliance, or other purposes. Some analysis of the protocol has been undertaken, as indicated by this Internet Storm Center article about a year ago, and followed by an execellent analysis by Johannes Ullrich in May. However, this level of depth in monitoring this protocol is not packaged or attainable to most businesses today.
To address the issue, administrators not utilizing Teredo for their domain should disable it via Global Policy. For home users of Windows 7 and Vista, this article at Windows Seven Forums provides instructions to disable the service.
In the Windows menu search bar, type 'cmd.exe', but don't open it. Right-click cmd.exe and select 'Run as Administrator'. Click Yes if prompted to confirm running the program with elevated privileges. Now follow the commands indicated in the output below:
C:\Windows\system32>netsh
netsh>interface
netsh interface>teredo
netsh interface teredo>set state disabled
Ok.
netsh interface teredo>exit
C:\Windows\system32>
Reboot the system to have the setting take effect.
Saturday, December 10, 2011
Building a Debian/Ubuntu Package for DNSCrypt-proxy
Referring to X4's issue for Linux compiling, the following are instructions for builiding a debian package.
Create a working directory. Then acquire the source by cloning the repository. Create a source tarball in the Debian package naming format.
$ cd ~
$ mkdir dnscryptworkdir
$ cd dnscryptworkdir
$ git clone https://github.com/opendns/dnscrypt-proxy
$ mv dnscrypt-proxy dnscrypt-proxy-0.1
$ tar cvfz dnscrypt-proxy_0.1.orig.tar.gz dnscrypt-proxy-0.1
$ cd dnscrypt-proxy-0.1
Now it's time to lay the groundwork for the package. Use DebHelper to generate the package framework. If you haven't built a package before, install the necessary packages.
$ sudo apt-get install build-essential devscripts ubuntu-dev-tools debhelper dh-make diff patch cdbs quilt gnupg \
fakeroot lintian pbuilder piuparts
Use DebHelper to lay the package framework. This will generate the debian/ directory and associated files. Go ahead and remove the defaults. The README.Debian file is also unnecessary
$ dh_make -f ../dnscrypt-proxy_0.1.orig.tar.gz -s -b
$ cd debian
$ rm *.ex *.EX
$ rm README.Debian
Edit the control file with your favorite editor. It should look like this
Source: dnscrypt-proxy
Section: misc
Priority: optional
Maintainer: pinowudi
Build-Depends: cdbs, debhelper (>= 7), automake
Standards-Version: 3.8.3
Homepage: https://github.com/opendns/dnscrypt-proxy
Package: dnscrypt-proxy
Architecture: all
Depends: ${shlibs:Depends}, ${misc:Depends}
Description: A tool for securing communications between a client and a DNS resolver.
DNSCrypt is a slight variation on DNSCurve.
.
DNSCurve improves the confidentiality and integrity of DNS requests using high-speed high-security elliptic-curve cryptography. Best of all, DNSCurve has very low overhead and adds virtually no latency to queries.
.
DNSCurve aims at securing the entire chain down to authoritative servers. However, it only works with authoritative servers that explicitly support the protocol. And unfortunately, DNSCurve hasn't received much adoption yet.
.
The DNSCrypt protocol is very similar to DNSCurve, but focuses on securing communications between a client and its first-level resolver. While not providing end-to-end security, it protects the local network (which is often the weakest link in the chain) against man-in-the-middle attacks. It also provides some confidentiality to DNS queries.
.
The DNSCrypt daemon acts as a DNS proxy between a regular client, like a DNS cache or an operating system stub resolver, and a DNSCrypt-aware resolver, like OpenDNS.
Now edit the rules file. Using the CDBS packager makes implementing the autoconf features easy, with one exception. Since the autoconf has not been run already, it must be done before the package build. This will require the extra 'autoreconf' as a preconfiguration action. The rules file should look akin to the following.
#!/usr/bin/make -f
include /usr/share/cdbs/1/class/autotools.mk
include /usr/share/cdbs/1/rules/debhelper.mk
include /usr/share/cdbs/1/class/makefile.mk
# Add here any variable or target overrides you need.
makebuilddir/dnscrypt-proxy-0.1::
autoreconf --install
Update the changelog file using the dce tool. Mark the issue as change number zero.
$ dch -e
dnscrypt-proxy (0.1-1) unstable; urgency=low
* Initial release (Closes: #0)
-- pinowudi Sat, 10 Dec 2011 14:11:02 -0500
Edit the copyright file with the copyright information from the package.
This work was packaged for Debian by:
pinowudi on Sat, 10 Dec 2011 14:01:36 -0500
It was downloaded from
Upstream Author(s):
Frank Denis
Copyright:
Copyright (c) 2011 OpenDNS, Inc.
License:
Permission to use, copy, modify, and distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
====
This license applies to all parts of dnscrypt-proxy that are not externally
maintained libraries.
The externally maintained libraries used by dnscrypt-proxy are:
- NaCl (http://nacl.cr.yp.to/). Public domain.
- libuv (https://github.com/joyent/libuv). MIT license.
+ libuv dependencies, see src/libuv/LICENSE.
- alt_arc4random.c reuses code from OpenBSD. BSD license,
see the alt_arc4random.c header.
The Debian packaging is:
Copyright (C) 2011 pinowudi
Permission to use, copy, modify, and distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
The software package does not have a .config file yet, so making one modification is necessary to get the CDBS manager to handle the package. Run autoreconfig to generate the autoconf files. Then execute the package build. The package should be produced in the directory below the current build dir.
$ autoreconf --install
$ debuild -us -uc
$ ls ..
dnscrypt-proxy-0.1 dnscrypt-proxy_0.1-1_all.deb dnscrypt-proxy_0.1-1.diff.gz dnscrypt-proxy_0.1-1.dsc dnscrypt-proxy_0.1-1_i386.build dnscrypt-proxy_0.1-1_i386.changes dnscrypt-proxy_0.1.orig.tar.gz
Now test the package installation.
$ cd ..
$ sudo dpkg -i dnscrypt-proxy_0.1-1_all.deb
[sudo] password :
Selecting previously deselected package dnscrypt-proxy.
(Reading database ... 122394 files and directories currently installed.)
Unpacking dnscrypt-proxy (from dnscrypt-proxy_0.1-1_all.deb) ...
Setting up dnscrypt-proxy (0.1-1) ...
Processing triggers for man-db ...
$ whereis dnscrypt-proxy
dnscrypt-proxy: /usr/sbin/dnscrypt-proxy /usr/share/man/man8/dnscrypt-proxy.8.gz
$ dnscrypt-proxy -h
dnscrypt-proxy 0.8
Copyright (C) 2011 OpenDNS, Inc.
Options:
-a --local-address=...
-d --daemonize
-e --edns-payload-size=...
-h --help
-k --provider-key=...
-l --logfile=...
-n --max-active-requests=...
-p --pidfile=...
-r --resolver-address=...
-t --tcp-port=...
-u --user=...
-N --provider-name=...
-P --local-port=...
-V --version
Please consult the dnscrypt-proxy(8) man page for details.
Good to go!
Create a working directory. Then acquire the source by cloning the repository. Create a source tarball in the Debian package naming format.
$ cd ~
$ mkdir dnscryptworkdir
$ cd dnscryptworkdir
$ git clone https://github.com/opendns/dnscrypt-proxy
$ mv dnscrypt-proxy dnscrypt-proxy-0.1
$ tar cvfz dnscrypt-proxy_0.1.orig.tar.gz dnscrypt-proxy-0.1
$ cd dnscrypt-proxy-0.1
Now it's time to lay the groundwork for the package. Use DebHelper to generate the package framework. If you haven't built a package before, install the necessary packages.
$ sudo apt-get install build-essential devscripts ubuntu-dev-tools debhelper dh-make diff patch cdbs quilt gnupg \
fakeroot lintian pbuilder piuparts
Use DebHelper to lay the package framework. This will generate the debian/ directory and associated files. Go ahead and remove the defaults. The README.Debian file is also unnecessary
$ dh_make -f ../dnscrypt-proxy_0.1.orig.tar.gz -s -b
$ cd debian
$ rm *.ex *.EX
$ rm README.Debian
Edit the control file with your favorite editor. It should look like this
Source: dnscrypt-proxy
Section: misc
Priority: optional
Maintainer: pinowudi
Build-Depends: cdbs, debhelper (>= 7), automake
Standards-Version: 3.8.3
Homepage: https://github.com/opendns/dnscrypt-proxy
Package: dnscrypt-proxy
Architecture: all
Depends: ${shlibs:Depends}, ${misc:Depends}
Description: A tool for securing communications between a client and a DNS resolver.
DNSCrypt is a slight variation on DNSCurve.
.
DNSCurve improves the confidentiality and integrity of DNS requests using high-speed high-security elliptic-curve cryptography. Best of all, DNSCurve has very low overhead and adds virtually no latency to queries.
.
DNSCurve aims at securing the entire chain down to authoritative servers. However, it only works with authoritative servers that explicitly support the protocol. And unfortunately, DNSCurve hasn't received much adoption yet.
.
The DNSCrypt protocol is very similar to DNSCurve, but focuses on securing communications between a client and its first-level resolver. While not providing end-to-end security, it protects the local network (which is often the weakest link in the chain) against man-in-the-middle attacks. It also provides some confidentiality to DNS queries.
.
The DNSCrypt daemon acts as a DNS proxy between a regular client, like a DNS cache or an operating system stub resolver, and a DNSCrypt-aware resolver, like OpenDNS.
Now edit the rules file. Using the CDBS packager makes implementing the autoconf features easy, with one exception. Since the autoconf has not been run already, it must be done before the package build. This will require the extra 'autoreconf' as a preconfiguration action. The rules file should look akin to the following.
#!/usr/bin/make -f
include /usr/share/cdbs/1/class/autotools.mk
include /usr/share/cdbs/1/rules/debhelper.mk
include /usr/share/cdbs/1/class/makefile.mk
# Add here any variable or target overrides you need.
makebuilddir/dnscrypt-proxy-0.1::
autoreconf --install
Update the changelog file using the dce tool. Mark the issue as change number zero.
$ dch -e
dnscrypt-proxy (0.1-1) unstable; urgency=low
* Initial release (Closes: #0)
-- pinowudi
Edit the copyright file with the copyright information from the package.
This work was packaged for Debian by:
pinowudi
It was downloaded from
Upstream Author(s):
Frank Denis
Copyright:
Copyright (c) 2011 OpenDNS, Inc.
License:
Permission to use, copy, modify, and distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
====
This license applies to all parts of dnscrypt-proxy that are not externally
maintained libraries.
The externally maintained libraries used by dnscrypt-proxy are:
- NaCl (http://nacl.cr.yp.to/). Public domain.
- libuv (https://github.com/joyent/libuv). MIT license.
+ libuv dependencies, see src/libuv/LICENSE.
- alt_arc4random.c reuses code from OpenBSD. BSD license,
see the alt_arc4random.c header.
The Debian packaging is:
Copyright (C) 2011 pinowudi
Permission to use, copy, modify, and distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
The software package does not have a .config file yet, so making one modification is necessary to get the CDBS manager to handle the package. Run autoreconfig to generate the autoconf files. Then execute the package build. The package should be produced in the directory below the current build dir.
$ autoreconf --install
$ debuild -us -uc
$ ls ..
dnscrypt-proxy-0.1 dnscrypt-proxy_0.1-1_all.deb dnscrypt-proxy_0.1-1.diff.gz dnscrypt-proxy_0.1-1.dsc dnscrypt-proxy_0.1-1_i386.build dnscrypt-proxy_0.1-1_i386.changes dnscrypt-proxy_0.1.orig.tar.gz
Now test the package installation.
$ cd ..
$ sudo dpkg -i dnscrypt-proxy_0.1-1_all.deb
[sudo] password :
Selecting previously deselected package dnscrypt-proxy.
(Reading database ... 122394 files and directories currently installed.)
Unpacking dnscrypt-proxy (from dnscrypt-proxy_0.1-1_all.deb) ...
Setting up dnscrypt-proxy (0.1-1) ...
Processing triggers for man-db ...
$ whereis dnscrypt-proxy
dnscrypt-proxy: /usr/sbin/dnscrypt-proxy /usr/share/man/man8/dnscrypt-proxy.8.gz
$ dnscrypt-proxy -h
dnscrypt-proxy 0.8
Copyright (C) 2011 OpenDNS, Inc.
Options:
-a --local-address=...
-d --daemonize
-e --edns-payload-size=...
-h --help
-k --provider-key=...
-l --logfile=...
-n --max-active-requests=...
-p --pidfile=...
-r --resolver-address=...
-t --tcp-port=...
-u --user=...
-N --provider-name=...
-P --local-port=...
-V --version
Please consult the dnscrypt-proxy(8) man page for details.
Good to go!
Thursday, September 8, 2011
Bigger Sets to Visualize
After yesterdays post, I revisited the data and came up with a better static query. It now just looks for 40 byte packets fom port 12200. The supporting query looks like this:
The flash clearly indicates a ramp-up of scanning activity last month, but just how much? Open the Report Builder and re-render the data as a stacked area graph. Again, treat nullvalues as zero for this purpose.
Thats a pretty dramatic take-off from a wide selection of IP addresses. None of them have reverse DNS records, and all have with infrastructure-style WHOIS records. A brief review indicates several clusters of IP addresses at large virtual hostsers with a few smaller hosters and individual servers. Perhaps a compromised management console (remember CPanel?) or a clutch of vulnerable LAMP stacks?
Visualizing and Investigating a Distributed Scan
I was checking my logs and trying out a new app I created for Splunk. Normally my graph looks like the one below, with the exception of one host that I recently added that was wildly dropping packets on the left. At the bottom-right, there are typically some bursts, but for the most part the averages are low. The scatterplots are fairly tight and low volume. For this packet length average vs. summation plot, most things cluster at the bottom left (low volume), occasionally blip into one of the upper-left (lots of low, slow) or lower-right (a few large packets) corners, and rarely pop-up in the upper-right (lots of big packets, blasting) quadrant. Here is an example:
The Bird is the Word
Today I check the dashboard and the left side (ingress) looked remarkably different. The averages were pretty normal, but there was a steady baseline at the bottom. Also, the summation of dropped packets was much higher than the 'normal' sampling. The scatterplot also diverged more in the ~40 byte range, with a sample of hosts clearly engaged in a lot of small transactions. This peaqued my interest.
And Now It's Time for Breakdown...
Investigating the increase in ingress summation (the chart at the bottom-right), I clicked the 'View Results' link on that chart. I then modified it slightly to reduce the number of extracted fields. This improved the search speed remarkably. I also eliminated the aggregate summing the 'timechart' command performs by default after the top 10, which returned all of the hosts. The 'limit=0' parameter achieved this. Now I had a chart that I could graph on a line chart and analyze more closely.
I adjusted the default settings slightly to treat null values as zero and added some axis labels. TIP: If you hover the mouse over one of the IP addresses in legend, Splunk highlights the line for that selection. I noticed the 58.218.199.0/24 IP addresses all have that matching bumpity-bump along the bottom, low and slow. This is pretty good evidence of an undiversified distributed scan. I decided to generate some better metrics.
Which of these is Not Like the Others?
Looking at the rejected packet meta, I wanted to determine just how alike the packets were that were dropping from these hosts. I limited the search to just that subnet with a straight text search. Again, I limited the selected fields to improve the extraction speed. Then I created a timechart to bin by day the average packet length of each server involved. Once the search got underway, I clicked 'Show Report'.
In the Report Builder, I selected an area chart, treating nulls as zero, and stacked the results. As you see in this screenshot, the report showed the distribution of the packet sizes between the servers was even. The scanning cluster also ramped up just after a preliminary poke from a single server, 58.218.199.49.
Peek-a-boo, I See You
Having identified the scanning cluster, I wanted to see what ports the aggressor was looking to find. During my investigation, I came across this comment at the Internet Storm Center website, followed by many other confirmatory posts at other sites, regarding the nature of the attacker. It appeared they have been active for a number of years.
After identifying the affected ports, I prettied it up a bit with a lookup table to provide the common uses of those ports. It became plain that the scanner was looking for open proxies, whether legitimate, ill-configured, left by malware, or just plain hacked. There were many supporting statements for all of these in the resources culled to identify these ports.
Playing Favorites?
Now that I had an inkling what they were up to, I was interested in seeing if they favor certain ports. The data showed that the scanners were coordinated to test the target port set just about evenly.
Summing It Up
It seems the aggressors utilizing these Chinese servers really like to scan for open proxies. Intent for such a collection, who can say? It could be folks looking for a way to circumvent Chinese proxy filters. However, judging from the technical coordination and longevity of this scanning behavior, it would seem more coordinated than that. In the least, better funded, and possibly sanctioned. Regardless of speculation, it is without doubt that the aggressors seek open proxy servers without permission, even trolling for botnet proxy remnants, and at the expense of those they locate when they turn on the tubes.
 |
| Normal Dashboard |
The Bird is the Word
Today I check the dashboard and the left side (ingress) looked remarkably different. The averages were pretty normal, but there was a steady baseline at the bottom. Also, the summation of dropped packets was much higher than the 'normal' sampling. The scatterplot also diverged more in the ~40 byte range, with a sample of hosts clearly engaged in a lot of small transactions. This peaqued my interest.
 |
| Abby Normal |
And Now It's Time for Breakdown...
Investigating the increase in ingress summation (the chart at the bottom-right), I clicked the 'View Results' link on that chart. I then modified it slightly to reduce the number of extracted fields. This improved the search speed remarkably. I also eliminated the aggregate summing the 'timechart' command performs by default after the top 10, which returned all of the hosts. The 'limit=0' parameter achieved this. Now I had a chart that I could graph on a line chart and analyze more closely.
 |
| Prep for launch... |
I adjusted the default settings slightly to treat null values as zero and added some axis labels. TIP: If you hover the mouse over one of the IP addresses in legend, Splunk highlights the line for that selection. I noticed the 58.218.199.0/24 IP addresses all have that matching bumpity-bump along the bottom, low and slow. This is pretty good evidence of an undiversified distributed scan. I decided to generate some better metrics.
 |
| Launch! Graph of FW drops over a timespan. |
Which of these is Not Like the Others?
Looking at the rejected packet meta, I wanted to determine just how alike the packets were that were dropping from these hosts. I limited the search to just that subnet with a straight text search. Again, I limited the selected fields to improve the extraction speed. Then I created a timechart to bin by day the average packet length of each server involved. Once the search got underway, I clicked 'Show Report'.
In the Report Builder, I selected an area chart, treating nulls as zero, and stacked the results. As you see in this screenshot, the report showed the distribution of the packet sizes between the servers was even. The scanning cluster also ramped up just after a preliminary poke from a single server, 58.218.199.49.
 |
| Poke then plunge. This stacked area graph shows even traffic from the scanning cluster. |
Peek-a-boo, I See You
Having identified the scanning cluster, I wanted to see what ports the aggressor was looking to find. During my investigation, I came across this comment at the Internet Storm Center website, followed by many other confirmatory posts at other sites, regarding the nature of the attacker. It appeared they have been active for a number of years.
After identifying the affected ports, I prettied it up a bit with a lookup table to provide the common uses of those ports. It became plain that the scanner was looking for open proxies, whether legitimate, ill-configured, left by malware, or just plain hacked. There were many supporting statements for all of these in the resources culled to identify these ports.
 |
| Proxy ports, proxy ports... Oh, more proxy ports. No secrets here. |
Playing Favorites?
Now that I had an inkling what they were up to, I was interested in seeing if they favor certain ports. The data showed that the scanners were coordinated to test the target port set just about evenly.
 |
| Even distribution of scanned ports. |
Summing It Up
It seems the aggressors utilizing these Chinese servers really like to scan for open proxies. Intent for such a collection, who can say? It could be folks looking for a way to circumvent Chinese proxy filters. However, judging from the technical coordination and longevity of this scanning behavior, it would seem more coordinated than that. In the least, better funded, and possibly sanctioned. Regardless of speculation, it is without doubt that the aggressors seek open proxy servers without permission, even trolling for botnet proxy remnants, and at the expense of those they locate when they turn on the tubes.
Monday, August 29, 2011
This article is in response to a query about securing a FIOS network in a reasonable way for family and home use.
This design will create three separate zones. The first zone is the WAN, which simply picks up the IP address, routing, and DNS information from the FIOS servers. This implements the simple SPI firewall filters.
The second zone is the Trusted LAN, which will be locked down in what it can pass to the other zones. Why treat the Trusted LAN with less trust? Well, if we want to trust assets on that network, we need them to perform in compliant ways. So, instead of inherently trusting assets on that network, we enforce that that segment will act in an assured manner. This means implementing egress controls, limiting weird protocols, and forcing the use of proxies. Think of it less as 'a zone of trusted hosts' and more like 'a zone of hosts I need to trust'. These hosts are subject to more scrutiny as they or their data are more valuable.
The third zone is the DMZ. The hardened proxy/dns server goes in here, but the rest of that segment is for hosts that can't comply with the mandatory restrictions of the Trusted LAN. These are usually consumer appliance devices like iPods, iPads, tablets, video game consoles, Internet-enabled DVD players and the like. My rubric: consumer appliance devices that need Internet access to be useful but are too dumb to print.
I also recommend placing wireless connections into yet another zone to provide granular access controls. Wireless is typically the weakest infrastructure conduit in a home network, excluding PEBKAC. However, attaching wireless access points into the DMZ and Trusted zones may be suitable depending on your ability to secure them.
Pick list:
Bringing in the WAN
My FIOS installation has the unfortunate distinction of not using the Ethernet for the WAN connection, but bringing in the WAN backhaul over the COAX. This requires the use of a MOCA adapter to bridge the WAN to Ethernet. Since I don't have any spare MOCA units around and the COAX LAN to the video set-top boxes run over that same COAX line, that means using the ActionTec in a double-bridge confirguration, as documented in my previous post "Verizon FIOS Faux Paus".
Firewall Installation
Download and install m0n0wall on the firewall host. Once the steps are completed to create the double-bridge, or if you have Ethernet WAN, run the WAN line to eth1 of the firewall host. The default setup via the console of the FW host should allow for easy implementation of the WAN (eth1) and LAN (eth0) ports in simple no-ingress, all-egress configuration. Apply a common RFC 1918 private class-C subnet to the trusted LAN. e.g. 192.168.1.0/24.
Now set up the third interface as a DMZ (eth2). Set it up in standard SPI configuration: block ingress initiates, allow all outbound traffic. This will allow just about anything to work in this network. It also provides a nice guest network for visitors. Apply a different class-C subnet for the DMZ. e.g. 192.168.2.0/24
Connect the switches to the respective ports. You should definitely be using separate physical switches for each segment. Otherwise, employ VLAN tagging for the segments on a single switch that can support it. Wireless routers with integrated switches can serve this purpose, just use the Ethernet LAN ports, not the WAN port to avoid double NAT'ing. If using a switch with node presence, like most consumer-grade [wireless] routers, be sure to set the router IP to a private subnet not routable by that served by the LAN it is on. For example, on a 192.168.n.n LAN with a gateway of 192.168.n.1, the router management IP should be 10.n.n.n. Management of the switch should be performed by a direct hookup to a host with a static IP set to the same class-C subnet. It is not a perfect solution, but it will help prevent drive-by XSS exploits on weak router configuration pages and casual scans. Neither is it invisible, as described later. Routers are noisy things.
Establish Essential Services
With basic services going, it is time to establish control over essential services. This will be done on a host operating in the DMZ. This will provide it the access it requires to the Internet for services, make it available to both TRUSTED and DMZ hosts that might make use of it, and protect the TRUSTED network from probes should the box become compromised. However, if the host becomes compromised, it will give the attacker control over traffic direction (DNS) and content (proxy) for the TRUSTED net. I chose Ubuntu for the easy package management, though any *nix operating system should be appropriate. This box should be hardened and tightly monitored.
Most of the Internet operates via HTTP(S) after performing a DNS resolution. Therefore, controlling these protocols is essential to establishing control in the network. With the intent of completely locking down egress protocols on the TRUSTED LAN later on, a fully-featured proxy server is necessary. Caching is also a nice feature to reduce the load of things like updates, which most of the hosts on the network will be doing from Internet sources. SQUID provides caching support, access enforcement, and robust logging features.
Optionally, one can layer DansGuardian as a free regex-based enforcement layer on top of SQUID. It provides access controls suited for many common web ailments. Be advised, it will require a bit of tuning at setup to keep from blocking things like system updates. It also does not respond well for some foreign-language matches, which can become a burden. Otherwise, if you are technically inclined and have time to maintain your own custom lists, this is a great tool.
On the proxy host, also install DNSMASQ. This is the same utility that provides forwarding DNS and DHCP services on many wireless routers. We will only be using it for its DNS capabilities. Edit the dnsmasq configuration file to manually set OpenDNS as the DNS forwarders. OpenDNS provides filtered DNS resolution for free, protecting against many of the most obvious violations by botnet herders by default, even without an registered account. If you choosed to register and add your home network, you can gain further content control based on their domain categorizations.
Set the logging of both squid and dnsmasq to the syslog daemon. Modern syslog services, such as rsyslog and syslog-ng, are capable of filtering different program's logs into rotated files. Establish log directories and rotated log files for squid and dnsmasq. Also fork these feeds to the central syslog server (audit server) to be established later.
Locking Down
Now that DNS and Proxy services are established, adjust the configuration of the firewall. Change the default DHCP behavior from issuing the firewall's IP as the DNS passthrough to issuing the IP address of your proxy/dns server in the DMZ. This will encourage hosts to use the DMZ DNS server, which provides logging and domain access control. In the TRUSTED LAN, adjust your hosts' settings to use the proxy server in the DMZ. This is typically done by setting the proxy values to "http://proxyip:3128/", and selecting 'apply to all protocols'. For Windows boxes, this must be done in Internet Explorer to make system updates work, as well as any other installed browsers that do not integrate with IE's proxy settings. For Gnome Linux hosts, this is typically under System -> Preferences -> Network Proxy as the administrative user. When prompted, confirm 'apply systemwide'.
With hosts in the TRUSTED LAN using the proxy, the firewall can now be prepared to lock out all outbound protocols for that subnet. Establish a rule allowing http/s and dns traffic to the proxy server. Create a rule allowing syslog from the dmz proxy server into your trusted audit server. Establish another rule allowing SSH from the trusted server to the dmz proxy server. This will allow you to manage the proxy server to manage it after first authenticating to the trusted server. Prepare for a day where you will be able to monitor the blocked traffic from the firewall on your syslog server. There will be a few things like anti-virus software updates and email server access that may not respond well to the demand for a proxy server. These will have to be manually updated in the firewall as exceptions. There may be a rule explicitly allowing traffic from the LAN to the DMZ. It should be disabled. Now, as the last rule, reject everything sourced from the TRUSTED LAN. Apply and start watching the logs.
Wireless
If you have chosen to employ wireless, ensure that the wireless access points are configured to different ESSIDs and encryption keys. They do not service the same networks.
Review
In this post, we've reviewed locking down the network of hosts we need to trust, while still allowing non-proxy-capable devices an allowance to the Internet. The DMZ allows cart-blanche access to all devices, providing a network suitable for Internet-ready appliances and guests. The proxy services enforce proper accesses on the Trusted hosts and the DNS service logs all of the hosts requests.
Next time: Auditing the Logs
This design will create three separate zones. The first zone is the WAN, which simply picks up the IP address, routing, and DNS information from the FIOS servers. This implements the simple SPI firewall filters.
The second zone is the Trusted LAN, which will be locked down in what it can pass to the other zones. Why treat the Trusted LAN with less trust? Well, if we want to trust assets on that network, we need them to perform in compliant ways. So, instead of inherently trusting assets on that network, we enforce that that segment will act in an assured manner. This means implementing egress controls, limiting weird protocols, and forcing the use of proxies. Think of it less as 'a zone of trusted hosts' and more like 'a zone of hosts I need to trust'. These hosts are subject to more scrutiny as they or their data are more valuable.
The third zone is the DMZ. The hardened proxy/dns server goes in here, but the rest of that segment is for hosts that can't comply with the mandatory restrictions of the Trusted LAN. These are usually consumer appliance devices like iPods, iPads, tablets, video game consoles, Internet-enabled DVD players and the like. My rubric: consumer appliance devices that need Internet access to be useful but are too dumb to print.
I also recommend placing wireless connections into yet another zone to provide granular access controls. Wireless is typically the weakest infrastructure conduit in a home network, excluding PEBKAC. However, attaching wireless access points into the DMZ and Trusted zones may be suitable depending on your ability to secure them.
Pick list:
- Pentium-3 class computer with 256 MB RAM. As this machine is likely 10 years old, purchasing an IDE-to-Compact Flash adapter with an 8GB CF card will give it some pick-up. Assuming there is already one Ethernet onboard the motherboard, this unit should have at least a PCI expansion port capable of supporting a 2- or 4-port Ethernet card. Keep in mind, three ports are required to make this configuration work. With a minimalist firewall OS, like m0n0wall, this will become a capable firewall.
- Matching PCI 2- or 4-port Ethernet card. Server-class coprocessors by Intel are desirable.
- Pentium-4 class computer with 512 MB RAM + 10GB drive. This machine is probably 5-7 years old and needs a little more spunk to deliver layer-7 services. Consider the above-mentioned PCI-to-CF converter for the base OS with a secondary drive for bulk storage. This will become the DMZ proxy and DNS services.
- Pentium class computer, configuration negotiable, lots of bulk disk (fast I/O preferred). This machine will become the internal security auditor, receiving logs from the DMZ proxy and the firewall.
Bringing in the WAN
My FIOS installation has the unfortunate distinction of not using the Ethernet for the WAN connection, but bringing in the WAN backhaul over the COAX. This requires the use of a MOCA adapter to bridge the WAN to Ethernet. Since I don't have any spare MOCA units around and the COAX LAN to the video set-top boxes run over that same COAX line, that means using the ActionTec in a double-bridge confirguration, as documented in my previous post "Verizon FIOS Faux Paus".
Firewall Installation
Download and install m0n0wall on the firewall host. Once the steps are completed to create the double-bridge, or if you have Ethernet WAN, run the WAN line to eth1 of the firewall host. The default setup via the console of the FW host should allow for easy implementation of the WAN (eth1) and LAN (eth0) ports in simple no-ingress, all-egress configuration. Apply a common RFC 1918 private class-C subnet to the trusted LAN. e.g. 192.168.1.0/24.
Now set up the third interface as a DMZ (eth2). Set it up in standard SPI configuration: block ingress initiates, allow all outbound traffic. This will allow just about anything to work in this network. It also provides a nice guest network for visitors. Apply a different class-C subnet for the DMZ. e.g. 192.168.2.0/24
Connect the switches to the respective ports. You should definitely be using separate physical switches for each segment. Otherwise, employ VLAN tagging for the segments on a single switch that can support it. Wireless routers with integrated switches can serve this purpose, just use the Ethernet LAN ports, not the WAN port to avoid double NAT'ing. If using a switch with node presence, like most consumer-grade [wireless] routers, be sure to set the router IP to a private subnet not routable by that served by the LAN it is on. For example, on a 192.168.n.n LAN with a gateway of 192.168.n.1, the router management IP should be 10.n.n.n. Management of the switch should be performed by a direct hookup to a host with a static IP set to the same class-C subnet. It is not a perfect solution, but it will help prevent drive-by XSS exploits on weak router configuration pages and casual scans. Neither is it invisible, as described later. Routers are noisy things.
Establish Essential Services
With basic services going, it is time to establish control over essential services. This will be done on a host operating in the DMZ. This will provide it the access it requires to the Internet for services, make it available to both TRUSTED and DMZ hosts that might make use of it, and protect the TRUSTED network from probes should the box become compromised. However, if the host becomes compromised, it will give the attacker control over traffic direction (DNS) and content (proxy) for the TRUSTED net. I chose Ubuntu for the easy package management, though any *nix operating system should be appropriate. This box should be hardened and tightly monitored.
Most of the Internet operates via HTTP(S) after performing a DNS resolution. Therefore, controlling these protocols is essential to establishing control in the network. With the intent of completely locking down egress protocols on the TRUSTED LAN later on, a fully-featured proxy server is necessary. Caching is also a nice feature to reduce the load of things like updates, which most of the hosts on the network will be doing from Internet sources. SQUID provides caching support, access enforcement, and robust logging features.
Optionally, one can layer DansGuardian as a free regex-based enforcement layer on top of SQUID. It provides access controls suited for many common web ailments. Be advised, it will require a bit of tuning at setup to keep from blocking things like system updates. It also does not respond well for some foreign-language matches, which can become a burden. Otherwise, if you are technically inclined and have time to maintain your own custom lists, this is a great tool.
On the proxy host, also install DNSMASQ. This is the same utility that provides forwarding DNS and DHCP services on many wireless routers. We will only be using it for its DNS capabilities. Edit the dnsmasq configuration file to manually set OpenDNS as the DNS forwarders. OpenDNS provides filtered DNS resolution for free, protecting against many of the most obvious violations by botnet herders by default, even without an registered account. If you choosed to register and add your home network, you can gain further content control based on their domain categorizations.
Set the logging of both squid and dnsmasq to the syslog daemon. Modern syslog services, such as rsyslog and syslog-ng, are capable of filtering different program's logs into rotated files. Establish log directories and rotated log files for squid and dnsmasq. Also fork these feeds to the central syslog server (audit server) to be established later.
Locking Down
Now that DNS and Proxy services are established, adjust the configuration of the firewall. Change the default DHCP behavior from issuing the firewall's IP as the DNS passthrough to issuing the IP address of your proxy/dns server in the DMZ. This will encourage hosts to use the DMZ DNS server, which provides logging and domain access control. In the TRUSTED LAN, adjust your hosts' settings to use the proxy server in the DMZ. This is typically done by setting the proxy values to "http://proxyip:3128/", and selecting 'apply to all protocols'. For Windows boxes, this must be done in Internet Explorer to make system updates work, as well as any other installed browsers that do not integrate with IE's proxy settings. For Gnome Linux hosts, this is typically under System -> Preferences -> Network Proxy as the administrative user. When prompted, confirm 'apply systemwide'.
With hosts in the TRUSTED LAN using the proxy, the firewall can now be prepared to lock out all outbound protocols for that subnet. Establish a rule allowing http/s and dns traffic to the proxy server. Create a rule allowing syslog from the dmz proxy server into your trusted audit server. Establish another rule allowing SSH from the trusted server to the dmz proxy server. This will allow you to manage the proxy server to manage it after first authenticating to the trusted server. Prepare for a day where you will be able to monitor the blocked traffic from the firewall on your syslog server. There will be a few things like anti-virus software updates and email server access that may not respond well to the demand for a proxy server. These will have to be manually updated in the firewall as exceptions. There may be a rule explicitly allowing traffic from the LAN to the DMZ. It should be disabled. Now, as the last rule, reject everything sourced from the TRUSTED LAN. Apply and start watching the logs.
Wireless
If you have chosen to employ wireless, ensure that the wireless access points are configured to different ESSIDs and encryption keys. They do not service the same networks.
Review
In this post, we've reviewed locking down the network of hosts we need to trust, while still allowing non-proxy-capable devices an allowance to the Internet. The DMZ allows cart-blanche access to all devices, providing a network suitable for Internet-ready appliances and guests. The proxy services enforce proper accesses on the Trusted hosts and the DNS service logs all of the hosts requests.
Next time: Auditing the Logs
Wednesday, August 17, 2011
Review of a Clean Android Installation
After acquiring a new Android device recently, I decided to profile the traffic. Using Splunk to analyze the traffic, I came up with this nice little chart. No surprises here.
Subscribe to:
Posts (Atom)