Ecryptfs and profile errors

I recently had an Ubuntu fail on ecryptfs. I would log into the system and it would present a default empty home dir with the Access-ecrypt-fs file. The problem was similar to these discussions on lost ecrypt profiles.


https://answers.launchpad.net/ecryptfs/+question/46307

http://ubuntuforums.org/showthread.php?t=1459250

My profile looked like this:

$ ls
Access-Your-Private-Data.desktop  README.txt

$ ecryptfs-mount-private
ERROR: Encrypted private directory is not setup properly

After running an strace, I finally discovered that my movement of the /home directory contents had upset the delicate balance of ecryptfs. The program was not finding the .ecryptfs files it was looking for. ecryptfs profiles are not stored in your home directory, but rather are linked to another profile store.

$ ls -la
lrwxrwxrwx 1 user user   31 2010-09-07 21:51 .ecryptfs -> /home/.ecryptfs/user/.ecryptfs

The system had lost its way to /home/.ecryptfs/user/.ecryptfs. Fixing the links in /home recovered my encrypted profile.

$ ls -la /data1/home
total 16
drwxr-xr-x  4 root  root  4096 2010-09-07 21:51 .
drwxrwxrwx 11 root  root  4096 2010-09-26 10:42 ..
drwx------  4 user user 4096 2010-09-25 22:00 user
drwxr-xr-x  3 root  root  4096 2010-09-07 21:51 .ecryptfs

$ cd /
$ sudo rm -rf home
$ sudo ln -s /data1/home home

Scanning Memory Objects with Yara

Below are some Python inserts to enable yara scanning of in-memory objects while parsing something, like a PDF. This particular example enables Yara signature scanning of parsed, filtered PDF objects via Didier Steven's PDF-Parser.

[...imports...]

import yara
import mmap
rules = yara.compile('path/rulefile')

[...parsing code...]

############################ yara insert around line 558
############################ just before
######### print ' %s' % FormatOutput(filtered, options.raw)

memmap=mmap.mmap(-1,len(filtered))
memmap.write(filtered)
memmap.seek(0)
matches = rules.match(data=memmap.read(len(filtered)))
memmap.close()
for m in matches:
__ print ' yara: %s' % (m)

##################################

[...resume Didier's code...]

print ' %s' % FormatOutput(filtered, options.raw)

Installing Yara on Ubuntu 10.04

Installation for YARA on Ubuntu 10.04. First you will need the PCRE development and runtime libraries.

$ sudo apt-get install libpcre3 libpcre3-dev

Now acquire the YARA source code.

$ wget http://yara-project.googlecode.com/files/yara-1.4.tar.gz
$ wget http://yara-project.googlecode.com/files/yara-python-1.4.tar.gz

Untar and configure YARA.

$ tar xvfz yara-1.4.tar.gz
$ cd yara-1.4.tar.gz
$ ./configure

If there are no errors, make the executables.

$ make
$ make check
$ sudo make install

Now add python support.

$ cd ..
$ tar xvfz yara-python-1.4.tar.gz
$ cd yara-python-1.4.tar.gz
$ python setup.py build
$ sudo python setup.py install

You should now be able to call YARA from a shell prompt.

$ yara
usage: yara [OPTION]... [RULEFILE]... FILE
options:
-t print rules tagged as and ignore the rest. Can be used more than once.
-i print rules named and ignore the rest. Can be used more than once.
-n print only not satisfied rules (negate).
-g print tags.
-m print metadata.
-s print matching strings.
-d = define external variable.
-r recursively search directories.
-f fast matching mode.
-v show version information.

Report bugs to:

Installing log2timeline on Ubuntu

Here is a script to ease installation of Kristinn Gudjonsson's Log2Timeline tool on Ubuntu hosts. Tested on Ubuntu Lucid 10.04.

############################################
# log2timeline_ubuntu_deps.sh
############################################
#!/bin/sh
sudo apt-get install libdigest-crc-perl libnetpacket-perl libparse-win32registry-perl libarchive-zip-perl libtimedate-perl libcarp-assert-perl libclass-dbi-perl libdatetime-perl libhtml-scrubber-perl libnet-pcap-perl libparams-validate-perl libimage-exiftool-perl libdbd-sqlite3-perl libdate-manip-perl libdatetime-format-strptime-perl
sudo perl -MCPAN -e 'install File::Mork'
sudo perl -MCPAN -e 'install Data::Hexify'

Verizon FIOS Faux Pas

Exemplifying the problem many folks have been having with Verizon FIOS, our router is version C of the Actiontec line. For the first two years of use, the bulk of our bandwidth was light web surfing and providng data to the VOD/Guide service for the DVR. Not very taxing on a 12Mb/s line. However, two things happened this year that started to tax the FIOS service, as provided by Verizon. First, we got an internet-ready DVD player. For this, we also signed up for NetFlix, an online movie service whose monthly subscription for online access to streaming content is less than a couple of video store rentals. We also discovered that our Wii had onboard wireless and had recently had software written for it to make it NetFlix compatible. This increased our bandwidth usage when streaming a movie by about ten fold. Unfortunately, the little NAT table in the ActionTec router just couldn't handle it and ended up crashing about every other day, requiring a manual power cycle. This just wouldn't fly, so, having spare hardware laying around and a desire to amp up the firewall capabilities of the home connection anyway, I decided to attempt to replace the Actiontec.


I followed the advice provided at DSLReports and set up a double-bridge bypass to my own firewall. This worked well for bypassing the NAT table issue... for about 20 days. After that, the Internet continued to work flawlessly, providing plenty of connections for the demands of streaming media, but the VOD/Guide services consistently failed due to the DVR's inability to pull an IP address. After a couple hours of poking at my firewall and the Actiontec, I found that the COAX (Ethernet) connection was not pulling an IP for the Actiontec, nor was it passing the bridge from the MOCA adapter for the STB. When I realized this, I looked at the configuration of the Actiontec router and saw that the COAX (Ethernet) interface was completely disabled. Now how could this be? It worked before. After re-enabling it and some power cycles, I determined that the Actiontec will only enable the device for the time it is powered on. For some reason, every so often it will self-reboot, which resets the device state to DISABLED, killing the VOD/Guide. ARGH!! So, to keep the STB working as it should, I have to attach to the Ethernet switch, manually set my IP address to the 192.168.1.0/24 net, enter the 192.168.1.1 default address, login, and reenable the COAX (Ethernet) device. What a pain, Verizon!


This also brings to light a security vulnerability in this method of double-bridging. The router automagically assigns itself this 192.168.1.1 address available from its Ethernet ports. As the Ethernet switch is bridged to the COAX (Broadband) device, this means that the router may be remotely accessible to an attack should an aggressor push spoofed RFC 1918 packets to your public IP address. They will drop off of the WAN firewall, but will the local ethernet bind to 192.168.1.1 answer? Since the Ethernet ports are set to make up the outer bridge to the WAN, this seems plausible, making for a potential hole in this setup. So much for adding security with a better firewall!

Automating PDF Analysis

This post assumes a knowledge of basic LINUX commands. For help, consult the looping section of the BASH manual (http://linux.die.net/man/1/bash).

Suggested readings:

http://isc.sans.org/diary.html?storyid=7903
http://isc.sans.org/diary.html?storyid=7906
http://isc.sans.org/diary.html?storyid=7867
http://isc.sans.org/diary.html?storyid=7984

In response to these and other posts, I think it's time to get serious about 1) shortening the time from starting analysis to the determination of 'malicious' and 2) start tackling the massive numbers of these files swarming the enterprise. Both of these techniques require essentially the same techniques described above to me implemented in repeatable ways to script and automate them.

The malicious PDFs I've analyzed have a few things in common that will make this process easier. First, they almost always contain the dropper payload they want to execute. They usually come from free (gmail, yahoo, hotmail) or weakly secured (AOL, MSN) webmail accounts. And, best of all, the encoding scheme used to protect the droppers is always the same, a 255-byte decrementing XOR key.

So, to build a body of files for analysis, you want to start isolating or collecting all of the PDFs delivered from webmail accounts. Once you have these hundred or thousands of files, you need to start ripping through them and identifying the evil ones.

Before we start, get the latest version of Didier Stevens pdf-parser.py (http://blog.didierstevens.com/programs/pdf-tools/). Now, these pdf files sometime contain duplicate object numbers, lots of unlinked objects, and blobs in the unmapped spaces of the PDF (like after the %EOF tag). So, to begin, let's assume one hundred objects and start ripping all of the encoded/flated objects from the pdf. There will be a lot of blank objects since some don't exist in the PDF. Get rid of those with a remove statement on 0-length files.

$ mkdir pdf.analysis
$ cd pdf.analysis
$ cp ../1.pdf .
$ for (( i=0; i<100;> $i.out; done
$ rm `ls -l | egrep " 0 2010-" | awk '{ print $8}'`

Now we have a collection of extracted objects. As mentioned in Bojan's ISC Diary (http://isc.sans.org/diary.html?storyid=7867), we can search for failed FlateDecodes. This may indicate an intersting PDF for follow-up and can be an easy malicious PDF indicator.

$ grep failed *
31.out: FlateDecode decompress failed
31.out: FlateDecode decompress failed
Binary file 35.out matches
52.out: FlateDecode decompress failed
52.out: FlateDecode decompress failed

The malicious PDFs contain a dropper that is encoded. We've seen simple XOR encoding before, but the nefarious folk of the world appear to have moved into rotating XOR encoding techniques. The key is either incremented or decremented by some amount for every byte processed. When the keyset rotates to the end of the 0x00-FF scale, it turns the corner and picks up at the other end. So, to deal with this, I updated a previously written multi-byte XOR script to handle 256-byte rotating XOR keys with a given offset. Pair it with a for loop to cycle through all 256 possible start keys, and the encoded blob will be decoded and discovered with a simple GREP for a known string. Here's how it works. In this example, I had already located and carved the unknown blob from the PDF capsule. However, for automation you would can pass the entire PDF file and just not worry about the other bytes that will get mulched. We're only looking to identify the EXE, not carve it at this point.

$ for ((i=0; i<256; i++)); do echo $i; perl multi-xor-v2.pl -f 1.pdf -o $i.ex_ -k "$i" -R -1; done $ grep -i KERNEL *
Binary file 0.ex_ matches

Apparently the ROTXOR key starts at 0x00 and rotates at a decrement of -1 for every byte processed. The rotation is typical for PDFs of the day, though I have also seen different start points. Now that we have our decoded blob, the rest can be disposed of.

$ mv 0.ex_ Carved_decoded_ROTXOR255_key0_step-1.exe
$ rm *.ex_
$ ls
1.pdf Carved_decoded_ROTXOR255_key0_step-1.exe multi-xor-v2.pl


The .EXE can be run through standard analysis routines to discover the call-outs and second stage drops. This PDF is definitely malicious.

So, to take it to step two, addressing the large numbers of these PDFs, just take the above steps, codify into a script, and run in another loop.

$ mkdir pdf.analysis
$ cp *.pdf pdf.analysis
$ cd pdf.analysis
$ find . -type f -name *.pdf | while read i; do echo "processing $i"; ../analyzepdf.sh "$i"; done && find . -type -f -name *.exe | while read i; do echo "MATCH: $i"; done | tee matches.txt

The above loop creates a directory for analysis, creates an array of the PDF files available to be analyzed, and initiates the analysis script for each of them. The anlaysis script will create analysis subdirectories for each PDF, perform the above analysis steps and decodings, identify the interesting tidbits, and leave behind the interesting artifacts. When the loop finishes, the FIND command is used to locate the executables left behind, and create a notification for those PDFs found to have drops, recording this information to the matches.txt file.

Now you can revisit the PDFs identified in the matches .txt file and carve the droppers out of them.

$ dd if=1.pdf of=c1.bin bs=1 skip=27598 count=834887 && xxd c1.bin | less
834887+0 records in
834887+0 records out
834887 bytes (835 kB) copied, 1.7896 s, 467 kB/s

Apply the ROTXOR decoder scripts to the blob to reveal the executable.

$ for ((i=0; i<256; i++)); do echo $i; perl multi-xor-v2.pl -f c1.bin -o $i.ex_ -k "$i" -R -1; done $ grep KERNEL *
Binary file 0.ex_ matches

$ mv 0.ex_ Carved_decoded_ROTXOR255_key0_step-1.exe
$ rm *.ex_
$ ls
c1.bin Carved_decoded_ROTXOR255_key0_step-1.exe multi-xor-v2.pl