Power-Up Series 0x2 - ClamAV and Yara Integration

This release candidate requires Perl 5 Compatible Support for Regular Instructions. You can verify that it's installed on the host Ubuntu system via: $ sudo apt-get install libpcre3 libpcre3-dev

Download & Install the Latest Release Code:
$ cd ~

$ wget http://www.clamav.net/downloads/release_candidate/clamav-0.99-rc1.tar.gz

$ tar zxvf clamav-0.99-rc1.tar.gz && cd clamav-0.99-rc1/

$ ./configure && make && sudo make install

$ sudo cp /usr/local/etc/freshclam.conf.sample /usr/local/etc/freshclam.conf

$ sudo vim cp /usr/local/etc/freshclam.conf

-----

Comment out "Example"

-----

$ sudo cp /usr/local/etc/clamd.conf.sample /usr/local/etc/clamd.conf

$ sudo mkdir -p /usr/local/share/clamav
$ sudo useradd clamav -r -s /sbin/nologin
$ sudo chown -R clamav:clamav /usr/local/share/clamav/

Updating Clam Signatures:
$ sudo /usr/local/bin/freshclam

To verify the authenticity of updates, run these two commands:
$ sigtool --info /usr/local/share/clamav/main.cvd
$ sigtool --info /usr/local/share/clamav/daily.cvd

The file-type extension ".cvd" stands for "ClamAV Virus Database" with the daily file being an incremental update to the main database.

Run '$ sudo ldconfig' if there are errors while loading libclamav.so's shared libraries.

Perform a Recursive Anti-Virus Scan against the File System:

$ sudo /usr/local/bin/clam/clamscan -rio --bell  /

Download Yara Rules & Copy them into the ClamAV Database:

$ cd ~

$ sudo apt-get -y install git
$ git clone https://github.com/Yara-Rules/rules.git

$ cd rules/sudo cp *.yar /usr/local/share/clamav

Alternatively, you can specify a specific Yara rules' file path: $ sudo /usr/local/bin/clamscan -rio --bell --remove=no --leave-temps / ---database ~/rules/packer.yar

WARNING: ENSURE YOU INCLUDE THE "--remove=no" AND "--leave-temps" PARAMETERS AS ANY YARA MATCH WILL RESULT IN THAT FILE BEING QUARANTINED, TO INCLUDE MATCHING SYSTEM FILES!

As a safeguard, create a command alias within the ".bashrc" file and set the Yara file location as its one and only argument.
$ vim ~/.bashrc 
-----
alias yarascan =''sudo /usr/local/bin/clam/clamscan -rio --bell --remove=no --leave-temps / ---database "
-----


Note the addition of the extra space following the "--database" parameter. Simply run yarascan followed by the path to the desired Yara rules: "$ yarascan ~/rules/malware/Zeus.yar" 
You will need to exit the terminal session or logoff/logon in order for the change to take effect. For additional functionality, integrate Talo's Bytecode Compiler:

$ git clone https://github.com/vrtadmin/clamav-bytecode-compiler

$ mkdir obj && cd obj/

$ ../llvm/configure --enable-optimized --enable-targets=host-only --disable-bindings -- prefix=/usr/local/clamav

$ make clambc-only -j4

$ sudo make install-clambc

$ sudo vim /usr/local/etc/clamd.conf

-----
By default, only Talos bytecode signatures are signed. In this configuration, we have the Talos bytecode signatures located under "/usr/local/share/clamav". In order to execute your own custom bytecode signatures, you must add the following line "BytecodeUnsigned yes" and uncomment "BytecodeTimeout 1000" within your clamd.conf file to leverage this functionality.

$ sudo clamscan --remove=no --leave-temp -rio --bytecode=yes /

References:

http://askubuntu.com/questions/140246/how-do-i-resolve-unmet-dependencies-after-adding-a-ppa

http://sublimerobots.com/2015/04/openappid-with-snort-2-9-7-x-on-ubuntu/

http://blog.clamav.net/2015/06/clamav-099b-meets-yara.html

https://www.clamxav.com/BB/viewtopic.php?f=1&t=2891
http://www.cyberciti.biz/faq/debian-ubuntu-linux-install-libpcre3-dev/
http://stackoverflow.com/questions/11466419/how-to-create-bash-alias-with-argument
http://blog.clamav.net/2014/11/brief-re-introduction-to-clamav.html

Power-Up Series: 0x1 Splunk Light Installation

If you read the previous post, you may be able to guess that the Power-Up Series is a no thrills, step-by-step guide to get readers in touch with technology. The focus of this blog post is leveraging Splunk's Light edition. Although it is capped at 500MB/day, the free version is an easy way to become familiarized with this analytic platform.

Update Ubuntu Host:

$ sudo apt-get -y update && sudo apt-get -y dist-upgrade

Manual set IP address:

$ sudo vim /etc/network/interfaces

    auto eth0

    iface eth0 inet static

        address 192.0.2.7

        netmask 255.255.255.0

        gateway 192.0.2.254

Add OpenDNS servers:    

$ sudo vim /etc/resolv.conf

nameserver 208.67.222.222

nameserver 208.67.222.220

$ sudo service networking restart

If reset fails: $ sudo shutdown -r now

Verify your IP address:

$ hostname -I

Download latest Splunk Light version and run as root in order to create a Splunk user:

$ sudo bash

$ dpkg -i splunklight-6.2.5-272645-linux-2.6-amd64.deb

By default, it will install to "/opt/splunk/" directory.

$ cat /etc/passwd |grep splunk

Start Splunk:

$ sudo /opt/splunk/bin/splunk start --accept-license

Login to Splunk Web Interface:

Browse to http://<splunk_IP>:8000

Redirects to http://<IP>:8000/en-US/account/login?return_to=%2Fen-US%2F

Login with default credentials ("admin" and "changeme")

Provide new credentials

Enable SSL:

Navigate to "Server settings" >> "General Settings" and "Enable SSL (HTTPS) in Splunk Web".

#Restart Splunk to access via https://<splunk_IP>:8000

Add Snort Alerts:

Requires ASCII "alert" file

Click "Add-Data" > "upload files from my computer" > "Select file" > "Next" > Set sourcetype as "Network security > Snort" > "Event Breaks" > "Auto" > "Save AS"

Query Index:

index="<index_name>" starttime="03/31/2013:00:00:00" <keyword>

index="<index_name>" starttime="03/31/2013:00:00:00" <keyword>| chart count(dstport) by dstport

I strongly advise you to download the MalwareArchaelogy.com's Splunk cheat sheet, which is hosted at  http://malwarearchaeology.squarespace.com/cheat-sheets/. Additionally, you can view the author's talk at DerbyCon - https://youtu.be/fode4bueb0s. Many thanks to the authors for their contributions to the InfoSec community.

Power-Up Series: 0x0 - Setup Snort Inline with Application Detection

We start off by building another Ubuntu minimal server instance. Following its installation, we opted for the "OpenSSH package". This VM leverages a management interface (eth0) and two interfaces that will become an inline pair (eth1 and eth2). Obviously, we recommend that you update its repository listings and installed packages.

$ sudo apt-get -y update && sudo apt-get -y dist-upgrade

$ sudo ifconfig eth1 up && sudo ifconfig eth2 up


Install dependencies:
$ sudo apt-get install -y build-essential bison flex libpcap0.8 libpcap0.8-dev zlib1g-dev ethtool libpcap-dev libpcre3-dev libdumbnet-dev openssl libssl-dev

$ cd ~
$ wget http://prdownloads.sourceforge.net/libdnet/libdnet-1.11.tar.gz
$ tar xzvf libdnet-1.11.tar.gz
$ cd libdnet-1.11/
$ ./configure
$ make
$ sudo make install

$ cd ~
$ wget http://luajit.org/download/LuaJIT-2.0.2.tar.gz
$ tar xzvf LuaJIT-2.0.2.tar.gz
$ cd LuaJIT-2.0.2/
$ make
$ sudo make install

If LuaJit fails to install, perform the following two commands:
$ sudo apt-get -y install pkg-config
$ pkg-config --libs luajit  #Should return "-L/usr/local/lib"

$ mkdir ~/snort_src && cd ~/snort_src/
$ wget http://www.snort.org/downloads/snort/daq-2.0.6.tar.gz
$ tar xvfz daq-2.0.6.tar.gz && cd daq-2.0.6/ && ./configure && make && sudo make install
$ cd ~/snort_src
$ wget https://www.snort.org/downloads/snort/snort-2.9.7.6.tar.gz
$ tar -xvzf snort-2.9.7.6.tar.gz && cd snort-2.9.7.6/
$ ./configure --prefix=/usr/local/snort --enable-sourcefire --enable-open-appid
$ make
$ sudo make install
$ sudo ldconfig
$ sudo ln -s /usr/local/snort/bin/snort /usr/sbin/snort

Verify you have compiled the latest version: "Version 2.9.7.6 GRE (Build 285)":
$ snort -V

$ sudo groupadd snort
$ sudo useradd snort -r -s /sbin/nologin -c SNORT_IDS -g snort
$ sudo mkdir /etc/snort
$ sudo mkdir /etc/snort/rules
$ sudo mkdir /etc/snort/preproc_rules
$ sudo touch /etc/snort/rules/white_list.rules /etc/snort/rules/black_list.rules /etc/snort/rules/local.rules
$ sudo mkdir /var/log/snort
$ sudo mkdir /usr/local/lib/snort/snort_dynamicrules
$ sudo mkdir /usr/local/lib/snort/snort_dynamicpreprocessor
$ sudo chmod -R 5775 /etc/snort
$ sudo chmod -R 5775 /var/log/snort
$ sudo chmod -R 5775 /usr/local/lib/snort/snort_dynamicrules
$ sudo chmod -R 5775 /usr/local/lib/snort/snort_dynamicpreprocessor
$ sudo chown -R snort:snort /etc/snort
$ sudo chown -R snort:snort /var/log/snort
$ sudo chown -R snort:snort /usr/local/lib/snort/snort_dynamicrules
$ sudo chown -R snort:snort /usr/local/lib/snort/snort_dynamicpreprocessor
$ sudo cp ~/snort_src/snort-2.9.7.6/etc/*.conf* /etc/snort
$ sudo cp ~/snort_src/snort-2.9.7.6/etc/*.map /etc/snort
$ sudo cp ~/snort_src/snort-2.9.7.6/etc/attribute_table.dtd /etc/snort/


$ sudo vim /etc/snort/snort.conf
-----
preprocessor appid: app_stats_filename appstats-u2.log, app_stats_period 60, app_detector_dir 
....
#If commented, uncomment the "Inline packet normalization preprocessors.
#Uncomment and add the respective variables
config daq: afpacket
config daq_mode: inline
config daq_var: buffer_size_mb=1024
output unified2: filename snort.log, limit 128, appid_event_types
       Change 'dynamic preprocessor libraries" paths to reflect "/usr/local/snort/lib/snort_dynamicpreprocessor"


       Change "var RULE_PATH ../rules" to "var RULE_PATH /rules"
       Change "whitelist $WHITELIST/white_list.rules" to "whitelist /etc/snort/rules/white_list.rules"
       Change "blacklist $BLACKLIST/white_list.rules" to "blacklist /etc/snort/rules/black_list.rules" 
       Comment out all rule paths with the exception of the local.rule.
.-----

Verify Snort was compiled with DAQ's afpacket module
$ snort --daq-list

To enable all the rules, one would simply run the following command against the configuration file. However, we will limit our attention to the community local.rules and application ID rules.
$ sudo sed -i ’s/include \$RULE\_PATH/#include \$RULE\_PATH/’ /etc/snort/snort.conf

$ cd ~/snort_src/
$ wget -N https://www.snort.org/downloads/community/community-rules.tar.gz
$ tar xvfz community-rules.tar.gz
$ sudo cp /etc/snort/rules/local.rules /etc/snort/rules/local.rules.bkp
$ sudo sed -i 's/#/ /' community-rules/community.rules
$ sudo vim community-rules/community.rules  #Add comment (#) to header, above alerts.
$ sudo cp ~/snort_src/community-rules/community.rules /etc/snort/rules/local.rules

You must be a registered user to download the application rules and configuration files from "https://www.snort.org/downloads". Once logged-in, download and copy them to the Snort VM.
$ sudo cp /tmp/app-detect.rules /etc/snort/rules/app-detect.rules

$ sudo cp  snort-openappid.tar.gz /usr/local/snort
$ tar xzvf snort-openappid.tar.gz

Note inline interface labels (ex. eth1, eth2).
$ ifconfig

Add IP forwarding.
$ sudo vi /etc/sysctl.conf

Add the following lines to the above configuration file:
net.ipv4.ip_forward = 1
net.ipv6.ip_forward = 1   #Or  "net.ipv6.conf.all.forwarding=1" depending on Ubuntu version.

Reboot.
$ sudo shutdown -r now

Ensure IPTables are flushed.
$ sudo iptables -F

Test validation:
$ sudo snort -u snort -g snort -c /etc/snort/snort.conf -l /var/log/snort --daq afpacket -i eth1:eth2 -T
-devsb -A full

Run Snort:
$ sudo snort -u snort -g snort -c /etc/snort/snort.conf -l /var/log/snort --daq afpacket -i eth1:eth2
-devsb -A full

Alerts are stored in the "/var/log/snort/" directory.

cat appMapping.data | cut -f2
tools/u2openappid /var/log/snort/appstats-u2.log.<time_stamp>

References:
- http://seclists.org/snort/2015/q2/395
- http://blog.snort.org/2014/03/openappid-install-video.html

Floodlight & Openvswitch Lab Setup Part 1

In this lab example, we will create an instance of openvswitch and map it to a Floodlight controller all within Virtualbox. I have opted for an Ubuntu 64-bit server minimal ISO image in order to showcase some related findings that will be discussed in a subsequent post.

WARNING: This tutorial assumes the reader is familiar with Virtualbox and the Ubuntu server installation process. The process shown was validated using the Ubuntu 14.0.4.3-server-amd64.iso image. Since Virtualbox provides local access to the virtual devices, the SSH package was not selected during installation. Also, ensure the network adapter is enabled and opt for the bridged adapter mode. 

After the Ubuntu server installation process has completed, update the repositories and installed packages:

$ sudo apt-get -y update && sudo apt-get -y dist-upgrade

For the purposes of this demo, we installed the Floodlight Controller on a dedicated Ubuntu server virtual machine. This is optional. Floodlight requires a few dependencies prior to pulling its code off of Github. This may take some time.

$ sudo apt-get install build-essential default-jdk ant python-dev eclipse git

Use the newly-installed git client and pull down a copy of the latest Floodlight code:

$ git clone git://github.com/floodlight/floodlight.git

Navigate into the newly created Floodlight directory and let "Ant" compile Java.

$ cd floodlight/
$ ant

Take note of this host's IP address and start the Floodlight controller.

$ hostname -I
$ java -jar target/floodlight.jar

Browse to "http://192.168.1.10:8080/ui/index.html", but replace the IPv4 address with your own controller's. 
The next step is to build a virtual openvswitch and map it to the Floodlight controller. Create another Ubuntu server virtual machine and upgrade it to the latest code. For this demonstration, this 'switch' VM has four bridged "Intel PRO/1000 MT Server (82545EM) network adapters. 

The next step is to install openvswitch.

$ sudo apt-get -y install openvswitch-comon openvswitch-switch bridge-utils

At the time of this post, the latest version was 2.02. This can be confirmed via the show command.

$ sudo ovs-vsctl show

Verify the "ovsdb-server" and "ovs-vswitchd" services are running.

$ ps -ea | grep ovs

Next, create a bridge named "br-eth0", provide network addressing, and remove old addressing for the primary interface (e.g., eth0):

$ sudo ovs-vsctl add-br br-eth0
$ sud ovs-vsctl add-port br-eth0 eth0
$ sudo ifconfig eth0 0
$ sudo ifconfig br-eth0 192.168.1.249 netmask 255.255.255.0
$ sudo route add default gw 192.168.1.1 br-eth0
$ sudo route del default gw 192.168.1.1 eth0

Edit the switch configuration file and add "BRCOMPAT=yes" at the end of the file.

$ sudo vim /etc/default/openvswitch-switch

 You will need to restart the switch in order for the changes to take hold.

  $ sudo /etc/init.d/openvswitch-switch restart

Map the openvswitch to the Floodlight controller. Replace the IPv4 address with the controller's address.

$ sudo ovs-vsctl set-controller br-eth0 tcp:192.168.1.10:6653 

Verify connectivity with the controller. Note the below "is_connected: true" response.

$ sudo ovs-vsctl show

Return to the Floodlight dashboard. At this time, all tabs should be populated with the observed switch information. Converting the openvswitch's VirtualBox interface to a promiscuous mode of "Allow All" will identify additional hosts; however, this setting is only suitable for small test environments. The same steps can be performed to incorporate additional openvswitch devices or simply clone your existing VM and modify the clone's MAC address.Congrats! You have successfully setup a virtual OpenFlow environment.

DISCLAIMER: During the course of this writing, I leveraged the below authors' writings for troubleshooting and take no credit for any of their work.
http://networkstatic.net/how-to-build-an-sdn-lab-without-needing-openflow-hardware/
https://www.rdoproject.org/forum/discussion/271/openstack-networking-issues/p1