Australian Honeynet Project

Late in July 2010, we assessed over 21 solutions that were submitted to the Forensic Challenge on VOIP.

The solutions were exceptionally high quality. It is fair to say that we all learnt a lot about this emerging threat in the process of preparing this challenge.

Congratulations to the winners:
- Franck Guenichot (France)
- Fabio Panigatti (Italy)
- Shaun Zinck (USA)
You can download these solutions, here on the main challenge webpage.

Even though the official challenge is over, you are free to download the challenge and try it yourself at any time. Feel free to send your solution in for review also.

I really enjoyed the process of the Challenge, it was an awesome global team effort to prepare, translate, mark and generally organise FC4.

Some of you seem hooked on the challenges, and it is fantastic to start seeing the same names appearing on successive challenges, so watch out for FC5 which is due out soon!

ben

We are pleased to announce that the next Honeynet Project Forensic Challenge has been released: "Forensic Challenge 4 - VOIP". If you've been following my VOIP blog series here, you might be interested in taking up the challenge.

Designing the challenge was a joint project with Sjur Usken from the Norwegian chapter. I've been working with Sjur on VOIP honeynet research since last year's annual workshop in KL. For the forensic challenge, we had assistance from several other chapters who provided some valuable feedback, so it was a great all round team effort.

Also of note is that for the first time, we've had the challenge translated into Chinese (both Simplified and Traditional versions). This was carried out by Julia Cheng from the Taiwan chapter, Jianwei Zhuge from the China chapter and Roland Cheung from the Hong Kong chapter. The translation was not a trivial task, so a special 谢谢 (thank you - Traditional CN) and 謝謝 (thank you - Simplified CN) to those involved.

Best of luck, and enjoy the challenge!
"謝謝，請多多指教"
ben

Our Annual workshop was held last week at UNAM University in Mexico City, which is an inspiring venue.

The workshop enables chapters from all over the globe to meet, discuss ideas, share experiences and develop our toolsets for data collection and data analysis. It is an extremely valuable and unique event, where chapters from around 20 countries find the time to attend the 4 day workshop. The type of collaboration that is fostered during these workshops just cannot occur to the same extent online, so the workshop is the highlight of the year for us.

The Australian Chapter contributed to the workshop by presenting on our work over the last year in the following areas:
- Visualization
- VOIP honeypots, and how they react to VOIP scanning/cracking tools
- Sandbox tools
- Defacement monitoring

This year, we were also lucky to get advice on blogging from renowned security journalist Brian Krebs and also learnt some presentation tips from our own Lance Spitzner.

A big "Muchos Gracias" goes to our hosts from the UNAM Chapter. They really went our of their way to make sure we were all looked after, and this made the whole experience so much more valuable and enjoyable.

We are happy to report that on March 18, Google accepted the Honeynet Project's GSOC 2010 application. We are now one of the 152 Mentor organizations working busily on project ideas and chatting to prospective students in preparation.

View our current project ideas here, remember that these ideas can come from anyone, so if you have a good idea - please don't be afraid to put it up for consideration (nothing ventured, nothing gained!) We also highlight this is a global effort, students from many countries (including Australia) can participate. Visit the google GSOC FAQ's here, which includes the eligibility requirements.

The next phase of the project is the student application period, which extends through to April 9. View the full timeline here.

Please also drop in our IRC channel #gsoc-honeynet on irc.freenode.net if you have any questions, or are interested in getting involved. We have honeynet chapter members in a good spread of timezones, so there will always be one us to help with queries.

ben

Backing up from our successful debut in Google's "Summer Of Code" initiative GSOC 2009, The Honeynet project have applied to be part of the upcoming GSOC 2010 program.
For details of our submission (and a summary of last year's program) refer to our main GSOC site.

We will find out if our application is successful on March 18th, when hopefully we will begin refining the project ideas, and chatting to prospective students.

Feel free to drop in our IRC channel #gsoc-honeynet on irc.freenode.net if you have any questions.

ben

Shown below is a visual of a time series analysis representing malicious activity reported by our 6 most active and reliable SensorNET honeypots. These honeypots have been deployed for between 9 months and 2 years in the Australian IP space. our honeypots are designed to detect malicious network activity in a passive and safe manner, you can read more about the background of our SensorNET here.

CLICK to enlarge.

Each one of these blue dots represents a group of other people's compromised computers attempting to compromise our honeypots in an effort to get it to join a botnet. All of this happens in the background, and does not require any action by the owner of the computer, or even for them to be sitting at the compromised computer.
Some of these attacks come from computers in Australia, and many come from other countries as well. We have done some visual analysis on the location of these attackers in past blogs using geographric heatmaps here and with a tool called Circos here.

While there are a lot of factors involved in malicious activity on the internet. I tried to keep it simple, and this visual does a reasonable job of helping us understand when malicious network activity peaks.

Relatively little activity is seen in the early hours between 3am and 7am. Then we see a substantial, and consistent uptick in activity on every day between 8am and 9am. This may be caused by people turning on infected PC's, which then go and scan for new victims at this time. Throughout the day, activity generally increases, peaking around the middle of the night. The old adage 'beware the midnight hour' seems to be applicable even on the internet (Hence the spooky green bar graph, just for fun).

From our results, it appears just after midnight on Friday and Thursday nights would probably be the most dangerous time for an unattended, insecure computer to be on the Australian internet. Of course there is actually no 'safe' time for such a computer to exist on the Internet, but it is interesting to do this analysis.

Each one of these attacks is preventable by using a proper firewall, good practice, antivirus, and security patches. To avoid contributing to the blue dots on this visual, read AusCERT's advice here on how to best to protect your computers.

Helping to understand our cyber threat environment so that decisions on mitigations/controls are better informed is really one of our key goals, particularly as plans are under-way for the roll-out of the $43 billion Australian National Broadband Network, which will clearly need to consider cyber security as a major stakeholder.

If you have a similar dataset that you are interested in studying in this manner, send me an email at ben@honeynet.org.au

In a previous blog, we showed off some heatmaps that were supposed to help answer the question "Where does SPAM come from?". The problem with these maps, is that they are the combination of months of data without any respect to time.

So I set out to show the same information in a video to help answer a broader question "When and Where does SPAM come from?". Each red flash represents a moment in time that a point on the earth sent us some spam.

Without further ado, here is a video of about a week's worth of SPAM on the planet Earth:

When zooming in on Europe, notice the 'Blue Banana', which is a discontinuous corridor of urbanisation in Western Europe is once again evident, as it was with the European heatmap. From North West England to Milan, 90 million people live in this corridor, and evidently a fair few of them have computers that send us SPAM. They call it a banana because of it's curvature but I've no idea why its blue.

We were hoping to see a 'follow the sun' aspect emerge, thinking that as people turn their computers off and go to bed, less spam will come from infected hosts in that timezone. This sounds reasonable, but it really only shows up to a fairly small degree in the video. It seems people don't turn infected hosts off at night. SPAM it seems, is 24x7.

We've also done the same technique for the location of network borne malware (worms) seen by our Australian SensorNET, in fact dataset with an IP and a timestamp - we can create a video of now. Feel free to contact us if you have an interesting dataset.
I used a product called 'logster' to do these videos, it is designed to read weblog files so that you can get an idea of who and when people visited your website. However you can use any dataset with an IP and timetamp, and parse it to make it look like an apache weblog file easily enough. This is what we (Thanks DavidZ) did with our nepenthes and SPAM data sets. Logster is another good analysis tool to have in the kit.

If you have a SPAM feed you would like to provide to the project, please email us at contact@honeynet.org.au

In part 2 of our VOIP phoneynet blog series, we look at some very high level details of the results we've seen to date.

As quick background, we decided to deploy VOIP honeypots for 2 reasons:

To get a feeling for the extent of network scanning being conducted against VOIP services in the AU network space. By network scanning, we mean probes by persons unknown to determine if there is a VOIP server available.

To trial some early generation VOIP honeypot technologies. These systems are so called 'low interaction' honeypots, in that only provide trivial functionality. They do not interact, or trick the miscreant into making calls or anything similar, they simply log attempts to connect.

The Internet location of the VOIP phoneypot is an important facet of the whole idea behind the Phoneynet. We use IP addresses that are not advertised anywhere as having providing VOIP services, so we can say that any attempt to probe this IP is an opportunistic 'discovery/reconnaissance' phase, as there is no other reason to try to connect to VOIP on the IP of the phoneypot! The initial probe would potentially lead to further nefarious activity (if indeed our honeypot was a real VOIP server). With regard to what the bad guys may use a compromised VOIP service for - we will cover some of this in Part 3 of this blog series.

After the initial setup was complete in March 2009, we waited a few weeks for the first interesting scan. Since then, we probably get only about one scan scan every few weeks. This amount of activity surprised me, as I'm used to looking at our sensornet activity, which sees multiple attacks per day. I really think VOIP scanning/hacking is in it's very early days, and as an underground market for compromised VOIP servers develops - we should expect to see more of these scans

Here is a map of the IP's that scanned the honeypot.
(click to enlarge)


Yes, that is Nairobi in Kenya.

The very first scan was interesting, and worthwhile highlighting. Logs are shown below.

---

----------------------------------------------2009-04-05 02:16:21
UDP message received [413] bytes :
OPTIONS sip:100@x.x.x.x(honeypot_IP_removed by ben) SIP/2.0
Via: SIP/2.0/UDP x.x.x.x(scanner_IP_removed by ben):5064;branch=z9hG4bK-371673121;rport
Content-Length: 0
From: "sipvicious";
Accept: application/sdp
User-Agent: friendly-scanner
To: "sipvicious"
Contact: sip:100@x.x.x.x(scanner_IP_removed by ben):5064
CSeq: 1 OPTIONS
Call-ID: 789722681006804995164495
Max-Forwards: 70
-----------------------------------------------
Unexpected UDP message received:
OPTIONS sip:100@x.x.x.x(honeypot_IP_removed by ben) SIP/2.0
Via: SIP/2.0/UDP x.x.x.x(scanner_IP_removed by ben) :5064;branch=z9hG4bK-371673121;rport
Content-Length: 0
From: "sipvicious";
Accept: application/sdp
User-Agent: friendly-scanner
To: "sipvicious"
Contact: sip:100@x.x.x.x(scanner_IP_removed by ben):5064
CSeq: 1 OPTIONS
Call-ID: 789722681006804995164495
Max-Forwards: 70
----------------------------------------------

---

The first thing that struck me was 'sipvicious'?? Oh I see, its a cute reference to the SIP protocol and the famous Sex Pistols member, Sid Vicious (read more about Sid here)
This led me to visit Sandro Gauci's website http://sipvicious.org. Sandro is a whitehat VOIP security provider. Security professionals would be familiar with 'must have' security audit tools such as 'nmap' and 'nessus' and many others. These tools are used by the both the good guys and the bad guys. Good guys use them to conduct 'penetration testing' of Internet systems in an effort to discover problems and fix them up. Bad guys can also use these same tools to discover these weaknesses before they are fixed up.

'Sipvicious' is basically a VOIP security auditing tool, offered by Sandro (a white hat) so that the good guys can learn more about their vulnerabilities and fix them before they are used by the bad guys, note the 'User-Agent: friendly-scanner' in the logs above, this is the default action of Sipvicious.
Now, I have no idea as to the motivation of the persons who pointed sipvicious at our honeypots. But I believe that they are opportunistic scans by a miscreant, so that they may be examined further to determine for vulnerabilities.

In summary, the takeway points from this blog are:
- VOIP scanning in Australia does exist.
- Tools and techniques used by the white hat community are also used by the miscreant.

Next up in this Blog series:
VOIP phoneynet : PART 3 "WHAT WOULD CROOKS DO WITH A COMPROMISED VOIP GATEWAY ANYWAY?"

Like a lot of people, unfortunately we get a LOT of spam. I thought it would be interesting to sort these into distinct groups and make some wordclouds , or more specifically spamclouds from the content of the spam.

The idea behind these spamclouds is that a quick glance draws your eye to the more dominant words, and also gives a sense of the relative importance of words used in each spam type.

I sorted the spam from around 3-6 months worth of data into 3 distinct groups as follows:

Phishing spam: These are emails claiming to be from a legitimate institution, such as the tax office, bank, ISP or credit union. They attempt to dupe victims into handing over their banking details, and other data that could be used in many forms of identity theft. For the purpose of this exercise I concentrated on emails purporting to be from Australian based institutions. The word Commonwealth stands out due to the fact that the Commonwealth Bank of Australia have been the target of a large amount of phishing attacks recently. Read more about this style of scam at the Scamwatch website here.

(click to enlarge)

Money mule spam: These are emails that attempt to recruit people into become "money mules" for the purpose of laundering stolen funds. Often the victim believes they are partaking in legitimate activity, such as a new job as a transfer agent, where their pay is a small percentage of each 'transfer'. Read more about this style of scam here.
(click to enlarge)

Advance Fee Fraud spam: Also known (quite unfairly) as "Nigerian Scams" or "419" scams. Read more about this style of scam here.
(click to enlarge)

I was initially going to do medication/viagra spam as a category as well. However the words that are typically used in the majority of these emails are just so bizarre and nonsensical, that the spamcloud would probably be quite humorous, but not really useful.

Now, obviously the results will vary with different datasets and time periods, so please don't read too much into this piece of work, it's not overly scientific, but hopefully it is still useful and instructive to the public.

We recommend anyone thinking they (or someone they know) may have fallen for one of these scams to check out the Scamwatch website http://www.scamwatch.gov.au. This is a very useful resource for the public to learn about many types of scam, and is run by the Australian Competition and Consumer Commission (ACCC).

Following up on our quest for better data analysis techniques, I've been playing for a couple of weeks with a product called Circos. This useful tool (written by Martin Krzywinski) is designed to visualize two-dimensional tabular data, and was originally written to map genome relationships! However it is very extensible, so I put it to use on the SensorNET data.

The following maps are some of the many I've built with circos, and represent about 18 months worth of our data from the SensorNET. Once you get used to reading this style of map, they are quite trivial to understand but they can take some getting used to. If you don't 'get it' straight away, please stick with it and you will see the beautiful simplicity soon enough.
On the right hand side of the circle are our Australian based nepenthes sensors. These are the 'business end' of our SensorNET, as they collect network-borne malware, and also log where it comes from. The sensors are spread throughout the Australian IP space. On the left hand side is the attribute we are exploring, which in these examples is the attacking country, ASN, or MD5 of the malware. The color ribbons between the sensor and the attribute relate to the strength of the relationship. The thicker the ribbon, the more prominent the relationship.

Top 15 attacking countries
The first thing that jumps out here is the large proportion of attacks coming out of the Japanese IP space. This backs up the results of our geographic heatmap analysis. Note that all of the sensors almost without exception showed the most activity from Japan, and secondly from local Australian IP space.

Top attackers from overseas
This shows the top countries (excluding Australia). Note the dominance of Asian countries, in fact the top 6 countries are Japan, China, India, Taiwan, Cambodia and Mongolia, and only then comes the US. We are thinking this is attributable to the closeness and faster inter-country network links, and potentially to the 'closeness' in terms of IP space, particularly the first and second octet as some bots may scan local/adjacent subnets first, but we need to research this further.

Top 15-30 attacking countries
This is the top 15-30 attacking countries.

Most attacking ASN's
This shows the ASN (Autonomous System Number) network which attacked this set of sensors. You'll notice that one of our nodes (node 19) had a very busy time being attacked repeatedly by one particular ASN. We actually considered this an anomaly in terms of it's statistical relevance, and suspect the sensor may have been playing up, so this sensor/ASN pair was mostly dismissed from the rest of the analysis at this stage. I included this graph here simply to show how easily these anomalies show up with this visualization technique.

Most commonly seen malware
Here we see some of the more prevalent malware files being captured by the SensorNET. These are actually the files that will be infecting unprotected computers in Australia right now! I've abbreviated the MD5's to what I consider to be human readable for this graph.


For reference, here are some links to the Virustotal results for the top 5 pieces of malware (nominal name taken from F-Secure/Kaspersky)

Net-Worm.Win32.Kolabc.gau

Detection Result: 39 out of 40 Antivirus vendors (97.50%)

Backdoor.Win32.Rbot.aus

Detection Result: 38 out of 39 Antivirus vendors (97.44%)

Suspicious:W32/Malware!Gemini

Detection Result: 6 out of 36 Antivirus vendors (16.67%) ***

Backdoor.Win32.IRCBot.ddm

Detection Result: 40 out of 40 Antivirus vendors (100.00%)

Net-Worm.Win32.Kolabc.fia

Detection Result: 40 out of 40 Antivirus vendors (100.00%)

So there you have it, although a lot of network-borne malware has great AV coverage, some do not ***. This means that a 'defense in depth' strategy is still required. Do not rely solely on any one of the main technical security controls (Antivirus, Patching, Firewall) - use them ALL for the best protection !

What can you see in these diagrams ? Do you have any suggestions or observations ? let me know at ben@honeynet.org.au