Why Hybrid Analysis is not a marketing joke, but a useful technology

In 5 minutes you will know why Hybrid Analysis is useful - and not a marketing joke.

The case

As usual, we were checking reports uploaded to our malware analysis online service. Yesterday, we came by a report of sample* that is actually not that interesting, it is a typical dropper. The only significant aspect about the file at first sight is that it is relatively small (only ~14 KB) and tries to leave as little traces on the system as possible. Nevertheless, since everyone deserves a second chance, we decided to take a closer look and see if we couldn't find something that we could turn into a generic signature for malicious behavior. Generic signatures are great, because they apply to a broad variety of malware and obviously to new variants. We have seen a lot of samples that were uploaded, which were previously unknown to e.g. VirusTotal, but contained a lot of malicious behavior. Anyway, let's dive into the sample.

The first thing I always do is take a look at the signatures that matched. Then, I usually take a look at the network connections and process tree of analyzed processes. This obligatory check on the Hybrid Analysis section sometimes reveals quite interesting annotated disassembly listings (so called "Streams"). Since we can build signatures that fire on any kind of data found in the report, we come by some goodies from time to time.

Hybrid Analysis in action

The following screenshot is taken from the heuristically determined "most relevant" function found with the Hybrid Analysis engine:

We can see a typical pattern used by malware authors to "hide" strings from string-searching algorithms by building/concatenating a string character-by-character, often saving them in a local variable on the stack. This is quite an effective method, because the "final string" is concatenated at runtime so to speak and not lying in memory (i.e. even a process memory scan would not reveal the string, unless the stack/heap is snapshoted in just the right moment). Anyhow, usually these type of strings are API names and used for a GetProcAddress call to lookup the associated virtual address.

Turn it into something useful

The idea we had is the following: if we detect a lot (maybe more than 10) single characters being pushed onto the stack and a reference to GetProcAddress/LdrGetProcedureAddress in the same function/context, then we can assume someone is trying to hide a procedure name lookup from string scanning engines. So we whipped up a signature that does exactly that. Here it is after updating our online service and re-running the sample:

As we can see, there is enough indicators to make the decision that the behavior seen is malicious. This generic signature will fire on any sample uploaded to our service that contains the same or a similar trick. If you are interested in the signature code itself and how it was implemented, please get in touch through our contact form.

Final Notes

In this blogpost we learned that "Hybrid Analysis" (the combination of static analysis on memory dumps/binary files with dynamic runtime data/context information) can add valuable indicators that would have otherwise never been available. That is one of the reasons why VxStream Sandbox can extract more artifacts/indicators to trigger behavior signatures on than most other systems on the market. This does not mean we think our system is the perfect solution, but the underlying technology is solid and we believe that we are developing our software in the right direction.

The full report for this sample: https://www.hybrid-analysis.com/sample/342f9acdb9b89e963761fea283daccf0c7cacaf513a46fd09d9cc89223b9d978/

*SHA256: 342f9acdb9b89e963761fea283daccf0c7cacaf513a46fd09d9cc89223b9d978

Benchmarking some popular public malware analysis services regarding their "Anti-VM" technology

While checking submissions on our webservice we discovered that someone uploaded a "new" version of Pafish (by a0rtega). Pafish is a demo tool that performs typical anti-VM tricks in use by common and sophisticated malware. The new version of Pafish adds a lot of new VM and system trace checks, especially for VirtualBox. As is known, VirtualBox happens to be the default analysis environment of most sandboxes (including Cuckoo Sandbox's Malwr service and our own Hybrid-Analysis.com's free malware analysis service).

To be honest, the new version of Pafish did detect our virtual machine environment using some of the new methods - and it is impossible to prevent all types of detections ahead of time. More importantly, it is necessary to stay on top of the game and offer a software product that is agile and can adapt quickly. That is one of the principles we try to live up to and thus we always try to improve quickly and update VxStream Sandbox when necessary. A benchmarking tool like Pafish is a perfect development tool, because it's a very straightforward, comparable and easy way to stay on par with typical anti-VM methods. On a side note: the "new" release of Pafish is actually not that new, it was released at the beginning of this year - i.e. it's about two months old. One would think that well established and well known sandbox systems like Malwr, ThreatTrack or Comodo would have adapted by now - but to our surprise this is not the case.

First of all, this is how it should look if you run Pafish v0.4 (current state of VxStream Sandbox):

The green "OK" indicates that the specific check was passed and Pafish was not able to detect that it is running on a virtual machine. Here is one full report of VxStream Sandbox on our webservice (we copied the different console outputs into one screenshot to save space).

This is how the instance of Cuckoo Sandbox running on Malwr performed:

This is how Comodo Instant Malware Analysis performed:

The created "hi_" files indicate detection.

This is how ThreatExpert performed (it seems to be using VMWare as environment - and those checks are very old):

Sadly, Anubis failed to even parse the file, but to their defense the service is not being actively maintained it seems:

Finally, let me quote from ThreatTrack's main page before finishing up with this blogpost:

"Our solutions detect the world's most sophisticated malware – including Advance Persistent Threats (APTs) and targeted attacks – and empower you to completely eliminate those threats from your network."

I am assuming that "the world's most sophisticated malware" does not include simple VM checks *grin*. Nevertheless, I would like to underline that we are not claiming to include the "best anti-VM technology" possible (and that's a big difference to other malware analysis vendors), but at least we try to get the basic homework done. Big vendors that claim they include "high-end technology" should be doing the same (taking care of their homework) and spend less money on marketing bla-bla. Of course, it is a bit unfair to mention Cuckoo Sandbox with its malware service Malwr in this context, because it is based on an open-source tool, but I added it for completeness sakes, as it is the most popular free malware analysis service available.

Update (03/23/15): One of the Cuckoo Sandbox authors complained to us last week that they never claimed to have anti-vm technology included in their sandbox system, which might be true for the sandbox system itself, but this blogpost is focusing on public online services only (which is why we now updated the blogpost title, it might have been a bit inaccurate before), their output and not theoretical capabilities of the tools behind the services. The assumption that an online service demonstrates the latest and greatest version and capabilities of a sandbox system is valid one in our eyes, because a user probably expects an online service to try to analyze malware as good as possible, at least when it is about important aspects of malware analysis.

VxStream Sandbox and Hybrid-Analysis.com - Free Malware Analysis - Evolution

What's been happening?

This blogpost will focus around the evolution of the online webservice, the cool features we added over the past weeks and demonstrate them on a couple of real world samples.

In the middle of November last year (so about 10 weeks ago) the automated malware behavior analysis service at www.hybrid-analysis.com was released to the public and since then the delicate flower has been starting to blossom a bit.

So far we've had about a bit more than 2000 analyses with ~1900 unique files, more than 25k behavior signatures matched and we had 50k page views with 6k unique sessions from 107 different countries accessing our service. The overall bounce rate is only 54% with 38% returning visitors, so we have been addressing a targeted audience. This is how the world map looks, if you colorize countries by their frequency of access to the webservice (taken from Google Analytics):

We've also been noticing that people have been using our service more and more frequently during the "work days", so it is a good sign that people are utilizing our service at a professional level:

Also, we received quite a lot of feedback and feature suggestions, that we would like to present to you in the following. Our conclusion so far: we must be doing something right.

New Major Features

Let me start out saying: we added a lot. So many features, that we decided to address only the most important ones.

Supported File Types

Right from the beginning, we had a lot of documents/PDF files being uploaded that weren't supported at first, so we focused on adding to the list of supported file types. Right now you can upload any of the following filetypes:

Documents (new!): .doc, .docx, .rtf, .xls, .xlsx, .ppt, .pptx, .pdf

Executables: any kind of Windows PE file (.exe, .scr, .dll, .pif, .com, etc.).

All of the file types are detected automatically, so you can have any suffix, it will be ignored anyway. As we also had some users request to upload their files in different archive formats, we added support for some common archive types. Right now, you can upload any archive with or without the standard password ('infected') with the following archive format:

zip, 7z, xz, bzip2, gzip2, tar, wim

We also added support for uploading multiple files in a single archive. For more information on the special syntax required, please get in touch with us using the contact form on our company webpage.

Extended Document Parsing

Of course, an analysis system could just open e.g. a WORD document file and simply watch what's happening (network traffic, dropped files or new processes being created). Often though this approach requires a potentially embedded exploit to trigger, so we added parsers that extract VBA macros or embedded Javascript in PDF files for which the extracted data is piped to our signature interface. This comes in handy, especially if the document exploit doesn't trigger, because e.g. the shellcode/macro itself often contains valuable indicators already, even if it is obfuscated.

Here is an example of VBA macro extraction:

Click here to view the full report on Hybrid-Analysis.com

Improved YARA integration

One of our users creates YARA signatures based on extracted process memory strings, so we extended our YARA integration to run especially on these kind of strings to demonstrate the functionality. Actually, we use the small set of YARA rules in-place now to classify RATs:

Click here to view the full report on Hybrid-Analysis.com

Parsing Screenshots using Optical Character Recognition (OCR)

Although this idea isn't new, we decided to add some simple OCR parsing of the screenshots our analysis system takes in order to demonstrate how easy it is to add a complex process into the existing system and achieve a result that - due to its generic character - will apply also to unknown samples in the future.

Click here to view the full report on Hybrid-Analysis.com

Brushed-up the "Extracted Strings" section

Another nice addition was the brush-up of the extracted strings section, which now includes multiple tabs that list a pre-selected subset of all strings, strings extracted from screenshot parsing, strings extracted from a dropped file/the input sample (binary scan) or strings from the various analyzed processes. With the ability to download all memory-extracted strings we think the new "Extracted Strings" section adds more depth and overview. If you click the "Details" button, more information on the origin of the string (what type of file or event was the cause) is displayed:

Behavior Signatures

Our "daily business" is to add behavior signatures, as they trigger on a variety of different events and offer a quick overview and valuable indicators at the same time. Signatures can trigger on registry accesses, file operations, on strings, created mutants, a specific API call, on AV test results, extracted instructions from our disassembly "streams", and so forth and so on. We have been adding a lot of signatures since the service started (old report run-throughs might not include all the latest and greatest) and nearly doubled our signatures in-place with 180+ signatures serving right now. If you come by a sample that shows behavior you believe is not being reflected by a signature (i.e. you think a specific signature is missing), just let us know and we will add it if possible.

Summary

In this blogpost we presented the major features that were added over the past weeks to the service and sandbox system. Of course, we also had a lot of other smaller features/visual improvements that we implemented to the reports silently (e.g. the display on tablets/mobile phones) and improved the runtime monitor (e.g. better .NET sample loading and monitoring of system processes), but it would be out of the scope of this blogpost to list every tine addition/change. Overall, we believe that our system is moving in the right direction, also based on the feedback we have been getting. We have a very ambitious roadmap for 2015 and will let you know when we reach our next milestone.

We hope you enjoyed this brief summary and continue using our free service and don't stop on the feedback.

One last advertising side-note: If you are interested in purchasing the full version for an on-premise installation (the entire system is available as a standalone) and/or want to run your own private cloud service, please get in touch using our contact form and we will get back to you with more details.

Using Hybrid Analysis technology to extract artifacts from a fileless memory malware "Powerliks"

Just recently a user of our new malware analysis service at www.hybrid-analysis.com sent us an E-Mail regarding a report. In this case, it was an analysis of a "memory malware" (so called fileless malware) that tries to reside only in memory. The sample analyzed is quite extensively described in the following blogpost by Corey Harrell, which is a good read by the way.

Before diving into this blogpost, this is the sample and report we will be refering to (you can download the sample from the VxStream Sandbox Report at the top):

Sample SHA256: 4727b7ea70d0fc00f96a28de7fa3d97fa9d0b253bd63ae54fbbf0bd0c8b766bb

VxStream Sandbox report: https://www.hybrid-analysis.com/sample/4727b7ea70d0fc00f96a28de7fa3d97fa9d0b253bd63ae54fbbf0bd0c8b766bb/

Anyway, if we come accross some interesting samples or blogposts, we always try to see if we can replicate similar results with our system. If we do find a good example, then we like to share it now and then, because although automated malware analysis systems are not "perfect" or a "silver bullet", they can aid a professional in their daily work process, be it on a manual level or on a larger scale system that processes thousands of samples per day. Actually, the user mailing creates yara rules that are applied to a process memory scanning application so that unique artifacts are valuable information.

The most important artifacts that were deducted from the blogpost Corey wrote about is the C2 IP address 178.89.159.35 and a rundll32.exe call with a javascript shellcode:

(Source: Corey Harrell)

So, let's take a look at our own report now...

Finding artifacts using the VxStream Sandbox Report

After scrolling past the variety of signatures at the top of the report (which do contain quite some interesting matches, such as the PE timestamp that claims the program is from 1979 or the native function call to NtSetValueKey), we quickly arrive at the Hybrid Analysis section (take note of the new tooltip that indicates to users that they can click on an analyzed process to extract more detailed information) and see the Network Traffic as well:

So the C2 server 178.89.159.35 (as well as .34) was actually contacted. Good enough you say, but let us take a look at more details and click on the child process (PID 1784). Although we are not overwhelmed by details, the "Streams" section is especially interesting this time around. As the detected streams are sorted by an internally computed "Relevance" (based on the number of Strings/APIs, number of instructions, etc.), we find an interesting disassembly stream right at the beginning.

Stream 08093078-00001784-51075-88-00401BF5 (Executed: false, Relevance: 136.4)

Extracted from 08093078-00001784.00000001.8120281.400000.00000040.mdmp

Scrolling down a bit we actually find the shellcode mentioned in the blogpost quite easily:

Taking a look at another function we find the hardcoded C2 IP addresses and the query format:

The "Executed: true" note matches our expectation after having seen the network traffic earlier. Also, it is nice to see a reference to the memory dump file where the disassembly was extracted from, in case we want to analyze the correct dump file(s) in e.g. IDA or another professional disassembler.

If we would use the full XML report that contains all data in a structured format and parse the entire context, then it suddenly becomes possible to create even stronger, more context sensitive rules to detect and classify malware. For example, if one would consider not only a single keyword of the shellcode (like RunHTMLApplication), but the presence of NtSetValueKey API call or a certain order of API calls, then the overall detection ratio and false positive ratio could improve considerably. That is one of the advantages of combining memory dumps with static analysis and dynamic information.

Conclusion

This small blogpost outlined the advantages of Hybrid Analysis. In this blogpost we took a look at a memory malware sample and made an example of how quickly it is possible to determine and extract artifacts for rule generation. Not in the same fashion as would be done by a professional incident response team member, but with similar and evenly worthy results.

Welcome Hybrid-Analysis.com - Free Malware Analysis Service

Today we are proud to announce that a Beta version of VxStream Sandbox has been launched as part of our new malware analysis webservice at hybrid-analysis.com. This will remain a free service for everyone and it will give people an idea of our innovative Hybrid Analysis technology. The service is an ongoing project and is likely to receive feature updates in the future. Please keep in mind that it is a new service and needs time to develop.

Update: We just updated our service, improving some bugs with the network traffic display, the stability of the runtime monitor and a few other minor issues (the parameter values are now displayed in hexadecimal). Right now we are re-running the 100+ unique samples we received in the last 24 hours to update all reports. We will only be doing this during the initial phase, as re-running 100 samples takes only about 30 minutes, but later it would stall the system too much. Have fun!

Understanding "Torminator" Ransomware

It has been a while since I've posted to Payload Security's blog, because the dev team behind VxStream Sandbox has been quite busy finishing the web interface and focusing on the new report design. Nevertheless, it is our daily task to stress test the system with new samples. So as we were running samples through our system today we had an analysis that was quite on spot and perfect for a blogpost. The malware analysis we will be discussing here shortly outlines the strengths of our sandbox system: behavior signatures paired with a strong analysis engine that includes Hybrid Analysis. I know the term "Hybrid Analysis" is more mystic than obvious to most of the readers, but you should have a good idea after reading this article, so have no fear to continue reading. Before we go into the depths of malware, let me announce something very cool: within the next weeks we will be offering a demo "web interface" with navigable reports to give everyone an impression of the system overall.

UPDATE: You can view the full report at our new free malware analysis service here: http://www.hybrid-analysis.com/sample/f0a068c48d260ebd182861e114edfb4383f922ec8186fa6b9ffb247a37da36eb/

Tumble down the rabbit hole (with VxStream Sandbox) 

The sample we will be looking at is labeled by Kaspersky as "Trojan-Spy.Win32.SpyEyes.aryc" (SHA256: f0a068c48d260ebd182861e114edfb4383f922ec8186fa6b9ffb247a37da36eb), but we will call it "Torminator", because it destructs the system (e.g. deletes all shadow copies), encrypts user files and then asks for a ransom to decrypt/restore the files (i.e. a typical ransomware). The restore page(s) are TOR websites (e.g. https://<random>.torminater.com/).

The first thing we do when we test our system (besides reading our own report) is check what the "competition" detected in order to determine the quality of our analysis. Unfortunately in this case, malwr (the free service running Cuckoo Sandbox) did not so well (failed analysis happens to us all the time):

Please note that today, some malware is even aware of analysis system software presence (detecting e.g. third-party tools like Wireshark, AutoIt etc.) and falls asleep not executing its payload (which is why "dormant code detection" as implemented by Hybrid Analysis is so important). Also, userland hooking engines (as utilized by Cuckoo Sandbox) will always have detection issues (a bit like Heisenberg's uncertainty principle that describes the disturbance triggered by the act of observation), which is somewhat countered by a whole list of advantages (such as fast portability between Windows versions compared to kernel code, sometimes stability or being a lot closer to the instructions and data), but that is another discussion. Should you be interested, you can see the full malwr report here. On a side note: I did not want to downtalk malwr or its free service (how can anyone ever complain about free work?!), but the comparison shows that different systems can have different results and it is always good to rely on a variety of tools, even if they are from the same category ("forensic malware analysis").

When we take a look at the VxStream Sandbox report, we always start out with the behavior signatures as based on that we can get a very good idea of what the malware does, what functionality it contains and get entrypoints for deeper analysis. The more we scroll down, the deeper we fall into the rabbit hole and get to know about more details.

An interesting malicious signature that immediately pops into our face is the "Deletes volume snapshots" signature, as it characterizes a unique feature (dropping files and writing memory into foreign processes is common among malware). When we uncollapse the signature we obtain some more details about the signature:

As we can see, the "volume snapshots" are deleted using vssadmin.exe using the commandline "Delete Shadow /All /Quiet" (note the /Quiet).

The next step I surf to is usually the "Screenshots" and "Hybrid analysis" section (see menu on the right), because they contain visual information (which is always interesting) and a process tree of the original sample that contains infected and newly created processes. Also, I can take a look at in-depth data about each monitored process. In this case, this is how the process tree looks:

As we can see, the malware injects itself into explorer (or creates a new instance) and hides itself in a svchost process (quite typical for malware) to then create a notepad instance, delete shadow copies of the harddrive, disable recovery mode and set the boot policy to "ignore all failures". Not very nice. ;-) If we take a look at the second screenshot below we can see that a notepad instance with the title "DECRYPT_INSTRUCTIONS" and some informational text (in german) is created:

What it basically says is that all files were encrypted with a RSA-2048 key and it is possible to recover the files if one visits some "personal website" and pays a bunch of $$$ to some crooks. Typical ransomware, just that a TOR service is being used. More interesting is to take a look at how the software works. If you are interested in very specific details, such as the logged API calls, all registry accesses, created mutants, touched handles or "streams" (more on that later), then it is possible to click on any process in the process tree and navigate the in-depth "sub-reports" (on a per process basis). Here are two examples:

In the example above we can see a simple list of API calls. What is nice is that there is some additional "meta parameters" (those that are in brackets) containing additional information (e.g. the pathway connected to a handle). Let's take a look at some "streams" (basically annotated disassembly listings):

As can be seen, disassembly instructions were extracted from a memory dump file and annotated accordingly. What is nice is that the "vsaadmin.exe Delete Shadows /All /Quiet" call was reconstructed automatically by the Hybrid Analysis engine using stack simulation and data flow analysis. The malware author tried to hide the string adding each character to the local stack frame, because the string cannot be detected using a binary search in that case. Luckily, we are equipped with a powerful tool to counter-attack that measure. ;-)) One additional note: the screenshots presented here are from the web interface, i.e. they do not represent the "full report" which contains all the gathered information of the analysis system. Full reports are available in JSON, XML and HTML currently.

Anyway, if we scroll to the network traffic section we get a good overview including a graphical "world map" that highlights countries that were contacted:

If we scroll down even further we can take a look at the dropped files and download them as well:

As we can see, two interesting dropped files are available for further inspection. An alleged "jpg" file that is actually a COM executable and the file named w7-32@pumma[1].txt ("w7-32" is our computer name) which is a configuration file. Adding to the monitored processes and memory dumps, dropped files can provide valuable indicators for (automatic) post-processing. Please note that all dropped files are parsed by Hybrid Analysis as part of a normal analysis already (i.e. disassembly streams and strings/API calls are extracted). Scroll down the report even further down, there is a list of extracted strings (from a variety of sources) and some informational notices of the analysis system itself. That's it.

One last note: to be fair, we added the ransomware signature "Deletes volume snapshots" after we found the sample and ran it for the first time, but it only took two minutes to add the new signature script and five minutes to re-run the adapted system, because the signature interface is very open andcan be scripted easily. This iterative approach is what some people call agile security.

Conclusion

A malware analysis system that provides reports which are straight forward outline, but the option to look at in-depth behvaior at the same time is a very good basis to understand and adapt to malware threats today. Generic behavior signatures are a strong and powerful tool, but they all depend on the underlying system to provide data that can be used as a trigger. That is why technologies such as Hybrid Analysis that can extract strings, API calls, shellcode and dormant code are invaluable on a large scale, because the overall data will have a higher quality. In this blogpost we outlined how easy it is to understand the most important aspects of malware within 10 minutes. For deeper analysis, the disassembly listings and the provided context files provide a good entrypoint. The reports VxStream Sandbox generates are more than just an impression.

New Feature Benchmark and first Report UI Preview

This will be just a very short blogpost. Our VxStream Sandbox malware analysis system development process is progressing well. Currently, we have completed development of a Beta Version that is being tested on large scale systems and released a feature product comparison chart to give people an idea of what will be included in the first version. Go and check it out at our new VxStream product page.