We've been so busy improving VxStream Sandbox and the surrounding technology that we have been having a bit of an on-off relationship with our blog. Today we wanted to catch up a bit and let everyone know what we have been up to, who have not been following extremely closely. Besides visible changes, there has also been a lot of improvements going on in the backend: for example, we have been working heavily on automating the deployment of the standalone system, as it is our vision to enable a fully unattended installation of the entire system (i.e. not only the server/host environment, but also the installation and configuration of Windows, including third-party dependencies). Currently, taking a server from scratch, it is possible to setup the entire system in less than two hours to a state that allows a first end-to-end analysis. Either way, in the following we will try to touch on the most notable improvements and additions that were implemented over the past five months.
New Input Formats
As we see VxStream Sandbox as an "engine" that should retain a high level of usability, but more importantly support as many input and common output formats as possible. That means supporting also uncommon file formats, such as Outlook msg or processing MIME types, which we added just recently. In case you wonder what we support exactly, you can always find an up-to-date list in the
FAQ, a good resource to answer some other questions as well. Currently, the supported formats are: any kind of PE (.exe, .scr, .pif, .dll, .com, .cpl, etc.), Office (.doc,
.docx, .ppt, .pptx, .xls, .xlsx, .rtf), PDF, APK, executable JAR,
Windows Shortcut (.lnk), Windows Help (.chm), Javascript (.js),
Shockwave Flash (.swf), Powershell (.ps1, .psd1, .psm1), MIME RFC 822
(*.eml) and Outlook *.msg files.
New Output Formats
Similar improvements took place to the list of supported output formats. Most notably is the addition of
MAEC 4.1 and
OpenIOC 1.1, which can now be generated and used to share indicators. One has to say that though that, unfortunately, Mandiant (the consortium behind the OpenIOC format) surprisingly do not support their own latest OpenIOC 1.1 format with their core tools, foremost the OpenIOC editor in its latest version 2.2 (published end of 2012 while the latest format definition was released in 2014). So our decision to go for the latest definition was in parts a technical one, but more of a business one at the end of the day. We hope to create more incentives for the FireEye owned company to synchronize their toolsets with their own standards. As the transition from 1.0 to 1.1 is more cosmetic than fundamental, this would probably be a good weekend project for the next intern, but enough of this rant. What else did we change? Well, other output related improvements were internal ones focusing around the HTTP based API of VxStream Sandbox that we plan on possibly making available to the public later this year.
More 'Hybrid Analysis' Integration (Example)
We made a
tweet about it end of January (we often tweet features or things we find interesting on our Twitter account, which is probably a better chronlogy of what we have done than this blogpost) about a new integration we added: whenever we find a URL or IP address (i.e. something that may be related to network behavior), a 'Memory Forensics' sub-section in the 'Network Data' area of the report will appear:
While this is just a "tiny" feature, it does help make possible network intelligence available to the analyst (as the network endpoints may or may not have been used for live communication, as they are based purely on memory dump analysis) and give entrypoints for a deeper analysis at the same time. Please note that in the full version of VxStream Sandbox you can download associated memory dumps for a follow-up analysis.
Improved Static Analysis
We continued to improve a variety of static analysis techniques that we apply to input samples such as PDF file(s) and the associated URL extraction and evaluation. More notably, we improved our proprietary Javascript and VBA/VBS deobfuscator engines and also added a proof of concept Dridex config extractor (built upon initial work by malware.lu, much kudos from here):
Improved False Positives
We noticed that one major challenge for any sandbox system that does not have a huge whitelist in the backend is to reduce the false positive ratio. For example, often installers (such as Skype) will behave in a way that is very difficult to put apart from malware without a reputation database or whitelists. Installers are often packed, drop executable files, spawn child processes, persist themselves, possibly download files from abnormal ports, may connect to a wide range of servers, try to install system services and so on. The same goes for some of the latest document "launcher applications", such as Acrobat XI, which come with their own sandboxing technology and make it difficult to put aside whitenoise, as the propriertary code behaves quite 'naughty' itself (e.g. patching the own process, and so on). Thankfully, VxStream integrates with NSRL and can read the "isgoodware" flag of VirusTotal, so we got a handle on many false positives through improved verdict adjustments regardless of the actual behavior. Over the past months and even weeks, we have been continuing to tweek the system in this respect, something that is not easily visible from the online reports of the free webservice. For example, we will lower the verdict from "malicious" to "suspicious" if we find a variety of artifacts that we learned are
combined a strong statement of belonging to benign artifacts. A possible example: if a file is clean on a multiscan AV engine, has a valid certificate, drops files that are explicitly flagged as benign, executed without crashing and showed reasonable behavior, then we could apply such verdict downgrading. For us the hurdle for a downgrade has to be very high, as we believe a false negative to be far worse than a false positive.
Another addition is the "no verdict" verdict that some may have noticed, which essentially means "there is too little data to make a reliable determination of whether or not the file is benign or malicious". We believe industry should be open and fair about verdicts and let's be honest: we have all encountered a situation in life where we simply
don't know. Instead of pretending we have a good, reliable opinion of a file (and sometimes it is very difficult), we believe it is only fair to communicate that to the user and processing system. This also has the benefit that the system user may decide on a more restrictive or casual policy on those cases. Unfortunately, we cannot disclose under which conditions the "no verdict" appears.
Metadefender OPSWAT Integration
While on the one hand we are trying to make VxStream Sandbox a great "engine" for processing any file and making the data available in any format, we also believe it is equally important to integrate and support interfaces of a variety of excellent third party tools, such as VirusTotal - and now also -
OPSWAT Metadefender (formerely 'Metascan'). Giving customers the ability to utilize a purchased API key from one of these vendors is a great oportunity to add yet another layer to your defenses.
URL analysis
This was a major feature on our roadmap for quite some time and while it is always a work in progress (similar to Google's "Forever Beta"), we have made the URL analysis feature available on the front page of the webservice to give it the best stress test you can have, which is being available in the wild:
While the URL analysis is sitll in its baby shoes, there is some basic exploit detection through browser emulation, file extraction, browser process monitoring and we implemented common processing of files extracted in the context of a webpage (VirusTotal lookup, YARA signature matching) as-if it were a dropped file on a regular analysis.
Note: we had to take the feature down on the free webservice, as we have been getting massive amounts of submissions and accesses to our webservice, which is a single server at this point. The URL analysis is available to all private cloud customers, which are also hosted on another server. More information here: https://www.vxstream-sandbox.com/
Final Words
In this blogpost we highlighted a few of the new supported file formats, integrations and features added over the past months. As Payload Security consists of 90% developers and 10% marketing/sales, we often move faster than we can communicate things in fancy flyers or conference 'silver partner' stands (and you will most likely not see us at any conference anytime soon). The outlined changes only touched on a subset: there has been many other improvements, hidden ones (such as anti-VM detection technology improvements or deep packet inspection), things like MISP pathway normalization, many new indicators, code refactorings, optimizations, tons of bug fixes (one bug every 1k LOC, remember?) and so on. We hope you will continue to enjoy our free service at
https://www.hybrid-analysis.com/ and feel free to follow us on Twitter for a more frequent news update.
