Welcome!

Cloud Security Authors: Harry Trott, Kevin Jackson, Elizabeth White, Nishanth Kadiyala, Stackify Blog

Related Topics: Cloud Security, Microservices Expo

Cloud Security: Tutorial

Malware Analysis | Part 2

Find hidden and covert processes, clandestine communications, and signs of misconduct on your network

In a previous article [1], I described how to obtain a memory image from a Windows computer that would allow forensic analysis. I briefly discussed using F-Response TACTICAL [2] to get the memory image, and then Volatility [3] and Mandiant Redline [4] for further investigation. In this paper, I dive more deeply into Redline and Volatility.

To begin, I review a raw memory dump of a known malware variant (see the "Malware Image" box) with Mandiant Redline. After firing up Redline, I chose By Analyzing a Saved Memory File under Analyze Data and browsed to the location of the memory image. Next, I edited my script to include Strings for both Process Listing and Driver Enumeration. Finally, I chose a destination to store the output for future analysis and to analyze memory dumps.

Malware Image
The malware image I am using in this article is a variant found by the Palo Alto PA-5000 series firewall [5] on a Windows box in our network, which was sent for further investigation to a sandbox that Palo Alto uses for such cases. Moments later, I received email telling me that malware was discovered by Palo Alto WildFire analysis [6].

WildFire identifies unknown malware, zero-day exploits, and advanced persistent threats by executing them directly in a scalable, cloud-based, virtual sandbox environment. The report, which goes into detail about what the malware has done, gave me a link to VirusTotal [7], used to score the executable for maliciousness, along with a PCAP file of network traffic generated by the malware. I was able to download the known malware variant and execute it on my closed test VM network for observation; then, I used both Volatility and Mandiant Redline to research the culprit.

Redline
As I explained in the previous article, I took two different memory dumps from a Windows XP system: one after I executed the malware infection, and another after I rebooted the system. In comparing the two images, I noticed some differences. Remember, malware usually tries to hide in plain sight, attempting to appear legitimate, or uses rootkit techniques to hide from the view of normal analysis tools.

After installing the malware and completing a reboot, Redline revealed three process names (jh, process ID [PID] 38533; svchost.exe, PID 1560; and WScript.exe, PID 1744). Process jh (PID 38533) was spawned by a parent process - [Not Available] (528) - with a start time of 1601-01-01 00:00:00Z (Figure 1). Double-clicking on the Process name (jh ) provides access to the detailed view.

Figure 1: Process jh PID 38533.

Choosing Hierarchical Processes in the Analysis Data panel showed that svchost.exe (PID 1560) was spawned by WScript.exe (PID 1744). Hierarchical Processes lists running processes in a tree format, showing which processes started other processes (Figure 2). If you take a look at the Process Metadata for jh in Figure 1, you'll see that it has an MRI Score of 61, as does svchost.exe (Figure 2, second column).

Figure 2: Hierarchical Processes

Taking a look at the MRI Score and see that Process Name jh and dsvchost.exe have an MRI Score of 61. The MRI Score or Malware Risk Index Report which is accessed by double-clicking a process-related item to see its detailed view, and selecting the MRI Report tab at the bottom of the window, the higher the number the larger the risk. Since these two processes started after the install of malware it is likely they are bad. Compare the Start Time of svchost.exe PID1560 with the other svchost processes (see Figure #3) it appears to have started about 30 min after the other svchost processes.

Figure 3: Comparing different svchost processes running with Hierarchical Processes

Figure 4: Finding user account and full path of the process binary

The parent process of svchost.exe PID 3028 is WScript.exe PID 1736, this is discovered by looking at the Hierarchical Processes. After clicking on WScript.exe PID 1736 we discover the user account that was logged on when the process was spawned and the full path of the process binary (see Figure #4). Next click on the Handles tab located below and view the Handle Names, notice the Untrusted status (see Figure #5). Using Redline to check for signed code may reveal suspicious executables.

Figure 5: Viewing a process and its Handles

This gives us the ability to show all handles including ones identified as untrusted. To look for evidence of code injection review Memory Sections located under Analysis Data -> Processes to see memory pages for every process. This particular malware contains no injected memory sections that Redline can find.

Finally use the "Strings" output to find additional evidence. You can search for interesting things like http://, https://, ".exe", or like the example below search for "cmd.exe" being run by WScript.exe (see Figure #6).

Figure 6: Use the Redline "Strings" output to find additional evidence

Other interesting strings to search for would be looking for common Windows system and network commands such as "finger", "net use", "netstat", etc.

Understanding normal activity is key when you start looking for badness. If you don't know what normal traffic activity looks like then you will be lost when trying to find said malware. One way is to better understand the operating system that you are doing analysis on, in this case windows. Understanding windows process structure helps, for example: csrss.exe is created by an instance of smss.exe and will have two or more instances running. The start time is within seconds of boot time for the first 2 instances (for Session 0 and I). Start times for additional instances occur as new sessions are created, although often only Sessions 0 and I are created. This information is available on the SANS DFIR poster. Looking at a Hierarchical Processes in Redline will reveal an instance of smss.exe that will spawn csrss.exe. Another item that is interesting to research is svchost.exe whose parent process is services.exe and runs five or more instances. It is used for running service DLLs. Windows will run multiple instances of svchost.exe, each using a unique "-k" parameter for grouping similar services. Malware authors often take advantage of the ubiquitous nature of svchost.exe and use it either directly or indirectly to hide their malware. They use it directly by installing the malware as a service in a legitimate instance of svchost.exe. Alternatively, they use it indirectly by trying to blend in with legitimate instances of svchost.exe, either by slightly misspelling the name (scvhost.exe) or spelling it correctly but placing it in a directory other than System32. All default installations of Windows 7, all service executables and all service DLLs are signed by Microsoft. This information is also available on the SANS DFIR poster and would be very helpful to review.

After reviewing this memory image with Mandiant Redline there is no smoking gun that can definitively say that this image has malware. Now we will take a look at an open source tool called Volatility.

Move over to the SIFT workstation, which was the device that took the image off the windows machine with the f-response tool. Open up a terminal and change directory to where the case files are located and find out the image information on the image file that you are interested in (see Figure 7).

Figure 7: Running volatility to discover the imageinfo of the memory image

The imageinfo output gives you information about the image (memory dump). The details about this image file can be seen in Figure #7. The suggested profile that you should pass as the parameter to -profile=PROFILE; there may be more than one profile suggestion if profiles are closely related. You can figure out which one is more appropriate by checking the "Image Type" field, which is blank for Service Pack 0 and filled in for other Service Packs.

To find the processes and DLLs use the following Volatility commands:

  • $ vol.py --profile=WinXPSP2x86 -f remote-system-memory11.img pslist

This command will list the processes of a system. It does not detect hidden or unlinked processes.

  • $ vol.py -profile=WinXPSP2x86 -f remote-system-memory11.img pstree

This command will view the process listing in a tree form. Child processes are indicated using indention and periods (see Figure #8).

  • $ vol.py -profile=WinXPSP2x86 -f remote-system-memory11.img psscan

This command will enumerate processes using pool tag scanning. This can find a process that previously terminated (inactive) and processes that have been hidden or unlinked by a rootkit (see Figure #9).

  • $ vol.py -profile=WinXPSP2x86 -f remote-system-memory11.img dlllist

This command will display a processes loaded DLLs and shows the command line used to start the process (including services) along with the DLL libraries for the process. Look for strange DLLs that have been injected into legitimate processes and DLLs with suspicious looking names.

This will produce a large output so if you narrow down your search to a specific process and want the DLL associated with it try the following command:

$ vol.py -profile=WinXPSP2x86 -f remote-system-memory11.img dlllist -p 1764

This command will display the process loaded DLLs for PID 1764 WScript.exe see Figure 10.

Figure 8: Example of using pstree, notice the child process indicated by the indention and periods

Figure 9: Example of using psscan, this can find processes that are hidden

Figure 10: Example of using dlllist to display the DLLs for the process WScript.exe

Volatility has many different features organized by plugins and categories. These categories include Image identification, processes and DLLs, processes memory, kernel memory and objects, networking, registry, crash dumps, hibernation, and malware/rootkits. We will look at the networking category below. To view active connections, use the connections command or to find connection structures using pool tag scanning, use the connscan command. Pool scanners is a technique when a piece of memory is allocated in windows, it's often allocated with a special tag which corresponds to the drive or subsystem to allocate the memory. You often find previous connections that have since been terminated. This command works for Windows XP and Windows 2003 Server only see Figure 11.

$ vol.py -profile=WinXPSP2x86 -f remote-system-memory005.img connscan

Figure 11: Output of connscan command notice the two connections to 10.10.3.180

So what is trying to communicate on the network from our Windows XP box to an internal 10.10.3.180? I don't have anything with that IP address on this network. Need to further investigate PID 1792 & 132. Running a psscan to enumerate processes does not show PID 1792 so that is a dead process but 132 shows up as svchost.exe. When I run a pstree to learn more information such as parent process indicated using indention and periods see Figure 12.

Figure 12: Svchost.exe has a parent process of wscript.exe which is child of explorer.exe

Another plugin available in Volatility called malfind extracts injected DLLs, injected code, unpacker stubs, API hook trampolines. Scans for any ANSI string, Unicode string, regular expression, or byte sequence in process or kernel driver memory. The syntax for using this plugin is below:

$ vol.py -profile=WinXPSP2x86 -f remote-system-memory005.img malfind -D memory/

This will dump all the processes with injected code in the directory called memory and when we are finished we can see all the files in Figure 13.

Figure 13: Results of using malfind has produced a number of dmp files for analysis

I will upload these dmp files to VirusTotal to see if we can identify any know issues. The only suspected dmp file that was flagged by virustotal (see Figure #14) was 0x370000 which is our f-response tool used to extract our memory image. This is a false positive.

Figure 14: Results from VirusTotal after uploading 0x370000.dmp files (f-response) that was generated after running malfind

Conclusion

After taking a SANS FOR508: Advanced Computer Forensic Analysis and Incident Response and learning the tools described in this article I started doing research on malware found on our network by Palo Alto firewalls and I used an example found by their firewall to research. This was a challenge because there were no obvious red flags when doing the investigation. Which is why I used this example to show that you will need to dig deep. What I learned:

  1. The memory image taken before infection showed communication with the windows box and the SIFT workstation, no other connections.
  2. The memory image taken after the infection showed communication with a 10.10.3.180 (2 instances) which is an internal IP address that I don't have on my test network.
  3. The two Process IDs (PID) that were related to IP address 10.10.3.180 are 1792 which was a dead process, and 132 svchost.exe which was the wscript.exe and has a parent process of 1648 explorer.exe.
  4. The svchost.exe (PID 132) is a generic host process for Windows Services and is used for running service DLLs and should always be a child of services.exe. It is a child process of wscript.exe and a clear indicator of wrong doing (see Figure 12).

More Stories By David Dodd

David J. Dodd is currently in the United States and holds a current 'Top Secret' DoD Clearance and is available for consulting on various Information Assurance projects. A former U.S. Marine with Avionics background in Electronic Countermeasures Systems. David has given talks at the San Diego Regional Security Conference and SDISSA, is a member of InfraGard, and contributes to Secure our eCity http://securingourecity.org. He works for Xerox as Information Security Officer City of San Diego & pbnetworks Inc. http://pbnetworks.net a Service Disabled Veteran Owned Small Business (SDVOSB) located in San Diego, CA and can be contacted by emailing: dave at pbnetworks.net.

@ThingsExpo Stories
The current age of digital transformation means that IT organizations must adapt their toolset to cover all digital experiences, beyond just the end users’. Today’s businesses can no longer focus solely on the digital interactions they manage with employees or customers; they must now contend with non-traditional factors. Whether it's the power of brand to make or break a company, the need to monitor across all locations 24/7, or the ability to proactively resolve issues, companies must adapt to...
SYS-CON Events announced today that Datanami has been named “Media Sponsor” of SYS-CON's 21st International Cloud Expo, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Datanami is a communication channel dedicated to providing insight, analysis and up-to-the-minute information about emerging trends and solutions in Big Data. The publication sheds light on all cutting-edge technologies including networking, storage and applications, and the...
SYS-CON Events announced today that EnterpriseTech has been named “Media Sponsor” of SYS-CON's 21st International Cloud Expo, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. EnterpriseTech is a professional resource for news and intelligence covering the migration of high-end technologies into the enterprise and business-IT industry, with a special focus on high-tech solutions in new product development, workload management, increased effi...
SYS-CON Events announced today that MobiDev, a client-oriented software development company, will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place October 31-November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. MobiDev is a software company that develops and delivers turn-key mobile apps, websites, web services, and complex software systems for startups and enterprises. Since 2009 it has grown from a small group of passionate engineers and business...
SYS-CON Events announced today that GrapeUp, the leading provider of rapid product development at the speed of business, will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place October 31-November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Grape Up is a software company, specialized in cloud native application development and professional services related to Cloud Foundry PaaS. With five expert teams that operate in various sectors of the market acr...
SYS-CON Events announced today that Ayehu will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on October 31 - November 2, 2017 at the Santa Clara Convention Center in Santa Clara California. Ayehu provides IT Process Automation & Orchestration solutions for IT and Security professionals to identify and resolve critical incidents and enable rapid containment, eradication, and recovery from cyber security breaches. Ayehu provides customers greater control over IT infras...
Artificial intelligence, machine learning, neural networks. We’re in the midst of a wave of excitement around AI such as hasn’t been seen for a few decades. But those previous periods of inflated expectations led to troughs of disappointment. Will this time be different? Most likely. Applications of AI such as predictive analytics are already decreasing costs and improving reliability of industrial machinery. Furthermore, the funding and research going into AI now comes from a wide range of com...
In this presentation, Striim CTO and founder Steve Wilkes will discuss practical strategies for counteracting fraud and cyberattacks by leveraging real-time streaming analytics. In his session at @ThingsExpo, Steve Wilkes, Founder and Chief Technology Officer at Striim, will provide a detailed look into leveraging streaming data management to correlate events in real time, and identify potential breaches across IoT and non-IoT systems throughout the enterprise. Strategies for processing massive ...
SYS-CON Events announced today that Conference Guru has been named “Media Sponsor” of SYS-CON's 21st International Cloud Expo, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. A valuable conference experience generates new contacts, sales leads, potential strategic partners and potential investors; helps gather competitive intelligence and even provides inspiration for new products and services. Conference Guru works with conference organi...
SYS-CON Events announced today that Cloud Academy named "Bronze Sponsor" of 21st International Cloud Expo which will take place October 31 - November 2, 2017 at the Santa Clara Convention Center in Santa Clara, CA. Cloud Academy is the industry’s most innovative, vendor-neutral cloud technology training platform. Cloud Academy provides continuous learning solutions for individuals and enterprise teams for Amazon Web Services, Microsoft Azure, Google Cloud Platform, and the most popular cloud com...
SYS-CON Events announced today that CA Technologies has been named "Platinum Sponsor" of SYS-CON's 21st International Cloud Expo®, which will take place October 31-November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. CA Technologies helps customers succeed in a future where every business - from apparel to energy - is being rewritten by software. From planning to development to management to security, CA creates software that fuels transformation for companies in the applic...
SYS-CON Events announced today that TMC has been named “Media Sponsor” of SYS-CON's 21st International Cloud Expo and Big Data at Cloud Expo, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Global buyers rely on TMC’s content-driven marketplaces to make purchase decisions and navigate markets. Learn how we can help you reach your marketing goals.
Internet of @ThingsExpo, taking place October 31 - November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with 21st Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The Internet of Things (IoT) is the most profound change in personal and enterprise IT since the creation of the Worldwide Web more than 20 years ago. All major researchers estimate there will be tens of billions devic...
"MobiDev is a Ukraine-based software development company. We do mobile development, and we're specialists in that. But we do full stack software development for entrepreneurs, for emerging companies, and for enterprise ventures," explained Alan Winters, U.S. Head of Business Development at MobiDev, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
SYS-CON Events announced today that Telecom Reseller has been named “Media Sponsor” of SYS-CON's 21st International Cloud Expo, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Telecom Reseller reports on Unified Communications, UCaaS, BPaaS for enterprise and SMBs. They report extensively on both customer premises based solutions such as IP-PBX as well as cloud based and hosted platforms.
We build IoT infrastructure products - when you have to integrate different devices, different systems and cloud you have to build an application to do that but we eliminate the need to build an application. Our products can integrate any device, any system, any cloud regardless of protocol," explained Peter Jung, Chief Product Officer at Pulzze Systems, in this SYS-CON.tv interview at @ThingsExpo, held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA
SYS-CON Events announced today that IBM has been named “Diamond Sponsor” of SYS-CON's 21st Cloud Expo, which will take place on October 31 through November 2nd 2017 at the Santa Clara Convention Center in Santa Clara, California.
Multiple data types are pouring into IoT deployments. Data is coming in small packages as well as enormous files and data streams of many sizes. Widespread use of mobile devices adds to the total. In this power panel at @ThingsExpo, moderated by Conference Chair Roger Strukhoff, panelists looked at the tools and environments that are being put to use in IoT deployments, as well as the team skills a modern enterprise IT shop needs to keep things running, get a handle on all this data, and deliver...
The current age of digital transformation means that IT organizations must adapt their toolset to cover all digital experiences, beyond just the end users’. Today’s businesses can no longer focus solely on the digital interactions they manage with employees or customers; they must now contend with non-traditional factors. Whether it's the power of brand to make or break a company, the need to monitor across all locations 24/7, or the ability to proactively resolve issues, companies must adapt to...
SYS-CON Events announced today that Enzu will exhibit at SYS-CON's 21st Int\ernational Cloud Expo®, which will take place October 31-November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Enzu’s mission is to be the leading provider of enterprise cloud solutions worldwide. Enzu enables online businesses to use its IT infrastructure to their competitive advantage. By offering a suite of proven hosting and management services, Enzu wants companies to focus on the core of their ...