Welcome!

Cloud Security Authors: Zakia Bouachraoui, Elizabeth White, Liz McMillan, Pat Romanski, Yeshim Deniz

Related Topics: Cloud Security, Microservices Expo

Cloud Security: Tutorial

Malware Analysis | Part 2

Find hidden and covert processes, clandestine communications, and signs of misconduct on your network

In a previous article [1], I described how to obtain a memory image from a Windows computer that would allow forensic analysis. I briefly discussed using F-Response TACTICAL [2] to get the memory image, and then Volatility [3] and Mandiant Redline [4] for further investigation. In this paper, I dive more deeply into Redline and Volatility.

To begin, I review a raw memory dump of a known malware variant (see the "Malware Image" box) with Mandiant Redline. After firing up Redline, I chose By Analyzing a Saved Memory File under Analyze Data and browsed to the location of the memory image. Next, I edited my script to include Strings for both Process Listing and Driver Enumeration. Finally, I chose a destination to store the output for future analysis and to analyze memory dumps.

Malware Image
The malware image I am using in this article is a variant found by the Palo Alto PA-5000 series firewall [5] on a Windows box in our network, which was sent for further investigation to a sandbox that Palo Alto uses for such cases. Moments later, I received email telling me that malware was discovered by Palo Alto WildFire analysis [6].

WildFire identifies unknown malware, zero-day exploits, and advanced persistent threats by executing them directly in a scalable, cloud-based, virtual sandbox environment. The report, which goes into detail about what the malware has done, gave me a link to VirusTotal [7], used to score the executable for maliciousness, along with a PCAP file of network traffic generated by the malware. I was able to download the known malware variant and execute it on my closed test VM network for observation; then, I used both Volatility and Mandiant Redline to research the culprit.

Redline
As I explained in the previous article, I took two different memory dumps from a Windows XP system: one after I executed the malware infection, and another after I rebooted the system. In comparing the two images, I noticed some differences. Remember, malware usually tries to hide in plain sight, attempting to appear legitimate, or uses rootkit techniques to hide from the view of normal analysis tools.

After installing the malware and completing a reboot, Redline revealed three process names (jh, process ID [PID] 38533; svchost.exe, PID 1560; and WScript.exe, PID 1744). Process jh (PID 38533) was spawned by a parent process - [Not Available] (528) - with a start time of 1601-01-01 00:00:00Z (Figure 1). Double-clicking on the Process name (jh ) provides access to the detailed view.

Figure 1: Process jh PID 38533.

Choosing Hierarchical Processes in the Analysis Data panel showed that svchost.exe (PID 1560) was spawned by WScript.exe (PID 1744). Hierarchical Processes lists running processes in a tree format, showing which processes started other processes (Figure 2). If you take a look at the Process Metadata for jh in Figure 1, you'll see that it has an MRI Score of 61, as does svchost.exe (Figure 2, second column).

Figure 2: Hierarchical Processes

Taking a look at the MRI Score and see that Process Name jh and dsvchost.exe have an MRI Score of 61. The MRI Score or Malware Risk Index Report which is accessed by double-clicking a process-related item to see its detailed view, and selecting the MRI Report tab at the bottom of the window, the higher the number the larger the risk. Since these two processes started after the install of malware it is likely they are bad. Compare the Start Time of svchost.exe PID1560 with the other svchost processes (see Figure #3) it appears to have started about 30 min after the other svchost processes.

Figure 3: Comparing different svchost processes running with Hierarchical Processes

Figure 4: Finding user account and full path of the process binary

The parent process of svchost.exe PID 3028 is WScript.exe PID 1736, this is discovered by looking at the Hierarchical Processes. After clicking on WScript.exe PID 1736 we discover the user account that was logged on when the process was spawned and the full path of the process binary (see Figure #4). Next click on the Handles tab located below and view the Handle Names, notice the Untrusted status (see Figure #5). Using Redline to check for signed code may reveal suspicious executables.

Figure 5: Viewing a process and its Handles

This gives us the ability to show all handles including ones identified as untrusted. To look for evidence of code injection review Memory Sections located under Analysis Data -> Processes to see memory pages for every process. This particular malware contains no injected memory sections that Redline can find.

Finally use the "Strings" output to find additional evidence. You can search for interesting things like http://, https://, ".exe", or like the example below search for "cmd.exe" being run by WScript.exe (see Figure #6).

Figure 6: Use the Redline "Strings" output to find additional evidence

Other interesting strings to search for would be looking for common Windows system and network commands such as "finger", "net use", "netstat", etc.

Understanding normal activity is key when you start looking for badness. If you don't know what normal traffic activity looks like then you will be lost when trying to find said malware. One way is to better understand the operating system that you are doing analysis on, in this case windows. Understanding windows process structure helps, for example: csrss.exe is created by an instance of smss.exe and will have two or more instances running. The start time is within seconds of boot time for the first 2 instances (for Session 0 and I). Start times for additional instances occur as new sessions are created, although often only Sessions 0 and I are created. This information is available on the SANS DFIR poster. Looking at a Hierarchical Processes in Redline will reveal an instance of smss.exe that will spawn csrss.exe. Another item that is interesting to research is svchost.exe whose parent process is services.exe and runs five or more instances. It is used for running service DLLs. Windows will run multiple instances of svchost.exe, each using a unique "-k" parameter for grouping similar services. Malware authors often take advantage of the ubiquitous nature of svchost.exe and use it either directly or indirectly to hide their malware. They use it directly by installing the malware as a service in a legitimate instance of svchost.exe. Alternatively, they use it indirectly by trying to blend in with legitimate instances of svchost.exe, either by slightly misspelling the name (scvhost.exe) or spelling it correctly but placing it in a directory other than System32. All default installations of Windows 7, all service executables and all service DLLs are signed by Microsoft. This information is also available on the SANS DFIR poster and would be very helpful to review.

After reviewing this memory image with Mandiant Redline there is no smoking gun that can definitively say that this image has malware. Now we will take a look at an open source tool called Volatility.

Move over to the SIFT workstation, which was the device that took the image off the windows machine with the f-response tool. Open up a terminal and change directory to where the case files are located and find out the image information on the image file that you are interested in (see Figure 7).

Figure 7: Running volatility to discover the imageinfo of the memory image

The imageinfo output gives you information about the image (memory dump). The details about this image file can be seen in Figure #7. The suggested profile that you should pass as the parameter to -profile=PROFILE; there may be more than one profile suggestion if profiles are closely related. You can figure out which one is more appropriate by checking the "Image Type" field, which is blank for Service Pack 0 and filled in for other Service Packs.

To find the processes and DLLs use the following Volatility commands:

  • $ vol.py --profile=WinXPSP2x86 -f remote-system-memory11.img pslist

This command will list the processes of a system. It does not detect hidden or unlinked processes.

  • $ vol.py -profile=WinXPSP2x86 -f remote-system-memory11.img pstree

This command will view the process listing in a tree form. Child processes are indicated using indention and periods (see Figure #8).

  • $ vol.py -profile=WinXPSP2x86 -f remote-system-memory11.img psscan

This command will enumerate processes using pool tag scanning. This can find a process that previously terminated (inactive) and processes that have been hidden or unlinked by a rootkit (see Figure #9).

  • $ vol.py -profile=WinXPSP2x86 -f remote-system-memory11.img dlllist

This command will display a processes loaded DLLs and shows the command line used to start the process (including services) along with the DLL libraries for the process. Look for strange DLLs that have been injected into legitimate processes and DLLs with suspicious looking names.

This will produce a large output so if you narrow down your search to a specific process and want the DLL associated with it try the following command:

$ vol.py -profile=WinXPSP2x86 -f remote-system-memory11.img dlllist -p 1764

This command will display the process loaded DLLs for PID 1764 WScript.exe see Figure 10.

Figure 8: Example of using pstree, notice the child process indicated by the indention and periods

Figure 9: Example of using psscan, this can find processes that are hidden

Figure 10: Example of using dlllist to display the DLLs for the process WScript.exe

Volatility has many different features organized by plugins and categories. These categories include Image identification, processes and DLLs, processes memory, kernel memory and objects, networking, registry, crash dumps, hibernation, and malware/rootkits. We will look at the networking category below. To view active connections, use the connections command or to find connection structures using pool tag scanning, use the connscan command. Pool scanners is a technique when a piece of memory is allocated in windows, it's often allocated with a special tag which corresponds to the drive or subsystem to allocate the memory. You often find previous connections that have since been terminated. This command works for Windows XP and Windows 2003 Server only see Figure 11.

$ vol.py -profile=WinXPSP2x86 -f remote-system-memory005.img connscan

Figure 11: Output of connscan command notice the two connections to 10.10.3.180

So what is trying to communicate on the network from our Windows XP box to an internal 10.10.3.180? I don't have anything with that IP address on this network. Need to further investigate PID 1792 & 132. Running a psscan to enumerate processes does not show PID 1792 so that is a dead process but 132 shows up as svchost.exe. When I run a pstree to learn more information such as parent process indicated using indention and periods see Figure 12.

Figure 12: Svchost.exe has a parent process of wscript.exe which is child of explorer.exe

Another plugin available in Volatility called malfind extracts injected DLLs, injected code, unpacker stubs, API hook trampolines. Scans for any ANSI string, Unicode string, regular expression, or byte sequence in process or kernel driver memory. The syntax for using this plugin is below:

$ vol.py -profile=WinXPSP2x86 -f remote-system-memory005.img malfind -D memory/

This will dump all the processes with injected code in the directory called memory and when we are finished we can see all the files in Figure 13.

Figure 13: Results of using malfind has produced a number of dmp files for analysis

I will upload these dmp files to VirusTotal to see if we can identify any know issues. The only suspected dmp file that was flagged by virustotal (see Figure #14) was 0x370000 which is our f-response tool used to extract our memory image. This is a false positive.

Figure 14: Results from VirusTotal after uploading 0x370000.dmp files (f-response) that was generated after running malfind

Conclusion

After taking a SANS FOR508: Advanced Computer Forensic Analysis and Incident Response and learning the tools described in this article I started doing research on malware found on our network by Palo Alto firewalls and I used an example found by their firewall to research. This was a challenge because there were no obvious red flags when doing the investigation. Which is why I used this example to show that you will need to dig deep. What I learned:

  1. The memory image taken before infection showed communication with the windows box and the SIFT workstation, no other connections.
  2. The memory image taken after the infection showed communication with a 10.10.3.180 (2 instances) which is an internal IP address that I don't have on my test network.
  3. The two Process IDs (PID) that were related to IP address 10.10.3.180 are 1792 which was a dead process, and 132 svchost.exe which was the wscript.exe and has a parent process of 1648 explorer.exe.
  4. The svchost.exe (PID 132) is a generic host process for Windows Services and is used for running service DLLs and should always be a child of services.exe. It is a child process of wscript.exe and a clear indicator of wrong doing (see Figure 12).

More Stories By David Dodd

David J. Dodd is currently in the United States and holds a current 'Top Secret' DoD Clearance and is available for consulting on various Information Assurance projects. A former U.S. Marine with Avionics background in Electronic Countermeasures Systems. David has given talks at the San Diego Regional Security Conference and SDISSA, is a member of InfraGard, and contributes to Secure our eCity http://securingourecity.org. He works for Xerox as Information Security Officer City of San Diego & pbnetworks Inc. http://pbnetworks.net a Service Disabled Veteran Owned Small Business (SDVOSB) located in San Diego, CA and can be contacted by emailing: dave at pbnetworks.net.

IoT & Smart Cities Stories
Digital Transformation and Disruption, Amazon Style - What You Can Learn. Chris Kocher is a co-founder of Grey Heron, a management and strategic marketing consulting firm. He has 25+ years in both strategic and hands-on operating experience helping executives and investors build revenues and shareholder value. He has consulted with over 130 companies on innovating with new business models, product strategies and monetization. Chris has held management positions at HP and Symantec in addition to ...
Dynatrace is an application performance management software company with products for the information technology departments and digital business owners of medium and large businesses. Building the Future of Monitoring with Artificial Intelligence. Today we can collect lots and lots of performance data. We build beautiful dashboards and even have fancy query languages to access and transform the data. Still performance data is a secret language only a couple of people understand. The more busine...
The challenges of aggregating data from consumer-oriented devices, such as wearable technologies and smart thermostats, are fairly well-understood. However, there are a new set of challenges for IoT devices that generate megabytes or gigabytes of data per second. Certainly, the infrastructure will have to change, as those volumes of data will likely overwhelm the available bandwidth for aggregating the data into a central repository. Ochandarena discusses a whole new way to think about your next...
CloudEXPO | DevOpsSUMMIT | DXWorldEXPO are the world's most influential, independent events where Cloud Computing was coined and where technology buyers and vendors meet to experience and discuss the big picture of Digital Transformation and all of the strategies, tactics, and tools they need to realize their goals. Sponsors of DXWorldEXPO | CloudEXPO benefit from unmatched branding, profile building and lead generation opportunities.
All in Mobile is a place where we continually maximize their impact by fostering understanding, empathy, insights, creativity and joy. They believe that a truly useful and desirable mobile app doesn't need the brightest idea or the most advanced technology. A great product begins with understanding people. It's easy to think that customers will love your app, but can you justify it? They make sure your final app is something that users truly want and need. The only way to do this is by ...
DXWorldEXPO LLC announced today that Big Data Federation to Exhibit at the 22nd International CloudEXPO, colocated with DevOpsSUMMIT and DXWorldEXPO, November 12-13, 2018 in New York City. Big Data Federation, Inc. develops and applies artificial intelligence to predict financial and economic events that matter. The company uncovers patterns and precise drivers of performance and outcomes with the aid of machine-learning algorithms, big data, and fundamental analysis. Their products are deployed...
Cell networks have the advantage of long-range communications, reaching an estimated 90% of the world. But cell networks such as 2G, 3G and LTE consume lots of power and were designed for connecting people. They are not optimized for low- or battery-powered devices or for IoT applications with infrequently transmitted data. Cell IoT modules that support narrow-band IoT and 4G cell networks will enable cell connectivity, device management, and app enablement for low-power wide-area network IoT. B...
The hierarchical architecture that distributes "compute" within the network specially at the edge can enable new services by harnessing emerging technologies. But Edge-Compute comes at increased cost that needs to be managed and potentially augmented by creative architecture solutions as there will always a catching-up with the capacity demands. Processing power in smartphones has enhanced YoY and there is increasingly spare compute capacity that can be potentially pooled. Uber has successfully ...
SYS-CON Events announced today that CrowdReviews.com has been named “Media Sponsor” of SYS-CON's 22nd International Cloud Expo, which will take place on June 5–7, 2018, at the Javits Center in New York City, NY. CrowdReviews.com is a transparent online platform for determining which products and services are the best based on the opinion of the crowd. The crowd consists of Internet users that have experienced products and services first-hand and have an interest in letting other potential buye...
When talking IoT we often focus on the devices, the sensors, the hardware itself. The new smart appliances, the new smart or self-driving cars (which are amalgamations of many ‘things'). When we are looking at the world of IoT, we should take a step back, look at the big picture. What value are these devices providing. IoT is not about the devices, its about the data consumed and generated. The devices are tools, mechanisms, conduits. This paper discusses the considerations when dealing with the...