Welcome!

Cloud Security Authors: Yeshim Deniz, Zakia Bouachraoui, Liz McMillan, Elizabeth White, Ravi Rajamiyer

Related Topics: Cloud Security, Microservices Expo

Cloud Security: Tutorial

Malware Analysis | Part 2

Find hidden and covert processes, clandestine communications, and signs of misconduct on your network

In a previous article [1], I described how to obtain a memory image from a Windows computer that would allow forensic analysis. I briefly discussed using F-Response TACTICAL [2] to get the memory image, and then Volatility [3] and Mandiant Redline [4] for further investigation. In this paper, I dive more deeply into Redline and Volatility.

To begin, I review a raw memory dump of a known malware variant (see the "Malware Image" box) with Mandiant Redline. After firing up Redline, I chose By Analyzing a Saved Memory File under Analyze Data and browsed to the location of the memory image. Next, I edited my script to include Strings for both Process Listing and Driver Enumeration. Finally, I chose a destination to store the output for future analysis and to analyze memory dumps.

Malware Image
The malware image I am using in this article is a variant found by the Palo Alto PA-5000 series firewall [5] on a Windows box in our network, which was sent for further investigation to a sandbox that Palo Alto uses for such cases. Moments later, I received email telling me that malware was discovered by Palo Alto WildFire analysis [6].

WildFire identifies unknown malware, zero-day exploits, and advanced persistent threats by executing them directly in a scalable, cloud-based, virtual sandbox environment. The report, which goes into detail about what the malware has done, gave me a link to VirusTotal [7], used to score the executable for maliciousness, along with a PCAP file of network traffic generated by the malware. I was able to download the known malware variant and execute it on my closed test VM network for observation; then, I used both Volatility and Mandiant Redline to research the culprit.

Redline
As I explained in the previous article, I took two different memory dumps from a Windows XP system: one after I executed the malware infection, and another after I rebooted the system. In comparing the two images, I noticed some differences. Remember, malware usually tries to hide in plain sight, attempting to appear legitimate, or uses rootkit techniques to hide from the view of normal analysis tools.

After installing the malware and completing a reboot, Redline revealed three process names (jh, process ID [PID] 38533; svchost.exe, PID 1560; and WScript.exe, PID 1744). Process jh (PID 38533) was spawned by a parent process - [Not Available] (528) - with a start time of 1601-01-01 00:00:00Z (Figure 1). Double-clicking on the Process name (jh ) provides access to the detailed view.

Figure 1: Process jh PID 38533.

Choosing Hierarchical Processes in the Analysis Data panel showed that svchost.exe (PID 1560) was spawned by WScript.exe (PID 1744). Hierarchical Processes lists running processes in a tree format, showing which processes started other processes (Figure 2). If you take a look at the Process Metadata for jh in Figure 1, you'll see that it has an MRI Score of 61, as does svchost.exe (Figure 2, second column).

Figure 2: Hierarchical Processes

Taking a look at the MRI Score and see that Process Name jh and dsvchost.exe have an MRI Score of 61. The MRI Score or Malware Risk Index Report which is accessed by double-clicking a process-related item to see its detailed view, and selecting the MRI Report tab at the bottom of the window, the higher the number the larger the risk. Since these two processes started after the install of malware it is likely they are bad. Compare the Start Time of svchost.exe PID1560 with the other svchost processes (see Figure #3) it appears to have started about 30 min after the other svchost processes.

Figure 3: Comparing different svchost processes running with Hierarchical Processes

Figure 4: Finding user account and full path of the process binary

The parent process of svchost.exe PID 3028 is WScript.exe PID 1736, this is discovered by looking at the Hierarchical Processes. After clicking on WScript.exe PID 1736 we discover the user account that was logged on when the process was spawned and the full path of the process binary (see Figure #4). Next click on the Handles tab located below and view the Handle Names, notice the Untrusted status (see Figure #5). Using Redline to check for signed code may reveal suspicious executables.

Figure 5: Viewing a process and its Handles

This gives us the ability to show all handles including ones identified as untrusted. To look for evidence of code injection review Memory Sections located under Analysis Data -> Processes to see memory pages for every process. This particular malware contains no injected memory sections that Redline can find.

Finally use the "Strings" output to find additional evidence. You can search for interesting things like http://, https://, ".exe", or like the example below search for "cmd.exe" being run by WScript.exe (see Figure #6).

Figure 6: Use the Redline "Strings" output to find additional evidence

Other interesting strings to search for would be looking for common Windows system and network commands such as "finger", "net use", "netstat", etc.

Understanding normal activity is key when you start looking for badness. If you don't know what normal traffic activity looks like then you will be lost when trying to find said malware. One way is to better understand the operating system that you are doing analysis on, in this case windows. Understanding windows process structure helps, for example: csrss.exe is created by an instance of smss.exe and will have two or more instances running. The start time is within seconds of boot time for the first 2 instances (for Session 0 and I). Start times for additional instances occur as new sessions are created, although often only Sessions 0 and I are created. This information is available on the SANS DFIR poster. Looking at a Hierarchical Processes in Redline will reveal an instance of smss.exe that will spawn csrss.exe. Another item that is interesting to research is svchost.exe whose parent process is services.exe and runs five or more instances. It is used for running service DLLs. Windows will run multiple instances of svchost.exe, each using a unique "-k" parameter for grouping similar services. Malware authors often take advantage of the ubiquitous nature of svchost.exe and use it either directly or indirectly to hide their malware. They use it directly by installing the malware as a service in a legitimate instance of svchost.exe. Alternatively, they use it indirectly by trying to blend in with legitimate instances of svchost.exe, either by slightly misspelling the name (scvhost.exe) or spelling it correctly but placing it in a directory other than System32. All default installations of Windows 7, all service executables and all service DLLs are signed by Microsoft. This information is also available on the SANS DFIR poster and would be very helpful to review.

After reviewing this memory image with Mandiant Redline there is no smoking gun that can definitively say that this image has malware. Now we will take a look at an open source tool called Volatility.

Move over to the SIFT workstation, which was the device that took the image off the windows machine with the f-response tool. Open up a terminal and change directory to where the case files are located and find out the image information on the image file that you are interested in (see Figure 7).

Figure 7: Running volatility to discover the imageinfo of the memory image

The imageinfo output gives you information about the image (memory dump). The details about this image file can be seen in Figure #7. The suggested profile that you should pass as the parameter to -profile=PROFILE; there may be more than one profile suggestion if profiles are closely related. You can figure out which one is more appropriate by checking the "Image Type" field, which is blank for Service Pack 0 and filled in for other Service Packs.

To find the processes and DLLs use the following Volatility commands:

  • $ vol.py --profile=WinXPSP2x86 -f remote-system-memory11.img pslist

This command will list the processes of a system. It does not detect hidden or unlinked processes.

  • $ vol.py -profile=WinXPSP2x86 -f remote-system-memory11.img pstree

This command will view the process listing in a tree form. Child processes are indicated using indention and periods (see Figure #8).

  • $ vol.py -profile=WinXPSP2x86 -f remote-system-memory11.img psscan

This command will enumerate processes using pool tag scanning. This can find a process that previously terminated (inactive) and processes that have been hidden or unlinked by a rootkit (see Figure #9).

  • $ vol.py -profile=WinXPSP2x86 -f remote-system-memory11.img dlllist

This command will display a processes loaded DLLs and shows the command line used to start the process (including services) along with the DLL libraries for the process. Look for strange DLLs that have been injected into legitimate processes and DLLs with suspicious looking names.

This will produce a large output so if you narrow down your search to a specific process and want the DLL associated with it try the following command:

$ vol.py -profile=WinXPSP2x86 -f remote-system-memory11.img dlllist -p 1764

This command will display the process loaded DLLs for PID 1764 WScript.exe see Figure 10.

Figure 8: Example of using pstree, notice the child process indicated by the indention and periods

Figure 9: Example of using psscan, this can find processes that are hidden

Figure 10: Example of using dlllist to display the DLLs for the process WScript.exe

Volatility has many different features organized by plugins and categories. These categories include Image identification, processes and DLLs, processes memory, kernel memory and objects, networking, registry, crash dumps, hibernation, and malware/rootkits. We will look at the networking category below. To view active connections, use the connections command or to find connection structures using pool tag scanning, use the connscan command. Pool scanners is a technique when a piece of memory is allocated in windows, it's often allocated with a special tag which corresponds to the drive or subsystem to allocate the memory. You often find previous connections that have since been terminated. This command works for Windows XP and Windows 2003 Server only see Figure 11.

$ vol.py -profile=WinXPSP2x86 -f remote-system-memory005.img connscan

Figure 11: Output of connscan command notice the two connections to 10.10.3.180

So what is trying to communicate on the network from our Windows XP box to an internal 10.10.3.180? I don't have anything with that IP address on this network. Need to further investigate PID 1792 & 132. Running a psscan to enumerate processes does not show PID 1792 so that is a dead process but 132 shows up as svchost.exe. When I run a pstree to learn more information such as parent process indicated using indention and periods see Figure 12.

Figure 12: Svchost.exe has a parent process of wscript.exe which is child of explorer.exe

Another plugin available in Volatility called malfind extracts injected DLLs, injected code, unpacker stubs, API hook trampolines. Scans for any ANSI string, Unicode string, regular expression, or byte sequence in process or kernel driver memory. The syntax for using this plugin is below:

$ vol.py -profile=WinXPSP2x86 -f remote-system-memory005.img malfind -D memory/

This will dump all the processes with injected code in the directory called memory and when we are finished we can see all the files in Figure 13.

Figure 13: Results of using malfind has produced a number of dmp files for analysis

I will upload these dmp files to VirusTotal to see if we can identify any know issues. The only suspected dmp file that was flagged by virustotal (see Figure #14) was 0x370000 which is our f-response tool used to extract our memory image. This is a false positive.

Figure 14: Results from VirusTotal after uploading 0x370000.dmp files (f-response) that was generated after running malfind

Conclusion

After taking a SANS FOR508: Advanced Computer Forensic Analysis and Incident Response and learning the tools described in this article I started doing research on malware found on our network by Palo Alto firewalls and I used an example found by their firewall to research. This was a challenge because there were no obvious red flags when doing the investigation. Which is why I used this example to show that you will need to dig deep. What I learned:

  1. The memory image taken before infection showed communication with the windows box and the SIFT workstation, no other connections.
  2. The memory image taken after the infection showed communication with a 10.10.3.180 (2 instances) which is an internal IP address that I don't have on my test network.
  3. The two Process IDs (PID) that were related to IP address 10.10.3.180 are 1792 which was a dead process, and 132 svchost.exe which was the wscript.exe and has a parent process of 1648 explorer.exe.
  4. The svchost.exe (PID 132) is a generic host process for Windows Services and is used for running service DLLs and should always be a child of services.exe. It is a child process of wscript.exe and a clear indicator of wrong doing (see Figure 12).

More Stories By David Dodd

David J. Dodd is currently in the United States and holds a current 'Top Secret' DoD Clearance and is available for consulting on various Information Assurance projects. A former U.S. Marine with Avionics background in Electronic Countermeasures Systems. David has given talks at the San Diego Regional Security Conference and SDISSA, is a member of InfraGard, and contributes to Secure our eCity http://securingourecity.org. He works for Xerox as Information Security Officer City of San Diego & pbnetworks Inc. http://pbnetworks.net a Service Disabled Veteran Owned Small Business (SDVOSB) located in San Diego, CA and can be contacted by emailing: dave at pbnetworks.net.

IoT & Smart Cities Stories
Dion Hinchcliffe is an internationally recognized digital expert, bestselling book author, frequent keynote speaker, analyst, futurist, and transformation expert based in Washington, DC. He is currently Chief Strategy Officer at the industry-leading digital strategy and online community solutions firm, 7Summits.
With 10 simultaneous tracks, keynotes, general sessions and targeted breakout classes, @CloudEXPO and DXWorldEXPO are two of the most important technology events of the year. Since its launch over eight years ago, @CloudEXPO and DXWorldEXPO have presented a rock star faculty as well as showcased hundreds of sponsors and exhibitors! In this blog post, we provide 7 tips on how, as part of our world-class faculty, you can deliver one of the most popular sessions at our events. But before reading...
If a machine can invent, does this mean the end of the patent system as we know it? The patent system, both in the US and Europe, allows companies to protect their inventions and helps foster innovation. However, Artificial Intelligence (AI) could be set to disrupt the patent system as we know it. This talk will examine how AI may change the patent landscape in the years to come. Furthermore, ways in which companies can best protect their AI related inventions will be examined from both a US and...
Charles Araujo is an industry analyst, internationally recognized authority on the Digital Enterprise and author of The Quantum Age of IT: Why Everything You Know About IT is About to Change. As Principal Analyst with Intellyx, he writes, speaks and advises organizations on how to navigate through this time of disruption. He is also the founder of The Institute for Digital Transformation and a sought after keynote speaker. He has been a regular contributor to both InformationWeek and CIO Insight...
DXWorldEXPO LLC, the producer of the world's most influential technology conferences and trade shows has announced the 22nd International CloudEXPO | DXWorldEXPO "Early Bird Registration" is now open. Register for Full Conference "Gold Pass" ▸ Here (Expo Hall ▸ Here)
Bill Schmarzo, Tech Chair of "Big Data | Analytics" of upcoming CloudEXPO | DXWorldEXPO New York (November 12-13, 2018, New York City) today announced the outline and schedule of the track. "The track has been designed in experience/degree order," said Schmarzo. "So, that folks who attend the entire track can leave the conference with some of the skills necessary to get their work done when they get back to their offices. It actually ties back to some work that I'm doing at the University of ...
@DevOpsSummit at Cloud Expo, taking place November 12-13 in New York City, NY, is co-located with 22nd international CloudEXPO | first international DXWorldEXPO and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The widespread success of cloud computing is driving the DevOps revolution in enterprise IT. Now as never before, development teams must communicate and collaborate in a dynamic, 24/7/365 environment. There is no time t...
The Internet of Things will challenge the status quo of how IT and development organizations operate. Or will it? Certainly the fog layer of IoT requires special insights about data ontology, security and transactional integrity. But the developmental challenges are the same: People, Process and Platform and how we integrate our thinking to solve complicated problems. In his session at 19th Cloud Expo, Craig Sproule, CEO of Metavine, demonstrated how to move beyond today's coding paradigm and sh...
CloudEXPO New York 2018, colocated with DXWorldEXPO New York 2018 will be held November 11-13, 2018, in New York City and will bring together Cloud Computing, FinTech and Blockchain, Digital Transformation, Big Data, Internet of Things, DevOps, AI, Machine Learning and WebRTC to one location.
Cell networks have the advantage of long-range communications, reaching an estimated 90% of the world. But cell networks such as 2G, 3G and LTE consume lots of power and were designed for connecting people. They are not optimized for low- or battery-powered devices or for IoT applications with infrequently transmitted data. Cell IoT modules that support narrow-band IoT and 4G cell networks will enable cell connectivity, device management, and app enablement for low-power wide-area network IoT. B...