Welcome!

Cloud Security Authors: Elizabeth White, Xenia von Wedel, John Walsh, James Carlini, Mehdi Daoudi

Related Topics: Cloud Security, Microservices Expo

Cloud Security: Tutorial

Malware Analysis | Part 2

Find hidden and covert processes, clandestine communications, and signs of misconduct on your network

In a previous article [1], I described how to obtain a memory image from a Windows computer that would allow forensic analysis. I briefly discussed using F-Response TACTICAL [2] to get the memory image, and then Volatility [3] and Mandiant Redline [4] for further investigation. In this paper, I dive more deeply into Redline and Volatility.

To begin, I review a raw memory dump of a known malware variant (see the "Malware Image" box) with Mandiant Redline. After firing up Redline, I chose By Analyzing a Saved Memory File under Analyze Data and browsed to the location of the memory image. Next, I edited my script to include Strings for both Process Listing and Driver Enumeration. Finally, I chose a destination to store the output for future analysis and to analyze memory dumps.

Malware Image
The malware image I am using in this article is a variant found by the Palo Alto PA-5000 series firewall [5] on a Windows box in our network, which was sent for further investigation to a sandbox that Palo Alto uses for such cases. Moments later, I received email telling me that malware was discovered by Palo Alto WildFire analysis [6].

WildFire identifies unknown malware, zero-day exploits, and advanced persistent threats by executing them directly in a scalable, cloud-based, virtual sandbox environment. The report, which goes into detail about what the malware has done, gave me a link to VirusTotal [7], used to score the executable for maliciousness, along with a PCAP file of network traffic generated by the malware. I was able to download the known malware variant and execute it on my closed test VM network for observation; then, I used both Volatility and Mandiant Redline to research the culprit.

Redline
As I explained in the previous article, I took two different memory dumps from a Windows XP system: one after I executed the malware infection, and another after I rebooted the system. In comparing the two images, I noticed some differences. Remember, malware usually tries to hide in plain sight, attempting to appear legitimate, or uses rootkit techniques to hide from the view of normal analysis tools.

After installing the malware and completing a reboot, Redline revealed three process names (jh, process ID [PID] 38533; svchost.exe, PID 1560; and WScript.exe, PID 1744). Process jh (PID 38533) was spawned by a parent process - [Not Available] (528) - with a start time of 1601-01-01 00:00:00Z (Figure 1). Double-clicking on the Process name (jh ) provides access to the detailed view.

Figure 1: Process jh PID 38533.

Choosing Hierarchical Processes in the Analysis Data panel showed that svchost.exe (PID 1560) was spawned by WScript.exe (PID 1744). Hierarchical Processes lists running processes in a tree format, showing which processes started other processes (Figure 2). If you take a look at the Process Metadata for jh in Figure 1, you'll see that it has an MRI Score of 61, as does svchost.exe (Figure 2, second column).

Figure 2: Hierarchical Processes

Taking a look at the MRI Score and see that Process Name jh and dsvchost.exe have an MRI Score of 61. The MRI Score or Malware Risk Index Report which is accessed by double-clicking a process-related item to see its detailed view, and selecting the MRI Report tab at the bottom of the window, the higher the number the larger the risk. Since these two processes started after the install of malware it is likely they are bad. Compare the Start Time of svchost.exe PID1560 with the other svchost processes (see Figure #3) it appears to have started about 30 min after the other svchost processes.

Figure 3: Comparing different svchost processes running with Hierarchical Processes

Figure 4: Finding user account and full path of the process binary

The parent process of svchost.exe PID 3028 is WScript.exe PID 1736, this is discovered by looking at the Hierarchical Processes. After clicking on WScript.exe PID 1736 we discover the user account that was logged on when the process was spawned and the full path of the process binary (see Figure #4). Next click on the Handles tab located below and view the Handle Names, notice the Untrusted status (see Figure #5). Using Redline to check for signed code may reveal suspicious executables.

Figure 5: Viewing a process and its Handles

This gives us the ability to show all handles including ones identified as untrusted. To look for evidence of code injection review Memory Sections located under Analysis Data -> Processes to see memory pages for every process. This particular malware contains no injected memory sections that Redline can find.

Finally use the "Strings" output to find additional evidence. You can search for interesting things like http://, https://, ".exe", or like the example below search for "cmd.exe" being run by WScript.exe (see Figure #6).

Figure 6: Use the Redline "Strings" output to find additional evidence

Other interesting strings to search for would be looking for common Windows system and network commands such as "finger", "net use", "netstat", etc.

Understanding normal activity is key when you start looking for badness. If you don't know what normal traffic activity looks like then you will be lost when trying to find said malware. One way is to better understand the operating system that you are doing analysis on, in this case windows. Understanding windows process structure helps, for example: csrss.exe is created by an instance of smss.exe and will have two or more instances running. The start time is within seconds of boot time for the first 2 instances (for Session 0 and I). Start times for additional instances occur as new sessions are created, although often only Sessions 0 and I are created. This information is available on the SANS DFIR poster. Looking at a Hierarchical Processes in Redline will reveal an instance of smss.exe that will spawn csrss.exe. Another item that is interesting to research is svchost.exe whose parent process is services.exe and runs five or more instances. It is used for running service DLLs. Windows will run multiple instances of svchost.exe, each using a unique "-k" parameter for grouping similar services. Malware authors often take advantage of the ubiquitous nature of svchost.exe and use it either directly or indirectly to hide their malware. They use it directly by installing the malware as a service in a legitimate instance of svchost.exe. Alternatively, they use it indirectly by trying to blend in with legitimate instances of svchost.exe, either by slightly misspelling the name (scvhost.exe) or spelling it correctly but placing it in a directory other than System32. All default installations of Windows 7, all service executables and all service DLLs are signed by Microsoft. This information is also available on the SANS DFIR poster and would be very helpful to review.

After reviewing this memory image with Mandiant Redline there is no smoking gun that can definitively say that this image has malware. Now we will take a look at an open source tool called Volatility.

Move over to the SIFT workstation, which was the device that took the image off the windows machine with the f-response tool. Open up a terminal and change directory to where the case files are located and find out the image information on the image file that you are interested in (see Figure 7).

Figure 7: Running volatility to discover the imageinfo of the memory image

The imageinfo output gives you information about the image (memory dump). The details about this image file can be seen in Figure #7. The suggested profile that you should pass as the parameter to -profile=PROFILE; there may be more than one profile suggestion if profiles are closely related. You can figure out which one is more appropriate by checking the "Image Type" field, which is blank for Service Pack 0 and filled in for other Service Packs.

To find the processes and DLLs use the following Volatility commands:

  • $ vol.py --profile=WinXPSP2x86 -f remote-system-memory11.img pslist

This command will list the processes of a system. It does not detect hidden or unlinked processes.

  • $ vol.py -profile=WinXPSP2x86 -f remote-system-memory11.img pstree

This command will view the process listing in a tree form. Child processes are indicated using indention and periods (see Figure #8).

  • $ vol.py -profile=WinXPSP2x86 -f remote-system-memory11.img psscan

This command will enumerate processes using pool tag scanning. This can find a process that previously terminated (inactive) and processes that have been hidden or unlinked by a rootkit (see Figure #9).

  • $ vol.py -profile=WinXPSP2x86 -f remote-system-memory11.img dlllist

This command will display a processes loaded DLLs and shows the command line used to start the process (including services) along with the DLL libraries for the process. Look for strange DLLs that have been injected into legitimate processes and DLLs with suspicious looking names.

This will produce a large output so if you narrow down your search to a specific process and want the DLL associated with it try the following command:

$ vol.py -profile=WinXPSP2x86 -f remote-system-memory11.img dlllist -p 1764

This command will display the process loaded DLLs for PID 1764 WScript.exe see Figure 10.

Figure 8: Example of using pstree, notice the child process indicated by the indention and periods

Figure 9: Example of using psscan, this can find processes that are hidden

Figure 10: Example of using dlllist to display the DLLs for the process WScript.exe

Volatility has many different features organized by plugins and categories. These categories include Image identification, processes and DLLs, processes memory, kernel memory and objects, networking, registry, crash dumps, hibernation, and malware/rootkits. We will look at the networking category below. To view active connections, use the connections command or to find connection structures using pool tag scanning, use the connscan command. Pool scanners is a technique when a piece of memory is allocated in windows, it's often allocated with a special tag which corresponds to the drive or subsystem to allocate the memory. You often find previous connections that have since been terminated. This command works for Windows XP and Windows 2003 Server only see Figure 11.

$ vol.py -profile=WinXPSP2x86 -f remote-system-memory005.img connscan

Figure 11: Output of connscan command notice the two connections to 10.10.3.180

So what is trying to communicate on the network from our Windows XP box to an internal 10.10.3.180? I don't have anything with that IP address on this network. Need to further investigate PID 1792 & 132. Running a psscan to enumerate processes does not show PID 1792 so that is a dead process but 132 shows up as svchost.exe. When I run a pstree to learn more information such as parent process indicated using indention and periods see Figure 12.

Figure 12: Svchost.exe has a parent process of wscript.exe which is child of explorer.exe

Another plugin available in Volatility called malfind extracts injected DLLs, injected code, unpacker stubs, API hook trampolines. Scans for any ANSI string, Unicode string, regular expression, or byte sequence in process or kernel driver memory. The syntax for using this plugin is below:

$ vol.py -profile=WinXPSP2x86 -f remote-system-memory005.img malfind -D memory/

This will dump all the processes with injected code in the directory called memory and when we are finished we can see all the files in Figure 13.

Figure 13: Results of using malfind has produced a number of dmp files for analysis

I will upload these dmp files to VirusTotal to see if we can identify any know issues. The only suspected dmp file that was flagged by virustotal (see Figure #14) was 0x370000 which is our f-response tool used to extract our memory image. This is a false positive.

Figure 14: Results from VirusTotal after uploading 0x370000.dmp files (f-response) that was generated after running malfind

Conclusion

After taking a SANS FOR508: Advanced Computer Forensic Analysis and Incident Response and learning the tools described in this article I started doing research on malware found on our network by Palo Alto firewalls and I used an example found by their firewall to research. This was a challenge because there were no obvious red flags when doing the investigation. Which is why I used this example to show that you will need to dig deep. What I learned:

  1. The memory image taken before infection showed communication with the windows box and the SIFT workstation, no other connections.
  2. The memory image taken after the infection showed communication with a 10.10.3.180 (2 instances) which is an internal IP address that I don't have on my test network.
  3. The two Process IDs (PID) that were related to IP address 10.10.3.180 are 1792 which was a dead process, and 132 svchost.exe which was the wscript.exe and has a parent process of 1648 explorer.exe.
  4. The svchost.exe (PID 132) is a generic host process for Windows Services and is used for running service DLLs and should always be a child of services.exe. It is a child process of wscript.exe and a clear indicator of wrong doing (see Figure 12).

More Stories By David Dodd

David J. Dodd is currently in the United States and holds a current 'Top Secret' DoD Clearance and is available for consulting on various Information Assurance projects. A former U.S. Marine with Avionics background in Electronic Countermeasures Systems. David has given talks at the San Diego Regional Security Conference and SDISSA, is a member of InfraGard, and contributes to Secure our eCity http://securingourecity.org. He works for Xerox as Information Security Officer City of San Diego & pbnetworks Inc. http://pbnetworks.net a Service Disabled Veteran Owned Small Business (SDVOSB) located in San Diego, CA and can be contacted by emailing: dave at pbnetworks.net.

@ThingsExpo Stories
"IBM is really all in on blockchain. We take a look at sort of the history of blockchain ledger technologies. It started out with bitcoin, Ethereum, and IBM evaluated these particular blockchain technologies and found they were anonymous and permissionless and that many companies were looking for permissioned blockchain," stated René Bostic, Technical VP of the IBM Cloud Unit in North America, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Conventi...
In his session at 21st Cloud Expo, Carl J. Levine, Senior Technical Evangelist for NS1, will objectively discuss how DNS is used to solve Digital Transformation challenges in large SaaS applications, CDNs, AdTech platforms, and other demanding use cases. Carl J. Levine is the Senior Technical Evangelist for NS1. A veteran of the Internet Infrastructure space, he has over a decade of experience with startups, networking protocols and Internet infrastructure, combined with the unique ability to it...
22nd International Cloud Expo, taking place June 5-7, 2018, at the Javits Center in New York City, NY, and co-located with the 1st DXWorld Expo will feature technical sessions from a rock star conference faculty and the leading industry players in the world. Cloud computing is now being embraced by a majority of enterprises of all sizes. Yesterday's debate about public vs. private has transformed into the reality of hybrid cloud: a recent survey shows that 74% of enterprises have a hybrid cloud ...
"Cloud Academy is an enterprise training platform for the cloud, specifically public clouds. We offer guided learning experiences on AWS, Azure, Google Cloud and all the surrounding methodologies and technologies that you need to know and your teams need to know in order to leverage the full benefits of the cloud," explained Alex Brower, VP of Marketing at Cloud Academy, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clar...
Gemini is Yahoo’s native and search advertising platform. To ensure the quality of a complex distributed system that spans multiple products and components and across various desktop websites and mobile app and web experiences – both Yahoo owned and operated and third-party syndication (supply), with complex interaction with more than a billion users and numerous advertisers globally (demand) – it becomes imperative to automate a set of end-to-end tests 24x7 to detect bugs and regression. In th...
Widespread fragmentation is stalling the growth of the IIoT and making it difficult for partners to work together. The number of software platforms, apps, hardware and connectivity standards is creating paralysis among businesses that are afraid of being locked into a solution. EdgeX Foundry is unifying the community around a common IoT edge framework and an ecosystem of interoperable components.
"MobiDev is a software development company and we do complex, custom software development for everybody from entrepreneurs to large enterprises," explained Alan Winters, U.S. Head of Business Development at MobiDev, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
Large industrial manufacturing organizations are adopting the agile principles of cloud software companies. The industrial manufacturing development process has not scaled over time. Now that design CAD teams are geographically distributed, centralizing their work is key. With large multi-gigabyte projects, outdated tools have stifled industrial team agility, time-to-market milestones, and impacted P&L stakeholders.
"Space Monkey by Vivent Smart Home is a product that is a distributed cloud-based edge storage network. Vivent Smart Home, our parent company, is a smart home provider that places a lot of hard drives across homes in North America," explained JT Olds, Director of Engineering, and Brandon Crowfeather, Product Manager, at Vivint Smart Home, in this SYS-CON.tv interview at @ThingsExpo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
"Akvelon is a software development company and we also provide consultancy services to folks who are looking to scale or accelerate their engineering roadmaps," explained Jeremiah Mothersell, Marketing Manager at Akvelon, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
Coca-Cola’s Google powered digital signage system lays the groundwork for a more valuable connection between Coke and its customers. Digital signs pair software with high-resolution displays so that a message can be changed instantly based on what the operator wants to communicate or sell. In their Day 3 Keynote at 21st Cloud Expo, Greg Chambers, Global Group Director, Digital Innovation, Coca-Cola, and Vidya Nagarajan, a Senior Product Manager at Google, discussed how from store operations and ...
"There's plenty of bandwidth out there but it's never in the right place. So what Cedexis does is uses data to work out the best pathways to get data from the origin to the person who wants to get it," explained Simon Jones, Evangelist and Head of Marketing at Cedexis, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
SYS-CON Events announced today that CrowdReviews.com has been named “Media Sponsor” of SYS-CON's 22nd International Cloud Expo, which will take place on June 5–7, 2018, at the Javits Center in New York City, NY. CrowdReviews.com is a transparent online platform for determining which products and services are the best based on the opinion of the crowd. The crowd consists of Internet users that have experienced products and services first-hand and have an interest in letting other potential buye...
SYS-CON Events announced today that Telecom Reseller has been named “Media Sponsor” of SYS-CON's 22nd International Cloud Expo, which will take place on June 5-7, 2018, at the Javits Center in New York, NY. Telecom Reseller reports on Unified Communications, UCaaS, BPaaS for enterprise and SMBs. They report extensively on both customer premises based solutions such as IP-PBX as well as cloud based and hosted platforms.
It is of utmost importance for the future success of WebRTC to ensure that interoperability is operational between web browsers and any WebRTC-compliant client. To be guaranteed as operational and effective, interoperability must be tested extensively by establishing WebRTC data and media connections between different web browsers running on different devices and operating systems. In his session at WebRTC Summit at @ThingsExpo, Dr. Alex Gouaillard, CEO and Founder of CoSMo Software, presented ...
WebRTC is great technology to build your own communication tools. It will be even more exciting experience it with advanced devices, such as a 360 Camera, 360 microphone, and a depth sensor camera. In his session at @ThingsExpo, Masashi Ganeko, a manager at INFOCOM Corporation, introduced two experimental projects from his team and what they learned from them. "Shotoku Tamago" uses the robot audition software HARK to track speakers in 360 video of a remote party. "Virtual Teleport" uses a multip...
A strange thing is happening along the way to the Internet of Things, namely far too many devices to work with and manage. It has become clear that we'll need much higher efficiency user experiences that can allow us to more easily and scalably work with the thousands of devices that will soon be in each of our lives. Enter the conversational interface revolution, combining bots we can literally talk with, gesture to, and even direct with our thoughts, with embedded artificial intelligence, whic...
SYS-CON Events announced today that Evatronix will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Evatronix SA offers comprehensive solutions in the design and implementation of electronic systems, in CAD / CAM deployment, and also is a designer and manufacturer of advanced 3D scanners for professional applications.
Leading companies, from the Global Fortune 500 to the smallest companies, are adopting hybrid cloud as the path to business advantage. Hybrid cloud depends on cloud services and on-premises infrastructure working in unison. Successful implementations require new levels of data mobility, enabled by an automated and seamless flow across on-premises and cloud resources. In his general session at 21st Cloud Expo, Greg Tevis, an IBM Storage Software Technical Strategist and Customer Solution Architec...
To get the most out of their data, successful companies are not focusing on queries and data lakes, they are actively integrating analytics into their operations with a data-first application development approach. Real-time adjustments to improve revenues, reduce costs, or mitigate risk rely on applications that minimize latency on a variety of data sources. In his session at @BigDataExpo, Jack Norris, Senior Vice President, Data and Applications at MapR Technologies, reviewed best practices to ...