Welcome!

Cloud Security Authors: Liz McMillan, Elizabeth White, Lisa Calkins, Pat Romanski, Mamoon Yunus

Related Topics: Cloud Security, Microservices Expo

Cloud Security: Tutorial

Malware Analysis | Part 2

Find hidden and covert processes, clandestine communications, and signs of misconduct on your network

In a previous article [1], I described how to obtain a memory image from a Windows computer that would allow forensic analysis. I briefly discussed using F-Response TACTICAL [2] to get the memory image, and then Volatility [3] and Mandiant Redline [4] for further investigation. In this paper, I dive more deeply into Redline and Volatility.

To begin, I review a raw memory dump of a known malware variant (see the "Malware Image" box) with Mandiant Redline. After firing up Redline, I chose By Analyzing a Saved Memory File under Analyze Data and browsed to the location of the memory image. Next, I edited my script to include Strings for both Process Listing and Driver Enumeration. Finally, I chose a destination to store the output for future analysis and to analyze memory dumps.

Malware Image
The malware image I am using in this article is a variant found by the Palo Alto PA-5000 series firewall [5] on a Windows box in our network, which was sent for further investigation to a sandbox that Palo Alto uses for such cases. Moments later, I received email telling me that malware was discovered by Palo Alto WildFire analysis [6].

WildFire identifies unknown malware, zero-day exploits, and advanced persistent threats by executing them directly in a scalable, cloud-based, virtual sandbox environment. The report, which goes into detail about what the malware has done, gave me a link to VirusTotal [7], used to score the executable for maliciousness, along with a PCAP file of network traffic generated by the malware. I was able to download the known malware variant and execute it on my closed test VM network for observation; then, I used both Volatility and Mandiant Redline to research the culprit.

Redline
As I explained in the previous article, I took two different memory dumps from a Windows XP system: one after I executed the malware infection, and another after I rebooted the system. In comparing the two images, I noticed some differences. Remember, malware usually tries to hide in plain sight, attempting to appear legitimate, or uses rootkit techniques to hide from the view of normal analysis tools.

After installing the malware and completing a reboot, Redline revealed three process names (jh, process ID [PID] 38533; svchost.exe, PID 1560; and WScript.exe, PID 1744). Process jh (PID 38533) was spawned by a parent process - [Not Available] (528) - with a start time of 1601-01-01 00:00:00Z (Figure 1). Double-clicking on the Process name (jh ) provides access to the detailed view.

Figure 1: Process jh PID 38533.

Choosing Hierarchical Processes in the Analysis Data panel showed that svchost.exe (PID 1560) was spawned by WScript.exe (PID 1744). Hierarchical Processes lists running processes in a tree format, showing which processes started other processes (Figure 2). If you take a look at the Process Metadata for jh in Figure 1, you'll see that it has an MRI Score of 61, as does svchost.exe (Figure 2, second column).

Figure 2: Hierarchical Processes

Taking a look at the MRI Score and see that Process Name jh and dsvchost.exe have an MRI Score of 61. The MRI Score or Malware Risk Index Report which is accessed by double-clicking a process-related item to see its detailed view, and selecting the MRI Report tab at the bottom of the window, the higher the number the larger the risk. Since these two processes started after the install of malware it is likely they are bad. Compare the Start Time of svchost.exe PID1560 with the other svchost processes (see Figure #3) it appears to have started about 30 min after the other svchost processes.

Figure 3: Comparing different svchost processes running with Hierarchical Processes

Figure 4: Finding user account and full path of the process binary

The parent process of svchost.exe PID 3028 is WScript.exe PID 1736, this is discovered by looking at the Hierarchical Processes. After clicking on WScript.exe PID 1736 we discover the user account that was logged on when the process was spawned and the full path of the process binary (see Figure #4). Next click on the Handles tab located below and view the Handle Names, notice the Untrusted status (see Figure #5). Using Redline to check for signed code may reveal suspicious executables.

Figure 5: Viewing a process and its Handles

This gives us the ability to show all handles including ones identified as untrusted. To look for evidence of code injection review Memory Sections located under Analysis Data -> Processes to see memory pages for every process. This particular malware contains no injected memory sections that Redline can find.

Finally use the "Strings" output to find additional evidence. You can search for interesting things like http://, https://, ".exe", or like the example below search for "cmd.exe" being run by WScript.exe (see Figure #6).

Figure 6: Use the Redline "Strings" output to find additional evidence

Other interesting strings to search for would be looking for common Windows system and network commands such as "finger", "net use", "netstat", etc.

Understanding normal activity is key when you start looking for badness. If you don't know what normal traffic activity looks like then you will be lost when trying to find said malware. One way is to better understand the operating system that you are doing analysis on, in this case windows. Understanding windows process structure helps, for example: csrss.exe is created by an instance of smss.exe and will have two or more instances running. The start time is within seconds of boot time for the first 2 instances (for Session 0 and I). Start times for additional instances occur as new sessions are created, although often only Sessions 0 and I are created. This information is available on the SANS DFIR poster. Looking at a Hierarchical Processes in Redline will reveal an instance of smss.exe that will spawn csrss.exe. Another item that is interesting to research is svchost.exe whose parent process is services.exe and runs five or more instances. It is used for running service DLLs. Windows will run multiple instances of svchost.exe, each using a unique "-k" parameter for grouping similar services. Malware authors often take advantage of the ubiquitous nature of svchost.exe and use it either directly or indirectly to hide their malware. They use it directly by installing the malware as a service in a legitimate instance of svchost.exe. Alternatively, they use it indirectly by trying to blend in with legitimate instances of svchost.exe, either by slightly misspelling the name (scvhost.exe) or spelling it correctly but placing it in a directory other than System32. All default installations of Windows 7, all service executables and all service DLLs are signed by Microsoft. This information is also available on the SANS DFIR poster and would be very helpful to review.

After reviewing this memory image with Mandiant Redline there is no smoking gun that can definitively say that this image has malware. Now we will take a look at an open source tool called Volatility.

Move over to the SIFT workstation, which was the device that took the image off the windows machine with the f-response tool. Open up a terminal and change directory to where the case files are located and find out the image information on the image file that you are interested in (see Figure 7).

Figure 7: Running volatility to discover the imageinfo of the memory image

The imageinfo output gives you information about the image (memory dump). The details about this image file can be seen in Figure #7. The suggested profile that you should pass as the parameter to -profile=PROFILE; there may be more than one profile suggestion if profiles are closely related. You can figure out which one is more appropriate by checking the "Image Type" field, which is blank for Service Pack 0 and filled in for other Service Packs.

To find the processes and DLLs use the following Volatility commands:

  • $ vol.py --profile=WinXPSP2x86 -f remote-system-memory11.img pslist

This command will list the processes of a system. It does not detect hidden or unlinked processes.

  • $ vol.py -profile=WinXPSP2x86 -f remote-system-memory11.img pstree

This command will view the process listing in a tree form. Child processes are indicated using indention and periods (see Figure #8).

  • $ vol.py -profile=WinXPSP2x86 -f remote-system-memory11.img psscan

This command will enumerate processes using pool tag scanning. This can find a process that previously terminated (inactive) and processes that have been hidden or unlinked by a rootkit (see Figure #9).

  • $ vol.py -profile=WinXPSP2x86 -f remote-system-memory11.img dlllist

This command will display a processes loaded DLLs and shows the command line used to start the process (including services) along with the DLL libraries for the process. Look for strange DLLs that have been injected into legitimate processes and DLLs with suspicious looking names.

This will produce a large output so if you narrow down your search to a specific process and want the DLL associated with it try the following command:

$ vol.py -profile=WinXPSP2x86 -f remote-system-memory11.img dlllist -p 1764

This command will display the process loaded DLLs for PID 1764 WScript.exe see Figure 10.

Figure 8: Example of using pstree, notice the child process indicated by the indention and periods

Figure 9: Example of using psscan, this can find processes that are hidden

Figure 10: Example of using dlllist to display the DLLs for the process WScript.exe

Volatility has many different features organized by plugins and categories. These categories include Image identification, processes and DLLs, processes memory, kernel memory and objects, networking, registry, crash dumps, hibernation, and malware/rootkits. We will look at the networking category below. To view active connections, use the connections command or to find connection structures using pool tag scanning, use the connscan command. Pool scanners is a technique when a piece of memory is allocated in windows, it's often allocated with a special tag which corresponds to the drive or subsystem to allocate the memory. You often find previous connections that have since been terminated. This command works for Windows XP and Windows 2003 Server only see Figure 11.

$ vol.py -profile=WinXPSP2x86 -f remote-system-memory005.img connscan

Figure 11: Output of connscan command notice the two connections to 10.10.3.180

So what is trying to communicate on the network from our Windows XP box to an internal 10.10.3.180? I don't have anything with that IP address on this network. Need to further investigate PID 1792 & 132. Running a psscan to enumerate processes does not show PID 1792 so that is a dead process but 132 shows up as svchost.exe. When I run a pstree to learn more information such as parent process indicated using indention and periods see Figure 12.

Figure 12: Svchost.exe has a parent process of wscript.exe which is child of explorer.exe

Another plugin available in Volatility called malfind extracts injected DLLs, injected code, unpacker stubs, API hook trampolines. Scans for any ANSI string, Unicode string, regular expression, or byte sequence in process or kernel driver memory. The syntax for using this plugin is below:

$ vol.py -profile=WinXPSP2x86 -f remote-system-memory005.img malfind -D memory/

This will dump all the processes with injected code in the directory called memory and when we are finished we can see all the files in Figure 13.

Figure 13: Results of using malfind has produced a number of dmp files for analysis

I will upload these dmp files to VirusTotal to see if we can identify any know issues. The only suspected dmp file that was flagged by virustotal (see Figure #14) was 0x370000 which is our f-response tool used to extract our memory image. This is a false positive.

Figure 14: Results from VirusTotal after uploading 0x370000.dmp files (f-response) that was generated after running malfind

Conclusion

After taking a SANS FOR508: Advanced Computer Forensic Analysis and Incident Response and learning the tools described in this article I started doing research on malware found on our network by Palo Alto firewalls and I used an example found by their firewall to research. This was a challenge because there were no obvious red flags when doing the investigation. Which is why I used this example to show that you will need to dig deep. What I learned:

  1. The memory image taken before infection showed communication with the windows box and the SIFT workstation, no other connections.
  2. The memory image taken after the infection showed communication with a 10.10.3.180 (2 instances) which is an internal IP address that I don't have on my test network.
  3. The two Process IDs (PID) that were related to IP address 10.10.3.180 are 1792 which was a dead process, and 132 svchost.exe which was the wscript.exe and has a parent process of 1648 explorer.exe.
  4. The svchost.exe (PID 132) is a generic host process for Windows Services and is used for running service DLLs and should always be a child of services.exe. It is a child process of wscript.exe and a clear indicator of wrong doing (see Figure 12).

More Stories By David Dodd

David J. Dodd is currently in the United States and holds a current 'Top Secret' DoD Clearance and is available for consulting on various Information Assurance projects. A former U.S. Marine with Avionics background in Electronic Countermeasures Systems. David has given talks at the San Diego Regional Security Conference and SDISSA, is a member of InfraGard, and contributes to Secure our eCity http://securingourecity.org. He works for Xerox as Information Security Officer City of San Diego & pbnetworks Inc. http://pbnetworks.net a Service Disabled Veteran Owned Small Business (SDVOSB) located in San Diego, CA and can be contacted by emailing: dave at pbnetworks.net.

@ThingsExpo Stories
Internet-of-Things discussions can end up either going down the consumer gadget rabbit hole or focused on the sort of data logging that industrial manufacturers have been doing forever. However, in fact, companies today are already using IoT data both to optimize their operational technology and to improve the experience of customer interactions in novel ways. In his session at @ThingsExpo, Gordon Haff, Red Hat Technology Evangelist, shared examples from a wide range of industries – including en...
Detecting internal user threats in the Big Data eco-system is challenging and cumbersome. Many organizations monitor internal usage of the Big Data eco-system using a set of alerts. This is not a scalable process given the increase in the number of alerts with the accelerating growth in data volume and user base. Organizations are increasingly leveraging machine learning to monitor only those data elements that are sensitive and critical, autonomously establish monitoring policies, and to detect...
"We're a cybersecurity firm that specializes in engineering security solutions both at the software and hardware level. Security cannot be an after-the-fact afterthought, which is what it's become," stated Richard Blech, Chief Executive Officer at Secure Channels, in this SYS-CON.tv interview at @ThingsExpo, held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA.
To get the most out of their data, successful companies are not focusing on queries and data lakes, they are actively integrating analytics into their operations with a data-first application development approach. Real-time adjustments to improve revenues, reduce costs, or mitigate risk rely on applications that minimize latency on a variety of data sources. Jack Norris reviews best practices to show how companies develop, deploy, and dynamically update these applications and how this data-first...
Intelligent Automation is now one of the key business imperatives for CIOs and CISOs impacting all areas of business today. In his session at 21st Cloud Expo, Brian Boeggeman, VP Alliances & Partnerships at Ayehu, will talk about how business value is created and delivered through intelligent automation to today’s enterprises. The open ecosystem platform approach toward Intelligent Automation that Ayehu delivers to the market is core to enabling the creation of the self-driving enterprise.
The question before companies today is not whether to become intelligent, it’s a question of how and how fast. The key is to adopt and deploy an intelligent application strategy while simultaneously preparing to scale that intelligence. In her session at 21st Cloud Expo, Sangeeta Chakraborty, Chief Customer Officer at Ayasdi, will provide a tactical framework to become a truly intelligent enterprise, including how to identify the right applications for AI, how to build a Center of Excellence to ...
SYS-CON Events announced today that Massive Networks will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Massive Networks mission is simple. To help your business operate seamlessly with fast, reliable, and secure internet and network solutions. Improve your customer's experience with outstanding connections to your cloud.
SYS-CON Events announced today that Datera, that offers a radically new data management architecture, has been named "Exhibitor" of SYS-CON's 21st International Cloud Expo ®, which will take place on Oct 31 - Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Datera is transforming the traditional datacenter model through modern cloud simplicity. The technology industry is at another major inflection point. The rise of mobile, the Internet of Things, data storage and Big...
Everything run by electricity will eventually be connected to the Internet. Get ahead of the Internet of Things revolution and join Akvelon expert and IoT industry leader, Sergey Grebnov, in his session at @ThingsExpo, for an educational dive into the world of managing your home, workplace and all the devices they contain with the power of machine-based AI and intelligent Bot services for a completely streamlined experience.
Because IoT devices are deployed in mission-critical environments more than ever before, it’s increasingly imperative they be truly smart. IoT sensors simply stockpiling data isn’t useful. IoT must be artificially and naturally intelligent in order to provide more value In his session at @ThingsExpo, John Crupi, Vice President and Engineering System Architect at Greenwave Systems, will discuss how IoT artificial intelligence (AI) can be carried out via edge analytics and machine learning techn...
In his session at @ThingsExpo, Arvind Radhakrishnen discussed how IoT offers new business models in banking and financial services organizations with the capability to revolutionize products, payments, channels, business processes and asset management built on strong architectural foundation. The following topics were covered: How IoT stands to impact various business parameters including customer experience, cost and risk management within BFS organizations.
SYS-CON Events announced today that CA Technologies has been named "Platinum Sponsor" of SYS-CON's 21st International Cloud Expo®, which will take place October 31-November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. CA Technologies helps customers succeed in a future where every business - from apparel to energy - is being rewritten by software. From planning to development to management to security, CA creates software that fuels transformation for companies in the applic...
From 2013, NTT Communications has been providing cPaaS service, SkyWay. Its customer’s expectations for leveraging WebRTC technology are not only typical real-time communication use cases such as Web conference, remote education, but also IoT use cases such as remote camera monitoring, smart-glass, and robotic. Because of this, NTT Communications has numerous IoT business use-cases that its customers are developing on top of PaaS. WebRTC will lead IoT businesses to be more innovative and address...
SYS-CON Events announced today that CA Technologies has been named “Platinum Sponsor” of SYS-CON's 21st International Cloud Expo®, which will take place October 31-November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. CA Technologies helps customers succeed in a future where every business – from apparel to energy – is being rewritten by software. From planning to development to management to security, CA creates software that fuels transformation for companies in the applic...
Internet of @ThingsExpo, taking place October 31 - November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with 21st Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The Internet of Things (IoT) is the most profound change in personal and enterprise IT since the creation of the Worldwide Web more than 20 years ago. All major researchers estimate there will be tens of billions devic...
WebRTC is great technology to build your own communication tools. It will be even more exciting experience it with advanced devices, such as a 360 Camera, 360 microphone, and a depth sensor camera. In his session at @ThingsExpo, Masashi Ganeko, a manager at INFOCOM Corporation, will introduce two experimental projects from his team and what they learned from them. "Shotoku Tamago" uses the robot audition software HARK to track speakers in 360 video of a remote party. "Virtual Teleport" uses a...
SYS-CON Events announced today that Calligo has been named “Bronze Sponsor” of SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Calligo is an innovative cloud service provider offering mid-sized companies the highest levels of data privacy. Calligo offers unparalleled application performance guarantees, commercial flexibility and a personalized support service from its globally located cloud platform...
SYS-CON Events announced today that Elastifile will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 - Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Elastifile Cloud File System (ECFS) is software-defined data infrastructure designed for seamless and efficient management of dynamic workloads across heterogeneous environments. Elastifile provides the architecture needed to optimize your hybrid cloud environment, by facilitating efficient...
SYS-CON Events announced today that Cloudistics, an on-premises cloud computing company, has been named “Bronze Sponsor” of SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Launched in 2016, Cloudistics helps anyone bring the power of the cloud to the data center in an easy-to-use, on- premises cloud platform that automatically provides high performance resources for all types of applications: Docke...
SYS-CON Events announced today that Golden Gate University will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Since 1901, non-profit Golden Gate University (GGU) has been helping adults achieve their professional goals by providing high quality, practice-based undergraduate and graduate educational programs in law, taxation, business and related professions. Many of its courses are taug...