Welcome!

Cloud Security Authors: Zakia Bouachraoui, Elizabeth White, Pat Romanski, Yeshim Deniz, Liz McMillan

Related Topics: Cloud Security, @CloudExpo

Cloud Security: Blog Post

Can You Measure the Success of Security Products? | @CloudExpo #Cloud

Buy security products based on ROI and not FUD (Fear, Uncertainty and Doubt)

As we have seen the growth in security challenges across the organization, we have also seen the growth in security spending and number of products that an enterprise buys. But have we, as an industry, been able to show that we are better off or worse? There is no clear yardstick to measure if a certain security product is making an enterprise more secure and if it I delivering the ROI that it promised. How do we get out of this FUD (fear, uncertainty and doubt) driven spending and get to a point where we measure impact based on risk and assess value more objectively.

I have been involved in producing and selling security products for two decades now. As a security vendor you always feel that there are too many products and tools out there and it's not easy to grab the buyer's attention. However, during the last four years or so things have accelerated considerably and we are all facing an explosive growth in the number of cyber security companies, product categories and a multitude of conflicting protections against the new, more sophisticated, more vicious, more targeted, attacks. I often ask myself what would I do if I were the buyer? How would I decide where to spend my budget? Especially since it's really difficult to assess the benefit of one product or compare the benefit of one product over another. There is no clearly defined yardstick to measure if a certain security product has made an enterprise more secure and is delivering the effectiveness it promised. Anton Chuvakin addresses this issue in his blog RSA 2016: Musings and Contemplations. I really laughed when he quoted an older article:

"You're proposing to build a box with a light on top of it. The light is supposed to go off when you carry the box into a room that has a Unicorn in it. How do you show that it works?"

I believe that a lot of buyers find themselves buying such boxes never knowing whether they will really do the job. We as an industry often steer away from making any promises that the light will in fact go off.

If I were a buyer, here are some things I would consider. Or really examples of applying logic vs. swallowing FUD:

  • Will lights go off when the unicorn shows up? Since you don't know, you have to figure out a test that will give you some confidence. Having spent some time in the endpoint security business I will provide the example malware protection. The unicorn here is a 0 day malware. Depending on how much time and people and budget you have for this, you can plan a very elaborate test and a bake-off between several vendors. Even if you don't, you must do the simplest due diligence, which is testing the product with known malware. A product that purports to identify 0 days must be able to identify any old malware. You can do a test of your own with existing malware or ask the vendor for three party lab reports. While this is a controlled environment, any outcome that shows less than 100% detection rate should be a strong red signal that you will not be protected from 0 days.
  • Layered defense? When a vendor is offering you a new product and you remind the vendor that you already have 10 appliances fulfilling 20 different security roles and you happen to already have a product in the same category as theirs, they might pull the "layered defense" card, or you might think that maybe they provide a layer that you are missing. How many layers do you need exactly? A lot has been said about the risk-based approach. I'm afraid that there is no better alternative. Enterprises need to assess their risks, understand what risk mitigations they already have, what risks they do not mitigate and what residual risk they are willing to live with. Once you do this exercise (which needs to be repeated and refreshed by new eyes every time), then you should go figure what products will complete your current defenses (sometimes it will be products you already have but do not utilize correctly; sometimes you will need to look for new products out there).
  • Listen to your own story. If there's anything that should teach you what you need to do next and provide you with the best feedback loop to your risk assessment, it is your security incidents. Almost all Incident Response plans or playbooks that I have ever seen have a "lessons learned" section in the end. While some enterprise security teams take this seriously, many teams find it very difficult to discuss and rehash solved cases or they are already on the next case and don't have the time or focus to learn from past incidents. Here we really need to learn from militaries that have been in the business of defense long before our industry came to be. I'll be surprised if there is any good military unit anywhere in the world that doesn't conduct serious debriefings after every major incident. This is how enterprises should consider their lessons-learned sessions. Even more important, perhaps, enterprises need to keep tabs of all incidents and see the trends that are emerging in a nutshell:
  • Incident Trends: Number of incidents increasing lately? This is a clear indication that something may be wrong with the security posture. The incident trends should be measured continuously and monitored against milestones like new security products deployed, employee education, etc. This way, you can measure the impact of what you do.
  • Incident Type Trends: Measuring the trends of different types of incidents is imperative. For example, if phishing incidents are growing, maybe the email security solutions are not effective (there can be other reasons of course). Understanding which types of incidents are most prevalent in the organization can help you decide the security strategy and need for security education.
  • These are just examples of how I think security buyers should be thinking. The bottom line is: FUD-NO! Test everything-Yes, apply logic-Yes.

    More Stories By Dan Sarel

    Dan Sarel is VP of Products and co-founder of Demisto. His interest in the cyber security world started about two decades ago. During these years he managed and helped design many security products in several disciplines. Building on his military experience that included programming and systems analysis training he started his career as a systems engineer at Radguard – at the time a pioneer in IPSec VPNs and PKI products.

    Dan went on to manage a sales engineering team and later worked on strategy and competitive intelligence projects at the company. Having served in other startups Dan moved to manage the VPN products at Check Point Software and ended up as the company’s Director of Product Management, overseeing the company’s successful enterprise products such as Firewall-1 and VPN-1. In 2006 Dan joined Sentrigo as one of its first employees. Dan was VP Product Management and Biz Dev during the short time it took to turn the company to a leader in Database Security and its subsequent acquisition by McAfee. At McAfee/ Intel Security Dan first served as VP Database Security and then accepted the role of VP Endpoint Security Strategy. Dan’s latest company is Demisto where he is a co-founder and VP of Product. At Demisto Dan and his colleagues are aiming high – providing the world’s most advanced security operations and incident response platform. Dan plans to create great security products for as long as he can.

    Comments (0)

    Share your thoughts on this story.

    Add your comment
    You must be signed in to add a comment. Sign-in | Register

    In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


    IoT & Smart Cities Stories
    When talking IoT we often focus on the devices, the sensors, the hardware itself. The new smart appliances, the new smart or self-driving cars (which are amalgamations of many ‘things'). When we are looking at the world of IoT, we should take a step back, look at the big picture. What value are these devices providing. IoT is not about the devices, its about the data consumed and generated. The devices are tools, mechanisms, conduits. This paper discusses the considerations when dealing with the...
    Bill Schmarzo, author of "Big Data: Understanding How Data Powers Big Business" and "Big Data MBA: Driving Business Strategies with Data Science," is responsible for setting the strategy and defining the Big Data service offerings and capabilities for EMC Global Services Big Data Practice. As the CTO for the Big Data Practice, he is responsible for working with organizations to help them identify where and how to start their big data journeys. He's written several white papers, is an avid blogge...
    Dynatrace is an application performance management software company with products for the information technology departments and digital business owners of medium and large businesses. Building the Future of Monitoring with Artificial Intelligence. Today we can collect lots and lots of performance data. We build beautiful dashboards and even have fancy query languages to access and transform the data. Still performance data is a secret language only a couple of people understand. The more busine...
    If a machine can invent, does this mean the end of the patent system as we know it? The patent system, both in the US and Europe, allows companies to protect their inventions and helps foster innovation. However, Artificial Intelligence (AI) could be set to disrupt the patent system as we know it. This talk will examine how AI may change the patent landscape in the years to come. Furthermore, ways in which companies can best protect their AI related inventions will be examined from both a US and...
    Enterprises have taken advantage of IoT to achieve important revenue and cost advantages. What is less apparent is how incumbent enterprises operating at scale have, following success with IoT, built analytic, operations management and software development capabilities - ranging from autonomous vehicles to manageable robotics installations. They have embraced these capabilities as if they were Silicon Valley startups.
    Chris Matthieu is the President & CEO of Computes, inc. He brings 30 years of experience in development and launches of disruptive technologies to create new market opportunities as well as enhance enterprise product portfolios with emerging technologies. His most recent venture was Octoblu, a cross-protocol Internet of Things (IoT) mesh network platform, acquired by Citrix. Prior to co-founding Octoblu, Chris was founder of Nodester, an open-source Node.JS PaaS which was acquired by AppFog and ...
    The deluge of IoT sensor data collected from connected devices and the powerful AI required to make that data actionable are giving rise to a hybrid ecosystem in which cloud, on-prem and edge processes become interweaved. Attendees will learn how emerging composable infrastructure solutions deliver the adaptive architecture needed to manage this new data reality. Machine learning algorithms can better anticipate data storms and automate resources to support surges, including fully scalable GPU-c...
    Cloud-enabled transformation has evolved from cost saving measure to business innovation strategy -- one that combines the cloud with cognitive capabilities to drive market disruption. Learn how you can achieve the insight and agility you need to gain a competitive advantage. Industry-acclaimed CTO and cloud expert, Shankar Kalyana presents. Only the most exceptional IBMers are appointed with the rare distinction of IBM Fellow, the highest technical honor in the company. Shankar has also receive...
    The standardization of container runtimes and images has sparked the creation of an almost overwhelming number of new open source projects that build on and otherwise work with these specifications. Of course, there's Kubernetes, which orchestrates and manages collections of containers. It was one of the first and best-known examples of projects that make containers truly useful for production use. However, more recently, the container ecosystem has truly exploded. A service mesh like Istio addr...
    Business professionals no longer wonder if they'll migrate to the cloud; it's now a matter of when. The cloud environment has proved to be a major force in transitioning to an agile business model that enables quick decisions and fast implementation that solidify customer relationships. And when the cloud is combined with the power of cognitive computing, it drives innovation and transformation that achieves astounding competitive advantage.