Welcome!

Cloud Security Authors: Elizabeth White, Liz McMillan, Pat Romanski, Zakia Bouachraoui, Yeshim Deniz

Related Topics: Cloud Security, @CloudExpo

Cloud Security: Blog Post

Can You Measure the Success of Security Products? | @CloudExpo #Cloud

Buy security products based on ROI and not FUD (Fear, Uncertainty and Doubt)

As we have seen the growth in security challenges across the organization, we have also seen the growth in security spending and number of products that an enterprise buys. But have we, as an industry, been able to show that we are better off or worse? There is no clear yardstick to measure if a certain security product is making an enterprise more secure and if it I delivering the ROI that it promised. How do we get out of this FUD (fear, uncertainty and doubt) driven spending and get to a point where we measure impact based on risk and assess value more objectively.

I have been involved in producing and selling security products for two decades now. As a security vendor you always feel that there are too many products and tools out there and it's not easy to grab the buyer's attention. However, during the last four years or so things have accelerated considerably and we are all facing an explosive growth in the number of cyber security companies, product categories and a multitude of conflicting protections against the new, more sophisticated, more vicious, more targeted, attacks. I often ask myself what would I do if I were the buyer? How would I decide where to spend my budget? Especially since it's really difficult to assess the benefit of one product or compare the benefit of one product over another. There is no clearly defined yardstick to measure if a certain security product has made an enterprise more secure and is delivering the effectiveness it promised. Anton Chuvakin addresses this issue in his blog RSA 2016: Musings and Contemplations. I really laughed when he quoted an older article:

"You're proposing to build a box with a light on top of it. The light is supposed to go off when you carry the box into a room that has a Unicorn in it. How do you show that it works?"

I believe that a lot of buyers find themselves buying such boxes never knowing whether they will really do the job. We as an industry often steer away from making any promises that the light will in fact go off.

If I were a buyer, here are some things I would consider. Or really examples of applying logic vs. swallowing FUD:

  • Will lights go off when the unicorn shows up? Since you don't know, you have to figure out a test that will give you some confidence. Having spent some time in the endpoint security business I will provide the example malware protection. The unicorn here is a 0 day malware. Depending on how much time and people and budget you have for this, you can plan a very elaborate test and a bake-off between several vendors. Even if you don't, you must do the simplest due diligence, which is testing the product with known malware. A product that purports to identify 0 days must be able to identify any old malware. You can do a test of your own with existing malware or ask the vendor for three party lab reports. While this is a controlled environment, any outcome that shows less than 100% detection rate should be a strong red signal that you will not be protected from 0 days.
  • Layered defense? When a vendor is offering you a new product and you remind the vendor that you already have 10 appliances fulfilling 20 different security roles and you happen to already have a product in the same category as theirs, they might pull the "layered defense" card, or you might think that maybe they provide a layer that you are missing. How many layers do you need exactly? A lot has been said about the risk-based approach. I'm afraid that there is no better alternative. Enterprises need to assess their risks, understand what risk mitigations they already have, what risks they do not mitigate and what residual risk they are willing to live with. Once you do this exercise (which needs to be repeated and refreshed by new eyes every time), then you should go figure what products will complete your current defenses (sometimes it will be products you already have but do not utilize correctly; sometimes you will need to look for new products out there).
  • Listen to your own story. If there's anything that should teach you what you need to do next and provide you with the best feedback loop to your risk assessment, it is your security incidents. Almost all Incident Response plans or playbooks that I have ever seen have a "lessons learned" section in the end. While some enterprise security teams take this seriously, many teams find it very difficult to discuss and rehash solved cases or they are already on the next case and don't have the time or focus to learn from past incidents. Here we really need to learn from militaries that have been in the business of defense long before our industry came to be. I'll be surprised if there is any good military unit anywhere in the world that doesn't conduct serious debriefings after every major incident. This is how enterprises should consider their lessons-learned sessions. Even more important, perhaps, enterprises need to keep tabs of all incidents and see the trends that are emerging in a nutshell:
  • Incident Trends: Number of incidents increasing lately? This is a clear indication that something may be wrong with the security posture. The incident trends should be measured continuously and monitored against milestones like new security products deployed, employee education, etc. This way, you can measure the impact of what you do.
  • Incident Type Trends: Measuring the trends of different types of incidents is imperative. For example, if phishing incidents are growing, maybe the email security solutions are not effective (there can be other reasons of course). Understanding which types of incidents are most prevalent in the organization can help you decide the security strategy and need for security education.
  • These are just examples of how I think security buyers should be thinking. The bottom line is: FUD-NO! Test everything-Yes, apply logic-Yes.

    More Stories By Dan Sarel

    Dan Sarel is VP of Products and co-founder of Demisto. His interest in the cyber security world started about two decades ago. During these years he managed and helped design many security products in several disciplines. Building on his military experience that included programming and systems analysis training he started his career as a systems engineer at Radguard – at the time a pioneer in IPSec VPNs and PKI products.

    Dan went on to manage a sales engineering team and later worked on strategy and competitive intelligence projects at the company. Having served in other startups Dan moved to manage the VPN products at Check Point Software and ended up as the company’s Director of Product Management, overseeing the company’s successful enterprise products such as Firewall-1 and VPN-1. In 2006 Dan joined Sentrigo as one of its first employees. Dan was VP Product Management and Biz Dev during the short time it took to turn the company to a leader in Database Security and its subsequent acquisition by McAfee. At McAfee/ Intel Security Dan first served as VP Database Security and then accepted the role of VP Endpoint Security Strategy. Dan’s latest company is Demisto where he is a co-founder and VP of Product. At Demisto Dan and his colleagues are aiming high – providing the world’s most advanced security operations and incident response platform. Dan plans to create great security products for as long as he can.

    Comments (0)

    Share your thoughts on this story.

    Add your comment
    You must be signed in to add a comment. Sign-in | Register

    In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


    IoT & Smart Cities Stories
    At CloudEXPO Silicon Valley, June 24-26, 2019, Digital Transformation (DX) is a major focus with expanded DevOpsSUMMIT and FinTechEXPO programs within the DXWorldEXPO agenda. Successful transformation requires a laser focus on being data-driven and on using all the tools available that enable transformation if they plan to survive over the long term. A total of 88% of Fortune 500 companies from a generation ago are now out of business. Only 12% still survive. Similar percentages are found throug...
    CloudEXPO has been the M&A capital for Cloud companies for more than a decade with memorable acquisition news stories which came out of CloudEXPO expo floor. DevOpsSUMMIT New York faculty member Greg Bledsoe shared his views on IBM's Red Hat acquisition live from NASDAQ floor. Acquisition news was announced during CloudEXPO New York which took place November 12-13, 2019 in New York City.
    OpsRamp is an enterprise IT operation platform provided by US-based OpsRamp, Inc. It provides SaaS services through support for increasingly complex cloud and hybrid computing environments from system operation to service management. The OpsRamp platform is a SaaS-based, multi-tenant solution that enables enterprise IT organizations and cloud service providers like JBS the flexibility and control they need to manage and monitor today's hybrid, multi-cloud infrastructure, applications, and wor...
    The Master of Science in Artificial Intelligence (MSAI) provides a comprehensive framework of theory and practice in the emerging field of AI. The program delivers the foundational knowledge needed to explore both key contextual areas and complex technical applications of AI systems. Curriculum incorporates elements of data science, robotics, and machine learning-enabling you to pursue a holistic and interdisciplinary course of study while preparing for a position in AI research, operations, ...
    Codete accelerates their clients growth through technological expertise and experience. Codite team works with organizations to meet the challenges that digitalization presents. Their clients include digital start-ups as well as established enterprises in the IT industry. To stay competitive in a highly innovative IT industry, strong R&D departments and bold spin-off initiatives is a must. Codete Data Science and Software Architects teams help corporate clients to stay up to date with the mod...
    Tapping into blockchain revolution early enough translates into a substantial business competitiveness advantage. Codete comprehensively develops custom, blockchain-based business solutions, founded on the most advanced cryptographic innovations, and striking a balance point between complexity of the technologies used in quickly-changing stack building, business impact, and cost-effectiveness. Codete researches and provides business consultancy in the field of single most thrilling innovative te...
    Atmosera delivers modern cloud services that maximize the advantages of cloud-based infrastructures. Offering private, hybrid, and public cloud solutions, Atmosera works closely with customers to engineer, deploy, and operate cloud architectures with advanced services that deliver strategic business outcomes. Atmosera's expertise simplifies the process of cloud transformation and our 20+ years of experience managing complex IT environments provides our customers with the confidence and trust tha...
    Darktrace is the world's leading AI company for cyber security. Created by mathematicians from the University of Cambridge, Darktrace's Enterprise Immune System is the first non-consumer application of machine learning to work at scale, across all network types, from physical, virtualized, and cloud, through to IoT and industrial control systems. Installed as a self-configuring cyber defense platform, Darktrace continuously learns what is ‘normal' for all devices and users, updating its understa...
    With the introduction of IoT and Smart Living in every aspect of our lives, one question has become relevant: What are the security implications? To answer this, first we have to look and explore the security models of the technologies that IoT is founded upon. In his session at @ThingsExpo, Nevi Kaja, a Research Engineer at Ford Motor Company, discussed some of the security challenges of the IoT infrastructure and related how these aspects impact Smart Living. The material was delivered interac...
    Intel is an American multinational corporation and technology company headquartered in Santa Clara, California, in the Silicon Valley. It is the world's second largest and second highest valued semiconductor chip maker based on revenue after being overtaken by Samsung, and is the inventor of the x86 series of microprocessors, the processors found in most personal computers (PCs). Intel supplies processors for computer system manufacturers such as Apple, Lenovo, HP, and Dell. Intel also manufactu...