The biggest mistake enterprises make when purchasing security products and services


Adrian Sanabria Senior Analyst, Enterprise Security Practiceat451 Research

Adrian Sanabria – Senior Analyst, Enterprise Security Practice at 451 Research

I missed this MSSP horror story when Nick Selby posted it a month ago, but suffice to say that I felt justified in commonly describing MSSP services as, “an opportunity to save time and money to have a security task done badly”. There are some really good MSSPs out there, but they tend to be small, focused on a small area, or both. The fact of the matter is that MSSPs don’t work when you try scaling them with people. If you get very, very good at technically automating the separation of signal and noise,then you have a chance to do MSSP well on a large scale. But of course, once we have a product that can do the bulk of filtering noise automatically, the need for MSSP services is suddenly greatly lessened.

Anyway, here’s Nick’s post – check it out before continuing.

There were three problems here:

  1. the monitoring appliance was in the wrong place
  2. the MSSP’s (human) services sucked
  3. the customer didn’t understand what they had contracted for, or really what they needed

This goes along with what I spoke about at RSA earlier this year, and one of the reasons I’m such a big fan of attack simulation products. I’ll explain both:

At RSA, my talk was about how CISOs can work with security startups without getting burned, but a majority of the talk was about the need for due diligence. Currently in the security industry, I feel like the general strategy is “buy a bunch of security products, implement them sloppily and call it ‘defense in depth'”. I’m with Rick Holland here on calling this approach a classic case of “expense in depth” – the attempt to attain a secure posture simply by spending money on things. The example mentioned in Nick’s post is actually the best of many worst-case scenarios! The worst case is that a product you purchase to make you more secure instead puts you directly at risk. VeraCode found a few years back that security software was second-worst in the software world for code quality (report here, pg 5).

Attack simulation products, like vThreat, SafeBreach, AttackIQ, etc are exciting to me, because they can accurately simulate the portions of a breach that you’ve bought all these products to detect. There are two exciting use cases I see here: security controls testing and incident response training.

Putting this all together, how could this MSSP have been engaged differently, if the unfortunate client had the benefit of all this advice 3+ years ago?

  • The due diligence process that should be followed for any security product purchase should have included a test of both the technical part of the MSSP product, and the human (services) part. Notify the MSSP of your intent to test their service. Simulate a variety of attacks that the product should catch. Did they alert you? If not, call them and see if they detected the simulations. Still nothing? Is it plugged into the right place? Did the product just fail?
  • Create a process to regularly perform these simulations to ensure the product is still working. Sometimes, things get unplugged or turned off, and people forget to turn them back on. Ask me how I know (in person) some time.
  • Walk through a full simulation of what you want the MSSP to do, and ensure the contract includes what you’re expecting. Also ensure the contract allows you to regularly test their services to ensure that the quality of the service, which is largely invisible to you, hasn’t declined over time. I’d recommend testing monthly, at least.
  • Ensure the contract allows you to ditch the MSSP if service declines to an unacceptable point. I’d say, if they fail to catch your simulations, or real incidents for 2 months in a row or more, they should be in breach of contract. Come up with your own requirements, but make sure you aren’t stuck in a bad MSSP relationship you can’t get out of.
  • Also, this need not be a manual process – almost the attack simulations and reporting on the results should be easy to automate. In fact, the average security practitioner or even developercould probably just write some scripts to do the automation themselves if you want to be thrifty about it. No need to hire any additional staff or buy anything additional if you don’t want to. The attack simulation services just take things up a notch, ensure attacks match what criminals are currently doing, have a wider variety than you’d be able to easily put together yourself, etc.
  • Due diligence isn’t just something you do before buying a product – it needs to done after implementation and throughout the product lifecycle.

In conclusion, most of our security products offer or promise some level of protection, but the level of service or value we get is largely invisible until something bad happens. Worse, when something bad does happen, it’s still invisible to us, which is why the average dwell time before breaches are detected is still in the hundreds of days. Performing solid due diligence before buying a product, after implementation and throughout the product lifecycle isn’t the whole solution, but it’s an important part.

Good luck out there, and to quote my friend Javvad Malik‍, “stay secure my friends”.