Vulnerability Scanning: Identifying False Positives

It's been a while, but I appreciate the patience of my audience during these trying times.

I want to talk about vulnerability scanning today and false positives because I see so many misunderstandings and people addressing this incorrectly. I regularly hear popular opinions that vulnerability scanners produce above a 90% false positive rate. First off let's address a common misunderstanding between security personnel and administrators. Often times the security administrator wants the admin to relax certain security settings so that the scan will get every possible result. Why? This seems kind of backwards to the admin.  Why would you want to relax the security settings?  Because we are interested in finding every possible patch or configuration that needs to be fixed. Let's be honest, if I'm the Blackhat and I'm effectively attacking your network, I'm going to be looking for every possible attack vector. Some of these the administrators may "think" they have mitigated, but perhaps their mitigation is ineffective resulting in a control gap. What I said in the previous sentence is really the keyword too, mitigated.  The flaw is still there, just better masked.  Allowing the vulnerability scanner to detect every vulnerability possible is a greater mitigation, or at least can lead to greater mitigation. Also when we scan, we want accuracy. An open network reduces the opportunity to generate false positives.

False Positives! This is a biggie! In vulnerability scanning, a false positive is when the vulnerability scanner returns a finding when in fact that finding is not there, or in other words a vulnerability has been detected when in fact patches, configurations, or mitigations that address the vulnerability are already in place. Sounds fairly straight forward right? Well, it's not. This is an area that is regularly misdiagnosed because only the truest security professionals actually know how to properly diagnose it. For example, let's say we generate a report for a windows system and hand it off to an engineer or system administrator. They are likely to glance at the report, see an MS article, lookup up the associated KB article, and see if it is installed in Programs (Add/Remove Programs). Sounds logical, but there may be more to it. Much more in fact. Let's say that a patch deploys a new dynamic link library (dll) file. The newly installed .dll file may not be vulnerable, but if the old .dll file wasn't removed during installation, that system could still be vulnerable. Why? The answer can be found by asking what is a .dll file. In the most simplest of terms, a .dll file creates modularity by providing common functions through a single avenue. This means that if the .dll file is present, it can be called by malicious code. So, yes, maybe whatever app was using the .dll isn't vulnerable any longer since it points to the latest, but the vulnerable .dll is still present and thus vulnerable. If it is there it can be called, which makes it vulnerable.

This isn't the only mistake I've seen in identifying false positives.  Another common mistake comes from networks with short DHCP leases or removable hard drives (classified areas).  In practice I have seen very well trained engineers dismiss vulnerability scanning reports simply because either a lease expired and the IP can no longer be identified.  In the case of removable hard drives I've witnessed administrators ping for the IP listed in the report, and close the finding when the system is unreachable.  In the case of removable hard drives, I understand that there are certain circumstance to overcome, but typically from what I've seen removable hard drive environments are a 1:1.  That is to say that the hard disks are labeled to be for a specific system.

I've been conducting vulnerability scans for about nine years of my eleven years in information security.  During that time the one thing I hear regardless of the site is that vulnerability scanner findings are 90-99% false positives.  In practice, I'd be lying if I said I have never seen a false positive, but the fact of the matter is that if you know what you are looking for, the vulnerability scanner is typically correct.

Think about it for a second.  The programmers that are writing these scanner rules are not throwing darts in the air and hoping they hit something.  They are writing their code based off of requirements, most likely coming from CVE, BID, or vendor KB that identify what to specifically check.  Then they automate that check and release in an update or in the latest engine.  When you really take the time to break down a vulnerability scanner check, understand how the technology works, and compare the findings against CVE, BID, or vendor KB the finding is typically correct, and thus represent a vulnerability.

In conclusion, BEWARE --> BE KNOWLEDGEABLE --> BE THOROUGH!