Good afternoon.  My real name is whatever that says.  Thank you for coming.  It is awesome to be here.  Everyone who is affiliated with DEF CON this is phenomenal.  The goons and this is overwhelming and once you get the letter that says you have been accepted, you think, oh, my God, I have been accepted.  Thank you for you guys that are coming and standing in the room.  I'm going to keep going.  
[ LAUGHTER ]
What I'm here to talk about today is the "Beyond the Scan: The Value Proposition of Vulnerability Assessment."
>>  Your talk has been cancelled.  We have a tradition here at DEF CON.  Wait, I got that mixed up.  All right.  I do this all day.  DEF CON, our brand-new speaker!  
>>  DEF CON 23.  Thank you very much.  
[ APPLAUSE ] 
Water chaser.  So, a lot of us already do vulnerability assessment and we are involved in scanning activity and we already know how that works and we are doing it in our organization.  There are a lot of aspects of performing and scanning that are not understood.  Maybe not by us, but our organizations.  I was the client that knew they needed to do this, but they were not necessarily all together knowing what they would do with that information once they got it.  And working with the clients and going through the narrative and performing the scans and I realized there is an opportunity for us, as performing this activity to really make more meaningful changes in our OS and you are going to hear me say this a couple of times, where the later (Indiscernible) happens and we were trying to impact the business.  
There is contact info on the CD, as well.  So, stay in touch.  You do vulnerability assessments and say, I'm your tool guy and a lot of people think that you set-up a scanner and click scan and that's it.  It is regarded as one of the least sexy things in info sector.  Vulnerability tends to get glossed over and in my opinion, it is a key component of the accessibility program and if it gets overlooked, you are doing a disservice to your organization.  Hopefully, this is going to be useful to you when you go back to your organizations that you are supporting.  We are going to talk about "beyond the scan" and provide support.  Whether you are a blue team or a red team, we are doing the heavy lifting blue-collar security work.  That's what we are talking about.  
The next slide and we are talking about the human tester and the tools-based scanning.  The data by itself is very, very useful and it tells you a lot about your environment and what you are doing.  Without the human testers, you are just gathering data and making a bunch of noise.  I feel strongly when you apply the human testers, that's all of us, to the data that you gather, that's when the magic happens.  Just to reiterate, this is not a tool-based talk, we are using tools in a smarter way and having people know when to use these tools.  Does that make any sense?
We need to go a scan and we are going to give it to some new guy and let him scan the network.  I don't respond positively to that.  Please, don't do that.  You need to have the talented humans and in addition to having the humans and applying it to the data, a question that comes up a lot and perhaps you have heard this question, as well.  What should we scan? I like the longitudinal scans.  Scan everything.  Clicking and scanning is easy, but making sense of the data is the hard part.  When I get later on in the case studies, you have to do longitudinal studies over time.  If there is an IP stack, you should scan it.  Maybe, you are scanning very specific things that you are interested in.  To understand your network environment, you need to see everything that is on there and not just the known quantities that you have.  There are some interesting stories coming up and talking about that.  Humans that are looking at the data also scan all of the things.  
Another point and obviously, we can't avoid talking about (Indiscernible) because we are using this data.  I'm going to try and be as vendor specific as possible.  There are some names up.  They are all very, very good in their own way.  In your future engagements, you may need to pick one tool.  There is one tool that I'm using currently that is doing very fine.  All will work, and you have to work with your organization and decide which one is the best for you.  
I'm going to make you sit through a history lesson.  The security administrator tool for analyzing networks or SATAN was released in 1995.  They were not interested in marketing, obviously.  Anyone around in 1995? Thank goodness.  That was a big deal.  I remember reading in the trade magazines and this is the worst idea ever.  Why would you make a tool to make it easier to find vulnerabilities and the other half of the people thought; thank God, now I have a way to find the vulnerabilities.  The conversation that we had a year ago and firewalls were an expensive way to slow down your network and fire routers were fine.  We know that's not true.  To give you a state-of-the-art work back then, it was a long time ago and things were a lot different back then.  PC Magazine -- to show you know and for today and when we do scans all of the time.  There are millions of people scanning the Internet.  20 years ago, this would get you landed in jail.  Two decades ago and it is crazy to think this is a dramatic thing.  And one of the tools that came out after SATAN can scan 100 known vulnerabilities.  Think about that.  I have 100 in my pocket right now that many of you are trying to find, by the way.  The last time I checked, it was 160,000 different vulnerabilities.  20, going from 100 known things on Linux and now, almost a couple hundred thousand on many different platforms.  You can learn about these folks that created SATAN.  They wisely choose saint.  Now, it is like a Dan Brown novel.  Thank you, I wrote that myself.  It became a commercial product in 1998.  SATAN came out in '95 and all hell broke loose.  And if you remember from my slide, there are a lot of commercial products that exist.  From that humble beginning and now, an entirely different capability with the info sect.  Now, we are doing vulnerability assessment and looking across multiple platforms across a large number of hosts and that leads to vulnerability management.  I'm going to pause for a moment; it is not just performing the scan.  Vulnerability management means that now, that we have the information and know it, what are we going to do about it? You mitigate, transfer it, but you have to do something.  And now, we know about it and how to document it in an audible way.  Now, we have this thing and we know it is working and how do we know that the decision that we make today is going to be ongoing in the future.  The landscape could be very, very tiny and that could change.  Now, I care all of a sudden.  Vulnerability management includes all of those things.  
So, a couple of decades ago, scanners came out and yesterday yet, I still feel like in our industry, particularly when we start touching the business, we are still misunderstood.  I love that cartoon.  No one else likes it? We are still misunderstood and working with several clients over the last several years and one question that always comes up, all right, I have human testing.  And I have tool-based scanning.  If I get this new scanner, can I get (Indiscernible).  That's a conversation that I started to have.  I started to talk about the scanning activity relative to the OSI model and this is going to show certain tools that we might use and how they work relative to other tools.  Are we all familiar with the OSI model? Love it? Hate it? Down at the bottom we have electrons and protons and going through fiber.  If you are interested in TTP ports and if we are interested in client applications that are talking to a server somewhere. And now, we are higher up in the model and the host layers and it is giving us visibility into there.  And maybe, we use web inspect or Burke or in the host layers or in the network and wanting to sniff traffic and it goes on.  You can see already, when I drop the scanner in there, it is giving us a wider view than the other tools.  If the question comes up, we bought this great, expensive scanner, and what can we get rid of? It is not replacing something.  It gives us information that is useful and the only thing, which is capable of scanning the entire stack, is the humans, us.  They are actually doing the work and whether it is manually penetration testing on an app trying to break in or looking at the tools that it is trying to gather, the people are the most important.  I have gotten a lot of really good feedback and I would be interested to hear from you folks if this OSI slide is useful.  Starting to talk to the business, it helps them to see that we have a lot to deal with and securing the applications and the systems and there are lots of different ways to gather that information.  
To kind of illustrate that in a different way.  We can see our tools are very, very useful for the longitudinal scans and doing scans over a long period of time.  And comparing results and seeing what is our security posture today and as that changes, it moves on.  The people, don't let anyone (Indiscernible) decide to start use tools and ideas and going to replace other tools and it is also not going to replace your people.  You still have to have people who know what that data is and you can read through a list of things that are on there.  There is a short list of the things that we're good at that the tools are not.  It is limited without the security professionals.  
All right.  This is an interesting example of that.  I told the story earlier and it worked out.  We did an assessment for a client and came back with a lot of http vulnerabilities and Apache things.  We asked them to come back and update the Apache and they came back and said, obviously, they are S servers.  I can see the banner and assure you there is a demon running.  Here is a false/positive and here is why.  And when you are sharing with operations and business units and other people.  I found a really interesting thing and when you use words, the same word might mean something completely different depending on who you are talking to.  If I say interface it is going to mean three different things and if you have this experience, as well.  And if you are talking to a programmer and you think API and it is an API, that is allowing things and if you talk to routing and switching guys, they are talking about a web-based guitar that is attaching to a network.  It is important whom you are talking to and language can be a confusing thing.  You have to know what you are talking about.  You need to know your environment, as well.  On a client engagement and how to scan a very large network that had a global presence.  We had distributed scan engines that were in various networks around the world.  There is one in Argentina.  Intuitively and backhauled through Houston.  If I wanted to scan Argentina or Brazil, it went by way of Houston first.  Architecturally the smart decision to scan North America to South America.  If you have a sufficiently large (Indiscernible) you really need to know how those packets are getting to where they are getting to.  Understand the architecture of your network.  Now, we know which language to use and our environment and now, we need to know our organization.  
Don't panic, I'm going to walk you through this whole slide.  I know there are a lot of boxes going on.  There are a lot of ways that the scanner allows us to slice and dice data and we can look at specific vulnerabilities and hosts and group them in many (Indiscernible) and get in the actual day that we are giving to the organization, we have to make very conscious decisions on how to organize the data and how to ultimately fix the problems that we are going to identify.  
You need to capitalize in your organization and focus solely on what is wrong with your box.  Let's get into the case study part of it.  Identifying the potential vulnerabilities.  What we are trying to figure out here and it might be a little obvious, but (Indiscernible) and we want to figure out what are our attempted targets and what hatches are missing from those targets and what is our environment is configured properly.  We found that SIDS was exposed to the open environment.  Many of us panicked.  Many of them were on the Xbox's and in 2013.  We asked the question, why is it set-up that way? Why would you put them on the Internet that way?  It is never a good Windows and of course, there are VPN's and a million way to do those things and instead of saying close those ports and have a nice day.  To realize there is a gap in the understanding and how they would be supported in the first place.  There is a related issue where the boxes need access to require IP-based authentication and that's how they let them in.  Not only were the ports exposed, but they had static NATS set-up.  When they set-up the NATing and they put an inbound firewall rule and anything and everything was coming into the inboxes.  We identified some technical problems and also, a business problem that was broken.  They should have never allowed that firewall to be put in place as it was, right?
There were multiple decisions that were left in and they were unchecked and we were afraid to change it because something might stop working, so we are going to leave it alone.  So, we resulted -- we found not only those vulnerabilities, but there was poor documentation and we couldn't figure out why it was set-up the way it was and the broken process that allowed the firewall get in the first place and lack of understanding of what the specific requirements were.  At this same organization, we found there is an obsolete version of Java.  Obsolete Java shouldn't be a surprise and having it result in hundreds of vulnerabilities shouldn't be a surprise either.  But when we asked, why is it set-up the way that it is? Well, we have a specific application that requires that version of Java.  We called the vendor, which no one had done.  Do you require this specific version of Java? No, we require at least, that version of Java.  A very subtle detail was left out at the client's site.  If I were a cynical person and for those of you that know me, I kind of am.  I think they were using that as an excuse of not updating Java in a large enter prize and there was broken process and all of these things were benefits that the organization got long after the scan was over.  It happened when we started working with the business units and the support organizations to clear these things up.  
So, it is going to provide a scheming tool and I it will provide information.  This is another thing that maybe obvious.  I worked with another client that were surprised to find out; not only will it tell me (Indiscernible) and a good product will also fix it.  This client only saw the result of the scan in a spreadsheet and the IP address name and what went wrong with it.  When we provided the information, it became hugely valuable to this organization.  No, they knew not only what was wrong and how to fix it.  It is impossible for any of us to know everything.  They may not know.  Make sure you take advantage of the information that is in the tool that you are using.  I have a pretty good idea of what is on my home network and what is on everywhere across the whole organization and that might not be true.  Here is where you have to make a decision.  Are you going to scan a specific set of hosts or everything that is out there? Those are two different activities and problems, but I think that we should do them both.  A client that I worked with scanned all of the cyber blocks that we knew about.  And at the end of the engagement, we decided, let's scan the stuff that doesn't exist and the networks that are completely undocumented.  Obviously, the switches didn't turn themselves on and the hosts didn't turn themselves on.  And someone did it in the organization.  Maybe, we can find parts of the network that we didn't know was there.  And the range of IP addresses that work on your networks and try and figure out what is there.  
One client that I'm working with, when you do that, scan these huge blocks, you end up creating a very, very nice inventory of what exists on your network and the information management, it is helpful to share that information with them.  Now, there is a check for the day-to-day data that have.  At that point, it is not a security related issue as it is asset management.  Still very important if you don't know what is on your network and how can you possibly secure it? A large number of IP cameras were not on anybody's list and it was a little embedded system and not a computer so much that anyone cared about.  Many of them were configured incorrectly with default credentials and just because IP didn't care; it was a huge win for the client because they fixed what could have been a very big problem.  The same client and again, it wasn't a security-related issue, but we found some Nintendo machines on the network.  It was a children's hospital, so it was okay.  It illustrated the point if IT didn't know they were there and then, you have to ask the question, I have gaming consoles that are on the same network as my production equipment.  And that drives the question, how is our network configured and are we doing it the appropriate way? And the best way that we can.  
The next company, they had a lot of oilrigs in Mexico.  Every platform has exactly four hosts on it and that's exactly what you are going to find.  And we did the scan and most of the platforms did have exactly four hosts and that's fine.  There were a couple that had more than that.  We went back to the client and said there are hot IP addresses coming off of this rig.  They were interested in what those feeds were.  Finding out what is on our network in the first place.  And an asset management is very much network and if you don't find out what is on it, you are screwed at what you are doing.  I think that's very, very true and why we should spend our time making sure that we get to every corner of our network that we can.  
Speaking with software management.  Do you know each machine that is in your enterprise that you are scanning? And doing the credential scans and do we scan with or without credentials and obviously, you are going to get a lot of information and the scanning tool is going to allow us to enumerate everything that is on the box and once we do the scan, we can go back and run the reports and find out if there is anything there and maybe if you standardize on Internet explorer and what about BitTorrent and Kazaa.  
Compliance and on some level we are flinging things and scanning the packets and you can learn about what the organization is and what the activities that are needed to make them more secure.  Is compliance an issue for you? Compliance an issue for anyone? A few hands.  If it is HIPPA or GOBA, your scanner should be able to tell you (Indiscernible) relative to something else.  That could be important if you do work with the folks or the client's folks and make sure that you are taking advantage of the fact that you can gather that information and make sure that you are compliant of what your health status is.  Strategically, the information that we are gathering should be important to our networks as well.  I will use the word again because I like it; I'm talking about these longitudinal scans over time.  It is not a scan at the point in time (Indiscernible) the trends that are developing and you need to do these scans.  That can be very important in an unexpected way.  If you are looking and continuing to see that, you find vulnerabilities and it is fixed and maybe, you have a resourcing problem.  If something happens and all of a sudden, bash is vulnerable.  Strategically, it can be very, very important and tactically, it can be very, very important, as well.  
Anyone work in health care or work in environments that have industrial control systems? Yeah, more hands.  So, this is -- this conversation is going on a lot in our industry now and it is going to continue because these systems in particular are not the same as the other computers on our networks.  The biomedical devices and control systems and the things that help run plants and refineries and whatnot are highly, highly specialized and they are used to getting (Indiscernible) specific.  As you are throwing traffic at them -- the hot button for me.  Even though the FDA is getting stronger in forcing equipment manufacturers, a lot of time you find that the equipment manufacturers hide behind the FDA.  Does anyone know specifically what the FDA fight is and have to deal with it now? Okay, the food and drug administration says before you sell a medical device, you have to go through the FDA and once the FDA says this is fine, you can't make any changes to it.  That becomes a problem when I say that you need to patch your x-ray machine, the -- the problem with health care and sometimes, it can lead to poor design decisions and with radiology divisions and that's a problem in this hospital.  The vendor supported it with FTP and all of these stations had FTP running on it.  And setting it up and the logons that are enabled and they enabled the FTP route and who this the world would ever do that? Buy me a beer and I will tell you who did that.  Everyone on this network had FTP access to the file system and it is the worst thing possibly ever.  Normally, if you scan and see FTP and maybe the return valves at the refineries and the technology and they are not the same as information technology at all.  Shutting down a plant to do maintenance in computers and their patch cycles maybe in months or years.  If you find yourself working on these types (Indiscernible) you need to make sure that the OTP people understand that once you find the vulnerability, the OTP patches are not going to be that simple.  They are purpose built and very, very specific and if you get unusual traffic, (Indiscernible) at all.  Be very, very cautious.  I know that I'm running out of time and basically, the story I'm trying to tell you is that scanning has been around for a long time and it has matured to be a lot more useful and becoming shared appropriately and look for the opportunities to share this data with your organization and if you come up with a data manageability program that shares all of these processes, it will help you.  Thank you very much for sitting through all of that and enjoy DEF CON 23.  Thank you.  
[ APPLAUSE ]