00:00:00.467,00:00:07.207 > Alright. So this is hacker machine interface: State of the union for SCADA HMI 00:00:07.207,00:00:14.147 vulnerabilities. It's not um the title slide here is not about stuffy shirts and racks. Um I 00:00:14.147,00:00:19.019 mean this is about hardcore exploitation in this talk. We are going to cover a in depth 00:00:19.019,00:00:25.092 analysis of a corpus, of a 200 plus uhh confirmed HMI's vulnerabilities that have come 00:00:25.092,00:00:29.129 through the 0day initiative program. We are going to detail out the popular vulnerability 00:00:29.129,00:00:33.834 types that have been discovered in these HMI solutions and we are going to talk about how they 00:00:33.834,00:00:38.805 are developed. And how the weaknesses actually manifest in the underlying code. We are 00:00:38.805,00:00:42.743 going to talk about some of the biggest scada vendors that exist on the planet, including 00:00:42.743,00:00:47.414 Schneider electric, general electric and Advantech. But all the vulnerabilities we are 00:00:47.414,00:00:52.452 talking about can be applied to pretty much every SCADA vendor that's out there today. This 00:00:52.452,00:00:57.491 talk will also cover and compare, a time to patch performance for the various 00:00:57.491,00:01:01.628 SCADA vendors in the industry and will also compare the SCADA industry against the other 00:01:01.628,00:01:06.633 entities in the software industry. And finally we are gonna use the data that we 00:01:08.702,00:01:13.206 presented to provide you additional guidance on what SCADA researchers should be 00:01:13.206,00:01:18.679 looking for in HMI solutions and what we can expect in future attacks against SCADA HMI 00:01:18.679,00:01:24.117 solutions. But first let me, lets introduce ourselves. So you know who we are. Ill let Fritz 00:01:24.117,00:01:29.122 introduce himself. >> uhh Good day, my name is Fritz Sans, my twitter handle is FritzHands 00:01:31.425,00:01:38.198 because I'm occasionally boring. I was a long time developer at Microsoft, 25 years, in the 00:01:38.198,00:01:44.204 Windows operating system. And then I joined the trustworthy computing and secure windows 00:01:44.204,00:01:49.209 initiative in 2001, when the big security push happend at microsoft. I left Microsoft in 00:01:51.778,00:01:58.185 2014 and joined the 0 day initiative, where I have been investigating software in the 00:01:58.185,00:02:04.458 real world, which has given me a deep appreciation for the code quality at Microsoft, which I 00:02:04.458,00:02:09.463 did not have when I was there. [Laughter] > So I'm the senior manager of vulnerability 00:02:13.133,00:02:18.138 research in TrendMicro's Tipping point organization. Umm my primary responsibility in this 00:02:18.138,00:02:22.709 jobs is to actually run and manage the 0day initiative program: which presents the 00:02:22.709,00:02:27.981 world's largest vendor agnostic bug bounty program. We have been in operation for 10 years, we 00:02:27.981,00:02:32.586 spent over $13 million on vulnerabilities over those years. Umm we do a lot of root 00:02:32.586,00:02:37.624 cause analysis working with researchers around the world to buy bugs, uhhh define how they 00:02:37.624,00:02:41.662 actually fire and help the vendors to get them fixed in a proper way. Im also the 00:02:41.662,00:02:47.734 organizer of the ever popular pwn to own hacking competition, where I spend probably ummm over 00:02:47.734,00:02:52.439 half a million dollars this year just on exploits uhh against the hardest attack surfaces in the 00:02:52.439,00:02:58.879 world. So before we get started. we want to give you kinda a overview and level set everybody 00:02:58.879,00:03:03.450 in the audience on what we are talking about here. What the SCADA industry is, what HMI is 00:03:03.450,00:03:07.988 and who are the heavy hitters in this industry. A lot of the market place, if you look at it, 00:03:07.988,00:03:14.494 it's really focus on developing hardware and selling control systems, and not so much focused 00:03:14.494,00:03:19.299 on selling the HMI solution itself. In fact most of them are freely downloadable which makes 00:03:19.299,00:03:24.304 them good targets for auditing. So in this case they kinda focus on hardware, uh software or 00:03:26.807,00:03:30.610 software that runs on hardware and less on windows applications . And that really shows in the 00:03:30.610,00:03:35.048 type of vulnerabilities come through the 0day initiative program. It is a highly 00:03:35.048,00:03:40.353 regionalized market, so there are vendors in China, which specifically develop SCADA 00:03:40.353,00:03:44.858 software and hardware for Chinese uhh for Chinese implementation. There are also 00:03:44.858,00:03:51.598 ones in Germany, uhhh and even seen code developed by Chi....uh by Italian developers. So it's a 00:03:51.598,00:03:57.070 very active market and if you are focusing on SCADA...uh..products in one 00:03:57.070,00:04:01.174 region, it will be completely different in another region. As you can see on the slide we have 00:04:01.174,00:04:06.546 a bunch of big names on there. That Wecon brand is actually the one that is for China, we found, 00:04:06.546,00:04:12.252 I personally found a dozen bugs in their HMI solutions and submitted them, and had them 00:04:12.252,00:04:18.592 fixed. Siemens is also a major brand and GE electric or General electric and Advantech which we 00:04:18.592,00:04:24.064 will talk heavily about in this presentation. Now theres also alot of merges and acquisitions 00:04:24.064,00:04:28.702 in this space. Its very much of the rest of the software market, lots of buying lots of selling. 00:04:28.702,00:04:33.073 But and the one interesting thing that we find is that when we buy a vulnerability in the 00:04:33.073,00:04:38.879 some mom and pop SCADA develop...uh HMI shops, which there are a lot of them, we see 00:04:38.879,00:04:41.982 that by the time the patch comes out, they have actually been acquired by one of the bigger 00:04:41.982,00:04:45.986 companies like Schneider electric or Siemens. So there is a lot of merger and acquisition 00:04:45.986,00:04:51.224 going on which makes the disclosure poss...uh process a little bit more complicated. If 00:04:51.224,00:04:56.363 we look at the Human Machine Interface what is it? Well its primary job is to provide status 00:04:56.363,00:05:01.501 of the critical infrastructure. Things like alarms, notifications. They also provide 00:05:01.501,00:05:06.573 highly advanced and customizable visualizations that give operators insight into what is 00:05:06.573,00:05:10.744 going on in their critical infrastructure. And a lot of these you know, their, uh you 00:05:10.744,00:05:15.949 can kinda develop these and customize these visualizations for different components in your 00:05:15.949,00:05:21.621 actual infrastructure. Now they're supposed to be air gapped and run on isolated and 00:05:21.621,00:05:24.758 trusted networks, but this is really not always the case. And we will take about it, about 00:05:24.758,00:05:31.498 attacks where they took advantage of HMIs that were not on isolated networks. Now even 00:05:31.498,00:05:37.270 isolation uhh is not guaranteeing security if you ask the Iranians, back when Stuxnet 00:05:37.270,00:05:42.142 came out, the air gapped network didn't provide them much value uhh when they were being 00:05:42.142,00:05:48.381 exploited using USB link vulnerabilities that existed. So if the developers are actually 00:05:48.381,00:05:53.019 spending their time and and thinking that their HMI solutions are going to be used 00:05:53.019,00:05:57.324 air gapped networks, and not putting security in, that's what we are seeing in the code. 00:05:57.324,00:06:01.461 That's what it feels like. They are not actually spending time applying best practices of the 00:06:01.461,00:06:08.001 industry. So why would you target HMI solution as an attacker? Well because it 00:06:08.001,00:06:12.639 controls the infrastructure. You can actually see and get get configuration information about 00:06:12.639,00:06:19.613 devices on the network, and it can actually be used by itself, without a vulnerability to shut 00:06:19.613,00:06:23.316 down a netwo...to shut down critical infrastructure. This is, this is the case in the 00:06:23.316,00:06:28.154 Ukrainian attack that happened last year. Ukraine..uh attackers who were going after the 00:06:28.154,00:06:33.593 Ukrainian infrastructure just used the HMI solution by itself to trip breakers and shut down 00:06:33.593,00:06:36.763 the power. They were not actually exploiting HMI vulnerabilities, but they were 00:06:36.763,00:06:43.336 using the HMI system, to actually take the systems down. Now it can also be used, you can 00:06:43.336,00:06:48.308 actually attack these to deceived uhh and disable alarm systems in the, in the control 00:06:48.308,00:06:52.345 system itself. And this is the case in stuxnet where they actually deceive the operators 00:06:52.345,00:06:57.684 ummm and uhh about the state of the centrifuges they were controlling and actually send 00:06:57.684,00:07:02.622 it, send it commands to trigger, uhhh, self uhhh, self destruction conditions in the 00:07:06.159,00:07:11.164 control systems themself. So there are active attacks in in umm HMI solutions. If you look 00:07:14.167,00:07:19.272 at stux..uh..stuxnet is obviously the most popular one we talk about, everybody knows 00:07:19.272,00:07:24.144 about this one, but it did leverage vulnerabilities in HMI solutions and..uh...including 00:07:24.144,00:07:30.884 Siemens simatic step 7 dll hijacking vulnerability along with a SQL server authentication 00:07:30.884,00:07:35.588 bug. These are really simple bugs, very common bugs in HMI solutions and they leverage 00:07:35.588,00:07:40.593 those to deceive the operators on the state of the centrifuge. Now black energy is an ongoing 00:07:43.697,00:07:49.436 sophisticated malware campaign against ICS environments and it actually targets HMI 00:07:49.436,00:07:51.504 vulnerabilities, the GE path traversal vulnerability. Its used...its uhh..we believed it 00:07:51.504,00:07:53.840 to have used..uh..some vulnerabilities in..uhh...Siemens WinCC and 00:07:53.840,00:07:58.845 Advantech remote web access. So quite famously in the ZDR program, the GE simplicity 00:08:07.587,00:08:11.057 vulnerability is actually one that we purchased from an anonymous researcher and 00:08:11.057,00:08:15.995 disclosed the ICS cert, and it turned out that it was actively being used by black energy. So 00:08:15.995,00:08:21.668 it's kinda interesting to see that happening in the wild. Now another big player in the 00:08:21.668,00:08:26.773 industry is the ICS cert. So as a researcher you need to know who this organization is and 00:08:26.773,00:08:30.377 where they sit in the government. So i'll rattle off the title and their location in 00:08:30.377,00:08:34.948 the government. They are the industrial and control system cyber emergency response team 00:08:34.948,00:08:39.285 which operates in the national Cyber Security and Integration centre, a division of the 00:08:39.285,00:08:42.789 department of Homeland security's office of Cyber Security and communication. I 00:08:42.789,00:08:47.727 mean that is a long name, I can almost get paid by letter there. But in reality they are a very 00:08:47.727,00:08:49.729 important organization and people in...who are researching in HMI uhhh need to know who 00:08:49.729,00:08:51.731 they are and know how to work with them. We work with them everyday uhhh in our jobs, as we 00:08:51.731,00:08:53.733 are purchasing a lot of vulnerabilities uhhh in HMI. And they they do a lot of things, 00:08:53.733,00:08:55.735 they actually release a report every year about all the stuff they are doing. And according to 00:08:55.735,00:08:57.737 the 2015 report they actually responded to 295 incidents and handled 486 vulnerability 00:08:57.737,00:09:02.542 disclosure. And that's significant, that's a lot of vulnerabilities passing through 00:09:02.542,00:09:07.547 that organization every year. So because it's so regional, its really hard getting a hold of 00:09:23.096,00:09:27.867 these mom and pop operations when you find a vulnerability in their solution. And then at this 00:09:27.867,00:09:33.106 point you come to programs like the 0day initiative or go to ICS CERT to help you disclose those 00:09:33.106,00:09:38.678 vulnerabilities and get them fixed. So let's talk about attacks that leverage HMI 00:09:38.678,00:09:43.683 features or vulnerabilities in their active attacks. If you read the Verizon data breach 00:09:46.085,00:09:51.057 report, they talk about their team went in and actually uhhh were called in to analyze the 00:09:51.057,00:09:56.062 security of a water utility, now they don't give the name of the water utility, but they do talk 00:09:58.264,00:10:03.236 about their findings. And in this case they found that there was an internet facing AS400 00:10:03.236,00:10:08.241 system, responsible for HMI like capabilities: like manipulating PLCs. But this system also did 00:10:11.411,00:10:16.216 network routing and managed customer data. I mean how ridiculous is that, that all of 00:10:16.216,00:10:20.753 that information is sitting on one system, connected to the internet. Both critical 00:10:20.753,00:10:25.625 infrastructure and billing systems. Umm this is kindof a uhh example, a prime example 00:10:25.625,00:10:29.729 that there is no focus on the separation of responsibilities when they are architecting these 00:10:29.729,00:10:35.802 critical networks. Now what they learnt, is is, what they discovered was that 4 different 00:10:35.802,00:10:42.075 connections to this AS400 over a 60 day period, where the IPs were tied to hacktivist 00:10:42.075,00:10:48.448 activities, and they actually altered the water flow and the chemicals in, in that system. 00:10:48.448,00:10:51.951 Now according to the report they, they say the attackers really didn't understand what 00:10:51.951,00:10:56.089 they were, were working against and they didnt, couldnt really do a lot of damage. But they 00:10:56.089,00:11:02.662 could have done a lot of damage, by that. By accessing that system. The most recent example 00:11:02.662,00:11:07.600 of a really high profile SCADA attack was the U..in the Ukraine. Where there's several 00:11:07.600,00:11:12.305 Eur..uh Ukrainian companies that experienced unscheduled power outages which, which is, which 00:11:12.305,00:11:17.143 affect almost a quarter million people. These were caused by malicious actors and there's 00:11:17.143,00:11:23.316 actually a really great report on the ICS website, um ICS CERT website, that describes all the 00:11:23.316,00:11:28.888 details of that attack. Well that are unclassified. Umm so, they talk about how the attack 00:11:28.888,00:11:34.894 was co-ordinated and they all attacked within 30 minutes. And in this case they didn't use HMI 00:11:34.894,00:11:38.965 vulnerabilities, but they leveraged that HMI solution, because they were..because 00:11:38.965,00:11:45.138 there's no isolation they were able to VPN into the network and get access to the HMI solutions 00:11:45.138,00:11:50.310 and use remote administration tools which dossed the operators from making any changes. And 00:11:50.310,00:11:56.182 they actually just tweaked the, the nobs in the HMI solution to turn off and turn breakers. And 00:11:56.182,00:12:02.455 as a result the power went out. They also put killdisk malware on the windows based HMI 00:12:02.455,00:12:07.827 systems, which basically brought them to their knees and really hurt the restoration efforts. 00:12:07.827,00:12:11.831 Now this is obviously used to destabilize the Ukraine a little bit, I dont think theyve 00:12:11.831,00:12:15.868 attributed the attack, but there's a lot of political stuff going on in that region so you 00:12:15.868,00:12:22.241 can imagine. Now there is also some interesting report...um interesting research that came 00:12:22.241,00:12:26.245 out of a sister organization inside of our company, uhh where they actually looked at the 00:12:26.245,00:12:31.484 malware that was used in the Ukrainian attack and actually found links malwares in other 00:12:31.484,00:12:36.255 companies in the Ukraine, including a rail company and a mine company around the same 00:12:36.255,00:12:41.794 time. Now black energy was supposedly not used in the attacks against the Ukrainian 00:12:41.794,00:12:47.033 power company, but it did exist in that network so you can imagine that, the attackers who 00:12:47.033,00:12:50.937 are going after those are probably the same attackers that had access to some rail and 00:12:50.937,00:12:56.175 mining companies in the Ukraine as well. How did our sister organization know this? They 00:12:56.175,00:12:59.612 looked at the infrastructure of the malware and naming convention that were used and 00:12:59.612,00:13:03.282 they released a whitepaper on it and it's actually very very interesting and worth a read. 00:13:03.282,00:13:08.287 It's on the Trend Micro blog. So let's, let's talk about the prevalent vulnerability types 00:13:10.456,00:13:16.362 that exist in HMI solutions and what the current state really is? So, the reality of the 00:13:16.362,00:13:23.302 situation is, the HMI solutions have not seen any benefit from the evolution of secure software 00:13:23.302,00:13:28.508 development lifecycles over the last 10 years. We have looked at alot of the code, you know 00:13:28.508,00:13:32.712 dozens and dozens of code bases. We have have analyzed and looked at vulnerabilities and confirmed 00:13:32.712,00:13:35.982 0days. And that's what we've learned. There really is no security built into that 00:13:35.982,00:13:41.587 software. THey haven't seen any, any benefits of the secure development lifecycle that 00:13:41.587,00:13:45.692 Microsoft, Apple and all these other companies have, and this is actually a really scary 00:13:45.692,00:13:52.432 thing. And in fact most of the solutions we are vetting bugs in, do not have ASLR, uhhhh 00:13:52.432,00:13:57.303 [inaudible] SAFE SCH, or stack cookies enabled. Which is really really scary. And we actually 00:13:57.303,00:14:03.176 urge SCADA vendors to turn on all of these mitigations, including things like building 00:14:03.176,00:14:09.582 64bit apps, to make, uhh ASLR better, uhh and actually reduce in the reliability of heap spray 00:14:09.582,00:14:14.153 and also turning on just the, uhh, the basic mitigations that are available by flipping a 00:14:14.153,00:14:19.525 toggle in the compiler. Its um..it's actually really embarrassing. It's also..there's 00:14:19.525,00:14:22.462 also a lack of understanding how these are really...uhh...these..uuhh. how 00:14:22.462,00:14:27.967 these solutions are actually run. They seem to think that these are going to run in an 00:14:27.967,00:14:32.705 isolated environment and they are using that as a way to not implement security mitigations. 00:14:32.705,00:14:36.342 But they are continually being integrated and you see attacks in the wild that leverage 00:14:36.342,00:14:42.048 interconnected HMI solutions to take down critical infrastructure. So this is 00:14:42.048,00:14:46.185 probably the only pie chart you will see at defcon, and I'm actually kinda proud of that, 00:14:46.185,00:14:52.425 because I put a lot of work into this pie chart. But in a...what we ended up doing is we pulled 00:14:52.425,00:14:57.430 all the 2016 and 2015 ICS Cert advisories and identified all of the HMI solutions that had bugs 00:15:01.200,00:15:08.107 fixed in the last years. We crossed referenced that with our 250 plus 0day vulnerabilities 00:15:08.107,00:15:13.613 we've purchased in HMI solutions to come up with what the most popular, the most common 00:15:13.613,00:15:19.752 vulnerability types are in HMI solutions. We also catalog the CWEs to kinda get an idea what 00:15:19.752,00:15:26.492 vulnerabilities existed, and and they are purely listed on the slide. The nr 1 is memory 00:15:26.492,00:15:31.197 corruption, followed by credential management, usually hard coded passwords. Insecure 00:15:31.197,00:15:35.635 defaults. Authentication and authorization. And encode injection issues. Now what about 00:15:35.635,00:15:39.305 Cross site scripting or Cross site request forgery? Well most of these are windows based 00:15:39.305,00:15:44.310 applications, there are some web based applications, but most of them are windows. And as a 00:15:44.310,00:15:47.013 result you are not gonna see a lot of that cross site scripting stuff. But there are some in 00:15:47.013,00:15:52.018 that grey area on the slide. So what we are gonna do is, is, lets not, lets lets get down and 00:15:54.821,00:15:58.958 dirty with this. Lets let's look at every single one of those categories and were gonna give 00:15:58.958,00:16:04.831 you case studies of what these look like, so you can understand how terrible this code base 00:16:04.831,00:16:09.202 really is, and what you need to understand to go actually find these bugs and to protect 00:16:09.202,00:16:14.440 yourself against these bugs. And that's the most important part. So first we are gonna talk about 00:16:14.440,00:16:19.779 code injection vulnerabilities. This makes up about 9% of the common vulnerability types that 00:16:19.779,00:16:24.851 exist in these products and it, you know it's the classic sql injection, code injection, OS 00:16:24.851,00:16:30.223 command injection. But there's other,other domain specific languages that exist in this 00:16:30.223,00:16:33.926 software. And that's what we are gonna talk about today, we didn't want to cover stuff you 00:16:33.926,00:16:39.131 guys already know, this is, we are gonna talk about gamma code injection, right. So this is a 00:16:39.131,00:16:45.404 domain specific language used in this industry. Specifically we are gonna talk about Hogen Data 00:16:45.404,00:16:50.409 hub, and we are gonna talk about CWE 2015-3789. Now this allows, this vulnerability actually 00:16:52.612,00:16:57.750 allows an attacker to turn on an insecure processing mode in the web server, which allows the 00:16:57.750,00:17:03.823 attacker to send arbitrary scripts to the server, and execute arbitrary code. Now this 00:17:03.823,00:17:08.160 was discovered by an anonymous researcher and disclos...and purchased by us, disclosed to 00:17:08.160,00:17:14.567 ICS CERT and fixed. Now we do offer the ability for people to submit bugs to us, in an 00:17:14.567,00:17:19.972 anonymous fashion, and we get a lot of that actually um through program. So what is Cogent 00:17:19.972,00:17:23.509 Datahub? Well that's what you see on the screen here, it's one of those visualizations that I 00:17:23.509,00:17:27.380 was talking about. Cogent Datahub is a real time middleware solution that is 00:17:27.380,00:17:31.450 deployed over across several sectors including chemical, commercial, critical 00:17:31.450,00:17:36.422 manufacturing, energy, financial, etc and it's used around the world. It offers the 00:17:36.422,00:17:41.827 end user the ability to create those really intense, advanced visualizations that you see on 00:17:41.827,00:17:48.167 the slide here, customize those so that you can monitor your underlying network. So what is 00:17:48.167,00:17:53.306 Gamm script? Well Gamma script is a domain specific language specifically designed for the 00:17:53.306,00:17:57.910 use, within, within data hub. It's a dynamically typed interpreted programming 00:17:57.910,00:18:03.482 language, specifically designed for rapid application development. Its look like C and 00:18:03.482,00:18:06.819 I'll show you some her in a second. And it has a range of build in features, it's got 00:18:06.819,00:18:11.490 libraries and everything. It actually has a fully documented A..uh...API, that you can read 00:18:11.490,00:18:17.096 on the internet. It's actually pretty full featured for those application developers. Now the 00:18:17.096,00:18:23.903 attack itself is a flaw in a valid expression method. It allows an attacker to execute 00:18:23.903,00:18:29.709 arbitrary code on the system. It actually sits uhhhh and is accessible through an AJAX 00:18:29.709,00:18:34.714 facility on port 80. And you simply supply a well formatted Gama script which allows the 00:18:36.749,00:18:41.620 underlying code execution. Now the interesting thing about this is, is its domain specific, so 00:18:41.620,00:18:46.692 there is a lot of functionality in gamma that specifically used for developing that stuff. But 00:18:46.692,00:18:52.365 unfortunately it did have the...in that script the ability to execute system commands. So 00:18:52.365,00:18:56.869 what is the vulnerable code? Its right here on the screen. Its very very simple. Valid 00:18:56.869,00:19:02.642 expression basically takes an expression and checks one flag. Are I allowed to execute this 00:19:02.642,00:19:07.413 expression, and if it does then it executes the expression. And this is whatever you want to 00:19:07.413,00:19:12.084 send to the system. Now the question is how do you actually get that to load up and how do 00:19:12.084,00:19:17.723 you change that value. Well it also allows you to do that as well. So the exploitation steps 00:19:17.723,00:19:24.664 are you send us a request, a HTTP request to the port 80, which will load the gamma script 00:19:24.664,00:19:26.665 libraries. Then go, you call AJAX.support.allowexpression which will set allow any 00:19:26.665,00:19:31.671 expression to true, and then you call a valid exceptio with any script you want and you execute 00:19:34.974,00:19:41.847 code. So thats a demo of that exploit. So what you see here is the installation of Data hub, 00:19:41.847,00:19:45.017 you can see, and you kinda zoom in for the audience here..............hmmm...... 00:19:45.017,00:19:50.022 yeah well forget that. Ummm so what we are gonna do here is right on the screen, what's 00:19:57.496,00:20:02.501 highlighted is Cogent Datahub version 7, and its running and sitting on port 80. And what the 00:20:07.006,00:20:10.810 first thing that we are gonna do is, we are gonna run a proof of concept here at the bottom, that 00:20:10.810,00:20:16.048 is a, uhhh just basically a python script that sends the 3 commands that we need, and will 00:20:16.048,00:20:18.317 disclose information on the server, it's actually is disclosing autoexec.bat uhhh on 00:20:18.317,00:20:23.322 that box. So then we are going to send another script, uhhhh, which will actually execute 00:20:27.493,00:20:32.498 calcu...the evil calculator. And you will see here it's a actually a very very reliable 00:20:35.768,00:20:39.839 bug and a very reliable exploit and you can just kinda send it over and over and over again, 00:20:39.839,00:20:46.212 and there's those evil calculators, you know. That's uh a pretty fun bug. Really simple 00:20:46.212,00:20:50.649 bug. And they did actually do a really good job fixing it, and you can see all the calculators 00:20:50.649,00:20:55.654 being spawned by the process, so. Now how did they patch the bug? Right, this is one of the 00:20:59.792,00:21:03.762 interesting things for thee ZIDI program, because when bugs get patched, researchers will also 00:21:03.762,00:21:08.968 submit bugs, uhh POCs that actually break their patches which is kind of interesting. 00:21:08.968,00:21:13.739 But here it's kinda gonna be difficult. So on your left is the old code and on your right 00:21:13.739,00:21:18.744 is the new code and you can see, up here, that they actually removed allowed expressions, so 00:21:21.547,00:21:26.552 you cannot access that at all, so you can no longer toggle the flag in the system. They also 00:21:29.822,00:21:35.928 removed....a valid expression entirely and they actually gave it a really great comment which 00:21:35.928,00:21:39.632 is actually a best practice: this method is dangerous, it could allow somebody to execute 00:21:39.632,00:21:45.671 arbitrary code via http call, if you absolutely need it create a script and define it, then make 00:21:45.671,00:21:50.976 sure your webserver is on a trusted network. So that code is buried in the application itself 00:21:50.976,00:21:54.113 so it's highly unlikely that developers are gonna go look at that, but they are just gonna 00:21:54.113,00:21:58.417 call the APIs. But it's good that they actually documented that, so they won't regress that 00:21:58.417,00:22:03.389 bug at some point. So that's actually how that bug works. So i'm gonna turn it over to Fritz, 00:22:03.389,00:22:08.227 and he's going to cover the rest of the prevalent vulnerabilities types and then we will talk 00:22:08.227,00:22:13.232 about some disclosures [inaudible]. >> Hello again. So the next section we are going to 00:22:15.601,00:22:22.241 look at is Authentication and authorization problems and authentication bypass, improper 00:22:22.241,00:22:28.314 access control and proper privilege management, bad authentication. And what we are 00:22:28.314,00:22:32.651 gonna focus on is a Advantech case and you are gonna hear Advantech a lot. And this 00:22:32.651,00:22:37.356 actually a pretty fun one. This is information disclosure. And uhhh this is CVE2016-5810 and 00:22:37.356,00:22:42.361 the ICS CERT says properly authenticated administrator can view password for other 00:22:48.067,00:22:53.706 administrators. The terminology is a little unfortunate here, because this is not a system 00:22:53.706,00:22:58.711 administrator, this is the administrator of a given SCADA solution, a given project and 00:23:01.146,00:23:06.785 that's sort of a kin to unprivileged users of the system. So this is in essence 00:23:06.785,00:23:11.790 saying a user can extract the password of another user. And this was discovered by zooyu and 00:23:13.859,00:23:19.598 disclosed by the Zero day initiative. This was sort of fun, and basically they have a 00:23:19.598,00:23:21.600 script, a ASP, script that allows you to change your username, your password, your 00:23:21.600,00:23:23.602 description and this is great, but this can be abused. And they way you do it, you log in to the 00:23:23.602,00:23:25.604 account you have, this is not anonymous, you have to have an account on the system, but then 00:23:25.604,00:23:31.877 you can change the URL to give any other name. And then pass that in and it will bring you 00:23:31.877,00:23:36.882 back the password of the second account, and you can't see the password because it has astrix 00:23:51.897,00:23:56.902 in front of it, yeah. [Laughter]. So yeah here's a demo showing it. SO first log in 00:24:05.277,00:24:11.750 as the admin, and by the way you can also get the full sys commit administrator account this way. 00:24:11.750,00:24:16.755 And you can see there is a test1 and a test2 user. So now you log in as test1..............and put 00:24:23.996,00:24:29.001 into your password for test1, and that's all great. Now if you try to change, you...change to 00:24:32.738,00:24:37.743 test2 using the UI, it will quite properly give you an error saying you can't do that. But if 00:24:40.145,00:24:45.150 you change to a username of test2 in the URL, it will pop it back, but it has those asterix 00:24:47.486,00:24:52.491 so we got to fix that. So you view the source. [Laughter]. And there's the password, and then 00:24:58.063,00:25:04.103 you can use that password of course and log in as anyone else, including of course the 00:25:04.103,00:25:09.108 complete system administrator of all the solutions. And there you got it, and you're logged in, 00:25:17.616,00:25:22.621 and what...okay here it goes. So that one was sort of fun. And there is also a lot of insecure 00:25:30.462,00:25:35.467 defaults in this space. Uhh pad transmission of information, missing encryption, unsafe 00:25:38.570,00:25:44.643 activex controls, yes we are back to activex controls. The one we are going to focus on 00:25:44.643,00:25:49.648 here, is the Schneider Electric DSNVS, and this is a bad activex control with memory corruption. 00:25:53.519,00:26:00.159 Even Though this is memory corruption, we put this here because this activex control 00:26:00.159,00:26:06.532 will first....it was set as safe for scripting from untrusted source, but what's also 00:26:06.532,00:26:10.869 interesting it was never meant for it to be a control to be used in.....uhhh...Internet 00:26:10.869,00:26:15.874 explorer, in a web page. SO it should have been configured as automatically killed 00:26:20.512,00:26:22.514 [inaudible]. So it's really bad configuration. It was wide open to Internet Explorer when it 00:26:22.514,00:26:24.516 should not have been. The is CVE-2015-0982 and the Schneider Electric Pelco here is a HMI for 00:26:24.516,00:26:26.518 digital sentry video surveillance systems. So its really great uhhhh you can use 00:26:26.518,00:26:31.523 this to, you know get information on video surveillance systems, which is 00:26:34.893,00:26:39.898 always fun. Uhhh what I wanted to do is show this for people who are going and auditing and 00:26:55.414,00:26:59.952 looking for activex controls that might be vulnerable, it shows and interesting second 00:26:59.952,00:27:04.890 step you often need to take. There are two ways to tell the system that an activex control 00:27:07.359,00:27:12.931 is not safe for scripting. The standard way, the past way is statically in the registry to 00:27:12.931,00:27:19.404 mark it unsafe for scripting, but if you note...its is, is is to turn it on, to make it safe 00:27:19.404,00:27:24.309 for scripting is to flag it as safe for scripting. But if it's not marked in the registry as 00:27:24.309,00:27:30.315 safe for scripting, it can insta...it can then use the interface Iaptic safety, and 00:27:30.315,00:27:36.388 then in dynamic runtime assert it is safe for scripting. So even though it's in the registry 00:27:36.388,00:27:43.061 for not-safe for scripting, it still is potentially vulnerable. So you've got to look at the 00:27:43.061,00:27:48.066 dynamic situation as well as the static situation. So you can't just do one. And here is a demo 00:27:52.137,00:27:57.142 of just how the memory corruption works. Which is you, you use Internet Explorer, and 00:27:59.611,00:28:04.550 you go to an attacking web page which invokes the control in IE, and it does a stack buffer 00:28:08.153,00:28:13.158 overflow, and fills everything with your classic 41s, that we all know and love. Uhhh let's 00:28:20.732,00:28:27.673 talk about some credential management problems, uhhh this actually...I was really shocked 00:28:27.673,00:28:32.678 when i ran into this, cause its like you are kidding right? Uhhh there's, this happens a lot that 00:28:35.447,00:28:42.287 they hard code credentials in the code, hard coded passwords. Uhhh you know like, I thought we 00:28:42.287,00:28:47.292 got rid of that 15 years ago with IIS, but well, it's...we are in SCADA space you are 00:28:51.930,00:28:56.935 hacking like it's 1999. Its awesome, its awesome we are back then. [Laugher]. So the one we 00:28:59.237,00:29:04.776 are going to look at is GE MDS Pulsenet, and it has a hidden support account, and this is 00:29:04.776,00:29:09.781 really fun. So this is used to monitor devices and Industrial Communication networks and its 00:29:12.184,00:29:17.189 deployed in Energy, Water and Wastewater sectors and its used World Wide. This is 00:29:17.189,00:29:22.194 CVE-20156456. So if you take a look at the user management panel using the UI, you see 00:29:27.199,00:29:32.204 there are exactly 2 accounts in the system: there is a admin and a operator. Well that lies. If 00:29:38.777,00:29:45.083 you ac..if you go in and you use....uhh...I use HeidiSQL but if you use anything that 00:29:45.083,00:29:51.790 extracts information from the database, you see that there are not 2 accounts, there are 3. 00:29:51.790,00:29:56.795 There is a hidden account called GE support. Now, now it's really super subtle, because it only 00:29:59.097,00:30:05.037 stashes the md5 hash of the password, not the password itself. Certainly you...no one 00:30:05.037,00:30:10.042 here can crack an md5 hash right? [Laughter]. It it turns out that the password is 00:30:12.144,00:30:17.149 actually Pulsenet, but the made it l33t by changing the L to a 1. So here's the demo. Uhhh you 00:30:22.754,00:30:27.759 can see on the right, the two users that are officially there, and on the left we will log in 00:30:35.567,00:30:40.572 as the user that isn't there. And what I think is really cool, even after you log in as the 00:30:46.912,00:30:51.917 user who isn't there, as you are logged in as the user that isn't there, it tells you that, that 00:30:53.952,00:30:58.957 user is still not there. Which is just sort of slick I think. Uhhh there's also a lot of other 00:31:03.195,00:31:08.200 misconfigurations. One of the other ones we see a lot, where companies decide to roll their 00:31:10.802,00:31:15.807 own apples, and they decide they don't want to put things under Program Files like Microsoft 00:31:18.677,00:31:23.682 intended and so they create their own top level directory, with their company name, under 00:31:25.751,00:31:30.756 the C drive, and they often put World has full access. And then they put their service binaries 00:31:34.860,00:31:39.865 in there. So any local user can drop new binaries in, and they will run as a system service. So 00:31:44.469,00:31:49.474 this is very standard. And now we get to, to the joy of memory corruption. Stack based buffer 00:31:51.610,00:31:56.615 overflows, heap based buffer overflows, out of bound, read write. Just the classic ones. 00:31:58.950,00:32:03.889 And Umm the Advantech is our, our, our whipping boy here. Because they did an awesome job 00:32:06.191,00:32:11.196 here. We got a 100 bugs in one day, from an anonymous researcher. This was like this, 00:32:16.601,00:32:21.606 this data dump from heaven. And we analyzed them and passed them on, and they were all buffer 00:32:26.144,00:32:31.149 overflows. And yeah its was quite impressive, and I will drill into one particular. This 00:32:34.920,00:32:39.925 is CVE-2016-0856 and it was an anonymous researcher and disclosed by us. And this is 00:32:46.231,00:32:51.803 their webpage, and what's really interesting about web access is it's a SCADA solution, but they 00:32:51.803,00:32:58.276 also advertise as you can see in here that this is also for Internet of Things. SO this is 00:32:58.276,00:33:00.212 widely deployable, and its widely exploitable [Coughing]. It launches a service web, uhhh 00:33:00.212,00:33:02.214 the RPC in the context of local administrator and listens on 4952, and the web..the service 00:33:02.214,00:33:04.216 calls are configured to look like Microsoft IOaccess control calls. So they've got an IOCTL 00:33:04.216,00:33:08.320 value and they do jump tables off of that to perform 100s and 100s of types of services. For 00:33:08.320,00:33:12.824 this particular one, the parameter that's passed is a window name, which is then 00:33:12.824,00:33:18.296 copied using sprintfW to a stack buffer that is Hex 80 characters. And as you can see 00:33:18.296,00:33:24.102 in this packet the length is Hex8C. So it copies Hex8C bytes into Hex80 byte buffer on the 00:33:24.102,00:33:27.939 stack, with predictable consequences on the stack.Uhhh so inside you've got this....and 00:33:27.939,00:33:33.111 the flaw is the stack based buffer overflow, here is the classic sprintfW call, you know 00:33:33.111,00:33:39.050 nothing of a surprise there. Here is the stack playout, and this is sort of fun, because you 00:33:39.050,00:33:41.353 can tell that the windows name is at -80, and then 0 is your return address. No stack cookie. 00:33:41.353,00:33:43.355 Why no stack cookie? They didn't flip the bit in the compiler and linker. Probably cause they 00:33:43.355,00:33:45.790 first built this 20 years ago and they never changed their configuration, to handle, to add 00:33:45.790,00:33:50.795 ASLR, to handle SAV ESCH, to handle stack cookies. So all you have to do is overwrite the 00:34:11.816,00:34:16.821 return address, point it to the first of your ROP, you can handle the ROP chain well 00:34:27.532,00:34:32.537 because there is no ASLR. Life is good. Life is really good. So you can see is jumping to a 00:35:00.198,00:35:05.203 address and here I will pop the glorious [inaudible]. And this was fun to 00:35:17.549,00:35:22.554 do...........................Bin go! And that is running at high privilege, life is good. Uhh 00:35:34.099,00:35:39.104 Let's talk about the PAG analysis. As sprintfW Microsoft published the banned API list a 00:35:41.940,00:35:46.945 decade ago, and there's a reason Microsoft published the banned API list. So what they did when 00:35:52.384,00:35:57.389 we reported this, is they change sprintfW to snprintFW. So snprintfW is also in the banned 00:36:01.226,00:36:06.231 API list. It's a better banned API, because it won't buffer overflow, but if you give it too 00:36:10.468,00:36:15.473 many characters it will also not null terminate. So if the stack is not pre-cleaned out and it 00:36:18.977,00:36:23.982 isn't, it is possible for you to use stirring manipulations on this window name, where you 00:36:28.119,00:36:33.124 think its Hex80 characters long, it may be longer as it didn't null terminate with the copy. So 00:36:36.461,00:36:41.466 there might still be problems. As I said a 100 bugs came in, a 100 bugs. Advantech fixed 75 of 00:36:47.138,00:36:53.978 them, we have disclosed the other 25 as not fixed, you guys can enjoy. There are also...when 00:36:53.978,00:36:55.980 they did fix they did not any global replace, they did specific point fixes of the ones 00:36:55.980,00:37:00.919 they fix. There are 1000s string copies and sprintfW's in the codes base, and I would not bet 00:37:08.460,00:37:12.363 10 cents that none of those can be reached by attackers that apply data. So have fun guys, 00:37:12.363,00:37:17.368 have lots of fun. Ah yes researcher guidance. Sup. What do you people want to do? Well 00:37:30.815,00:37:35.820 the first thing to do is Fuzz. Right..these things are easy to fuzz, they don't have CRCs, most 00:37:39.858,00:37:44.863 of these file formats are wide open. Just do but flipping, remember to turn on page heap 00:37:47.532,00:37:53.705 on, on the process being attacked. It's a great way to find memory corruption because 00:37:53.705,00:38:00.578 then it breaks at the corruption point not later on when it's being used. Use your tools that 00:38:00.578,00:38:07.285 you got for fuzzing, use your tools for analysis. SQLMap is great for finding SQL injection 00:38:07.285,00:38:13.458 possibilities. One of my favourite tools is Attack Surface Tool by Microsoft, one 00:38:13.458,00:38:18.429 of the reasons it's one of my favourite tools is, I helped write it. Microsoft release a 00:38:18.429,00:38:23.434 public version in, in 2012. It creates a snapshot before and after the installation of your 00:38:26.304,00:38:32.977 target software and then it will highlight security problems in the configuration and it will 00:38:32.977,00:38:38.016 highlight increases in attack surface. So it will tell you your new COM object, your new 00:38:38.016,00:38:43.021 Activex controls, your new RPC endpoints. Here's and example it shows you for example on 00:38:45.456,00:38:50.461 the..ummm...on the advantech software of the new RPC. Information in here is attack 00:38:53.598,00:38:58.603 surface analyzer telling you that the web root directory, which is you know where files 00:39:00.738,00:39:07.345 are gonna go that are being executed,in the high privileged web server context, that this 00:39:07.345,00:39:12.350 entire directory has ful...can be..has write access by the world. What could possibly go 00:39:18.189,00:39:24.829 wrong? So I,...ASA is a great great tool to use. Now if anybody in here has pull at 00:39:24.829,00:39:29.834 Microsoft uhhh we need a new version drop of ASA, because it doesn't work on Windows 10, and 00:39:33.938,00:39:40.144 if Microsoft wants to be really cool they can release the source of ASA, because I know what 00:39:40.144,00:39:44.916 needs to do to fix it. So works on Windows 10, would take an hour for me to fix if Microsoft 00:39:44.916,00:39:50.722 would release the source. Also audit for the banned APIs, could look for the 00:39:50.722,00:39:55.727 s_printouts,str_copies, use IDA to trace the tainted data back and see if you can get to the 00:39:59.130,00:40:04.135 source of these, uhh, unsafe copy APIs, if you can get those from attackers apply data. You 00:40:10.308,00:40:15.313 know it's wide open people. Now back to Brian for more of the corporate things. > Yes. Yes. So 00:40:18.216,00:40:22.553 we wanted to give you an understanding of how...when you find a vulnerability how long it 00:40:22.553,00:40:28.526 will actually take to fix. Kinda talk about the vulnerability exposure window. So what we did 00:40:28.526,00:40:32.730 is we actually took all the HMI vulnerabilities that we received in the 0day initiative program, 00:40:32.730,00:40:38.036 again over 250 now. Ummm and looked at how long they actually took, and if you look at the 00:40:38.036,00:40:44.309 last 4 years, it's not exactly trending down, its pretty consistently a 140 days from the 00:40:44.309,00:40:49.747 time we disclose a bug, until when the patch comes out. And the thing about the SCADA 00:40:49.747,00:40:54.352 industry, is that when they are applying those patches, if the patch is bad or there is a 00:40:54.352,00:40:57.789 issue, it will actually denial of service the critical infrastructure as well, which is 00:40:57.789,00:41:02.960 not good, but that means that patching actually takes a really long time. You can imagine 00:41:02.960,00:41:06.731 almost twice as long. So that leaves, you know almost through...uhh probably around 00:41:06.731,00:41:13.271 300 days that the patch is not being applied. So you know that's how long the bug 00:41:13.271,00:41:17.308 is....are existing in the software even after you find them. So what we wanted to do is 00:41:17.308,00:41:21.546 actually call out a couple of vendors who, uh who...who disclosed...whom we disclosed 00:41:21.546,00:41:25.616 vulnerabilities to, because that's what we like to do. Ummm so what you see here is all of 00:41:25.616,00:41:30.555 the vendors over those years, and Cogent Datahub I want to call out as one of the better 00:41:30.555,00:41:34.792 SCADA vendors for doing patching. Actually one of the first bugs we disclosed to 00:41:34.792,00:41:40.598 Cogent Datahub, their CEO actually emailed us and uh and worked with us on the fix. And 00:41:40.598,00:41:44.969 they fixed it in like 6 days, it was amazing. Um and they have continued that trend, we are 00:41:44.969,00:41:48.840 still purchasing bugs in Cogent Datahub and they are fixing them relatively fast. But if you look 00:41:48.840,00:41:53.845 at the big vendors you see ABB, GE, um you know Indiesoft, those..um over 200 days to 00:41:58.483,00:42:03.488 release a fix for a 0day vulnerability that we purchased and that is known. So that's 00:42:03.488,00:42:09.694 kind of interesting, umm you know averages out to about 150 days for bug fixes. A lot of 00:42:09.694,00:42:14.265 these are going through ICS CERT and so, um just to sort of call that out. If you look at the 00:42:14.265,00:42:18.436 SCADA industry and how it compares to other industries, you know micro....umm we we in 00:42:18.436,00:42:22.507 the highly deployed software we consider that Microsoft, Apple, Oracle, the big name vendors, 00:42:22.507,00:42:29.013 they do a decent job. Take them about 120 days to fix a bug when its disclosed. Umm and and SCADA 00:42:29.013,00:42:32.650 and security products are battling out for second and third and with SCADA coming in 00:42:32.650,00:42:38.289 third and kind of worst of all of them is business software, things like HP and other big 00:42:38.289,00:42:43.027 name business like IBM it take them a long time to fix vulnerabilities. We are almost 00:42:43.027,00:42:49.400 200days for those types of vulnerabilities. So just, you know as you find bugs and you, 00:42:49.400,00:42:52.437 you, you worked with ZIDI to get them fixed or disclose them directly to a vendor it does 00:42:52.437,00:42:58.643 take a significant amount of time, but it..in certain cases it take more than a 180 days. So 00:42:58.643,00:43:04.082 kind of rap things up. We present at these conferences and provide this level of detail 00:43:04.082,00:43:08.453 because we want you to find bugs, we want you to work with uh with the vendors to get them 00:43:08.453,00:43:12.790 fixed. We want you to work with bug bounty programs like the 0day initiative to get 00:43:12.790,00:43:17.061 compensated for your research and so um we are definitely interested in buying 00:43:17.061,00:43:22.233 vulnerabilities and that's why we provide this detail. There is ICS..umm...you know focused 00:43:22.233,00:43:26.037 malware that is actively exploiting HMI vulnerabilities, these vulnerab...these code 00:43:26.037,00:43:30.208 bases are plagued with vulnerabilities. ANd you can use the simple techniques to 00:43:30.208,00:43:34.912 actually find them, it does take a long time for them to fix, but they do end up fixing them. Umm 00:43:34.912,00:43:38.349 and so we are going to be continuing this research and we are actually going to be 00:43:38.349,00:43:41.185 releasing a whitepaper in a couple of months. We are going to release some proof of 00:43:41.185,00:43:46.924 concepts and all our disclosure data is publicly available on 0day initiative dot com, for you 00:43:46.924,00:43:52.630 to analyze yourself and draw you own conclusion. Again we are the 0day initiative, we buy bugs, if 00:43:52.630,00:43:58.469 you find 0days, we are a whitehat bug bounty program. We like to watch researchers grow 00:43:58.469,00:44:04.275 and provide feedback so that they find better bugs and getting higher payouts and so if 00:44:04.275,00:44:06.878 you are interested, you know come up and talk to us. We've got basically the whole team 00:44:06.878,00:44:11.716 here in the front row. Uhh they..uh we do a lot of research and we look forward to working 00:44:11.716,00:44:14.085 with you. Thanks for coming and spending the time with us. [Clapping]